Publication Date

5-2019

Date of Final Oral Examination (Defense)

2-25-2019

Type of Culminating Activity

Thesis

Degree Title

Master of Science in Organizational Performance and Workplace Learning

Department

Instructional and Performance Technology

Major Advisor

Seung Youn (Yonnie) Chyung, Ed.D.

Advisor

Donald J. Winiecki, Ed.D., Ph.D.

Advisor

Soo Jeoung “Crystal” Han, Ph.D.

Abstract

Organizational performance improvement cannot be expected without continuous monitoring and evaluations. Nonprofit organizations (NPOs) funded through government grants and other funder contracts face high expectations for evaluation and reporting of data as they are held accountable to their funders. However, NPOs share common struggles in both measuring and building evaluation capacity, which refers to the ability for an organization to conduct an effective evaluation that meets accepted standards of the discipline (Milstein, Chapel, Wetterhall, & Cotton, 2002). Thus, the purpose of this research was to answer the following research question: How does an NPO use an ECB framework and its tool as a guide while building evaluation capacity?

In order to gain a deep level of understanding of ECB, the researcher reviewed available literature on ECB frameworks and tools and conducted a case study within an NPO labeled “Foothills” (pseudonym) in the Northwest region of the United States to investigate how the organization used a selected ECB framework and its assessment tool towards engaging in ECB. The case study included a series of three focus groups with five key stakeholders of the organization during September through November of 2018 using the organizational evaluation capacity self-assessment developed by Bourgeois and Cousins (2013). The researcher selected this framework based on her assessment at that time that the organization might lack evaluation knowledge and skills and that the comprehensive nature of their framework and associated self-assessment tool would help the organization improve its evaluation capacity. The stakeholders completed the self-assessment tool individually first, and then engaged in focus groups to discuss their experience with the assessment tool, reflect on their current ECB capacity, and make their future ECB plans for continuous organizational improvement.

The focus groups revealed limited evaluation capacity among the stakeholders. Through all three focus groups, the participating stakeholders expressed concern about not understanding terminology and questions used in the assessment. Key areas that participating stakeholders struggled with were defining the evaluation unit, evaluation lead, and the various evaluation projects and processes that the organization had participated in, as indicated by an increase of ‘Don’t Know’ responses on the assessment and discussion within the focus groups. This supported the stakeholders’ determination that their foundational knowledge in evaluation was limited. This lack of knowledge in these terms and functions hindered the stakeholders’ ability to both answer some of the questions during the self-assessment and fully understand the role of evaluation within the organization.

From this research, it seems that foundational knowledge of evaluation and performance improvement practices may be a prerequisite for organizations to complete this type of assessments and the ECB process. However, this leaves NPOs that have low levels of evaluation capacity with few resources and options to build their evaluation capacity. To better fit the lack of resources and knowledge for NPOs with low and developing capacities, it may be worth establishing ECB resources to grow their foundational knowledge, prior to engaging in more advanced ECB work with the help of a skilled evaluator. This could also save the overall costs for NPOs with low and developing capacities as they use other available tools to build foundational knowledge before hiring a potentially expensive evaluation expert for the rest of the ECB and evaluation processes.

If researchers and practitioners work to build these tools and resources to help improve NPOs’ evaluation capacity, organizations should then be able to participate more fully in using human performance improvement (HPI) principles to better monitor their overall impact, identify performance issues, and measure the effectiveness of implemented solutions. This will lead to NPOs that are able to operate more effectively and efficiently, create increased funding, and produce greater impacts within the communities they support.

DOI

10.18122/td/1554/boisestate

Download

Share

COinS