Program leadership’s decision to include an evaluator during the program planning and design phase is the critical first step necessary for evaluators to provide the programmatic benefits associated with the evaluation profession. Several recent developments have promoted evaluator inclusion in program planning and design activities, including federal legislation that mandates evaluator inclusion and advocacy efforts from evaluation academics. However, the evaluation literature presents a collective frustration within the evaluation field due to ongoing exclusion from program planning and design activities. Utilizing the defensive attribution hypothesis, this quantitative study gathered responses from 260 American Evaluation Association members and 61 Project Management Institute members to determine an evaluator exclusion rate, develop a taxonomy of exclusion factors, and explore the extent to which program leaders and program evaluators demonstrate defensive attributions when rating these factors’ influence on evaluator exclusion in program planning and design activities. Results indicated an approximately 70% evaluator exclusion rate in respondents’ most recent program experiences. Furthermore, the defensive attribution hypothesis was not supported in the study, as program evaluators more strongly attributed their lack of inclusion to deficiencies outside of the evaluation practice, but program leaders also more strongly attributed evaluator exclusion to deficiencies outside of the evaluation practice. Program evaluators most strongly attributed their exclusion to program leaders’ insufficient training and knowledge on the role of evaluation during the program planning and design phase. Program leaders most strongly attributed evaluator exclusion to their own staffing decisions, indicating a preference to not include evaluators in program planning and design activities due to achieving previous program success without them, assigning evaluation activities to non-evaluation staff, and a funding process that allows the practice to occur. As the first study to explore evaluator exclusion in the program planning and design phase, it sets a foundation for future research studies to corroborate and build upon its findings, identify policies that encourage evaluator inclusion, and continue efforts to establish mutually beneficial relationships in the program planning and design phase.

Knowledge advancement occurs when the creation of new and useful knowledge encompasses and supersedes earlier knowledge. A rapidly growing number of scholars with state-of-the-art research tools has led to the growth of knowledge exploration in almost every field. It, however, has been observed that the findings of new studies frequently differ from previously established evidence and even disagree with one another. Conflicting and contradictory results prevail in the literature. This phenomenon has puzzled many people with respect to which findings are reliable and which should be considered as valid. Inconclusive results in the literature inhibit, rather than facilitate, knowledge advancement in sciences. Meta-analysis, which is referred to as the analysis of analyses, designed to synthesize findings from a large collection of quantitative analyses that produce inconsistent results has become a major research method in the fields of medicine, education, and psychology; however, the method has been slow to penetrate research in nonprofit and public management (NPM). This study, therefore, discusses how meta-analysis contributes to knowledge advancement in the fields of nonprofit and public management by using nonprofit commercialization as an example to examine its impact on nonprofit capacity and donations, respectively. The attention of this discussion is directed toward how the use of meta-regression models is able to offer new and useful knowledge that encompasses and supersedes earlier knowledge in the literature with evidence-based results. Moreover, this study examines whether the use of SEM-based meta-analysis produces equivalent results when compared with results from traditional meta-regression models. The comparison results suggest that the use of SEM-based meta-analysis is able to produce equivalent results even when missing data are present. Overall, this study makes at least two contributions. First, it introduces a newly-developed method for conducting meta-analysis to the field of NPM. This method is especially useful when there are missing data in data sets. Second and most importantly, this study demonstrates how knowledge advancement in NPM can be achieved by conducting meta-analysis.