Impact Evaluation of Development Programs

During the 1990s and 2000s, a policy known as Education with Community Participation (EDUCO) not only became the cornerstone of education reform in El Salvador but also became a global education policy, one which is known for decentralizing to rural families the responsibility for hiring and firing teachers. As is shown in this paper, its rise to fame was not only a product of the particular political-economic context in which it was borne, but was also a product of the impact evaluations produced by the World Bank, which served as the evidence base through which this and other international institutions could legitimately promote the neoliberal model of community involvement represented by EDUCO. Problematically, however, a reappraisal of these impact evaluations reveals, first, that their findings and conclusions around significant effects were not warranted and, second, that the entire impact evaluation enterprise is fundamentally flawed due to the financial-political-intellectual complex out of which these studies emerged and back into which they fed as they were used to advocate for market-oriented policy solutions around the world. Thus, in addition to explaining the reform dynamics that gave rise to EDUCO, this paper (a) systematically reviews the findings and limitations of each of the six impact studies that constitute the international knowledge base around this policy, (b) reconsiders what we can reasonably claim to know about EDUCO, (c) reflects on the national and international implications of the critical review presented here, and (d) remarks on the shortcomings of—and the alternatives to—impact evaluations as a
means to produce policy-relevant findings.

This paper presents an impact evaluation of a project supporting bamboo handicrafts in selected sites in two countries: Cambodia and Laos. The project was implemented uniformly in both countries as a randomized control trial, with nearly identical baseline and endline data collected in both countries. Outcomes analyzed include the intermediate effects of the project on bamboo handicraft outcomes and the ultimate effects of the project on gender and living standards outcomes. The analysis finds that the project had a positive impact on living standards outcomes in both countries and a generally positive impact on gender outcomes, with some important differences between countries. However, estimated project effects on intermediate bamboo handicraft outcomes are not entirely consistent with the estimates of the project’s living standards impact, although they are consistent with at least some of the qualitative research findings and with the impressions gained from regular monitoring visits by the research team. The paper’s findings underscore the need for careful impact evaluations that focus on both intermediate and ultimate outcomes.

- QCA is a case based method which allows evaluators to identify different combinations of factors that are critical to a given outcome, in a given context. This allows for a more nuanced understanding of how different combinations of factors can lead to success, and the influence context can have on success.

- QCA allows evaluators to test theories of change and answer the question ‘what works best, why and under what circumstances’ in a way that emerges directly from the empirical analysis, that can be replicated by other researchers, and is generalizable to other contexts.

- While it isn’t appropriate for use in all circumstances and has limitations, QCA also has certain unique strengths – including qualitatively assessing impact and identifying multiple pathways to achieving change which make it a valuable addition to the evaluation toolkit.

Hiring local artists to collaborate on efforts intending to change values and alter behavior is an approach used by aid organizations in international development. Soft-power campaigns targeting local attitudes are often seen by the development institution as positive collaborations between foreign humanitarians and local artists. “Song for Hawa,” a 2013 collaboration between Liberian artist/rapper Takun J, and the international NGO PCI Media Impact, is a cultural diplomacy campaign intending to engage Liberians about the difficult topic of child rape. This paper discusses concerns with this instrumental approach to culture in development, and difficulties with the impact evaluation of using art to affect social change.

Using J.W. Berry's acculturation framework, this papers examines the extent that indigenous migrant minorities attribute their identity formation to national state interventions. This paper is a preliminary ethnographic report, as  part of a Master's Research Methods class.

There is a need for embracing change that creates a transformative educational landscape resulting in highly innovative, engaging instruction and improved 21st-century competencies. Administrators are driving force behind these initiatives capable to create an engaging environment for active teaching-learning mechanisms and a vision of fostering lifelong learners with a global perspective. This descriptive method of research was conducted to determine the Information Communication Technology profile capabilities and management competencies of 30 school administrators in the selected schools in the Division of Imus City. The findings of the study showed that the availability of electrical service supports the entire schools which have sufficient, but unstable technology tools and equipment. School administrators were highly proficient in utilizing funds for teachers' training despite inadequate funding to maintain technology resources. Schools were highly capable of utilizing available technology and other instructional media in designing activities appropriate for diverse learners. However, school administrators were moderately capable of ICT leadership management and ICT instructional management with a mean of 3.34. Despite limited financial resources, administrators were highly capable of their blended learning fiscal management with a mean of 3.44. Meanwhile, this paper presents a development framework that underscore the ICT resource mechanisms geared towards the 21st century management competencies of school administrators which encompass the quality pedagogy and assessment; curriculum support system; propagation of digital tools; and finance allocation for rational internet connectivity and equitable access to technology resources in school-community that increasingly employed to ensure the sustainability of the ICT capability and competence.

Islam education curriculum is increasing steadily from time to time with the sparked of several proposals and changes for the purpose of comprehensive improvement. J-QAF program for example is manifestation of strength for the Islamic Education system through Pelan Induk Pembangunan Pendidikan (PIPP). J-QAF program was firstly introduced in the year 2005 and yet very little studies about it. Purpose of this study was to see several impact of its implementation either in Institution of Teacher Education or in school by especially. The forms of this survey use the quantitative method through questionnaires as primary data and support with qualitative study through data which are glean from open ended question. Five research questions proposed which include aspects; career perception, problems, learning satisfaction, satisfaction of implementation training program and satisfaction for lecturer’s teaching beside see the strength and weakness of j-QAF's program from the sight of j-QAF’s teacher. The study’s location cover the entire Malaysia, sample number n=497 election was made with the stratified random sampling technique. Quantitative research data are analyzed using descriptive statistics (measure of central tendency) while data which are collected via qualitative study are analyzed by theme and category. Research finding showed, above 97 percent impetus of j-QAF’s teacher selects profession in teaching field is because of their own desire, more than 86 percent also said their purpose for choosing to be a teacher is caused by desire to get job satisfaction, want to serve as government employees and want to get a more secure job, above 88 percent of j-QAF’s teachers have known the undeceive objective of j QAF’s program. From the aspects of management and relationship among j-QAF teachers with the school parties and community, have been discovered that above 80 percent of j-QAF teachers can cooperate with the school parties. From the aspect of student’s achievement, above 98 percent of j-QAF teachers always prioritizing student’s achievement/control, more than 94 percent of j-QAF teachers were proud becoming j-QAF teachers, happy with their duty as j-QAF teacher and get satisfaction from work those carried out. From the aspect of learning management implementation, pedagogy and evaluation, study found above 84 percent of j-QAF teachers using various education’s method / approach. From the aspect of implementation co curriculum in j-QAF’s program, have been discovered that above 82 percent school administrator always give encouragement and support to j-QAF co curriculum activity. From the aspect of ICT's implementation in the j-QAF's program, finding research showed over 80 percent of j-QAF teachers already use the information technology and communications facility in education session. From the aspect of support from the administrator, it was discovered that more than 79 percent school administrator always provide guidance and encouragement for  j-QAF teachers and from the aspects of the use of teaching and learning material and source, above 88 percent of j-QAF teachers always use teaching resource in their education's session in j-QAF classes. Lastly from the aspect value added in the j-QAF's program, study found over 69 percent of j-QAF teachers has made additional program/activity to j-QAF students. Teacher’s perception for j-QAF’s program from the aspect of career problem, study found 55 percent of j-QAF teachers mention the main problem is the burden replacing other teacher’s class and existence of obstacle from the school administrator to apply additional allocation to achieve activities for j-QAF’s program.  Teacher’s perception for j-QAF’s program from the aspect of learning interaction in Teaching Institutes, research finding showed over 72 percent of j-QAF teachers mentions that module content is useful to become an effective teacher. Teacher’s satisfaction from the aspect of training program discover above 71 j-QAF teachers feel very satisfied for all the course program which have been carried out.  Teacher’s satisfaction for j-QAF's lecturers teaching, finding research showed above 93 percent of j-QAF teachers feel satisfied for their interaction with lecturers in IPG from the aspect of theoretical knowledge transfer, practical knowledge, emphasis on application, innovation practical activity and creativity elements in their education.  Finding from qualitative data also show the main good or advantage of j-QAF’s program is j-QAF’s teacher have gotten experience especially from the aspect of knowledge input, real teaching in classroom, overcome students problems and can build self-confidence. They also said that curriculum objective of j-QAF’s program as those desired by Ministry of Education Malaysia have been achieved in producing students who acquire the religion knowledge, skilled in recitation and writings Jawi documentation, able to read Quran and adopt basic things of Fardhu Ain, especially Solat and pupils afford to understand and use simple words of Communication Arabic Language.  On the other hand, the most important weakness in the j-QAF's program is from the aspect of material and source for education and learning. Most of the j-QAF teachers mention that there is shortage of BBM and if there is, it did not reach to their hand. In addition, there are also weakness in the aspect of lacking cooperation and monitoring of officers from District Education Office (PPD) and State Education Department (NRD). J-QAF's teachers also not satisfied about their settlement in school without taking into account their family and less cooperation from the school administrator that stems from the school management who less understand the real idea of j-QAF’s program.  This study suggests adding financial allocation especially in Kem Bestari Solat activity, the necessary of frequent monitoring by officers in PPD, NRD, JAPIM, BPG and IPG, ensure material and source for j-QAF’s program reaches to the hands teacher, and facilitate j-QAF teacher’s settlement with take into account family, security and settlement distance inland.

Using J.W. Berry's acculturation framework, this papers examines the extent that indigenous migrant minorities attribute their identity formation to national state interventions. This paper is a research proposal, as  part of an exercise in a Master's Research Methods class.

The paper reports on both methodological and substantive findings. It presents a method for generating simplified representations for regional urban populations, their geographical sub-populations and communities. the method generates greatly simplified high-resolution socio-economic profiles of populated geographical areas from complex large census data sets.

Strand: Good governance

Background

Problem: Much of Zambia’s population is poor and depends on subsistence farming. However, farming that is dependent on rains is being adversely affected by the global climatic changes. The poor agricultural seasons has seriously reduced the opportunities for farming households to feed themselves and generate income. The government, some local and international non-governmental organisations, and development agencies have embarked on programs and projects to help the adversely affected citizens. The Direct Response Emergency Cash Transfer (DiRECT) project was implemented (September 2016 to June 2017) by a consortium of Concern Worldwide Zambia (CWZ) and Save the Children (SC). The project was designed as a pilot and learning initiative. Timeliness, community participation, effectiveness and efficiency of the poverty reduction and food security interventions are some of the challenges that project implementers and decision makers have faced.

Objective of the study: The study was for the implementing and funding institutions to assess the relevance, connectedness, efficiency, effectiveness, and impact of the social cash transfer project. Lessons from running the DiRECT were to be used by the project implementers to enhance the performance of future emergence projects. They were also to be used by the Government of Zambia to inform its decision makers on whether to scale up its Social Cash Transfer programme and how to enhance the design and implementation of other shock responsive emergence social protection schemes. Other organisation having or planning to implement Social Protection interventions were also interested in the lessons from the DiRECT’s evaluation with respect to programme strategy and approach. They wanted to use them in similar future emergence humanitarian interventions that respond to food insecurity.

Objective of the paper: This paper is an effort to share knowledge on measuring the impact of social cash transfers in Zambia. It could be useful to a lot of governments, social development agencies, philanthropists and poverty reduction non-governmental organisation that are contemplating using cash transfers in poverty alleviation and/or food security interventions. It could also help those already using cash transfers to vulnerable groups to get lessons that could be relevant for enhancing their programmes.

Methods: This paper is a case study based on a Zambian project. The impact evaluation study was undertaken in the Western and Southern provinces of Zambia where the project had beneficiaries.
The evaluation used a mixed methods approach which allowed for triangulation of data sources and results. It was conducted in two out of the ten provinces of Zambia.

Qualitative Data Analysis

In order to assess performance of the project across the Development Assistance Committee of The Organization for Economic Cooperation and Development (DAC OECD) evaluation variables i) Relevance and Connectedness, ii) Effectiveness iii) Efficiency) and components such as iv) Performance of the project against the planned output, outcome and impact results, v) Lessons and vi) Recommendations, the consultants used manual analysis of field notes. This involved coding of field notes around the key themes of the evaluation and developing story lines on the patterns of results that the coding was generating. The consultants also used the following rating method (Tables 6 and 7) to determine the performance of the key evaluation variables (Relevance, Effectiveness and Efficiency).  The same approach for assessment can be applied to Outcome and Impact Results.  The method is subjective and is informed by weighing positive and negative results around an evaluation variable. The quantitative survey results with actual figures on performance are used to complement the qualitative and subjective ratings.
Limitations of the qualitative data collection included that an important tool for assessing the performance of the planned output, outcome and impact results had fewer respondents than those that had been planned for.  The consultants therefore used other sources of data field data to infer on the performance of the planned results.

A dissemination workshop of the evaluation report was held by Concern Zambia where the major players in the social protection sector; Ministry of Development and Social Welfare, international development agencies and local civil society, were invited.

Results: The project’s impact results read: “Protect lives, well-being and livelihoods, stimulate markets, prevent negative coping strategies for 25,680 Zambian households affected by El Nino”.

The End line evaluation noted using the DAC criteria of evaluation that the programme:

(i) was very a relevant intervention;
(ii) average on effectiveness;
(iii) Efficient; and
(iv) had positive outcomes against the planned results.

4. Conclusions:

Limitations of the qualitative data collection included that an important tool for assessing the performance of the planned output, outcome and impact results had fewer respondents than those that had been planned for.  The consultants therefore used other sources of data field data to infer on the performance of the planned results.

It is important to measure the impact of social transfers on the beneficiaries and market. The implementing and funding institutions by knowing what works and does not in the emergence cash transfer schemes can then make informed decisions. The DiRECT project also can be used as a model by other interested institutions to enhance the performance of their current social cash transfer programmes or the design of their future programmes.

The use of cash transfers for social protection has been scaled up by the government in Zambia across the ten provinces.

Conflict of Interest: The work on which this presentation is based was fully funded by the Commissioner of the impact evaluation; Concern Worldwide. Being the proprietary organisation of the evaluation report permission was sort from its Country Manager (Zambia), Mr Richard Mwape, for the report to be used as a the source knowledge product for this educational presentation and he granted the permission on 7th May, 2018.

KEYWORDS: evidence-informed decision-making, evaluation, monitoring, social cash transfer programmes, disaster mitigation programmes, poverty alleviation, good governance, Zambia.

Abstract
This conceptual paper aims to draw the attention of educators in higher education, technical and vocational education and training to the eminent 4th industrial revolution (4IR) and its potential implications on learning, teaching and student success. In this paper, the author makes a case that educators live in a dynamic world, a world of evolutions and technological advancements and suggests exploitation of potential opportunities to enhance learning and improve teaching.
The past four centuries are characterised by paralleled industrial and educational revolutions, commencing with industrial revolutions (IR) 1.0 through 4.0. Parallel dynamisms and technological developments happened in the education fraternity, branded as Education1.0 through Education 4.0. These dynamisms, evolutions and revolutions exert pressure on the way peoples live, work, and relate to one another. Although this is like a fairy-tale, the current wave, code-named Education 4.0 sweeping across the education sector has the potential to render the services of some education practitioners redundant. Since the 21st century students (generation Z) differ from those of the 19th century, same principle applies to the 21st century skills requirements for the educators. Moreover, these ‘educational dynamisms’ have implications on curricula content, pedagogy and professional competencies. To cope, educators in higher education and TVET need to meet the demands and pressures exerted by educational and technological dynamisms through personal, social and professional transformation. To remain relevant, educators need to deliberately adjust the way they deliver learning and teaching activities in the lecture halls and classrooms. Therefore the article recommends that higher education and TVET institutions decisively react to the challenges posed by technological advancements through curricular transformation, embracing technological advancements and continuing professional development.
Key concepts: dynamism, education 4.0, generation z, industrial revolution 4.0, transformation, 4IR skills,

Conducting and communicating research is a responsible intellectual job. A researcher would go analysing immense literature and would search for valid evidence. Prior to narrating an authentic statement. Correspondingly while selecting a method for study it is vital to opt for an approach that is ethically suitable in a defined context. While applied on a set of a populace, generally denoted to as a sample and/or universe. Therefore, social research would be a more laborious task. Hence, the social context is ever-changing in terms of time and space. Diverse methods of social research are being invented yearly. To compete with consistently changing social phenomena and needs of evidence. Operational research, evaluation, and screen monitoring are the most prominent approaches of modern social research. Hence, these naïve approaches of social research would undergo learning and adaptation. Community development projects and retrospective studies thereof are also being synthesized, with existing methods of social research. An identical practice is an adaptation of the Randomized Controlled Trial hereinafter (RCT) approach to conduct impact evaluations of humanitarian and development projects. Hitherto, RCT was being widely used by health researchers as a clinical research approach. Hence, an adaptation of this clinical research approach for field studies, particularly for the evaluation of humanitarian projects. Those are being implemented to provide survival support to vulnerable communities. It would require this approach to undergo some ethical adaptations. This research paper is developed to commence a wider literary discourse on requisite ethical adaptations for RCT to use in the evaluation of humanitarian projects. This research paper brings the findings from i Correspondence: European Journal of Social Sciences Studies-Volume 6 │ Issue 2 │ 2021 78 desk and field. To discuss key questions; where and how we can use RCT, and what ethical adaptations are necessary not to be forgone? This discourse is established on the usefulness of RCT, ethics of social research, ethics of evaluation, and humanitarian principles. The overarching purpose of this research paper is to facilitate the adaptation of RCT in the field of impact evaluation. While considering the ethical principles of the development sector and evaluation.

The Support to Training and Employment Programme (STEP) for Women was launched in 1986 by the Central Government as one of the measures to empower women in the informal sector. It is a poverty alleviation programme that seeks to provide income generating activities to women from vulnerable groups to enable their economic empowerment.
STEP has the objective of extending training in new skills or the upgradation of existing skills to provide sustainable employment opportunities to women through a variety of action oriented projects. The scheme covers ten traditional sectors of employment namely, Agriculture, Small Animal Husbandry, Dairying, Fisheries, Handlooms, Handicrafts, Khadi and Village Industries, Sericulture, Social Forestry and Waste Land development.
The present study is based on two objectives. First, to review completed evaluations of STEP, using a feminist framework, to see what the recurring themes are in evaluations of STEP projects, and what the implications are for programme design and implementation. Second, to develop a framework for evaluation that uses a gender and equity lens that could inform
future evaluations.
The study assesses what makes an evaluation a gender responsive evaluation which is reflected by the findings on the mobility and accessibility of women to institutions and resources, their participation in political and decision-making bodies, their economic and social empowerment, reduction in drudgery, increase in awareness and confidence levels, better communication skills, and access to health and sanitation facilities.
Based on the findings, the framework synthesizes the common constraints/weaknesses through the evaluations and evaluation approaches that have appeared to have worked well. It also throws light on what makes it a gender responsive framework.

Using J.W. Berry's acculturation framework, this papers examines the extent that indigenous migrant minorities attribute their identity formation to national state interventions. This paper is a preliminary survey report, as  part of a Master's Research Methods class.

Objetivo: Evaluar el impacto del Programa de Microcrédito de MIDE en la macro Región Sur del Perú, analizando la relación entre la participación en el programa de microcréditos y los cambios en las condiciones socioeconómicas y de empoderamiento de las personas que en él participan.

This Policy Note presents the potential poverty impacts of the Philippine conditional cash transfer (CCT) program, Pantawid Pamilyang Pilipino Program (Pantawid Pamilya) using simple analytical tools and the government’s largest database of poor households to date, the National Household Targeting System for Poverty Reduction (NHTS-PR). In doing so, the Note also aims to encourage policy-makers and researchers to
not only use the NHTS-PR for its intended purpose, that is, for objective targeting of social assistance programs, but also for analysis to maximize the utility of information contained in the database. The Note finds that the Pantawid Pamilya can reduce poverty incidence among beneficiaries by up to 6.2 percentage points. In program areas, it can reduce poverty incidence by 2.6 percentage points and inequality by 6.6 percent. Increasing compliance of existing beneficiaries will enhance the poverty-reducing impacts of the Pantawid Pamilya. Expanding program coverage will produce higher impacts as long as targeting and program
implementation remains properly managed.

This article proposes an approach to monitoring and evaluating advocacy that integrates theory of change and storytelling. This approach, called Narrative Assessment, addresses the feasibility of objectivity and evidence in the complex context of advocacy and proposes an evaluation methodology rooted in alternative conceptualizations of rigour and of evaluator roles. The approach centres on practical judgment and the construction and examination of stories through interaction between advocates and monitoring and evaluation specialists. The article discusses how Narrative Assessment can be useful in the evaluation of advocacy in terms of (1) monitoring and evaluation specialists' orientation toward programmes; (2) the interpretation of outcomes; (3) the assessment of outcome relevance; (4) reflection and learning and (5) the communication of programme results. The approach builds on lessons drawn from evaluation of eight advocacy programmes in international development.

The purpose of this paper is to identify best practices for achieving positive, measurable impact in the for-purpose sector, which includes non-profit organizations and social enterprises. Through an analysis of secondary sources (case studies, evaluation reports, government documents, etc.), six processes were identified which differentiate successful organizations from organizations that failed. These processes are (1) analyzing relative advantage, (2) analyzing cultural compatibility, (3) identifying impact metrics, (4) identifying and addressing barriers to adoption, (5) instituting probationary trials, and (6) regularly reviewing social impact and cost effectiveness. These findings can serve as a framework for budding non-profit organizations and social enterprises, ensuring efficiency and high impact.

This chapter reflects on the use and value of a systemic theory-based and participatory mixed-methods approach to address the challenges of impact evaluation in complex development contexts. A Participatory Impact Assessment and Learning Approach (PIALA) was developed and piloted with the International Fund for Agricultural Development (IFAD) in Vietnam (2013) and Ghana (2015) to assess and debate systemic impacts on rural poverty together with partners and stakeholders. An action research was conducted around the pilots to learn about the processes and mechanisms that can make an impact evaluation using PIALA on a limited budget rigorous and inclusive. The main conclusion from the research is that inclusiveness and rigour can reinforce each other, even more so when participatory processes and methods are employed at a larger scale. Important attributes are methodological complementarity and consistency, and extensive and robust triangulation and cross-validation. Critical to deliver quality at scale is research capacity. Investing in such capacity helps reducing the cost over time while enhancing the value and uptake of evaluation findings.

In 2015, Oxfam commissioned an impact evaluation of its work on local inclusive governance for building resilient livelihoods in the Dry Zone in Myanmar as part of its Effectiveness Review series. This blog reflects on how PIALA was adopted and adapted for this impact evaluation.

In recent years, the African Development Bank’s (AfDB) Board of Executive Directors
has strongly underlined its will to increase accountability in project design and implementation.
The Bank´s Senior Management has also committed itself to proactively promote
impact evaluations in Bank operations. However, at the moment there exists no internal
user guidelines on how to internalize Impact Evaluation in the design of either regional
operations (ROs such as the Program for Infrastructure Development in Africa (PIDA)
transport corridors) or national infrastructure interventions (NOs), including rural roads.
This article discusses the main steps for creating such a “user guide”, in order to generate
robust quantitative evaluations of the effectiveness of infrastructure interventions, which
is due to increase accountability for intervention selection, implementation and sustainability.
Its aim is to contribute to making sure the Bank´s practitioners (e.g. The Operational
Departments’ Task managers) are able to internalize these guidelines in the design and costs
of their own transport infrastructure projects, thereby contributing to the Bank’s collective
knowledge of what works (i.e. what best can deliver) when routinely building-in evaluation
components in their own transport infrastructure/corridor projects.

The Syria crisis is now moving into its ninth year. The ongoing conflict has had enormous impacts on Syrians, host communities and neighbouring countries. Australia has supported the response since 2011 by working with partners to provide humanitarian protection and assistance. In 2016, Australia committed to a three-year $220 million Syria Crisis Humanitarian and Resilience Package. This was Australia’s first multiyear package in response to a protracted crisis.
This report details the findings and recommendations from an evaluation of Australia’s investments under the three-year Syria Package. The findings and recommendations are intended to inform Australia’s ongoing response to the Syria crisis, as well as multiyear humanitarian packages of assistance in other protracted crises.

Growing calls for accountability and efficiency in the discourse of development in
donor countries has afforded the practice of impact evaluation an enviable position.
Combined with the constraints of time, resource and capacity, impact evaluations face
complex challenges in their endeavor to produce action worthy knowledge (Chambers
2000). This thesis utilizes the concepts of Coercion and Commitment in the global aid
chain (Wallace et al., 2006), to unpack the factors that shape the ‘evidence’ produced
by evaluations, by using a case study of an evaluation being undertaken by IDS in
Pakistan.

The increasing demand for measures of economic and social impacts of culture is somehow connected to a declining social perception of its intrinsic value. Coupled with a declining consensus on its necessity, spending on culture has increasingly been justified by a growing number of politicians and decision makers in its instrumental dimension, and its proved usefulness. This has undermined the automatic legitimacy of relevant investments in the sector, unless returns could be expected (best if economic and in the short term). In those countries where the bulk of financial resources for culture is public, scarcity, spending review, economic crisis, combined with the lowering of the social appreciation of culture, have dealt an heavy blow on the sector, but the same can be said of those countries where culture derive its main resources from private support.
Initially welcomed as a deserved acknowledgement of the collateral merits of culture, the social and economic benefits it is expected to generate have progressively outgrown and replaced its intrinsic value.
It is in this context that measurement of the social and economic impacts of the arts and culture has become a recurring topic in the literature of the last two decades or more. It is also a common aspiration of many cultural organisations that depend on public or private funding.
As instrumental approaches to culture and its role have gained a growing weight on the intrinsic ones, evidence sought to provide a sound basis and reliable information, upon which funding decision and policies are made, tends to document the benefits of culture for the local or national economy, social inclusion and cohesion, reduction of inequalities, and so on, while cultural impacts remain virtually unaccounted for.
The paper elaborates on the idea that intrinsic value is not to be underrated, and discusses why cultural impacts are worth investigating.


Keywords – Cultural impact assessment, cultural indicators, cultural participation, European Capital of Culture.

Impact evaluation practice has fixated on internal validity of identification, paying insufficient attention to external validity. In this paper, we propose an approach to external validity appraisal anchored in a careful understanding of the Theory of Change (ToC) of a given development intervention. Our effort expands on, weaves together and seeks to unify three emerging, valuable but seemingly disjointed external validity appraisal efforts: Woolcock’s (2013) rich, new framework focusing on key facts, Cartwright and Hardie’s (2012) free-standing offering tied in with evidence-informed decision-making and an econometric literature that has centered on selection into program participation and population attribute differences. We mitigate concerns about Cartwright and Hardie’s deliberately underspecified support factors, Woolcock’s privileging of causal density and the insufficient attention to social, institutional and political dimensions of low-income settings by Woolcock and others. Social science theory, evidence and experiential insights guide the careful mapping of a development program’s ToC and help identify critical and substantive assumptions within it. These assumptions often centre on the mechanisms of a development program whose workings may display decisive variation across settings. Accordingly, a key task in external validity appraisal is to carefully examine whether inter-setting variation is decisive or not. We suggest that the ‘how’ of external validity is often not in program replication nor in systematic reviews nor in econometric efforts, but in more ex ante attention to the theory of change and evaluation design along with ex post attention to reporting on implementation processes and refining the ToC to aid collective learning.

This paper by Adinda Van Hemelrijck and Irene Guijt explores how impact evaluation can live up to standards broader than statistical rigour in ways that address challenges of complexity and enable stakeholders to engage meaningfully. A Participatory Impact Assessment and Learning Approach (PIALA) was piloted to assess and debate the impacts on rural poverty of two government programmes in Vietnam and Ghana funded by the International Fund for Agricultural Development (IFAD). We discuss the trade-offs between rigour, inclusiveness and feasibility encountered in these two pilots. Trade-offs occur in every impact evaluation aiming for more than reductionist rigour, but the pilots suggest that they can be reduced by building sufficient research and learning capacity.

Developing countries in general and Pakistan in particular, face a huge challenge in protecting and sustaining forests. Just in the last five years total area of forest has been reduced from 2.58 percent to 2.19 percent of total area of the country. Numerous programs have been launched at national and international level to conserve the 'green gold'. The project titled 'Afforestation of Blank Areas in state Forests of Murree Hills District Rawalpindi' is the continuity of the steps taken by the Punjab Government to protect and preserve provincial forests. The major objectives of project were afforestation of 5000 acres, protection of natural regeneration of 4000 acres and plantation of 10,000 Platinus Orientalis Varnacular (Chinar) trees in blank areas of Murree hills along with other ancillary work such as soil conservation of 7000 acres. Directorate General Monitoring and Evaluation (DGM&E), P&D department conducted third party evaluation of the project to check whether planned objectives of the project were achieved or not? A comprehensive and systematic statistical methodology for the estimation of proper sample size and testing of hypothesis (i.e. standard No. 27 plants per 50 * 50 plot have been planted or not) was developed to conduct this evaluation. Evaluation was conducted based on the developed evaluation design and methodology. The results indicated that in Murree Forest Division the overall average number of pits per acre were 471, average survival rate remained 79%, overall deviation from the standard plantation was recorded +40% and protection of the plants using stones found to be very useful.

This study is based on empirical research conducted in 2021, focusing on AFPS’ development interventions in the Jordan Valley and Halhul; namely support for date palm cultivation, cooling rooms, and a production line in the Jordan Valley; and support for natural grape juice production in Halhul. It utilizes the case studies to formulate a deeper understanding of the reality of farmers in these areas, how they have interacted with development interventions, and the relevant transformations linked to the settler colonial encounter. The study engages with the discussion of different notions of development, engaging with the discussion on the implications of survival based development within  the settler colonial conditions. Through this research, we hope to contribute to our collective understanding of these themes, to better conceptualize and understand the changing circumstances and challenges of Palestinian development.

Though we have recently witnessed the “exponential production of digital data to measure, analyze, and predict educational performance” (Salajan & Jules, this volume), there has not been sufficient attention given to the quantitative methods that are used to process and transform this data in order to arrive at findings related to “what works”. This chapter addresses this gap by discussing a range of constraints that affect the main methods used for this purpose, with these methods being known as “impact evaluation.” Specifically, this chapter addresses its purpose, first, by making explicit the methodological assumptions, technical weaknesses, and practical shortcomings of the two main forms of impact evaluation—regression analysis and randomized controlled trials. Although the idea of Big Data and the ability to process it is receiving more attention, the underlying point here is that these new initiatives and advances in data collection are still dependent on methods that have serious limitations. To that end, not only do proponents of Big Data avoid or downplay discussion of the methodological pitfalls of impact evaluation, they also fail to acknowledge the political and organizational dynamics that affect the collection of data. To the extent that such methods will increasingly be used to guide public policy around the globe, it is essential that stakeholders inside and outside education systems are informed about their weaknesses—methodologically and in terms of their inability to take the politics out of policymaking. While the promises of Big Data are seductive, they have not replaced the human element of decision making.

Evaluación para el desarrollo social : aportes para un debate abierto en América Latina / Miguel Vera (editor). p. cm. 9993969788 "Esta obra es el resultado de ... el Curso de Gerencia Social para Directivos "Evaluación para el Desarrollo Social", realizado en Guatemala como parte del programa de capacitación [del] INDES"-p. 15. Includes bibliographical references.

A variety of health financing schemes shaped on pre-payment scheme have been implemented across Sub-Saharan Africa (SSA) to address the Millennium Development Goals (MDGs). In Mauritania, the Obstetric Risk Insurance package (ORI) focusing on maternal and perinatal health has been progressively implemented at the health district level since 2002. Here, our main objective was to assess the effectiveness of the ORI in increasing facility-based delivery rates, as well as increases in family planning, antenatal and postnatal care, caesarean delivery and neonatal health, from demographic and health survey data between 2002 and 2011. We also examined whether the effects of the ORI varied between strata of the population. The study was based on a quasiexperimental before-and-after design to assess the causal link between availability of ORI and increase in use of maternal health services and neonatal mortality. In combination with geographical information system, difference-indifferences and odd ratio approaches were used to address our objectives. Indicators of access to care for pregnant women and neonatal health and improved in both non-intervention and intervention groups during the study period. There was no global effect of the availability of ORI on facility-based delivery rates, nor on the use of antenatal and postnatal care services, except for qualified antenatal services. However, delivery rates in local health centres with ORI increased more rapidly than in those with no ORI, the contrary was shown for hospitals. Caesarean delivery and family planning decreased with ORI. Although late neonatal mortality rates remained low in the country, a significant decrease was seen in districts without ORI. Except for some strata of the population, ORI has not really met its objective of attracting more pregnant women towards facility-based health care.

This is the second benefit incidence analysis of the Philippines’ conditional cash transfer program
conducted by a World Bank team that uses standard measures to assess the implementation
performance of the Pantawid Pamilyang Pilipino Program. We show that despite the program’s rapid
expansion since it was piloted in 2007, it maintains good targeting accuracy, progressivity, and cost-efficiency
in delivering assistance to the poor. We also show that Pantawid Pamilya helps reduce short-term
poverty by closing the income gap of beneficiary households by up to 7.4 percentage points,
helping them afford their basic needs. Among beneficiaries, the program reduces food poverty and
total poverty by up to 6.7 percentage points. To maintain the program’s relevance, adequacy of
assistance, sustained impact on beneficiary welfare, and adjustments in the benefit level and program
coverage at a sustainable level are recommended.

During the last decade the UK Government has used area-based initiatives as a key strategy to tackle deprivation, despite the criticisms that few of these initiatives have been evaluated. This paper evaluates the impact of one such area-based initiative on employment. The New Deal for Communities (NDC) program aids financially residents of 39 of the most deprived neighborhoods in the country to design and implement policies that could tackle unemployment and improve their living conditions. Using the difference-in-difference method the paper shows that the program increased the probability of jobless individuals entering employment in NDC areas compared to residents living in similar deprived areas used as control group. The program had the greatest effect on those who were in job-training, full time education and in receipt of incapacity benefit before the program began. If the NDC program increased the supply of skilled labor and encouraged inactive individuals to join the labor force, individuals in NDC areas already in employment could as a result be displaced into unemployment or less well paid jobs. However, the evidence presented here shows that the program did not cause displacement effects among those residents who were already employed in NDC areas before the program began.

In the context of the need for urgent interventions, the resources of national budgets are (to varying degrees) limited, and the pressing need for liquidity difficult to cover, given the slowdown in economic activity. As decisions and their effects do not appear instantly on tax increases and spending cuts, the European Commission's sources of support and liquidity are to offset national efforts and increase confidence in troubled economies, and in particular of the mountain economy. During this period (February-August 2020), the European Union intervened to support national monetary and fiscal policy measures through the Temporary framework of the European Commission; Fiscal policy instruments; Multiannual Financial Framework 2014-2020, Multiannual Financial Framework 2021-2027, European Investment Plan for European Green Transactions.
In this paper we aim to highlight both the first part of the crisis (March-April 2020), the European Commission launched support programs, as well as other European financial institutions (eg European Investment Fund, European Central Bank, etc.); some have only just been withdrawn due to the fact that these funds depend on projects for the reserved areas, as well as the second stage of the crisis, ie from May 2020, the European Commission has proposed and approved new measures. on economic recovery and recovery at the European level following the COVID 19 pandemic, supporting new programs and financial reallocations for the Member States.Finally, the paper reflects the importance of impact assessment and future steps, highlighting the growing importance of this practice at different levels and for different actors involved in the sustainable development of the mountain economy at the national and European level.

Keywords: financing, mountain economy, sustainable development

JEL classification: G23, Q19, Q56

This chapter addresses issues of definition and method. The first section takes on the challenge of defining what it means to conduct a critical review of impact evaluations within the political economy of global education reform. As discussed, one must go beyond a critical evaluation of the findings of impact evaluations and beyond a critical appraisal of the production of impact evaluations to also consider the role that impact evaluations play, once produced, in the field of global education policy. Considering this role means examining their influence at the level of poli-cymaking processes, organizational agendas, personal careers, and discur-sive context, among others. The latter sections of this chapter suggest methods that can be used to put this approach to critical review into practice. The specifics of how these methods were employed in the policy case from El Salvador (the focus of Chaps. 4, 5, and 6) are also detailed.

It is crucial that scholars, but also policymakers and practitioners , have a critical understanding of the conceptual and technical limitations of impact evaluations, as well as the ways that they are necessarily affected by organizational and political dynamics. In order to achieve a critical understanding of impact evaluations, this chapter directs its attention toward both their most common form—regression analysis—as well as the form that is seen to be more robust, that is, randomized control trials (RCTs). The methodological assumptions of each of these are discussed , first, in conceptual terms before moving on to a review of some more technical issues. The final section of the chapter turns to a consideration of how the production of impact evaluations is affected by organizational and political dynamics. In all, this chapter advocates a critical understanding of impact evaluations in five senses: conceptually, technically , contextually, organizationally, and politically.