Impact Evaluation of Government Projects

Лист вiд  Державної служби України з питань безпечності харчових продуктів та захисту споживачів №В-2151/Г-1695 вiд 06.10.21 року - це результат роботи порталу Урядова «гаряча лінія» України 1545, що є механізмом для взаємодії Уряду з громадянами, що дозволяє оперативно вирішувати проблемні питання, які порушуються у зверненнях громадян. "Звернення приймаються працівниками Урядового контактного центру за допомогою засобів телекомунікації і надсилаються органам виконавчої влади відповідно до компетенції, Секретаріату Кабінету Міністрів України, Офісу Президента України, державним колегіальним органам для вирішення порушених у них питань по суті, надання відповідних роз’яснень." - https://www.kmu.gov.ua/uryadova-garyacha-liniya-1545

In these recent years, the interoperability of e-Government applications has been an extremely important research and development ICT domain to facilitate the seamless exchange of information across public and private sectors department of government. Many researchers and practitioners have focused on the adaptation and design of Government Interoperability Frameworks to do a form of relationship in the E-Government implementations. The major elements relevant to E-Government interoperability are conformance and compliance and there are a large number of Government applications that are available through the web, but it is difficult to automate interoperation because of the heterogeneity in developing these E-Government applications, the heterogeneous platforms on which the applications run, the complexity of web interface that is not machine-friendly and the Non-standard format used for data input/output. In this paper web services and business process engines have been developed as a standard XML-based model for message exchange between heterogeneous applications by using a web services choreography language to remedy these problems.

This Impact Report identifies and summarises the diverse impacts, resulting from the £500m of UK funding of Science and Technology in 2013, using numerous quantitative metrics and short case study extracts.  It shows how the varied impacts of a national science and technology organization and its many funded programmes can be annually identified, captured, tracked, and summarised in a public document.

The report was produced for, and by, the Science and Technology Facilities Council (STFC) -  the UK Research Council responsible for funding: (1) all UK astronomy, particle and nuclear physics research, (2) related UK science and technology facilities (e.g. as Harwell and Daresbury), (3) the UK contribution to international science (e.g. CERN, international telescopes and space missions), (4) innovation and business spin-off programmes, and (5) UK public engagement activities.

The Report was co-researched and co-authored by the STFC Impact Team, including myself while Impact Evaluation Manager in 2013, with Dr. Claire Dougan-McCallie (lead author/Head of Impact) and Jenny Beard, with additional input from senior managers, and many data and case study contributions from other key managers across STFC.

Varied key impacts are identified and reported including:

• Research amongst the best (by citation impact) in astronomy, nuclear and particle physics.
• Supported research of 226 Principal Investigators in 70 universities, led to 1,100 peer-reviewed papers in 2012.
• Facilities (the Diamond Light Source, ISIS and CLF) were used in 2012/13 by over 4,200 unique users , in over 2,400 experiments, producing over 1,000 papers in peer-reviewed journals.

• Technology from CERN benefited the UK economy by over £100 billion every year.
• Funding the development of the UK’s space, internet and computer animation industries, returned over £500 billion to the UK economy per annum.
• Funded Innovation Campuses hosted over 230 enterprises and supported over 5,000 jobs.
• UK industry won £43 million in contracts from the international subscriptions funded, and had accumulated a total of £150 million since 2005.

• Funded research inspired future generations to study STEM subjects, attracting 90% of UK undergraduates to study physics, funded 782 PhD students  and 16,800 student training days.
• Between 2009 to 2012 public engagement programmes engaged people on 58 million occasions, with 2 million people engaged in face-to-face activities in 2013 alone.

This book posits the Scheduled Castes (SCs) and Scheduled Tribes (STs) vis-à-vis their upper-caste Hindu peers and
establishes how caste is a lived reality in everyday life in modern India.
It explores areas where caste and religious exclusion are most visible, such as human development, inequality, poverty,
educational attainments, child malnutrition, health, employment, wages, gender, and access to public goods. With an in-depth
theoretical foundation and empirical analysis, it establishes that in each of these sectors, the performance of upper caste
Hindu households is far better compared to that from the SC, ST, and Muslim households.

Assessing the returns of public sector investments in Information Technology (IT) has been the subject of a round of debates between practitioners and the academic researchers. The range and complexity of government Information Technology (IT) investments and broad community of benefit owners makes assessing investment returns a daunting prospect. This paper provides a short review of existing literature on Public Return on Investment (ROI) and the general approaches to measurement of such returns. It also presents some examples of successful Public ROI programs and practices in the United Arab Emirates (UAE). The intention of this paper is to share experience and knowledge with the wider research and practitioner community in different governments.

This study aimed to survey e-procurement in the Public Sector with a view to investigate the challenges in adoption of e-procurement. Current literature on Public Procurement indicates a shift towards adoption of new technologies in Supply Chain processes that include e-procurement. These technological changes in the external business environment are eliciting various responses from Supply Chain actors in the public sector. The level of public sector response is influenced by various factors posing as challenges to adoption of e-procurement. The existing literature indicates some of the challenges arising while responding to these technological changes include lack of employee competence, inadequate legal framework, inadequate technological infrastructure for integrating e-commerce with other systems and concerns about security of e-procurement business transaction data and information. These perceived challenges in informed the purpose for this study. The study population included state corporations under the Ministry of Finance. Due to time and financial constraints, the researcher carried out a simple random selection of three respondents from the procurement departments of all the 16 corporations giving a sample size of 48. Both primary and secondary data was used for the study. The research study used a questionnaire as a key instrument for primary data collection. Qualitative data was analyzed through content analysis. Quantitative data was analyzed through the use of frequency distribution, mean scores and standard deviations. In addition, one-way analysis of variance (ANOVA) tests was carried out.  From the study, it was revealed that employee competency, inadequate legal framework, inadequate technological infrastructure and security of procurement transaction data was a challenge to e-procurement adoption in the organizations under review. The study recommended that among others, due to continuous turnover of the employees', continuous training for the incoming staff is required on eprocurement. In addition, formal recognition backed by legislation of the electronic procurement transactions should be encouraged. Integration of the Organizations system and those of the suppliers, demonstration of the positive impact of the system, and installation of linkages between all Governments agencies should be encouraged.

Keywords: E-Procurement Adoption, Electronic Procurement, Public Procurement

This research was funded by, and implemented within, a UK national research organisation, while the author was the KE and Impact Evaluation Manager responsible for assessing impact on a £15m UK government-funded research programme.

This work was also submitted as the authors MBA dissertation which was accepted and awarded in 2009.

ABSTRACT: A methodology and methods ‘toolbox’ is proposed for conducting impact research and assessment from within research programmes. The emphasis is upon how to clarify and assess the wider impacts from the early stages onwards.

The approach explicitly acknowledges, and attempts to address, the known issues and challenges of impact assessment while drawing upon the resources and opportunities available to research managers and researchers within such programmes.

The study has two triangulating foundations. Firstly, it examines the academic literature to help identify and design the pilot methodology and methods. Secondly it trials these in a real programme to both test and improve them.

The study was conducted over one year within one leading UK research organisation, where two research programmes (and around 60 research projects of £15m total value) were being assessed for their impacts. The study draws upon actual impact management experience and reflection, prospective (ex ante) and retrospective (ex post) assessments, participatory contributions from leading researchers undergoing assessment, and consultation with external users. The findings of this study outline the pilot and its first iteration. Recommendations are made for improvement to the methodology and methods.

A subsequent follow-on bid to UK government incorporating the impacts identified from implementing this methodology, successfully led to a further £15m of government funding.

It is expected the pilot methodology and methods will be applicable to other organisations, conducting and funding such research programmes, required to research and assess the wider research impacts.

Although primary focus of implemetation trials was upon research in physical sciences, technology, high-tech engineering, computing/IT, and environment, the wider literature study, the methodology design and justification, the methods proposed and used, and the resulting general recommendations and lessons are expected to apply more broadly to many other research areas, where the impact is to be found more widely in the society, economy, government or public sector or within organizations, policies, professions or the public.

Interest in this work is expected from research programme managers, impact managers and assessors, management consultants advising in this area, and government funding managers, all seeking to embed a developing approach to understand, capture, and enhance wider impact.

This paper presents findings from the evaluation study carried out to review the UAE national ID card software. The paper consults the relevant literature to explain many of the concepts and frameworks explained herein. The findings of the evaluation work that was primarily based on the ISO 9126 standard for system quality measurement highlighted many practical areas that if taken into account is argued to more likely increase the success chances of similar system implementation projects.

The paper evaluates language policies of the ethnic republics of Russia titled aft er the ethnic groups speaking Finno-Ugric languages in order to understand why the policies had limited impact on their sociolinguistic situations. Th is is an empirical-analytical study based on quantitative research that investigates within the framework of policy analysis the link between policy outputs and outcomes in order to test the hypothesis that changes in behaviour and attitudes can be traced back to the patterns of language management. Th e study produces a systematized set of data on measures taken to create the conditions for language knowledge, use and attitudes in the republics and searches for correlations among variables. Th e sources of quantitative data include legal and other offi cial documents, accessible offi cial statistics and available sociological and sociolin-guistic surveys. Such a policy evaluation contributes to the theoretical understanding of the limits of the revivalist project pursued as a "top-down" public policy.

The Millennium Development Goals is a United Nation’s initiative that focuses on the eradication of poverty, environmental protection, human rights and the protection of the vulnerable. Philippines is one of the third world counties that aims to break the cycle of poverty. In 2007, the government come up with a program known as Ahon Pamilyang Pilipino (APP) Program under the administration of Pres. Gloria M. Arroyo which becomes Pantawid Pamilya Pilipino Program in 2012 under the administration of Pres. Benigno  C. Aquino.  The nature of the program was adapted from the countries in Latin America such as Brazil, Mexico and Columbia.  It is a conditional cash transfer program for the identified eligible beneficiaries and has been implemented nationwide for a decade. The program provides social assistance (financial – short term) and social development (human capital - long term).  Cash grants are based on compliance to the program’s conditionalities that focus on health, education and family development as an advocacy for empowerment especially the women. The major roles of the implementers are to monitor the beneficiary’s compliance and; provide referrals to whatever the family needs skills training, employment, capital assistance, medical assistance and more. The program is in partner with different stakeholders to fully provide the services needed and appropriate for every household. 

It has been proven that no program is perfect, there must be some loopholes during the course of the implementation.  Critics from different agencies sprout everywhere to evaluate and question the effectiveness of the program.  However, program advocates surrender their selfless dedication and passion believing that everyone deserves a chance to change and improve.  The main goal of the program is to uplift the level of well-being of every household from survival, subsistence to self-sufficient.  Whatever outcome or impact will the program have with the clients, is the government’s legacy.  This article shows milestone of every beneficiaries of the program.

The Root & Tuber Improvement and Marketing Programme (RTIMP) in Ghana was implemented by the Ministry of Food and Agriculture (MoFA) between 2007 and 2014, and supported through a US$ 18.83 million loan from the International Fund for Agricultural Development (IFAD). In anticipation of the completion of RTIMP and the start-up of the follow- up GASIP (Ghana Agricultural Sector Investment Program), the Ministry of Food and Agriculture (MoFA)and the IFAD Country Office jointly commissioned a major, nation-wide impact evaluation of RTIMP. The evaluation would serve as a learning exercise to flag emerging issues from RTIMP that merit closer attention in the GASIP, and it would therefore complementing RTIMP’s other M&E and supervision processes. The evaluation used the novel Participatory Impact Assessment & Learning Approach (PIALA), a theory-based and mixed-methods approach developed with support from IFAD and the Bill & Melinda Gates Foundation (BMGF).

Con el propósito de fundamentar la propuesta denominada “El Ciberempleo: un esquema laboral innovador entre las instituciones públicas del estado de Guanajuato”, se preparó este documento para apoyar la toma de decisiones institucionales al respecto, en donde se revisó el marco legislativo federal y estatal que normativamente podría incidir en el esquema laboral basado en Ciberempleo.

1. Background of the study According to the World Bank (2016a) numerous developing countries face common set of challenges in meeting the increasing economic and social demands of globalization. As a process of economic integration and market openness to external trade, globalization offers the possibility of accelerated economic growth. Yet to take advantage of the opportunities that globalization presents, countries must be able to increase their competitiveness by developing a competent and flexible workforce. The best known way of developing a well suited workforce to drive economic growth is by educating the population/citizens of a given country (UNDP, 2015).

This chapter reflects on the use and value of a systemic theory-based and participatory mixed-methods approach to address the challenges of impact evaluation in complex development contexts. A Participatory Impact Assessment and Learning Approach (PIALA) was developed and piloted with the International Fund for Agricultural Development (IFAD) in Vietnam (2013) and Ghana (2015) to assess and debate systemic impacts on rural poverty together with partners and stakeholders. An action research was conducted around the pilots to learn about the processes and mechanisms that can make an impact evaluation using PIALA on a limited budget rigorous and inclusive. The main conclusion from the research is that inclusiveness and rigour can reinforce each other, even more so when participatory processes and methods are employed at a larger scale. Important attributes are methodological complementarity and consistency, and extensive and robust triangulation and cross-validation. Critical to deliver quality at scale is research capacity. Investing in such capacity helps reducing the cost over time while enhancing the value and uptake of evaluation findings.

Information Communication Technological innovation is assumed to help economic growth. Over Last three decades investment on Information communication technology in Sri Lanka has rapidly increased with the objective of developing the economy, reducing poverty and ultimately improving the quality of life. These kinds of investment & project now become ubiquitous in development arena. But still, it is a question that what is the best way to evaluate & identify success or failure of Information Communication Technology (ICT) for development initiatives in human development dimension. 

This paper discusses the use of Capability Approach (CA) which conceived in 1980 by Amartya Sen as a frame work to evaluate welfare economic.  Sen (2001, p 38) identified “five type of instrumental freedom” is used to analyze the best Information Communication Technology for Development (ICT4D) applications (Winners of e-Swabhimani) developed in Sri Lanka. This paper concludes with a review of the high and low impact areas of capabilities advanced through ICT for Development initiatives.

About the course The course aimed to address an important prerequisite for incorporating impact evaluation (IE) into programme design: a theoretical and practical understanding of IE approaches to enable selection of appropriate methodologies, coupled with careful appraisal of the resulting evidence. The course introduced participants to current quantitative as well as qualitative evaluation techniques for impact evaluation and gave them an insight into the critical understanding of the roles they can play in the design and assessment of public policy and development interventions. The group enthusiastically tackled the difficulties of learning STATA,NVIVO and the intricacies of evaluation research designs and econometric techniques, all bolstered by the support of the experienced teaching team. Our trainers will be issued with trainees training manual with an pre-evaluation tests to be done for 30 minutes before you begin the training. There will also be a post training evaluation to be given by the course coordinator and the feedback will be given before issue of certificate and graduation. We hope you will enjoy our professional touch to this training. The theoretical modules will be integrated with the statistical analysis using SPSS, STATA and Nvivo to enhance learning retention and conceptualization of the key concepts. The trainee manual has gaps to enhance interaction with the facilitator so you will be expected to note down the key points discussed throughout the training. The manual has procedures of of various steps together with several well researched examples from practical real life project impact evaluation assignments out from our team of practitioners with over 10 years' experience evaluating development projects across several nations. Target Participants This course is targeted at researchers, project staff, managers, and development practitioners, policy makers in government, NGOs and development organizations who want to determine the impact of their interventions. Course duration September 9 th to 20 th Venue: Nairobi 10 Days USD 2000 exclusive VAT of 16% What you will learn By the end of this course the participants will be able to:

In recent years, the African Development Bank’s (AfDB) Board of Executive Directors
has strongly underlined its will to increase accountability in project design and implementation.
The Bank´s Senior Management has also committed itself to proactively promote
impact evaluations in Bank operations. However, at the moment there exists no internal
user guidelines on how to internalize Impact Evaluation in the design of either regional
operations (ROs such as the Program for Infrastructure Development in Africa (PIDA)
transport corridors) or national infrastructure interventions (NOs), including rural roads.
This article discusses the main steps for creating such a “user guide”, in order to generate
robust quantitative evaluations of the effectiveness of infrastructure interventions, which
is due to increase accountability for intervention selection, implementation and sustainability.
Its aim is to contribute to making sure the Bank´s practitioners (e.g. The Operational
Departments’ Task managers) are able to internalize these guidelines in the design and costs
of their own transport infrastructure projects, thereby contributing to the Bank’s collective
knowledge of what works (i.e. what best can deliver) when routinely building-in evaluation
components in their own transport infrastructure/corridor projects.

This paper by Adinda Van Hemelrijck and Irene Guijt explores how impact evaluation can live up to standards broader than statistical rigour in ways that address challenges of complexity and enable stakeholders to engage meaningfully. A Participatory Impact Assessment and Learning Approach (PIALA) was piloted to assess and debate the impacts on rural poverty of two government programmes in Vietnam and Ghana funded by the International Fund for Agricultural Development (IFAD). We discuss the trade-offs between rigour, inclusiveness and feasibility encountered in these two pilots. Trade-offs occur in every impact evaluation aiming for more than reductionist rigour, but the pilots suggest that they can be reduced by building sufficient research and learning capacity.

Out-of-school youth (OSY) rate is increasing nowadays due to the high cost of education. In order to alleviate this, the city government of Davao implemented the Scholarship on Tertiary Education Program (STEP) in 2008 to provide economic assistance to underprivileged yet deserving students of Davao City. The study aimed to examine the effectiveness and efficiency of STEP by identifying, evaluating, comparing the costs and benefits, and analyzing if it is worth pursuing on the next years. Data were gathered through personal interviews with the STEP in-charge and 5 of its 33 graduates. The ex-post impact of STEP was assessed through Cost Benefit Analysis (CBA) approach that compares the opportunity cost of the STEP graduates who went to college after high school, which is the salary they could have earned if they worked after high school, against the opportunity cost of the conceptualized counterfactual scenario which is the college degree foregone if they chose to work first before entering college The opportunity costs were measured in terms of the length of time STEP graduates can recuperate the 4-year tuition fees based on their annual income and the length of time that the non-STEP recipients will need to work in order for them to enroll to college based on Region XI's minimum wage. The Benefit-Cost Ratio (BCR) of 6.57 showed that STEP has a positive impact since it takes 0.632 years or 7.6 months for STEP graduates to recompense their tuition, while it takes 0.934 years or 11.2 months for high school graduates who chose to work first to gain enough money to proceed to college. Therefore, getting a college degree through scholarships like STEP is more beneficial than working before entering college. Thus, STEP is worth pursuing and should be given an increased budget to cater more students to lessen the number of OSY.

This article provides some insight into the implementation of the national ID programme in the United Arab Emirates (UAE). The fundamental aim is to contribute to the existing body of knowledge in the field, as it sheds light on some of the lessons learned from the programme that is believed to widening the knowledge horizons of those involved in such initiatives.

Though we have recently witnessed the “exponential production of digital data to measure, analyze, and predict educational performance” (Salajan & Jules, this volume), there has not been sufficient attention given to the quantitative methods that are used to process and transform this data in order to arrive at findings related to “what works”. This chapter addresses this gap by discussing a range of constraints that affect the main methods used for this purpose, with these methods being known as “impact evaluation.” Specifically, this chapter addresses its purpose, first, by making explicit the methodological assumptions, technical weaknesses, and practical shortcomings of the two main forms of impact evaluation—regression analysis and randomized controlled trials. Although the idea of Big Data and the ability to process it is receiving more attention, the underlying point here is that these new initiatives and advances in data collection are still dependent on methods that have serious limitations. To that end, not only do proponents of Big Data avoid or downplay discussion of the methodological pitfalls of impact evaluation, they also fail to acknowledge the political and organizational dynamics that affect the collection of data. To the extent that such methods will increasingly be used to guide public policy around the globe, it is essential that stakeholders inside and outside education systems are informed about their weaknesses—methodologically and in terms of their inability to take the politics out of policymaking. While the promises of Big Data are seductive, they have not replaced the human element of decision making.

Impact-what does it mean and how do we know what "counts"? We all want to do work that has an impact, and this is true of all sectors, whether that be government, public, private, not-forprofit, university, education, or community stakeholders. However, understandings of what it means in practice, what it takes to achieve, and how it can be tracked and calculated remain largely unclear and contested. While the rhetoric of "impact" and the "impact agenda" has become popular in the last decade or so, our practice and research appear to be lagging. In this introductory paper to the special issue on Impact into practice: Demonstrating applied public administration and policy improvement we outline how systems thinking approach can aid understanding of research and education impact on government practice. A systems approach reveals where reliance exists, where responsibility falls, and where new and deepened relationships are needed. While more needs to be done by all parties to acknowledge the collective nature of impact and the necessary reliance on one another, we argue that redistribution of responsibility is needed, including the government's significant role. Without collective recognition of reliance, responsibility, and relationships in the system of impact, our respective endeavors can only be expected to go so far. By thinking about impact as a system, we can end the "blame game" between university and government sectors, and encourage action within and across sectors, in the pursuit of better outcomes for citizens and society.

This study expounds the impact of the evaluation of academic projects with and without metrics. It is based on real-time data gathered from term-end academic projects of six batches of students at BIIT, Yelagiri Hills. It validates its result with tables and graphs to show the distinction between the two types of evaluation in software academic projects. The data for three years without measurement and with measurement confirm that the evaluation with measurement improves the students' performance in their project in terms of timeframe, output and quality. It concludes that measurement -based software engineering education helps the learners and trainers to identify learners' strengths and improve further.

Government 2.0. involves the use of Web 2.0 technologies by governments to create a transformative opportunity related to key public sector issues such as transparency, collaboration, communication and accountability. Despite its impressive potential, it has hardly been realized yet in practice. This study thus investigates the level of awareness and perceived motivation towards the use of Government 2.0. The study adopted a survey design approach and collected data using a questionnaire administered to citizens accessing government services delivered through Federal Inland Revenue Services (FIRS) and the Federal Road Safety Commission (FRSC) offices in Lagos and Ibadan, Nigeria. A total of 311 respondents participated in the study. Two research questions were answered using descriptive analysis. Three research hypotheses were also tested at 0.05 level of significance, using regression analyses. Findings show that respondents are more aware of the Twitter handle of FIRS (x=0.8505). The findings also revealed that there is a significant influence of awareness on continual usage intention of Government 2.0 (F =24.757; R=.274; R2=.075; p<.05), and that there is a significant influence of intrinsic motivation, extrinsic motivation on continual usage intention of Government 2.0 (F=86.602, R=.602; R2=.362; p<.05). The study recommends that motivational incentives, accessibility, open data and shared knowledge should be incorporated into government social media account as this will motivate citizens to use the services continually.

El Servicio de Impuestos Internos (SII) ha venido efectuando en los últimos 10 años una serie de innovaciones tecnológicas que han impactado al personal del SII, a los usuarios del Servicio (contribuyentes), que son las organizaciones que tributan y la gran mayoría de los ciudadanos del país. Pero más sorprendente ha sido el efecto en los servicios de impuestos internos de los países desarrollados por la gran eficiencia que han logrado por medio de la incorporación de la tecnología.
El presente trabajo tiene como objetivo medir el impacto que dichas innovaciones han tenido en la percepción de los contribuyentes, el grado de conocimiento de las mismas y uso de ellas por parte de los usuarios. También se pretende establecer que el avance tecnológico del SII ha significado un aumento en la cantidad de usuarios de Internet en el país.
*