2023 · Q2 · Approaches to policy and programme evaluation (20 marks)

Source: PUBLIC ADMINISTRATION 2023.pdf, PDF page 1 · Answer allocation: 20 marks

Original examination question

What are the different ways in which policies and programs can be evaluated? Discuss the approaches and explain the distinctions in these approaches.

Bullet outline

• Evaluation judges merit, results and value using explicit criteria.
• Distinguish ex ante, formative/process, outcome/impact and ex post evaluation.
• Compare experimental, quasi-experimental and theory-based approaches.
• Explain economic and participatory evaluation.
• Choose methods according to the question, ethics and evidence.

Full exam-ready answer

Introduction

Policy and programme evaluation systematically examines the design, implementation, results and value of public interventions. It differs from monitoring, which routinely tracks activities and indicators. Evaluation asks why results occurred, whether objectives were appropriate and whether an intervention deserves continuation or modification. Approaches differ by timing, purpose, causal method and evaluative criteria.

Timing and purpose

Ex ante evaluation assesses a proposal before implementation. It examines needs, alternative designs, feasibility and expected costs and benefits. Its purpose is to improve choices before resources are committed, although its conclusions depend on uncertain forecasts.

Formative evaluation supports improvement while a programme is being developed or implemented. Process evaluation examines whether activities reach intended participants and operate as designed. It can reveal delivery failures even before final outcomes become measurable.

Summative or ex post evaluation judges performance after a meaningful period of operation. Outcome evaluation examines changes in relevant conditions, while impact evaluation seeks to identify the changes attributable to the intervention rather than other influences. A rise in school attendance after a programme begins does not alone prove causation.

Approaches to causal explanation

Experimental evaluation uses random assignment, where feasible and ethical, to establish comparable treatment and control groups. It can strengthen causal inference, but may be costly, politically difficult or unsuitable for nationwide institutional reforms.

Quasi-experimental approaches use comparison groups or statistical designs without random assignment. Methods such as difference-in-differences can estimate effects, but their credibility depends on assumptions about comparability and trends. Weak data can undermine apparently precise results.

Theory-based evaluation examines the programme's causal logic: how activities are expected to generate outputs and outcomes, and under which conditions. Case studies, interviews and mixed methods help explain mechanisms and context. These approaches are valuable for complex interventions but require careful testing rather than simply accepting the programme's own narrative.

Criteria and stakeholder perspectives

Economic evaluation compares resources with results. Cost-effectiveness analysis compares costs per unit of a common outcome, while cost-benefit analysis seeks to express benefits and costs in monetary terms. Both support allocation decisions, but monetising dignity, equity or environmental values can be controversial.

Participatory evaluation involves affected communities and implementers in defining questions and interpreting findings. It can reveal neglected experiences and improve ownership, though power imbalances may shape whose views are heard.

Conclusion

No single approach answers every evaluation question. Pakistan's public organisations should match methods to purpose, data and ethical constraints, combining quantitative evidence with contextual understanding. Credible evaluation makes assumptions transparent, distinguishes contribution from demonstrated causation and communicates limitations. Its ultimate value lies in improving public decisions rather than merely producing favourable performance reports.

Study note: This is an original model answer, not an official FPSC marking scheme. The outline is for revision; the prose below it provides the complete answer. Marks guide relative depth rather than a prescribed word count.