Key Aspects of Evaluation: From Efficiency to Sustainability

When a government scheme, an NGO programme, or an urban infrastructure project comes to an end, one question matters more than any other: did it actually work? Answering that is the job of evaluation. But “did it work” is rarely a simple yes or no. A project can hit all its targets yet waste enormous resources. It can change lives in the short term and leave nothing behind a few years later. To make sense of all this, evaluators rely on a set of distinct aspects, each asking a different question about a project’s worth. These aspects build on the framework that the OECD Development Assistance Committee first laid out and which agencies worldwide now use. Understanding them helps you judge any intervention honestly, whether it is a metro rail line, a sanitation drive, or a skill-development programme.

Efficiency and effectiveness: two questions, not one 🔗

Effectiveness and efficiency sound similar, and they are often confused. They measure two completely different things. Keeping them apart is the first discipline of good evaluation.

Effectiveness: did we achieve what we set out to do? 🔗

Effectiveness measures the extent to which a project achieved its planned objectives and intended outputs. It compares results against the targets set during planning. If an urban housing scheme aimed to build 1,00,000 affordable units and delivered 85,000, its effectiveness can be roughly stated as 85 percent of the goal. The question is straightforward: were the planned results delivered?

But effectiveness is not only about hitting a headline number. A careful evaluator also asks whether the quality of those targets was good in the first place, and whether the benefits reached everyone they were supposed to reach. A scheme can report strong aggregate success while quietly excluding the poorest or most remote groups. This is why effectiveness evaluation increasingly looks at how results are distributed across different groups, not just the overall total. A water supply project that connects 90 percent of a city but leaves out every informal settlement is far less effective than the single number suggests.

Efficiency: did we get good value for the resources used? 🔗

Efficiency measures productivity in relation to resources. It compares the outputs a project produced against the inputs it consumed: money, time, staff, and materials. Efficiency examines whether a project delivered results in a cost-effective and timely way, weighing both the quantity and quality of outputs against what was spent to achieve them.

This is where the difference becomes clear. Two road-building agencies might each construct 50 kilometres of road, making them equally effective. But if one spent twice the budget and took twice the time, it was far less efficient. A project can be highly effective and deeply wasteful at the same time. Evaluators measure efficiency through tools like cost-benefit analysis, cost-effectiveness ratios, and timeliness assessments that check whether work was finished on schedule.

Efficiency matters enormously in public spending. In India, the Development Monitoring and Evaluation Office under NITI Aayog was created to help ensure the efficient and effective use of public resources, conducting independent evaluations of centrally sponsored schemes before they are renewed. The shift towards an Output-Outcome Monitoring Framework reflects this thinking: instead of simply tracking how much money was spent and how many physical units were built, the framework maps financial outlays to the outputs and outcomes they are meant to produce, making accountability for every rupee far clearer.

One caution is worth remembering. Efficiency can sometimes conflict with fairness. The cheapest way to deliver a service is not always the most equitable, because reaching marginalised or remote populations often costs more per person. A good evaluation does not treat efficiency as the only virtue; it weighs it against the other aspects.

Impact and sustainability: looking beyond the project’s end 🔗

Effectiveness and efficiency tell you how well a project ran while it was active. But two further aspects ask harder, longer-term questions. These are usually the most difficult to measure, and often the most important.

Impact: what really changed for people? 🔗

Impact goes beyond whether outputs were delivered. It asks about the deeper, often unintended changes a project produced in people’s lives and in the systems around them. The impact criterion is about the big “so what?” question: did the intervention create change that genuinely matters to people?

Consider a vocational training programme. Its output might be 5,000 youth trained. Its impact is a different matter entirely: how many found stable employment, whether household incomes rose, whether young women gained more say in family decisions, and whether the local economy shifted as a result. Impact includes effects that nobody planned for, both positive and negative. A new flyover may ease traffic (a positive impact) while also cutting a neighbourhood in two and reducing footfall for local shops (a negative one). Honest impact evaluation captures both.

Impact is hard to measure because change takes time and because it is difficult to prove that the project, rather than some other factor, caused the change. Evaluators use approaches like comparing the project area against a similar area that did not receive the intervention, or tracing the chain of cause and effect step by step. This is also why impact is frequently reported as the least satisfactorily implemented criterion among practitioners, since it is often confused with a single methodology and suffers from data gaps.

Sustainability: will the benefits last? 🔗

Sustainability asks whether the positive changes will continue after the project’s funding, staff, and attention move on. Sustainability is the extent to which the net benefits of an intervention continue, or are likely to continue, over the medium and long term.

This is the aspect that exposes weak projects. India’s urban landscape is full of examples: public toilets built under a sanitation drive that fall into disuse within a year because no one was made responsible for cleaning and maintaining them; water purification plants installed in villages that stop working once the warranty expires and no local technician can repair them. The infrastructure was effectively built and the impact was real for a while, but none of it was sustainable.

Evaluating sustainability means examining several kinds of staying power at once. Practitioners typically look at the financial, economic, social, environmental, and institutional capacities needed to keep benefits flowing. Financial sustainability asks whether money will be available for upkeep. Institutional sustainability asks whether some organisation or local body will own and run the asset. Environmental sustainability asks whether the gains harm the natural resource base they depend on. Social sustainability asks whether the community values and supports the change enough to maintain it.

A short-term success can quietly undermine long-term sustainability if a project is rushed or designed only to meet immediate targets. This tension between fast results and lasting ones is one of the central challenges every evaluator must weigh. Fittingly, NITI Aayog’s own stated vision for its monitoring and evaluation work is to improve sustainable outcomes and impacts of the government, placing these long-horizon aspects at the centre of public evaluation.

Qualitative and quantitative evaluation: numbers and meaning 🔗

Measuring any of these aspects requires data, and that data comes in two broad forms. Understanding the difference, and knowing how to combine them, separates a shallow evaluation from a credible one.

Quantitative evaluation: measuring what can be counted 🔗

Quantitative evaluation deals with numbers. It relies on surveys, statistics, and measurable indicators to answer questions about how much, how many, and to what extent. It is designed to produce findings with high statistical reliability that can be generalised across a large population. If you want to know whether a financial literacy programme improved household budgeting, a quantitative study would measure the change in budgeting practices across thousands of households using before-and-after surveys.

Quantitative methods are ideal for measuring outcomes, comparing groups, and establishing the scale of an effect. They answer “what happened” with precision. India’s output-outcome framework, with its roughly 6,000 indicators across hundreds of schemes, is essentially a vast quantitative monitoring exercise.

Qualitative evaluation: understanding the how and why 🔗

Qualitative evaluation deals with meaning, experience, and context. It uses interviews, focus group discussions, observation, and case studies to gather rich, detailed insights from a smaller number of people. Rather than statistical generalisation, it focuses on collecting in-depth information from a smaller number of cases to surface lived experiences and perspectives that numbers alone cannot reveal.

To continue the financial literacy example, where quantitative methods measure whether budgeting improved, a qualitative study would ask how women perceive their financial decision-making after the training. It explains the “why” behind the numbers. If a scheme’s results were disappointing, qualitative work often reveals the reasons: confusing instructions, distrust of officials, or barriers nobody anticipated. NITI Aayog’s own guidelines for quick assessment studies lean on qualitative tools like key informant interviews and focus groups to capture exactly this kind of insight.

Mixed methods: why the best evaluations use both 🔗

The debate over which approach is “better” misses the point. Each answers questions the other cannot. The strongest evaluations combine them in what is called a mixed-methods design. As the World Bank notes, much evaluation data is of limited use because of over-reliance on quantitative methods alone. Qualitative work can shape better survey questions, identify drivers that are hard to quantify, uncover unexpected issues, and trace the pathways through which impact actually happens.

In practice, mixed methods can be combined in different sequences. An evaluator might run a survey first and then use interviews to explain surprising results, or start with interviews to understand a community before designing a large survey, or collect both kinds of data at the same time. When numbers and narratives are brought together, they produce a more comprehensive understanding than either could alone. The quantitative side tells you the scale of the change; the qualitative side tells you what that change means to the people living it.

How the aspects fit together 🔗

These aspects are not a checklist to be ticked mechanically. They are a set of lenses, each focusing on a different dimension of a project’s value. Effectiveness and efficiency examine performance during the project. Impact and sustainability examine what endures after it. Qualitative and quantitative methods are the tools that let you measure all of them with both precision and depth.

A genuinely useful evaluation resists the temptation to celebrate a single strong number. A metro line that opened on budget (efficient) and carried its projected ridership (effective) is only a success if it also reshaped urban mobility (impact) and can be maintained and expanded for decades (sustainable). Used together, and supported by both numbers and human stories, these aspects turn evaluation from a formality into a tool for genuinely better decisions.

What do you think? If a public project in your city achieved all its physical targets on time and on budget but its benefits faded within two years, would you still call it a success? And when a project’s quantitative results and the lived experiences of its beneficiaries tell two different stories, which one should evaluators trust more?

References

1. https://www.oecd.org/en/publications/applying-evaluation-criteria-thoughtfully_543e84ed-en.html
2. https://academy.evalcommunity.com/applying-evaluation-criteria-thoughtfully-oecd/
3. https://www.activityinfo.org/blog/posts/2024-05-14-leveraging-the-oecd-dac-criteria-for-impact-evaluation-from-principles-to-practice.html
4. https://dmeo.gov.in/
5. https://dmeo.gov.in/content/output-outcome-monitoring-framework-oomf
6. https://ieg.worldbankgroup.org/sites/default/files/Data/DAC-Criteria/ConsultationReport_EvaluationCriteria.pdf
7. https://www.niti.gov.in/sites/default/files/2019-08/3_DMEO-StateGovtMeeting.pdf
8. https://www.evalcommunity.com/career-center/qualitative-vs-quantitative-research/
9. https://acf.gov/sites/default/files/documents/opre/qualitative.pdf
10. https://dmeo.gov.in/sites/default/files/2020-12/Quick_Assessment_Studies_Final.pdf
11. https://alnap.org/help-library/resources/combining-quantitative-and-qualitative-methods-for-program-monitoring-and-evaluation/
12. https://www.evalcommunity.com/career-center/evaluation-methodologies/