BackIn today’s world everything is changing constantly. Problems, the solutions, results – everything. Social problems, in particular, are large, complex and mutating constantly. Interventions addressing these problems need to be able to anticipate and keep up with these changes. To do so, it becomes critical to see, assess and learn from the change that interventions create, especially given the interconnectedness of issues and the corresponding systems.
But, why look at change at all? Why measure? Because, only when we see can we solve, and only when we solve in touch with contextual realities can we sustain. There is so much literature on measurement and evaluation that helps us see how important it is to understand the effectiveness of the interventions, how it can inform strategic decision-making, ensure accountability and so on. They are views heavily tilted towards the supply side. What if we take a step back and ask a fundamental question – ‘What’s really changing for the community?’
This big question leads to many more questions. Is there meaningful change in their lives? How do we constantly listen to the effect of those changes? How do we solve, ongoingly, with their experiences at the centre?
While we’ve all been told to measure and showcase ‘impact’, what does measuring ‘change’ really mean? If ‘why measure’ directly determines what gets measured, what would a demand-led model of solving and measuring look like?
Embracing Complexity
An estimated USD 8 billion is spent by NGOs, Governments and companies, on impact measurement and is anticipated to increase 300% by 2030. Yet, despite the rigour and resources dedicated to measuring impact, we may need to think beyond traditional models to truly understand what is changing and what is driving or impeding change.
When considering the most impactful initiatives, it’s easy to showcase impressive metrics and quick success stories. However, the reality of measuring true change is far more complex. Not everything that is changing is measurable in a straightforward way. How do we measure, for example, if the agency of people has been restored as a result of an intervention? This complexity is something Kuza has grappled with in Africa and South Asia. At Kuza, the independent impact report shows remarkable results – 67% increased farmer productivity and 95% improved incomes among those they work with. While these are incredibly proud achievements, they raise a couple of critical questions:
- Do these impact parameters – productivity and incomes – paint the full picture of change and the conditions in which it was created?
- Are we on the path to shifting the equilibrium for the community for good or are the interventions short lived?
- Can change ever be attributed to just one party, one intervention?
Attribution or Contribution?
In reality, the effects of various external factors make it difficult, if not impossible, to isolate the change created by one specific programme or initiative, especially in an attempt to bring about change at scale. This makes it difficult to attribute accurately.
Is it, therefore, better to consider that these interventions are one of many factors that influenced the outcomes, and aim at a broader and more realistic understanding of how interventions interact with other external factors to drive the intended change?
What if we think of ourselves as contributors to change? This can allow us to acknowledge and understand collaborative efforts of the entire ecosystem, and also attribute the individual leaders in the ecosystem in a way that can help foster collaboration?
However, for System Orchestrators, there may be a catch. While they may move to a contribution mindset, the ecosystem of support around them (board members, funders, partners) may still perceive attribution as a big indicator of change. Moving beyond vanity metrics to discover and use metrics that reflect real and meaningful change needs a mindset shift across all actors. Some questions to use as stepping stones could be:
- Do we want to measure what is right, or what is easy?
- What may be the first step in breaking this cycle and ‘question vanity’?
Creative Doing
Let’s move to ‘Doing’, or, the ‘how’ to measure change. Options range from LogFrame, Pre-post to Outcome mapping, Results based management, Randomized Control Trial (RCT) etc. Theoretically, some of these methods offer a clear, objective way to attribute changes in outcomes to a specific intervention, but reality is more complex. They are often time-consuming, difficult and expensive to execute in complex multidimensional contexts such as agriculture. For smaller organisations or those working in rapidly changing environments with budgets like USD 500K to USD 3M, the cost and complexity could also be a limitation.
As an example, imagine an RCT designed to measure the impact of a bundled input-and-advisory service to farmers. While the trial might compare farmers who receive the service to those who don’t, it cannot easily account for the spillover effects like nearby farmers adopting new practices informally, or service providers changing their pricing due to perceived demand etc. These indirect or network dynamics often where real change happens are filtered out by design. Also, in this case, the underlying assumption that there is a degree of stability in farming communities could be false as the communities are constantly adapting to market shifts, weather, and community dynamics. Hence, isolating a single intervention may give clarity but not necessarily useful insights to do-think-do.
- Can we get more creative, rapid and dynamic in our ‘doing’?
- Can we explore frameworks and technologies that move us away from the ‘feel good’ metrics and drive us closer to the ‘get real’ metrics that matter the most to the communities we serve?
A holistic approach
In Kuza’s case, the commitment to move to ‘get real’ metrics translated into embracing a holistic approach that combines technology with human touch. Kuza’s ‘OneNetwork’ in Kenya is an open network designed to onboard all service providers across the agricultural value chain, from inputs, finance, and post-harvest logistics to off-takers. This open model allows for flexibility, collaboration, and scalability in service of the Farmers.
At the heart of Kuza’s model are people – Farmers and also Agripreneurs, rural youth from local communities, who serve as the trusted connection points between farmers and service providers. These agripreneurs are the ones who bring the human touch, building relationships and trust that technology alone cannot achieve. They help farmers increase productivity and access markets, while also earning a sustainable income by providing valuable services.
This is a conscious effort to adopt a tech-enabled people-centric approach and not a tech-led one. The beauty of this approach is that it doesn’t rely on a single actor or intervention. It’s a collaborative model that recognizes the importance of multiple players working together to achieve shared goals. By empowering agripreneurs and creating an open, inclusive network, we’re making it possible for everyone involved to contribute to and benefit from meaningful change. This paves way to a more holistic approach to change, and understanding the impact this change is creating on the ground.
With this holistic approach backed by technology that can aid speed and scale, it is now possible to ask these broader questions to sustain the changes we attempt to create:
- How can we create systems that are adaptable, inclusive, and resilient?
- How can we foster collaboration rather than competition for credit?
Paradigm Shift – From Measuring Impact to Assessing Progress
Between change and impact, there is a journey of progress, the continuous movement on a time scale. As Zadie Smith rightly put “Progress is never permanent, will always be threatened, must be redoubled, restated and reimagined if it is to survive.”
Assessing this progress on the journey in a continuous fashion therefore becomes very critical. It will help understand what is changing, when, from where to where, how, in what direction, by how much etc. There is an urgent need for a paradigm shift – from traditional ways of measuring impact towards assessing progress – to Learn, Act and Progress rather quickly.
- From one-time, static, episodic, long-drawn, in point measurement towards multiple, active, continuous, repetitive, rapid, dynamic learning
- From recording programmatic inputs and outputs towards deeply listening to the voices of communities
- From quantitative to-the-point data points towards relevant qualitative aspects that can be quantified
- Not just with the objective of reporting to funders and stakeholders but with the intent to learn, act and progress together at scale
- Towards unearthing the deeper, system-wide shifts seen on the ground
This is the new thinking, a new framework we are co-exploring at the Centre for Exponential Change with Kuza and Global Development Incubator. We are asking some big questions:
- How do we intentionally design to assess the non-linear rate of change?
- How do we sense and make sense of the other positive changes that have been set off in motion for good?
- How do we measure network effects?
We’re curious about assessing the progress we make towards exponential change – one that tracks not just direct outcomes, but network effects, tipping points, and emergent change across entire value chains. The journey has just begun. We’re learning, experimenting, and listening, to data, to patterns, and most importantly, to the voices of farmers and agripreneurs, powered by Apurva.ai, a sense-making infrastructure. We don’t have all the answers but we’re asking better questions and we will get better at asking better questions as we move forward. We are committed to learning fast, improving as we go, and building models of impact that are as dynamic and interconnected as the communities we work with.
Albert Einstein’s words of wisdom are so apt here:
“Not everything that can be counted counts, and not everything that counts can be counted.” In the end, true impact is about more than just numbers – it’s about creating lasting, positive change in the lives of the people we serve.
