What is impact?

Impact has a very broad definition and it is used with many different meanings. For the purpose of this paper impact is defined as the “significant and lasting changes in people’s lives, communities or ecosystems brought about by a given action or series of actions” (Roche, 1999). As for impact measurement refers to the process of analysing, calculating and monitoring the
changes, either positive or negative, that result from a given intervention (for example of an initiative, program, project or organization).

Why measure impact?

In order to evidence its impact the venture must engage in impact measurement. A
systematic approach to impact measurement enables the organization not only to communicate
real social return to funders, investors and beneficiaries but also to maintain an informed
position on what it is genuinely achieving and to plan its allocation of resources and grow
accordingly. It is important to be open to learn with the process and identify what may need to
change so that you are able to maximise impact and/or optimize the allocation of resources to
the activities that generate the most value.
Measure impact serves three main purposes: (1) measure for learning – understand
performance and test assumptions; (2) measure for action – guide behaviour and
communication; measure for accountability – report performance and build relationships.
The benefits of measuring impact can be summarized as follows:
? Improved internal management: greater efficiency in planning and in the
implementation of the activities; efficiency gains through better management and
allocation of resources and / or activities;
? Improved external positioning: more clarity of the social value created; better
differentiation of the impact venture in relation to their peers; more attractive to
investors.
There are several methodologies and tools to assess impact, which serve different purposes
and different target audiences. So it is important to make an honest reflection about what is your
motivation for measuring impact and who will be using the data and analysis. There are
methodologies and tools more focused on the monetization of outputs and impacts often seeking
some form of proxy from which financial comparisons can be drawn and others are designed
to serve as a management tool and place a higher priority on performance indicators. In many
cases, the nature of the data collected and how it is analysed is quite different for these two
functions and, in all cases, the accuracy and nature of analysis should be tailored to the purpose
that it is intended to serve.

What is the Theory of Change and how should I build it?

In this chapter we will introduce and apply the Theory of Change, which is a tool that
has inspired other methodologies such as Social Return on Investment. Social Return on
Investment is an analytic tool for measuring and accounting for a much broader concept of
value, taking into account social, economic and environmental factors (Retrived from:
http://www.neweconomics.org/issues/entry/social-return-on-investment).
The theory of change describes a strategy for achieving the desired change. It identifies
the preconditions, pathways and interventions necessary for an impact venture’s success,
explaining how early changes relate to more intermediate changes and then to longer-term
change. So, a theory of change works mainly at the strategy level, mapping out the interventions
required to create outcomes that will lead to the ultimate impact. Sometimes outcomes are
closely related, or they may occur independently. These changes and connections are often
represented visually, for example through a chart, or a set of tables.
The steps in creating the theory of change, starts backward from the impact goal.
? Step 1. Determine impact goal(s) – what kind of change in society do you want to
promote?
It is important to articulate the change you are seeking to promote. The temptation is always
mentioning the outputs, for example, “we deliver bed nets in malaria-ridden locations”.
However, there is an underlying impact goal. For instance, bed nets are presumably intended to
reduce the incidence of malaria, improve health and possibly increase lifespan or reduce
poverty.
The desired impact should be clearly defined and stated in terms of the specific change desired
and the specific population who will benefit. It should be also realistic and succinct.
? Step 2. Develop a theory – what do you have to do to achieve the impact goal?
Once you have defined the impact goal, you should work backwards through the steps you think
are needed to achieve it. To do this it can help to think in terms of outcomes, which are the
changes that will contribute to achieve your impact goal.
Starting with the impact goal ask, “What outcomes must be brought about before we can
achieve our impact goal?” These outcomes are placed directly underneath the impact goal.
When you have captured that information, continue backwards mapping by repeating the
process for each of the outcomes you just identified. In the process, you should define the
pathways. A pathway is the sequence in which outcomes must occur to reach long-term goal.
Pathways can be depicted by vertical chains of outcomes connected to one another by arrows,
proceeding from early outcomes at the bottom to longer-term outcomes at the top. Then link
your activities to your outcomes and see if the flow makes sense. The theory of change rests on
a number of assumptions that must be met for the chain of events in the theory of change to
hold.

What kind of changes should I monitor?

The change can occur in several levels with different deepness degree. Below you can
find an example of the type of change that can occur and an example to illustrate:
? Changes in learning
Unemployed people improve their technical skills in a given area
? Changes in behavior
Unemployed people increase their motivation to look out for a job
? Changes in the relation
Unemployed people developed healthy relationships with their families
? Changes in the condition
Unemployed people get and maintain a job for a given time
Also, identify what is the level of our target audience:
? Individual
? Family
? Community
? Society
Finally, you should list the outputs directly produced by the activities promoted and relate
them to the outcomes of the solution. The outputs are the direct consequence of the core
activities of the solution (eg. a direct output of a workshop may be the level of attendance,
participation or utility of the tools provided). This gives you an idea of the performance of your
impact venture.
After this exercise you are able to build you theory of change (relationship between outcomes
and impact goals) and your logic model.

How to validate the logic?

After a theory is agreed upon, it should be validated. Your impact venture can do this in a
number of ways. First individuals with experience in the field should confirm the theorized
relationships. Then the initiative should look for peers active in the same cause to examine their
successes and failures. Trade, academic and government publications can also be important
sources of evidence regarding how your theory of change holds up. Finally, the theory should
be vetted with stakeholders – particularly beneficiaries – to determine whether they believe in
the theory. The process of producing the theory of change is as critical as the end result. It
brings every part of the venture to talk trough the tough questions and assess key activities and
it can contribute to build an engaged environment in the team. It guides team members to think
deeply about what they are doing and why it matters. Hence, your developed theory of change
should be:
? CREDIBLE – It should be based on previous experience and insight from your different
stakeholders or relevant research where appropriate.
? ACHIEVABLE – You will have the necessary resources to carry out the intervention as
detailed.
? TESTABLE – The model will provide clear, specific and comprehensive indicators that
you can use to assess progress and achievements.
? SUPORTED – Your stakeholders will be involved in defining and agreeing your theory
of change and you will have a common language with which to go forward.

What are the problems faced in an Impact Assessment Process?

It is important to keep in mind that a percentage of the results obtained can be attributed to other
organizations and factors. There are often positive and negative spillovers that must be considered in
attributing causal relationships.
Similarly, there may be results that could happen even in the absence of our programmes
(deadweight effect). For example, the evolution of children’s knowledge about healthy eating can
demonstrate a positive historical evolution. The challenge is to identify, within this positive
development, which is, in fact, the impact that can be attributed to the performance of our initiative.
To estimate this impact one can make predictions about the future development of children’s
knowledge about healthy eating, based on historical information. Another way to estimate the impact
is by establishing the control groups. For instance, it can be analysed the participation and interest of
the children for healthy eating habits at home by selection a sample of students who attended to
specific educational programme and other sample that didn’t. However, even if there are a correlation
between variables – for instance imagine that the programme student sample present 20% more
interest in cooking healthy meals that the other group – it does not mean that the relation is causal.
Other factors might contribute to this differential and a deeper analysis and research is always
influence the results.
Finally, the social entrepreneur may find a conflict exists between serving the most at risk
beneficiaries and the operational interests of the venture. Higher risk beneficiaries may be more
demanding of resources and result in slower turnarounds (for initiatives geared toward supplying
services), and they may also present high risks financially. However, these beneficiaries may also
bethe most excluded and underserved, and therefore those who stand to benefit most from the
venture’s work. For example, a venture focused on employment may find there is an operational
incentive to select those beneficiaries who are more capable, but in doing so, neglect those who are
most in need. So, it is important to reflect if the impact venture is exposed to this conflict of at risk
beneficiaries presenting at once higher impacts and potentially compromised operational viability.

 

If you wish to download the file, click here