Page 362 - DEDU504_EDUCATIONAL_MEASUREMENT_AND_EVALUATION_ENGLISH

Educational Measurement and Evaluation

Notes Step 3 : Creating an evaluation infrastructure to support formative evaluation as a learning,
change-oriented, developmental activity. This includes working with programme staff on an
ongoing basis to :
• create a culture that supports risk-taking, reduces fear of failure, and values lessons learned
from mistakes
• establish channels of communication that support the dissemination of information and
allow organisational members to learn from one another in ways that contribute to new
insights and shared understandings
• create new opportunities for shared learning and knowledge creation
• modify systems and structures that inhibit organisational learning
Step 4 : A fourth step entails finding out about the decision-making cycle, the different stakeholder
groups and their respective information needs and interests. These might include policy makers
and programme makers at central level, local site programme managers, and operational staff.
Each set of stakeholders will be asking different questions of the evaluation and have a preference
for the way that findings are presented and or/communicated. Where there is a lack of appropriate
mechanisms or opportunities for feedback, the evaluator will need to establish a structured way
to provide relevant stakeholders with feedback.
Step 5 : Formative evaluation involves an ongoing cycle of data gathering and analysis. The
choice of methods will be determined largely by the questions being addressed, and the
methodological preferences of different stakeholders. Most formative evaluations use a variety
of methods. Where a collaborative, participative approach is taken to formative evaluation, the
methods are likely to include those which foster and support interaction, dialogue, learning and
action.
Step 6 : There are different views as to whether the evaluator’s responsibility stops with feeding
back findings and facilitating processes of learning among programme actors, or whether she or
he also has a role to play in follow-through action. Where the evaluator is external to the
organisation, the role is likely to be limited to the former. Formative evaluators may however
be internally located, especially where the preferred model of formative evaluation is influenced
by organisational learning concepts and practices. In these circumstances, the formative evaluation
cycle is likely to include shared responsibility for implementing the action plan and monitoring
its progress.
31.1.5 Strengths and Limitations
Formative evaluation provides a rich picture of a programme as it unfolds. It is a source of
valuable learning not just prospectively for the programme but for future programmes as well.
Formative evaluation is highly complementary to summative evaluation and is essential for
trying to understand why a programme succeeds or fails, and what complex factors are at work.
Large scale programmes are often marked by a discrepancy between formal programme theory
and what is implemented locally. Formative evaluation can help determine whether the
substantive theory behind the programme is flawed, whether the evaluation was deficient, or if
implementation failed to pass some causal threshold.
To be effective and achieve its purpose of programme improvement, formative evaluation requires
strong support from the top as well as bottom-up support. It must be endorsed by programme
decision-makers and others who will need to act on its findings. Support may be withdrawn,
overtly or covertly, if the findings expose weaknesses in programme design or implementation,
especially where the organisational culture is one of blame and discourages innovation or learning
from mistakes. Research findings suggest that programme managers are more receptive to ‘bad
news’ that is communicated by internally located evaluators (‘one of us’), than by independent
evaluators.

356 LOVELY PROFESSIONAL UNIVERSITY