Page 359 - DEDU504_EDUCATIONAL_MEASUREMENT_AND_EVALUATION_ENGLISH

Unit 31 : Formative and Summative Evaluation

Notes

Formative evaluation activities include the collection and analysis of data over the
lifecycle of the programme and timely feedback of evaluation findings to programme
actors to inform ongoing decision-making and action (i.e. it is a form of operational
intelligence). It requires an effective data collection strategy, often incorporating
routinised monitoring data alongside more tailored evaluation activities.

Feedback is primarily designed to fine-tune the implementation of the programme although it
may also contribute to policy-making at the margins through piecemeal adaptation.
Evaluators conducting a formative evaluation ask many different kinds of questions and use a
variety of methods to address them. Questions are commonly open-ended and exploratory, aimed
at uncovering the processes by which the programme takes shape, establishing has changed from
the original design and why, or assessing soft organisational factors such as the extent of ‘buy in’
by practitioner staff to the programme’s goals and intended outcomes.
Formative evaluation questions also investigate the relationship between inputs and outcomes,
which can involve the formulation and measurement of early or short-term outcome measures.
These often have a process flavour and serve as interim markers of more tangible longer term
outcomes.
Methods which might be used include stakeholder analysis, concept mapping, focus groups,
nominal group techniques, observational techniques and input-output analysis. Formative
evaluation’s concern with the efficiency and effectiveness of project management can be addressed
through management - oriented methods like flow charting, PERT/CRM (Programme Evaluation
and Review Technique and Critical Path Method) and project scheduling. The measurement of
interim or short-term outcome measures, which capture steps in the theory of how change will
be achieved over the long term, may involve construction of qualitative or process indicators
and use of basic forms of quantitative measurement.
Formative evaluation may be planned and managed in a variety of ways. The prevailing practice
has been to prioritise the information needs of central staff (policy makers, programme managers)
as those primarily responsible for programme steerage, leaving unspecified the roles that local
staff (local site managers, local practitioners) and clients can play in reshaping plans and strategies
in response to feedback. Newer conceptions of formative evaluation (for example, the mutual
catalytic model of formative evaluation outlined by Chacon-Moscoso et al, 2002) emphasise a
more inclusive approach to the involvement of stakeholders, and as well seek to elicit their
participation as collaborators in the evaluation process rather than simply as providers of
information. The role of evaluator changes from one concerned with gathering data and
communicating evaluation findings to one of engaging programme participants in a form of
evaluative inquiry. Organisational actors are helped to generate their own data and feedback
through collective learning processes.

Formative evaluation lends itself most readily to a case study approach, using a
qualitative mode of inquiry. There is a preference for methods that are capable of
picking up the subtleties of reforms and the complexities of the organisational context
and wider policy environment.

31.1.1 Need for Formative Evaluation
Many commentators would argue that all Structural Fund initiatives operate in conditions of
uncertainty, and that formative evaluation is a desirable corrective or steerage component of all

LOVELY PROFESSIONAL UNIVERSITY 353