Page 186 - DLIS401_METHODOLOGY_OF_RESEARCH_AND_STATISTICAL_TECHNIQUES

Unit 14: Evaluation Research

Objectivist, elite, quasi-evaluation Notes

As a group, these approaches represent a highly respected collection of disciplined inquiry
approaches. They are considered quasi-evaluation approaches because particular studies legitimately
can focus only on questions of knowledge without addressing any questions of value. Such
studies are, by definition, not evaluations. These approaches can produce characterizations
without producing appraisals, although specific studies can produce both. Each of these approaches
serves its intended purpose well. They are discussed roughly in order of the extent to which
they approach the objectivist ideal.
Experimental research is the best approach for determining causal relationships between variables.
The potential problem with using this as an evaluation approach is that its highly controlled
and stylized methodology may not be sufficiently responsive to the dynamically changing
needs of most human service programs.

Notes Management information systems (MISs) can give detailed information about the
dynamic operations of complex programs. However, this information is restricted
to readily quantifiable data usually available at regular intervals.

Testing programs are familiar to just about anyone who has attended school, served in the
military, or worked for a large company. These programs are good at comparing individuals
or groups to selected norms in a number of subject areas or to a set of standards of performance.
However, they only focus on testee performance and they might not adequately sample what
is taught or expected.
Objectives-based approaches relate outcomes to prespecified objectives, allowing judgements
to be made about their level of attainment. Unfortunately, the objectives are often not proven
to be important or they focus on outcomes too narrow to provide the basis for determining the
value of an object.

Content analysis is a quasi-evaluation approach because content analysis judgements need not
be based on value statements. Instead, they can be based on knowledge. Such content analyses
are not evaluations. On the other hand, when content analysis judgements are based on values,
such studies are evaluations.

Objectivist, mass, quasi-evaluation
Accountability is popular with constituents because it is intended to provide an accurate
accounting of results that can improve the quality of products and services. However, this
approach quickly can turn practitioners and consumers into adversaries when implemented in
a heavy-handed fashion.

Objectivist, elite, true evaluation
Decision-oriented studies are designed to provide a knowledge base for making and defending
decisions. This approach usually requires the close collaboration between an evaluator and
decision-maker, allowing it to be susceptible to corruption and bias.
Policy studies provide general guidance and direction on broad issues by identifying and
assessing potential costs and benefits of competing policies. The drawback is these studies can
be corrupted or subverted by the politically motivated actions of the participants.

Objectivist, mass, true evaluation
Consumer-oriented studies are used to judge the relative merits of goods and services based
on generalized needs and values, along with a comprehensive range of effects. However, this

LOVELY PROFESSIONAL UNIVERSITY 181