Page 62 - DEDU504_EDUCATIONAL_MEASUREMENT_AND_EVALUATION_ENGLISH

Educational Measurement and Evaluation

Notes 5. Prepare a tentative scoring key in advance.
6. Alert students if penalties for spelling or grammar are given.
7. Evaluate essays anonymously by mixing papers and masking names.
8. Scan longer essays to presort into stacks of high, medium and low quality and then reread to
assign marks.
9. Score all answers to one question before scoring the next question.
10. Decide whether essays are to be scored holistically or analytically. Holistic scores are awarded
as a global measure of overall quality, i.e. a single mark is given on a 1 - 5 (A - E) or perhaps
a 10 or 12 point scale. An analytic score is the sum of several different ratings of features about
an essay. An example of analytic scoring is found on page 29.
When a combination of essay and multiple choice items are included in a test, the calculation of the
exam score can be handled in several ways. Perhaps the easiest method is to assign weights to each
section of the test. If the essay portion is worth 50 percent and the multiple choice worth 50 percent,
the essay might be rated on a scale from 0 to 10 points, either holistically or analytically. The total
number of essay points could be multiplied by 5 making a range of 0 to 50 possible points for the
essay. The difficulty in using a single scale from 0 to 50 or more points to initially rate essays is that
it is extremely difficult for a reader to reliably differentiate papers that receive, for example, 36 or 38
points. A second reading might well reverse the two scores.
The following discussion about essay examinations is particularly helpful in evaluating essay
questions. It is adapted from a bulletin published at Michigan State University.

The Writing of Essay Questions
It is commonly recognized that the reading and marking of essay examinations is a difficult and
time-consuming process; it is not always recognized that the preparation of essay questions requires
time and effort. The weaknesses which have frequently characterized the essay question have, in
part, resulted from a lack of awareness of or concern about these difficulties. Overemphasis on
factual information and on memorized extracts of textbooks and lectures, ambiguity in meaning
and an associated ambiguity of response, and variations in the standards used to judge student
performance result in difficulties in reading and in unreliable marking. Essay questions can be
markedly improved by conscious attention to the identification and elimination of these and other
deficiencies.
Perhaps the most general suggestion for improving essay testing is that the task set for the student
should first be clearly understood by the teacher and then clearly presented to the student. It is
unreasonable to present the student with the combined tasks of reading the instructor’s intent in the
question and meeting his expectation as to the answer. On the other hand, to make an essay question
highly specific and detailed is to reduce it to a type of objective test item involving a filling in of
blanks implied in the question. The poser of essay questions must tread the path between highly
general questions to which no adequate answer can be given in the time permitted—even assuming
the meaning is clear—and highly specific questions which involve word or phrases answers. The
question must state precisely what is wanted, and it should contain enough clues that students can
readily infer the direction their answer should take. It is also desirable that the question should state
or imply the approximate length and complexity of the desired response. Some examples may serve
to clarify this and other points.

5.7 Planning for Diagnostic Testing

The essential steps in educational diagnosis are :
(i) Identifying the students who are having trouble or need help.
(ii) Locating the errors or learning difficulties.
(iii) Discovering the causal factors of slow learning.

56 LOVELY PROFESSIONAL UNIVERSITY