Page 87 - DEDU504_EDUCATIONAL_MEASUREMENT_AND_EVALUATION_ENGLISH

Unit 7 : Reliability – Types , Methods and Usability

decision that is estimated in decision consistency reliability. If an examinee is classified as a Notes
master on both test administrations, or as a non-master on both occasions, the test is producing
consistent decisions. This approach can be used either with parallel forms or with a single form
administered twice in test-retest fashion.
Internal Consistency
The internal consistency measure of reliability is frequently used for norm referenced tests (NRTs).
This method has the advantage of being able to be conducted using a single form given at a
single administration. The internal consistency method estimates how well the set of items on a
test correlate with one another; that is, how similar the items on a test form are to one another.
Many test analysis software programs produce this reliability estimate automatically. However,
two common differences between NRTs and CRTs make this method of reliability estimation
less useful for CRTs. First, because CRTs are typically designed to have a much narrower range
of item difficulty, and examinee scores, the value of the reliability estimate will tend to be lower.
Additionally, CRTs are often designed to measure a broader range of content; this results in a set
of items that are not necessarily closely related to each other. This aspect of CRT test design will
also produce a lower reliability estimate than would be seen on a typical NRT.

Interrater Reliability
All of the methods for estimating reliability discussed thus far are intended to be used for
objective tests. When a test includes performance tasks, or other items that need to be scored by
human raters then the reliability of those raters must be estimated. This reliability method asks
the question, “If multiple raters scored a single examinee’s performance, would the examinee
receive the same score. Interrater reliability provides a measure of the dependability or consistency
of scores that might be expected across raters.

Self Assessment
1. Fill in the blanks :
(i) According to ______ , “The reliability of a test refers to the ______ of scores obtained by
the same individuals on different occassions or with different set of equivalent items.

(ii) ______ demonstrates the extent to which a test is able to produce stable, Consistent scores
across time.
(iii) The ______ measure of reliability is frequently used for norm referenced tests.
(iv) ______ reliability provides a measure of the dependability or consistency of scores that
might be expected across raters.

7.3 Methods of Reliability
The chief basis of ascertaining reliability is correlation. The relation between two equivalent or
parallel test scores is called reliability coefficient, which indicates the reliability of the test.
Following are the chief methods of finding out reliability coefficient :
1. Parallel form or alternate method,

2. Method of rational equivalence or K.R. formula,
3. Test-retest method,
4. Split-halt method.
1. Alternate Method or Parallel Form Method : Under this method, a test is constructed in two
parallel formats. The questions in these two parallel test are different from each other, but
they are equal from all other views. It essential in this method that both tests are parallel.
The parallel form ‘A’ and ‘B’ of many standardized tests are already available. In order to

LOVELY PROFESSIONAL UNIVERSITY 81