Page 276 - DLIS402_INFORMATION_ANALYSIS_AND_REPACKAGING

Unit 12: Content Analysis

Unfortunately, software can’t draw useful conclusions. Maybe in 10 years the software will be much Notes
cleverer, but at the moment there’s no substitute for human judgement - and that takes a lot of time.
Even so, if your units are not too large, and all the content is available as a computer file, you can
save time by delaying the coding till a later stage than usual. The time is saved when similar content
is grouped, and a lot of units can all be coded at once.
For example, if you were studying conflict, you could use software such as NVivo to find all units
that mentioned “conflict” and a list of synonyms. It would then be quite fast to go through all these
units and sort them into different codes.

Using judges
A common way to overcome coding problems is to appoint a small group of “judges” and average
their views on subjective matters. Though it’s easy to be precise about minor points (e.g. “the word
Violence was spoken 29 times”), the more general your analysis, the more subjective it becomes
(e.g., the concept of violence as generally understood by the audience).
Use judges when there are likely to be disagreements on the coding. This will be when any of these
conditions applies:
units are large (e.g., a whole TV program instead of one sentence)
you are coding vague concepts, such as “sustainability” or “globalization”
you are coding nonverbal items, such as pictures, sounds, and gestures
your findings are likely to be published, then challenged by others.
The more strongly these conditions apply, the more judges you need. Unless you are being incredibly
finicky (or the project has very generous funding!) 3 judges is often enough, and 10 is about the
most you will ever need. The more specific the coding instructions, the fewer the judges you will
need. If you only have one person coding each question, he or she is then called a “coder” not a
“judge” - though the work is the same.
Any items on which the judges disagree significantly should be discussed later by all judges and
revised. Large differences usually result from misunderstanding or different interpretations.
Maybe you are wondering how many judges it takes before you’re doing a survey, not content
analysis. 30 judges? 100?
Actually, it doesn’t work like that. Judges should be trained to be objective: they are trying to describe
the content, not give their opinions. All judges should agree as closely as possible. If there’s a lot of
disagreement among the judges, it usually means their instructions weren’t clear, and need to be
rewritten.
With a survey, respondents are unconstrained in their opinions. You want to find out their real
opinions, so it makes no sense to “train” respondents. That’s the difference between judging content
and doing a survey. However if you’re planning to do a content analysis that uses both large units
and imprecise definitions, maybe you should consider doing a survey instead (or also).

Make a note on issues of reliability and validity.

Self Assessment

Fill in the blanks:
3. ............. builds on conceptual analysis by examining the relationships among concepts in a
text.

LOVELY PROFESSIONAL UNIVERSITY 271