Page 56 - DMGT404 RESEARCH_METHODOLOGY

Research Methodology

Notes 4. Instrument Variation: Instrument variation effect is a threat to internal validity when
human respondents are involved. For example, an equipment such as a vacuum cleaner is
left behind, for the customer to use for two weeks. After two weeks the respondents are
given a questionnaire to answer. The reply may be quite different from what was given by
the respondent before the trail of the product. This may be because of two reasons:
(a) Some of the questions have been changed
(b) Change in the interviewer for pre-testing and post testing are different
The measurement in experiments will depend upon the instrument used to measure. Also
results may vary due to application of instruments, where there are several interviewers.
Thus, it is very difficult to ensure that all the interviewers will ask the same questions with
the same tone and develop the same rapport. There may be difference in response, because
each interviewer conducts the interview differently.

5. Selection Bias: Selection bias occurs because 2 groups selected for experiment may not be
identical. If the 2 groups are asked various questions, they will respond differently. If
multiple groups are participating, this error will occur. There are two promotional
advertisement A & B for "Ready to eat food". The idea is to find effectiveness of the two
advertisements. Assume that the respondent exposed to 'A' are dominant users of the
product. Now suppose 50% of those who saw 'Advertisement A' bought the product and
only 10% of those who saw 'Advertisement B' bought the product. From the above, one
should not conclude that advertisement 'A' is more effective than advertisement 'B'. The
main difference may be due to food preference habits between the groups, even in this
case, internal validity might suffer but to a lesser degree.
6. Experimental Mortality: Some members may leave the original group and some new
members join the old group. This is because some members might migrate to another
geographical area. This change in the members will alter the composition of the group.

Example: Assume that a vacuum cleaner manufacturer wants to introduce a new version. He
interviews hundred respondents who are currently using the older version. Let us assume that,
these 100 respondents have rated the existing vacuum cleaner on a 10 point scale (1 for lowest
and 10 for highest). Let the mean rating of the respondents be 7.
Now the newer version is demonstrated to the same hundred respondents and equipment is left
with them for 2 months. At the end of two months only 80 participant respond, since the
remaining 20 refused to answer. Now if the mean score of 80 respondents is 8 on the same 10
point scale. From this can we conclude that the new vacuum cleaner is better?

The answer to the above question depends on the composition of 20 respondents who dropped
out. Suppose the 20 respondents who dropped out had negative reaction to the product, then the
mean score would not have been 8. It may even be lower than 7. The difference in mean rating
does not give true picture. It does not indicate that the new product is better than the old product.
One might wonder, why not we leave the 20 respondent from the original group and calculate
the mean rating of the remaining 80 and compare. But this method also will not solve the
mortality effect. Mortality effect will occur in an experiment irrespective of whether the human
beings or involved or not.

Concomitant Variable

Concomitant variable is the extent to which a cause "X" and the effect "Y" vary together in a
predicted manner.

50 LOVELY PROFESSIONAL UNIVERSITY