Page 43 - DLIS401_METHODOLOGY_OF_RESEARCH_AND_STATISTICAL_TECHNIQUES

Methodology of Research and Statistical Techniques

Notes Finally, the experiments should be conducted in such a way that the only difference between
the experimental and the control group is the manipulation of a variable during the experiment.
Taken together, randomization or matching, and the fact that the manipulation during experimentation
is the only difference between the two groups, these techniques allow for the control of all
variables, other than the manipulated one, to interfere in the outcome of the experiment
(internal validity!).
Note on the One-Shot Case Study:

A single group is manipulated on an independent variable, and then measured on a dependent
variable. This method must involved pretest and posttest to be of any significance (otherwise
there is nothing to compare), i.e., the one-group pretest-posttest design, but then we are not
sure if it was the manipulated variable that caused the observed difference.
2. Internal Validity and External Validity
(a) Internal Validity: did the experimental treatment cause the observed difference?
The problem of internal validity refers to the logic of design, the fact whether other variables
that may intervene were controlled, i.e., the integrity of the study. The problem can be that the
conclusions of an experiment are not warranted based on what happened during the experiment.
This can come about because of: (a) accident: historical events can have occurred during the
experiment and affected its outcome; (b) time: people change, mature, during the period of
experimentation; (c) testing: the groups are aware of what is being researched; (d) instrumentation:
the techniques to measure pretest and posttest results are not identical (reliability); (e) statistical
regression: results are biased because the subjects started with extreme values on a variable;
and (f) other problems include, that the relationships are temporal but not causal, and that the
control group may be frustrated or stuff.
Randomization of subjects into an experimental and a control group (to ensure that only the
experimental manipulation intervened, while other variables are controlled), and reliable
measurements in pretest and posttest are guards against problems of internal validity.
(b) External Validity: are the results of the experiment generalizable?
The problem of external validity refers to the issue of generalizability: what does the experiment,
even when it is internally valid, tell us about the real, i.e., non-manipulated, world?
A good solution is a four-group experimental design, i.e., first an experimental and a control
group with pretest and posttest, and second, an experimental and a control group with posttest
only. And better than anything else is a two-group design with posttest only when there is
good randomization, since randomization ensures that all variables are evenly distributed
between experimental and control group so that we do not have to do a pretest.

Did u know? An experimental manipulation as close as possible to the natural conditions,
without destroying internal validity, are the best methods to ensure external
validity.

(c) Note on Ex-Post Facto Experiment
This is not a true experiment since there is (was) no control group. The manipulation of the
independent variable has naturally occurred (e.g., earthquake). We are of course not sure, say
when we compare with a group were the natural “manipulation” did not take place, that there
are (or are not) other variables involved (very bad on the control of variables).

38 LOVELY PROFESSIONAL UNIVERSITY