Simultaneous Examination and Method Analysis by Variance Algebra
William J. Moonan · The Journal of Experimental Education · 1955
CONSIDERABLE a 11 e ntion is of ten given to the construction of examinations which are to be utilized in some respect for educational and psychological research purposes. Frequently, these tests are used in method experiments of varying complexity. It is a matter of r e c o r d that very little attention is given to the examina tion after or during the execution of the experi ments. Frequently consistency coefficients are quoted from data obtained from extra-experi mental sources, but it is more important to re port these when they result from the imposition of the upon the experimental subjects. This is especially true, if the impose conditions which are somewhat different from those of the original examination standardization. Another important matter concerns whether or not any or a set of items interact, in the non-additive sense, with the different methods. In order to point out the relevancy to experiments, consider this hypothetical educa tional example. Suppose a 50 item test is given to two classes of equal size of arithmetic students who were taught by two different methods. If the students of the first class answered the first 25 items correctly and missed the last 25 items, while the students of the second class reversed this type of response, then the two sample means which are calculated from the total test scores of the two groups would be equal and the hypoth esis of equality of the two would be sus tained. Certainly we would not like to conclude that the of teaching were equivalent since we lack the necessary basis for judging the equiva lence with a test whose items so obviously inter act with the methods. This may happen on a smaller scale in real educational experiments and consequently should be guarded against. We need to use an examination with items which are effected by, but do not interact with, the factors of the experiment. Tests of hypotheses for item method interactions and consistency coefficients would be of great value in the analysis of exper iments utilizing examination data. It is the purpose of this paper to show how item-factor interactions may be tested in a 1-fac tor experiment where the variables are obtained as a result of an examination score and to evalu ate the internal consistency coefficient and the index of internal consistency of the examination. The factor will be referred to as methods in the discussion which follows. These ideas may be easily generalized to other designs typically used in educational and psychological research.