Log-linear modeling and two-sample CFA in the search of discrimination types
Mark Stemmler, C. Raymond Bingham · Psychology science · 2003
Summary Two-sample configurai frequency (CFA) is suggested as a useful statistical tool to compare data from pretest-posttest-designs. The investigated data may be difference or improvement scores. The above procedure is recommended because improvement scores from two dependent samples, although metrically scored, are usually non-normally distributed and therefore not suitable for parametric comparisons. The two-sample CFA is compared to log-linear modeling (LLM); the similarities and dissimilarites between the two statistical methods are presented. LLM takes a model fitting approach, that is LLM tests the goodness-of-fit of a null model, which assumes no interactions between the sample or grouping variable and the outcome variables. Instead of a global approach as used by LLM, CFA takes a local or cell level approach, searching for differences between the hypothesized (null) model and the empirical data. The Fisher-Yates test is introduced as a statistic to test for cell patterns or configurations which discriminate between the two samples under investigation. Real data from educational psychological research is used to demonstrate univariate and bivariate two-sample comparisons. Key words: Two-sample comparison, Configurai Frequency Analysis (CFA), twosample CFA, log-linear modeling (LLM), nonparametric testing, contingency table analysis 1. Introduction In a Lead Article of the journal Applied Psychology (von Bye, Spiel and Wood, 1996) the advantages of configurai frequency analysis (CPA) in applied psychological research were presented. This paper can be understood as a sequelae or a supplement, introducing CPA for the analysis of dependent samples, comparable to a dependent t-test in parametric data analysis, an application not outlined in the mentioned Lead Article. In experimental intervention research improvement scores Y = X^sub Time 2^ - X^sub Time 1^ are derived from treatment N^sub Treatment^ and control samples N^sub Control^ of N^sub Total^ individuals (e.g., children, adolescents) as part of prctcst-posttcst treatment designs. In such designs both samples are observed before and after an intervention, for example, before and after an enhancement program for children. When the improvement scores (Y) are scaled ordinally rather than metrically (or continuously) it is more appropriate to test for group differences using a nonparametric statistical approach such as the Mann-Whitney U test or a median test. However, ratings are often bimodal, one mode resulting from small improvements in the control group and the other mode resulting from large improvements in the treatment group. In such instances the U test (cf. Siegel and Castellan, 1988) should not be applied. Here, the application of other nonparametric tests, for instance, log-linear modeling (LLM) or the twosample Configurai Frequency Analysis (CPA), is recommended. Although CPA is a statistical technique with some tradition (Liencrt and Krauth, 1975; Netter, 1996), its merits for evaluation research have been widely neglected in the authors' opinions. In this paper, CFA is presented as a nonparametric statistical tool to test for differences between two dependent samples. The similarities and dissimilarities between CFA and LLM are illustrated using real data from educational psychology. 2. Configurai frequency analysis Configurai Frequency Analysis (CFA) is a nonparametric tool for the analysis of d-dimensional contingency tables (von Eye, 1990; 2002). In CPA the cells of a contingency table, called configurations, are analyzed by comparing expected frequencies to observed frequencies. The binomial test, Pearson's chi-square or asymptotic approximations to (he z-statistic arc the commonly used test statistics to compare the expected to the observed frequencies (Krauth, 1993 ; Lautsch and von Weber, 1995). Expected frequencies may be based on any hypothetical model and arc usually expressed in terms of a log-linear model, typically a main effect model (Mcllenbergh, 1996; von Eye and Nesselroade, 1992). …