Synthesis of non-overlap of all pairs using logistic transformation or binomial generalized linear mixed model


James E. Pustejovsky

Man Chen


April 21, 2022


American Educational Research Association annual convention


San Diego, CA

Available methods for meta-analysis of findings from single-case designs include one-stage methods involving modeling of raw data from across multiple studies and two-stage methods involving calculation of effect sizes and subsequent meta-analysis. The two-stage approach works well for some effect size measures, such as log response ratios, but performs inadequately for the non-overlap of all pairs index. NAP is an effect size in the family of non-overlap measures, which quantify effect magnitude in terms of pairwise rank comparisons of outcomes under different treatment conditions, and is thus a useful metric for outcomes that are not normally distributed and not on a ratio metric. We examine two alternative approaches to meta-analysis of NAP, based on either transforming the effect size estimates or on a binomial generalized linear mixed model. We demonstrate the approaches by re-analyzing data from a meta-analysis of SCEDs examining augmentative and alternative communication interventions and evaluate the performance of the approaches using an extensive simulation study. We find that neither approach performs adequately for synthesis of single-case data series with limited numbers of observations in the baseline and intervention phases.

Back to top