Synthesis of non-overlap of all pairs using logistic transformation or binomial generalized linear mixed model

Available methods for meta-analysis of findings from single-case designs include one-stage methods involving modeling of raw data from across multiple studies and two-stage methods involving calculation of effect sizes and subsequent meta-analysis. The two-stage approach works well for some effect size measures, such as log response ratios, but performs inadequately for the non-overlap of all pairs index. NAP is an effect size in the family of non-overlap measures, which quantify effect magnitude in terms of pairwise rank comparisons of outcomes under different treatment conditions, and is thus a useful metric for outcomes that are not normally distributed and not on a ratio metric. We examine two alternative approaches to meta-analysis of NAP, based on either transforming the effect size estimates or on a binomial generalized linear mixed model. We demonstrate the approaches by re-analyzing data from a meta-analysis of SCEDs examining augmentative and alternative communication interventions and evaluate the performance of the approaches using an extensive simulation study. We find that neither approach performs adequately for synthesis of single-case data series with limited numbers of observations in the baseline and intervention phases.