A comparison of hypothesis tests for homogeneity in meta-analysis with focus on rare binary events

Chiyu Zhang, Xinlei Wang, Min Chen, Tao Wang

Research output: Contribution to journalReview articlepeer-review

5 Scopus citations


Analysis of rare binary events is an important problem for biomedical researchers. Due to the sparsity of events in such problems, meta-analysis that integrates information across multiple studies can be applied to increase the efficiency of statistical inference. Although it is critical to examine whether the effect sizes are homogeneous across all studies, a comprehensive review of homogeneity tests has been lacking, and in particular, no attention has been paid to infrequent dichotomous outcomes. We systematically review statistical methods for homogeneity testing. By conducting an extensive simulation analysis and two case studies, we examine the performance of 30 tests in meta-analysis of rare binary outcomes. When using log-odds ratio as the association measure, our simulation results suggest that there is no uniform winner. However, we recommend the test proposed by Kulinskaya and Dollinger (BMC Med Res Methodol, 2015, 15), which uses a gamma distribution to approximate the null distribution, for its generally good performance; for very rare events coupled with small within-study sample sizes, in addition to the Kulinskaya–Dollinger test, we further recommend the conditional score test based on the random-effects hypergeometric model proposed by Liang and Self (Biometrika, 1985, 72:353–358). One should be cautious about the use of the Wald tests, the Lipsitz tests (Biometrics, 1998, 54:148–160), and tests proposed by Bhaumik et al (J Am Stat Assoc, 2012, 107:555–567).

Original languageEnglish (US)
Pages (from-to)408-428
Number of pages21
JournalResearch Synthesis Methods
Issue number4
StatePublished - Jul 2021


  • Wald test
  • bootstrap
  • conditional likelihood
  • fixed effect
  • generalized linear mixed-effects model
  • heterogeneity
  • random effects
  • score test

ASJC Scopus subject areas

  • Education


Dive into the research topics of 'A comparison of hypothesis tests for homogeneity in meta-analysis with focus on rare binary events'. Together they form a unique fingerprint.

Cite this