Should a Trial with Lost Allocation Key be Incorporated in Meta-analysis?

Abstract

Aims: On occasion randomized controlled studies may remain unpublished because the allocation key identifying which group received active treatment and which received placebo (or alternative treatment) has been lost. The purpose of this paper is to explore whether and under what circumstances an unpublished study like NCT00250237 whose allocation key has been irretrievably lost might be incorporated into a meta-analysis, allowing the information contained to contribute, however tentatively, to the evidence base around a clinical topic. Study Design: Review / meta-analysis Place and Duration of Study: Vancouver Coastal Health Research Institute and School of Population and Public Health, University of British Columbia and Department of Surgery, Rijnland Hospital, between 2009 and 2014. Methodology: For two published trials, we calculated separate effect measures for a simulated lost allocation key, under the alternative assumptions that Groups A or B received active treatment. These two effects were compared visually to the pooled effect from a meta-analysis of all other comparable studies. We present as well several quantitative methods for reconstituting a lost allocation code, involving frequentist meta-analysis, meta-regression, and Bayesian techniques. We apply these to an analysis of NCT00250237. Results: The simulated lost allocation keys of both trials were successfully identified through visual methods using meta-analysis. Reconstitution in this manner requires that the trial’s effect depart significantly from the null and that there exist a sufficient number of other high quality studies addressing the same clinical topic to allow unbiased calculation of a pooled summary effect. While direct reconstitution may sometimes be possible, our primary approaches are implicitly Bayesian and, as with NCT00250327, allow probabilities to be estimated for each allocation assignment. Conclusion: Under some circumstances studies with a lost allocation key may be incorporated cautiously into meta-analysis.