Do simple screening statistical tools help to detect reporting bias? - Archive ouverte HAL Access content directly
Journal Articles Annals of Intensive Care Year : 2013

Do simple screening statistical tools help to detect reporting bias?

(1, 2) , (2) , (2) , (1)
1
2

Abstract

BACKGROUND: As a result of reporting bias, or frauds, false or misunderstood findings may represent the majority of published research claims. This article provides simple methods that might help to appraise the quality of the reporting of randomized, controlled trials (RCT). METHODS: This evaluation roadmap proposed herein relies on four steps: evaluation of the distribution of the reported variables; evaluation of the distribution of the reported p values; data simulation using parametric bootstrap and explicit computation of the p values. Such an approach was illustrated using published data from a retracted RCT comparing a hydroxyethyl starch versus albumin-based priming for cardiopulmonary bypass. RESULTS: Despite obvious nonnormal distributions, several variables are presented as if they were normally distributed. The set of 16 p values testing for differences in baseline characteristics across randomized groups did not follow a Uniform distribution on [0,1] (p = 0.045). The p values obtained by explicit computations were different from the results reported by the authors for the two following variables: urine output at 5 hours (calculated p value < 10-6, reported p >= 0.05); packed red blood cells (PRBC) during surgery (calculated p value = 0.08; reported p < 0.05). Finally, parametric bootstrap found p value > 0.05 in only 5 of the 10,000 simulated datasets concerning urine output 5 hours after surgery. Concerning PRBC transfused during surgery, parametric bootstrap showed that only the corresponding p value had less than a 50% chance to be inferior to 0.05 (3,920/10,000, p value < 0.05). CONCLUSIONS: Such simple evaluation methods might offer some warning signals. However, it should be emphasized that such methods do not allow concluding to the presence of error or fraud but should rather be used to justify asking for an access to the raw data.
Fichier principal
Vignette du fichier
2110-5820-3-29.pdf (280.11 Ko) Télécharger le fichier
Vignette du fichier
2110-5820-3-29-S1.DOCX (39.9 Ko) Télécharger le fichier
Vignette du fichier
2110-5820-3-29-S2.DOCX (35.66 Ko) Télécharger le fichier
Vignette du fichier
2110-5820-3-29.xml (47.92 Ko) Télécharger le fichier
Origin : Publisher files allowed on an open archive
Format : Other
Format : Other
Format : Other
Loading...

Dates and versions

inserm-00868762 , version 1 (01-10-2013)

Identifiers

Cite

Romain Pirracchio, Matthieu Resche-Rigon, Sylvie Chevret, Didier Journois. Do simple screening statistical tools help to detect reporting bias?. Annals of Intensive Care, 2013, 3 (1), pp.29. ⟨10.1186/2110-5820-3-29⟩. ⟨inserm-00868762⟩
83 View
167 Download

Altmetric

Share

Gmail Facebook Twitter LinkedIn More