Tools used to assess the quality of peer review reports: a methodological systematic review

Cecilia Superchi 1, 2, 3, * José Antonio González 1 Ivan Solà 4, 5 Erik Cobo 1 Darko Hren 6 Isabelle Boutron 7
* Corresponding author
1 Department of Statistics and Operations Research [Barcelone, Espagne]
2 CRESS - U1153 - Equipe 5 : METHODS - Méthodes de l’évaluation thérapeutique des maladies chroniques
UPD5 - Université Paris Descartes - Paris 5, CRESS (U1153 / UMR_A_1125 / UMR_S_1153) - Centre de Recherche Épidémiologie et Statistique Sorbonne Paris Cité
3 UPD5 - Université Paris Descartes - Paris 5
4 Iberoamerican Cochrane Centre [Barcelone, Espagne]
5 CIBER de Epidemiología y Salud Pública (CIBERESP)
6 Department of Psychology [Split, Croatie]
7 Centre d'épidémiologie Clinique [Hôtel-Dieu]
Abstract : BACKGROUND: A strong need exists for a validated tool that clearly defines peer review report quality in biomedical research, as it will allow evaluating interventions aimed at improving the peer review process in well-performed trials. We aim to identify and describe existing tools for assessing the quality of peer review reports in biomedical research. METHODS: We conducted a methodological systematic review by searching PubMed, EMBASE (via Ovid) and The Cochrane Methodology Register (via The Cochrane Library) as well as Google® for all reports in English describing a tool for assessing the quality of a peer review report in biomedical research. Data extraction was performed in duplicate using a standardized data extraction form. We extracted information on the structure, development and validation of each tool. We also identified quality components across tools using a systematic multi-step approach and we investigated quality domain similarities among tools by performing hierarchical, complete-linkage clustering analysis. RESULTS: We identified a total number of 24 tools: 23 scales and 1 checklist. Six tools consisted of a single item and 18 had several items ranging from 4 to 26. None of the tools reported a definition of 'quality'. Only 1 tool described the scale development and 10 provided measures of validity and reliability. Five tools were used as an outcome in a randomized controlled trial (RCT). Moreover, we classified the quality components of the 18 tools with more than one item into 9 main quality domains and 11 subdomains. The tools contained from two to seven quality domains. Some domains and subdomains were considered in most tools such as the detailed/thorough (11/18) nature of reviewer's comments. Others were rarely considered, such as whether or not the reviewer made comments on the statistical methods (1/18). CONCLUSION: Several tools are available to assess the quality of peer review reports; however, the development and validation process is questionable and the concepts evaluated by these tools vary widely. The results from this study and from further investigations will inform the development of a new tool for assessing the quality of peer review reports in biomedical research.
Keywords : Methods Peer review Quality control Report Systematic review
Document type :
Journal articles
Complete list of metadatas

Cited literature [55 references]  Display  Hide  Download
https://www.hal.inserm.fr/inserm-02309545
Contributor : Myriam Bodescot <>
Submitted on : Wednesday, October 9, 2019 - 12:58:32 PM
Last modification on : Saturday, November 9, 2019 - 7:52:12 PM

File

s12874-019-0688-x.pdf
Publication funded by an institution

Identifiers

Collections

INRA | UNIV-PARIS5 | UNIV-PARIS7 | USPC

Citation

Cecilia Superchi, José Antonio González, Ivan Solà, Erik Cobo, Darko Hren, et al.. Tools used to assess the quality of peer review reports: a methodological systematic review. BMC Medical Research Methodology, BioMed Central, 2019, 19 (1), pp.48. ⟨10.1186/s12874-019-0688-x⟩. ⟨inserm-02309545⟩

Export

BibTeX TEI DC DCterms EndNote

Share

Metrics

Record views

45

Files downloads

35

Contact

CCSD