We searched PubMed for all reports of randomized trials indexed from 1 July to 31 December 2009 with the publication type ‘Randomized Controlled Trial’ (search as of 4 January 2010). We limited our search to the National Library of Medicine’s (NLM) set of 121 Core Clinical Journals (formerly published as the Abridged Index Medicus), all of which are published in English.

We included all non-primary reports of randomized trials, which reported a comparison between patient groups. This comparison could be between intervention groups as randomized, as in the primary trial publication, or between groups not randomized, such as a comparison between patient subgroups, perhaps across interventions. We defined a non-primary publication as one, which reported trial results other than the primary trial publication (that is, the first publication with consolidated data from all centers, including the results for the primary outcome). We excluded non-primary reports of randomized trials that did not include a comparative analysis (for example, those exploring risk factors in a particular patient group) and those which reported early phase trials (for example, pilot and feasibility studies), trial protocols, or interim analyses.

One person (SH) screened the titles and abstracts of all retrieved reports to exclude any obvious reports of non-randomized studies. A copy of the full article was then obtained for all remaining records and two people assessed and confirmed whether or not they met the eligibility criteria. Any additional material about the trial included as an appendix on the journal website was also obtained if available.

Data extraction was carried out by six reviewers working in pairs (in blocks of 25 articles allocated at random). Each reviewer independently extracted data from eligible reports; any differences between reviewers in a pair were resolved by discussion, with the involvement of an arbitrator if necessary. To ensure consistency between reviewers, we piloted the data extraction form using a sample of five papers from the sample under review. A data extraction manual was developed to provide guidance for each item on the data extraction form. Following piloting of the data extraction form the data extraction manual was modified slightly to ensure consistency in the data extraction process.

We extracted information on whether the comparison between treatment or patients groups was randomized, the journal type, source of funding, details of trial registration, reference to the trial protocol, whether the study was referred to as a non-primary publication in the abstract, the disease area and the type of intervention being investigated. We assessed the following study specific characteristics in relation to the non-primary publication: the number of study centers, the number of study groups, total sample size, and whether the non-primary publication analyzed all, or a subset of, randomized participants. We extracted data on the number of outcomes (where there was a comparison group) reported in the non-primary publication and, for the outcome which was the main focus of the non-primary publication, whether it was pre-specified or exploratory, the statistical methods used, how the results were reported and whether they were statistically significant. If more than one main outcome was reported we selected the one reported first in the methods section, but if not reported there we took the first outcome reported in the results section. We defined an outcome as a variable intended for comparison between groups; any outcomes assessed at multiple time points were classified as separate outcomes 6 . We also assessed the extent to which the non-primary publication reported information about the primary trial publication that reported the main results of the trial. Where non-primary publications provided a citation (or other unique identifier) to the primary publication of the trial, we compared the consistency of information reported in the two publications.

All analyses were descriptive. The primary analysis focused on the general characteristics of the non-primary publication and the reporting of their study outcomes and results. We compared reporting between non-primary publications where the comparison between groups was or was not as randomized. We also compared the consistency of trial information reported in non-primary publications with that reported in the full text of the primary publication.

Table 1 provides information on the general characteristics of the non-primary publications. The majority (n = 56; 81%) of reports were published in specialty journals with by far the most common medical area being cardiology (n = 35; 51%). Forty-five percent (n = 31) of non-primary publication reports were non-industry funded, 25% (n = 17) were part industry funded and 14% (n = 14) were solely industry funded. Around half (n = 35; 51%) of the reports investigated drugs as the primary intervention of interest, whereas 32% (n = 22) assessed surgical or procedural interventions and 16% (n = 11) assessed counseling or lifestyle interventions. Details of trial registration (n = 30; 43%) or where the trial protocol (n = 27; 39%) could be accessed were each reported in less than half of non-primary publication reports. The majority (n = 56; 81%) of trials were described as multicenter, and most had two study groups (n = 55; 80%).

^aIdentified as non-primary in the abstract: publication did not have an abstract (n = 1).

Overall most non-primary publications reported on either subgroup analyses (n = 27; 39%), analyzing a specific subgroup of patients from the randomized trial, or reported on secondary outcomes (n = 29; 42%); 19% (n = 13) reported outcomes during extended follow-up periods (see Additional file 1 for more detail on the types of study identified). Non-primary publications where the comparison between groups was as randomized were more likely to report analyses of secondary outcomes (n = 19; 45%), whereas non-primary publications where the comparison was not as randomized were more likely to report on subgroup analyses or a specific subset of patients (n = 16; 60%).

Table 2 provides information of the type of outcomes and results reported in the non-primary publication. Just under half (n = 29; 42%) of non-primary publications reported between one and five different outcomes (where there was a comparison group), with 16% (n = 11) reporting >11 different outcomes and 16% (n = 11) reporting >20; poor reporting meant that the number of outcomes assessed was sometimes unclear (n = 7; 10%). It was also often unclear (n = 29; 42%) whether the analyses described in the non-primary publication were pre-specified (that is, planned and documented before examination of the data preferably in the study protocol) or exploratory (that is, the hypothesis being tested was not specified before examination of the data). However, the majority of non-primary publications did provide sufficient information on the statistical methods (n = 68; 98%) used to compare groups and reported a summary results for the main outcome of the non-primary publication (n = 66; 96%) with estimated effect sizes and precision (n = 52; 75%); in 64% (n = 44) the results for the main outcome were statistically significant.

^aFor which there was a comparison between groups.

It was unclear from reading the abstract in 41% (n = 28) of non-primary publications that it did not report the main results of the trial. Half (n = 33; 48%) of non-primary publications cited only the primary trial report, if additional publications were cited this usually related to the study protocol (n = 19; 27%); five only cited the trial protocol despite the main results of the trial having already been published (Table 3). Less than half (n = 27; 39%) of non-primary publications reported the main outcome of the trial (that is, that reported in the primary publication) with only 6% (n = 4) reporting its numerical results and only 9% (n = 6) reporting on the method of randomization.

^aNumber of citations relating to primary publication: primary publication not published at time of non-primary publication (n = 1).

Finally, we compared the information reported in the non-primary publication with that reported in the primary publication (Table 4). The majority (n = 56; 81%) of non-primary publications were published in specialty journals whereas most (n = 46; 68%) primary publications were published in general medical journals, with one-quarter (n = 16; 23%) having the same corresponding author in both publications. Non-primary publications were more likely to report being non-industry funded (Non-primary: 45% versus Primary: 36%) whereas primary publications were more likely to report being solely (20% versus 30%) or partially industry funded (25% versus 33%). The source of funding was the same in both the non-primary and primary publication 65% (n = 45) of the time. In seven publications, the source of funding was reported in the primary publication (solely industry funded n = 4; non-industry funded n = 3) but omitted from the non-primary publication. Less than half of non-primary and primary publications reported details of trial registration or where the trial protocol could be accessed.

Most primary publications (n = 64; 93%) reported the time period in which the trial was conducted compared to around half of non-primary publications (n = 39; 57%). The median number of participants randomized in the primary publication was 1,452 (IQR 389 to 4,439, range 34 to 21,906). Only around one-third of publications (n = 26; 38%), reported the same number of participants in both the non-primary and primary publication. Just over half (n = 37; 53%) of primary publications reported a statistically significant result for main outcome of the trial. Interestingly, of the primary publications which reported a non-statistically significant result for the main outcome of the trial (n = 31; 45%), just over half (n = 17/31; 55%) reported the main outcome of the non-primary publication as being statistically significant.

Our study provides an overview of the information currently reported in non-primary reports of randomized trials published in the scientific literature, the type of analyses they perform, and the extent to which they report information about the main outcome of the randomized trial and its results. The majority (n = 56; 81%) of non-primary publications were large, multicenter trials, published in specialty journals. Most reported on either subgroup analyses, analyzing a specific subgroup of patients from the randomized trial, or reported on secondary outcomes or analyses; a small number reported evaluating primary outcomes during extended follow-up periods. Less than half of non-primary publications reported details of trial registration, where the trial protocol could be accessed, or made it clear in the abstract that the report was not the primary publication for the trial. This could be misleading and make it difficult, or in some cases impossible, to identify multiple publications for the same trial 5 7 .

Non-primary publications often reported results for multiple outcomes. It was frequently unclear, however, how many outcomes had actually been assessed or whether the analyses described in the non-primary publication were pre-specified or post-hoc exploratory analyses. Multiple testing, for example by performing multiple subgroup analysis, can be a problem because of the risk of false positive findings the more analyses that are performed 3 8 . This could be a particular problem for post-hoc analysis where it is often unclear how many analyses were undertaken and whether they were motivated by inspection of the data 9 10 11 . An investigator might also be tempted to ‘fish’ for, and selectively report, the results of statistically significant outcomes as opposed to non-significant outcomes 12 ; thus one should be cautious in the interpretation of such results 13 14 . There is some indication of selective reporting in our study whereby primary publications, which reported a non-statistically significant result for the main outcome of the trial, were more likely to report the main outcome of the non-primary publication as being statistically significant.

Readers of non-primary publications of randomized trials should be able to interpret the findings of these new analyses in the context of the previously published main results. Authors should therefore provide sufficient details about the study methods, sample selection, the primary outcome, and its results to enable readers to assess the new findings 15 . However, in our sample less than half of non-primary publications reported the main outcome of the randomized trial, with very few reporting its numerical results.

We are not aware of other similar studies assessing the characteristics of and types of analyses reported in non-primary reports of randomized trials. However, several studies have examined the reporting of subgroup analyses published in primary reports of randomized trials (that is, reporting the primary outcome) and have identified similar shortcomings 9 11 16 17 18 . For example, Wang and colleagues 11 reviewed 97 primary reports of randomized trials published in the New England Journal of Medicine between 2005 and 2006; 59 (61%) reported subgroup analyses, with larger trials and multicenter trials being more likely to report subgroup analyses than smaller trials and single-center trials. Among the trials which reported subgroup analyses, only 21 (36%) mentioned these analyses in the methods section and in 40 (68%) it was unclear whether the subgroup analyses were pre-specified or exploratory. Assmann 9 reported similar findings in a review of 50 trials published in 1997 in four leading medical journals, as did Hernandez and colleagues 18 in a review of 63 cardiovascular trials published between 2002 and 2004.

Our study has some limitations. First, we included only reports of non-primary publications identified in PubMed using the indexing term ‘Randomized Controlled Trial’. We will therefore have missed some non-primary reports that were not indexed using this term. The search was also limited to the National Library of Medicine’s set of Core Clinical Journals and so may not be representative of all journals. Second, we included only reports of non-primary publications which we identified as such from reading the abstract or full text of the article, we did not assess the trial protocol. Some non-primary publications may have been omitted where the distinction was not clear (that is, we identified them as primary reports) or where the authors changed the nature of the outcome from that specified in the protocol. For example Chan and colleagues 6 in a review of 102 reports of randomized trials, identified major discrepancies in the specification of outcomes when comparing the trial protocol with the published article. Given the limitations of our approach to this study, it is possible that we have underestimated the number of non-primary publications and the magnitude of the problem of poor reporting.

The CONSORT Statement, most recently updated in 2010, provides recommendations for reporting the findings of randomized trials 2 . While primarily aimed at reporting the primary results, it also gives some recommendations for when and how secondary outcomes and additional subgroup analyses should be reported within the context of the primary publication. We are not aware of any specific reporting guidelines for addressing non-primary reports of randomized trials (http://www.equator-network.org). In response to our findings, we identified some additional suggestions for what authors should describe when reporting these types of analyses in non-primary reports of randomized trials (see Table 5).

^aAdapted from 2010 CONSORT Statement.

In particular, authors should make clear in the abstract that it is not the primary publication for the trial and whether the analyses being reported were pre-specified or exploratory analyses. In the full text of the article, authors should specify the objectives or hypotheses being tested for example whether reporting subgroup analyses, secondary outcomes, or extended follow-up, and the time point at which they are being assessed. It is also important to set the study objectives within the context of the main trial, giving details of the number of study groups, the interventions, key aspects of trial methodology including the method of randomization, and the primary outcome and its numerical results. Other important information includes the number of participants in each group and whether the non-primary publication analyzed all, or a subset, of randomized participants. The number of outcomes assessed in the non-primary publication, whether each analysis was pre-specified or exploratory, and the statistical methods used to compare groups should be reported. As in the main report, for each analysis authors should report a result for each group and the estimated effect size and precision. Details of trial registration and the trial protocol are also important to help readers have greater understanding of what was planned and what was done, and to assess the validity and reliability of the new findings within context of main randomized trial.