Inadequate reporting of randomised controlled trials is common and hampers the appraisal of the validity and generalisability of results [22, 23]. To overcome such problems, the Consolidated Standards for reporting of Trials (CONSORT) Group developed the CONSORT statement [24] in 1996, which was followed by a revised version in 2001[25].
This can explain the high level of the mean Jadad score found in our study.
The main objective of this article was to evaluate the methodological quality of RCTs with a HRQOL component in gastrointestinal surgery.
Using the stated selection and eligibility criteria, we found 24 RCTs with HRQOL assessment which included some 3476 patients. HRQOL was a secondary end point in most trials (75%).
The aim of our study was not to compare results with other medical or surgical specialities but to describe how digestive surgeons use HRQOL in their trials and to find how to improve. However, compared with other HRQOL studies in other disease sites and treatments [12–19, 26, 27] (table 4) the overall quality of the reported trials is good, although there are a number of shortcomings with regard to the reporting of the HRQOL design and results.
Among the studies reviewed, there is generally poor details about the rationale for selecting a specific measure and instrument administration.
A justification for selecting the HRQOL measure was given in only five studies (20,8%). In others reviews this justification was given in 9,7 to 90% of the trials (table 4). This point has to be improved as well in surgery as for others specialities.
This is regarded as important because instrument selection is critical for reliability, validity, and reproducible results.
It is also important to standardize the instructions and completion procedures of assessment material administered to patients, particularly in RCTs, because often many researchers and institutions are involved in collecting HRQOL data. Thus, standard procedures help to ensure adequate data quality and minimize any possible bias in data collection. (For example, will patients answer the questionnaire at a clinic visit, by telephone, by mail? Is it to be done by the investigator, his colleagues, or a research nurse?)
Our study was in accordance with previous studies (table 4) where instrument administration was reported in 0 to 66,6%.
A major methodological drawback was a lack of a priori hypothesis about possible HRQOL changes before commencing the trial. Only ten RCTs (41,6%) explicitly stated an a priori hypothesis thus limiting spurious HRQOL results due to multiple significance testing. This results was similar and even better to the majority of previous study where the a priori hypothesis was stated in 13 to 72,7% of the trials (table 4).
A key consideration for future studies is the selection of a limited number of HRQOL indicators before commencing the trial, possibly basing this selection on previous related trials, or on a specific a priori research hypothesis about the impact of a given therapy.
One major issue is the reporting of compliance at baseline and the documentation of missing data. Three RCTs (12,5 %) did not provide the absolute number or the percentage of patients who completed the questionnaire before commencing the trial and 9 (37,5%) did not provide any details about HRQOL missing data during the course of the trial.
This result was similar to previous studies where missing data were documented in only 48,4 to 74,8% (table 4).
Although the majority of trials started with reasonable sample sizes, many were plagued with problems of patient drop-out. Such attrition often limits the general robustness of the results and reduces confidence in the HRQOL conclusions. Data are generally not missing at random and therefore bias can be introduced [17, 28–30]. The benefit of a intervention may be overestimated by comparison of group means as only individuals who remain well enough to fill in questionnaires provide data. The unreported details of missing data is a frequent problem in studies where HRQOL is measured [31] and previous works already proposed procedures to address this issue [32, 33]. More attention to improving compliance and reporting in future studies would be valuable.
Of the 23 eligible studies, 11 (47,8%) found some significant difference on HRQOL scales between arms. This would indicate that the HRQOL measures are valuable in providing additional data.
However, although HRQOL differences were observed, it is necessary to remember that, whereas many subscales are often used and compared over treatments and time, not all subscales will show a significant difference.
This underlines the need to declare in advance the HRQOL hypotheses and the importance of careful interpretation of multiple repeated statistical analyses.
A further trap in analysis of HRQOL data is the difference between statistical and clinical significance in changes of scores. It is acknowledged that, although analysis of large samples may reveal small changes that seem to be statistically significant, these changes may not be clinically meaningful to the patient and are, therefore, of limited value to the improvement of patient care.
An effort to determine if such small numerical differences have a clinical meaning from a patient’s perspective has been highlighted as an important aspect for determining the impact of a given treatment [28, 34].
Unfortunately, only six of these studies (26%) examined the clinical significance of apparent differences. It is highly desirable that future studies will routinely include the concept of clinical significance to help evaluate the value of HRQOL results.
Furthermore, in several RCTs, the HRQL results were not formally presented, but the main results were described in the text.
It is possible that this occurred because most trials used HRQOL as a secondary end point. In such trials, it is frequently observed that limited space is given to HRQOL data, with priority given to the primary clinical end point.
Two authors have overcome this difficulty by separately reporting clinical and HRQL trial outcomes.
This is an opportunity for adequate explanation and presentation of what may often be complex results.
However the disadvantage of splitting the HRQL data from the main trial paper is that surgeons are unlikely to read the HRQL paper once the main clinical message of a particularly trial has been published. If this occurs, then during the process of clinical decision making the HRQL impacts of treatment may be overlooked [19]. It is therefore recommended that clinical and HRQL outcomes are published together so that clinical decision making is based upon relevant patient-centered endpoints.
Whilst we identified the above reported methodological limitations, it was impressive that nearly all the studies used HRQOL valid measures and provided details on the HRQOL timing of assessment during the trial.
There were eleven trials (45,8%) with robust HRQL design, and statistically significant differences in HRQL were reported in six of these trials. Only one trial « very limited » (4%) could be invalidated by its lack of rigor in presenting HRQOL data.
Pertinently, the strict methodological approach to the assessment of the patient-based QOL criteria in the evaluation of therapeutic strategies can help, patients and their doctors, in medical decision making.
The Efficace’s checklist can be considered as a minimum standard; however, HRQOL design also greatly depends on the context and the specific research question of the trial; hence good reports may have different emphases and some issues might have different relevance according to the specific study questions [11].
If HRQL is considered to be a relevant outcome in a clinical trial and if HRQL is assessed robustly, then it will always contribute to clinical decision making regardless of the direction of the outcomes [11, 26]. Only if HRQL assessments are flawed (underpowered study, too many missing data items, invalid questionnaires) may they not contribute to clinical decision making.
Our study showed some limitations. We selected randomised controlled trials (RCTs) from 12 journals. This restrictive choice was led by the recognised quality of the four medical journals selected (leading journals that publish research reports in all fields and have a broad readership) and because the eight surgical journals comprised a good sample of surgery around the world. The purpose of this choice was to create a homogeneous group of publications and conditions that allowed standardised analysis. This arbitrary choice may have introduced a bias causing overestimation of the quality of the RCTs analysed. In addition, authors of original articles were not pursued for additional data or for clarification of points that were unclear about trial methodology.
We recognise also that our review is limited by its restriction to RCTs, but the checklist developed by Efficace and al [11]was originally devised only for this type of design. It could be interesting to develop an applicable and useful checklist for not randomized studies.
Despite these potential limitations, this paper suggests that surgeons are interested in HRQL outcomes and that HRQOL assessment in RCT settings has the potential to provide invaluable data for developing new treatments in gastrointestinal surgery.
