Molecular pathology tests on tumours are increasingly required by clinicians seeking targeted treatments for patients with cancers. The list of targeted therapies is rapidly expanding, and molecular tests are already mandatory to guide treatment decisions for patients with metastatic colorectal carcinomas with EGFR antibodies 1 2 , lung carcinomas with EGFR inhibitors 3 and metastatic melanomas with BRAF inhibitors 4 . These tests are usually performed on DNA extracted from formalin-fixed paraffin-embedded (FFPE) tumour samples. In 2005, trastuzumab was shown to improve the survival of patients with breast carcinomas 5 6 , and the evaluation of HER2 status became mandatory in patients with such tumours. However, in 2006, central testing for 2535 patients confirmed only 86% of the results obtained in local laboratories 7 , and there was still some debate about the reliability of testing for HER2 amplification in 2011 8 . Thus, since 2005, thousands of patients may have received inappropriate treatment due to false results in mutation tests. The history of in situ testing for HER2 should encourage physicians and health authorities to pay attention to the quality of mutation testing in cancers.

Disease-free and overall survival is improved by treatment with the specific inhibitor vermurafenib, in patients with advanced or metastatic melanomas with BRAF mutations 4 9 . This has led US and European authorities to approve its use for patients with p.V600E or p.V600 mutations, respectively. Clinical benefit has also been demonstrated for other BRAF and/or MEK inhibitors 10 11 , and clinical assays are currently underway in patients at earlier stages of disease. Thus, since June 2011the determination of BRAF mutational status has been obligatory, to determine the best treatment options for patients with late-stage melanomas.

In 2008, the French National Institute for Cancer (INCa) initiated a nationwide program for the development of regional platforms for molecular pathology testing for cancers 12 . This national network consists of 28 platforms involving 46 laboratories. These laboratories are supported by grants from INCa, and all the tests are performed free of charge. In 2011, 20,761 EGFR tests for lung non-small cell carcinomas and 17,153 KRAS tests for colorectal carcinomas (http://www.e-cancer.fr) were performed. In practical terms, any of the 63,703,191 inhabitants of France (http://www.insee.fr) can benefit from molecular pathology analysis for free, provided it is necessary for treatment, and this nationwide program can thus be considered a success. The French platforms performed 3,479 BRAF tests on melanoma samples in 2011 and 4,629 in 2012.

This study is the first evaluating the quality of the tests performed in this national network. We used three parameters to assess quality: BRAF mutation status, response times and conformity to the French recommendations for test reporting.

The EQA involved two independent tests, performed in May and November 2012. The protocol was identical for both tests (May and November): 10 formalin-fixed paraffin-embedded (FFPE) samples with massive lymph node metastases were obtained from the Ambroise Paré Centre for Biological Resources (Boulogne, France). The study was approved by the Ile de France 8 institutional review board (#12 01 08). The samples studied were taken from patients who had already died before the start of the study. Four patients had a p.V600E, one had a p.V600K and five had no BRAF p.V600 mutations. The reference mutational status was checked by Cobas 4 and pyrosequencing 13 14 . A dedicated computer program was developed (Lincoln, Boulogne, France) to anonymise each sample and each participating laboratory through the assignment of random code numbers. Each sample received three different identification numbers, to prevent the exchange of information between participating laboratories. For each test, six of the ten samples were selected at random, such that the final set of six samples contained at least two cases with and two cases without BRAF mutations. The participating laboratories, EQA organisers and sponsor were unaware of the samples chosen until the databases of collected responses were blocked.

Each of the ten selected FFPE samples was re-embedded in two twin blocks, and the six selected cases were then re-embedded again by an external pathology laboratory (IPP, Paris, France) for anonymisation. Serial 4 μm-thick sections were cut from the same FFPE block where possible, or from the twin block if the first block had already been used up. Sections were cut in a dedicated molecular pathology environment (specific room, equipment, reagents and trained technicians). Slides were received by the participating laboratories within four weeks of cutting. The participating laboratories were aware of the month, but not of the day of testing.

Participation in this external quality control (EQA) study was free of charge. The protocol was sent to all the French laboratories and to four other European laboratories. All the laboratories contacted agreed to participate (n=50). The results of the first test (May) were communicated to the laboratories (in July) before the second test was carried out (November). The French recommendations were available from the INCa web site (http://www.e-cancer.fr). Minor modifications to these recommendations were published between tests #1 and #2, and all the laboratories were informed of these changes.

All original molecular pathology reports were sent by post, fax and/or e-mail to Lincoln, for anonymisation. Anonymised reports were analysed by the organising laboratory and data the data were entered into the dedicated software. BRAF mutation status was entered twice, independently. Compliance with French recommendations was evaluated by analysing each original report and scoring 15 different parameters 0, 1, 2 or 3, corresponding to “absent”, “incomplete” and “complete”, respectively; Additional file 1: Table S1). Response time was determined as the interval between the day of delivery of the slide batches by the transporter and the day on which the corresponding report was received. The maximum acceptable response time was 40 days in test #1 and 28 days in test #2. A “good” response was defined as indicated in Table 1. This EQA did not evaluate the methods used by the various laboratories to assess BRAF status, but the collection of this information was optional.

The quality of the DNA obtained from the FFPE samples was assessed after the completion of both tests. DNA was quantified by spectrophotometry (Nanodrop) and concentrations were adjusted to 25 ng/μl. Real-time PCR, generating an 80-base pair amplicon 15 , was performed 14 times on each sample.

For the six samples of test #2, a deep sequencing analysis was performed, with the Ion AmpliSeq™ Cancer Hotspot Panel v2 primer pool and Ion AmpliSeq ™ Master Mix v2.0 (Ion Torrent, Life Technologies, Carlsbad, CA), according to the protocol recommended by the manufacturer. The multiplexed amplicon library concentration and size was determined with an Experion™ DNA analysis kit (Bio-Rad Laboratories Inc., Hercules, CA). Samples were barcoded with the Ion Xpress Barcode Adapters 1–16 Kit, according to the manufacturer’s instructions (Ion Torrent, Life Technologies) and multiplexed for emulsion PCR. Sequencing was performed with the Ion 316 Chip, on a Personal Genome Machine Sequencer (PGM, Ion Torrent, Life Technologies). The variants were characterised with the associated variant caller software.

Statistical analysis was performed with SAS software (SAS Institute, Cary, USA). Chi-squared or Fisher’s exact tests were used to assess differences for qualitative data, and analysis of variance or non-parametric Mann–Whitney tests were used to assess differences for quantitative data. All tests were two-tailed and a significance threshold of 5% was applied in all cases.

The 12 samples randomly selected for the tests corresponded to four cases with c.1799T>A, p.V600E, two cases with c.1798_1799GT>AA, p.V600K, and six cases without p.V600 BRAF mutations. The four European laboratories responded in due time in all cases and were evaluated only for BRAF status. The results were correct for 47/48, and false for one case (no mutation for a sample with a p.V600K mutation).

The following results concern only the 46 French laboratories, one of which participated only in the second test. We received 524 of the 546 responses expected within an acceptable timeframe (40 days for test #1, and 28 days for test #2). A technical failure of the determination of BRAF status was reported in four of these responses. Thus, overall, BRAF status was evaluated in an acceptable timeframe in 520 of 546 (95.2%) samples.

The health authorities of France (a country with 62 million inhabitants) have set up a network for the detection of somatic mutations in cancers, including BRAF p.V600 mutations in melanomas. This network carried out 3,479 tests for BRAF mutations in 2011 and 4,629 such tests in 2012. This study, the first EQA for this testing, revealed that only 2.7% of the results were false and had potential clinical implications, despite the use of techniques developed in the laboratory concerned and not certified, by most of the 46 laboratories. Most of the false results obtained related to two samples with a p.V600K mutation and poor DNA quality. Response time and pathological evaluations of samples improved significantly in the six months between the tests, as did the evaluation of tumour cell content.

The response time for tests of BRAF status is a major issue. Indeed, the median overall survival of patients with stage IV or advanced melanomas not treated with BRAF inhibitors is only six to 10 months 9 . Thus, treatment strategies, such as targeted or immune therapies, and/or possible inclusion in a clinical trial must be considered within the first month after diagnosis. This target in terms of timing is currently met by expert centres, but remains difficult to achieve in a nationwide network. The mean response time for test #1 was 22 days. This came as something of a surprise, because the expected response time was nine days, based on the data provided by each laboratory to INCa in 2011 (http://www.e-cancer.fr). The participating laboratories agreed to response decrease in the maximum response time from 40 to 28 days for test #2. In test #2, the mean response time was significant better, falling from 22 to 12 days (P<0.001). We attribute this important progress to the EQA and the discussion of the results obtained with all laboratories between tests #1 and #2. However, our findings cannot be considered to demonstrate a similar improvement of response times in routine practice. Quality management at all French medical laboratories, according to ISO 15189, and certification within the seven next years should also help to decrease response times.

This study revealed that almost all French laboratories are currently using tests developed in-house. Cost may be one of the chief reasons for this. Indeed, the use of a certified test to assess the BRAF status of tumour DNA costs at least twice as much as the use of in-house techniques 16 . However, the widespread use of non-certified detection methods might generate a high rate of false results. We show here that the false result rate was only 2.7%, only one third that for the pathological diagnosis of rare tumours in France (http://www.e-cancer.fr). Interestingly, the false result rate was not found to be related to the type of technique used, although the small number of samples (n=12) and the wide range of combinations of techniques used (n=17) limits our interpretation of this finding.

Twelve samples were tested, and 54%, 37% and 9% of the participating laboratories had no, one and two false results, respectively. Training at each laboratory is probably of considerable importance for limiting false results. Unfortunately, as the results were rendered anonymous for the EQA, we were unable to compare false result rates between laboratories as a function of their level of activity. The false result rate depended strongly on the sample and ranged from 0% to 27.3%. We therefore tried to identify parameters associated with a high false result rate. We found that the type of p.V600 mutation and the quality of the DNA obtained from the sample were significantly associated with the likelihood of false results. The false results rate was highest for the two cases with p.V600K mutations. These findings are consistent with those reported for another series, in which 30% of pV600K mutations were not detected by real-time PCR 17 . Real-time PCR with 25 ng/μl DNA revealed significant differences in the quality of the DNA obtained from the 12 samples (P<0.0001), with CT values from 27.4 to 31.2. The tissue must be fixed and embedded in paraffin for diagnosis in clinical practice, but these procedures may modify the nucleic acid 18 . Interestingly, false result rates of 1/260 and 24/261 were obtained for the six samples with the highest DNA quality and the six samples with the lowest DNA quality, respectively. There were too few samples to determine whether the type of p.V600 mutation and DNA quality had independent effects on the risk of a false result. However, our data suggest that the staff of these laboratories should be trained in the detection of rare p.V600 mutations, and consider routine testing the quality of DNA obtained from FFPE samples.

In conclusion, we show here, for the first time, that nationwide EQA can improve the quality of molecular tests on FFPE tumour samples. We also show that, despite the use of several combinations of in-house tests, the false result rate for BRAF testing in melanoma was low. Finally, our data suggest that the training of laboratory staff to detect rare mutations and assessments of DNA quality might limit the risk of false results.

INCa: French National Institute for Cancer; EQA: External quality assessment; EGFR: Epidermal growth factor receptor; DNA: Eeoxyribonucleic acid; FFPE: Formalin-fixed paraffin-embedded; ISO: International Organisation for Standardisation.

JFE, AL, PH, JCS and PLP received honoraria from Roche for expert advice on diagnosis and/or for studies on patients with melanomas. JFE received honoraria from Glaxo Smith Kline for expert advice on the diagnosis and/or treatment with BRAF inhibitors of patients with cancers. LB is employed by Roche.

JFE, JT and and LB designed the study, collected the data, contributed to data interpretation, wrote the manuscript. FN, SS, GF, AL, DL, ZHR, PH, JCS and PLP provided data, contributed to data interpretation. All authors read and approved the final manuscript. Members of the BRAF EQA Group provided data and contributed to data interpretation.