Skip to Main content Skip to Navigation
Journal articles

Objective Evaluation of Multiple Sclerosis Lesion Segmentation using a Data Management and Processing Infrastructure

Olivier Commowick 1, * Audrey Istace 2 Michael Kain 1 Baptiste Laurent 3 Florent Leray 1 Mathieu Simon 1, 2 Sorina Camarasu Pop 4 Pascal Girard 4 Roxana Ameli 2 Jean-Christophe Ferré 5, 1 Anne Kerbrat 1, 6 Thomas Tourdias 7 Frédéric Cervenansky 4 Tristan Glatard 8 Jeremy Beaumont 1 Senan Doyle 9 Florence Forbes 10, 9 Jesse Knight 11 April Khademi 12 Amirreza Mahbod 13 Chunliang Wang 13 Richard Mckinley 14 Franca Wagner 14 John Muschelli 15 Elizabeth Sweeney 15 Eloy Roura 16 Xavier Llado 16 Michel Santos 17 Wellington Santos 17 Abel Silva-Filho 17 Xavier Tomas-Fernandez 18 Hélène Urien 19 Isabelle Bloch 19 Sergi Valverde 16 Mariano Cabezas 16 Francisco Javier Vera-Olmos 20 Norberto Malpica 20 Charles Guttmann 21 Sandra Vukusic 22 Gilles Edan 1, 4 Michel Dojat 23 Martin Styner 24 Simon Warfield 18 François Cotton 2 Christian Barillot 1
* Corresponding author
1 VisAGeS - Vision, Action et Gestion d'informations en Santé
INSERM - Institut National de la Santé et de la Recherche Médicale : U1228, Inria Rennes – Bretagne Atlantique , IRISA-D5 - SIGNAUX ET IMAGES NUMÉRIQUES, ROBOTIQUE
2 CHLS - Centre Hospitalier Lyon Sud [CHU - HCL]
3 LaTIM - Laboratoire de Traitement de l'Information Medicale
4 CREATIS - Centre de Recherche en Acquisition et Traitement de l'Image pour la Santé
5 Département de Radiologie [CHU de Rennes]
6 CHU Pontchaillou [Rennes]
7 CHU Bordeaux [Bordeaux]
8 Concordia University [Montreal]
9 Pixyl Medical [Grenoble]
10 MISTIS - Modelling and Inference of Complex and Structured Stochastic Systems
Inria Grenoble - Rhône-Alpes, Grenoble INP - Institut polytechnique de Grenoble - Grenoble Institute of Technology, LJK - Laboratoire Jean Kuntzmann
11 University of Guelph
12 Ryerson University [Toronto]
13 KTH - Royal Institute of Technology [Stockholm]
14 University of Bern
15 Johns Hopkins Bloomberg School of Public Health [Baltimore]
16 VICOROB - Research institute of Computer Vision and Robotics [Girona]
17 UFPE - Universidade Federal de Pernambuco [Recife]
18 CRL - Computational Radiology Laboratory [Boston]
19 LTCI - Laboratoire Traitement et Communication de l'Information
20 URJC - Universidad Rey Juan Carlos [Madrid]
21 Brigham and Women's Hospital [Boston]
22 Service de Neurologie [Lyon]
23 GIN - Grenoble Institut des Neurosciences
24 UNC - University of North Carolina [Chapel Hill]
Abstract : We present a study of multiple sclerosis segmentation algorithms conducted at the international MICCAI 2016 challenge. This challenge was operated using a new open-science computing infrastructure. This allowed for the automatic and independent evaluation of a large range of algorithms in a fair and completely automatic manner. This computing infrastructure was used to evaluate thirteen methods of MS lesions segmentation, exploring a broad range of state-of-theart algorithms, against a high-quality database of 53 MS cases coming from four centers following a common definition of the acquisition protocol. Each case was annotated manually by an unprecedented number of seven different experts. Results of the challenge highlighted that automatic algorithms, including the recent machine learning methods (random forests, deep learning, …), are still trailing human expertise on both detection and delineation criteria. In addition, we demonstrate that computing a statistically robust consensus of the algorithms performs closer to human expertise on one score (segmentation) although still trailing on detection scores.
Document type :
Journal articles
Complete list of metadata

Cited literature [46 references]  Display  Hide  Download
https://www.hal.inserm.fr/inserm-01847873
Contributor : Gilles Edan <>
Submitted on : Tuesday, January 28, 2020 - 11:01:25 AM
Last modification on : Monday, March 29, 2021 - 2:44:17 PM

File

Vignette du document
s41598-018-31911-7.pdf
Publication funded by an institution

Identifiers

Collections

INSERM | UNIV-BREST | HCL | CHLS | CREATIS | UNIV-RENNES1 | INRIA | LJK | CENTRALESUPELEC | UNIV-ST-ETIENNE | UNIV-LYON1 | UNIV-RENNES | LATIM | INSA-GROUPE | PARISTECH | IRISA | CNRS | UDL | INSTITUT-TELECOM | UR1-UFR-ISTIC | LJK_PS | LJK_PS_MISTIS | INSA-RENNES | IDS | IMAGES | ECOGE | SES | I3-TPT | LTCI | S2A | POLITIQUES-PUBLIQUES | UGA

Citation

Olivier Commowick, Audrey Istace, Michael Kain, Baptiste Laurent, Florent Leray, et al.. Objective Evaluation of Multiple Sclerosis Lesion Segmentation using a Data Management and Processing Infrastructure. Scientific Reports, Nature Publishing Group, 2018, 8 (1), pp.13650. ⟨10.1038/s41598-018-31911-7⟩. ⟨inserm-01847873v3⟩

Export

BibTeX TEI DC DCterms EndNote

Share

Metrics

Record views

589

Files downloads

1391