Objective Evaluation of Multiple Sclerosis Lesion Segmentation using a Data Management and Processing Infrastructure

Olivier Commowick 1 Audrey Istace 2 Michael Kain 1 Baptiste Laurent 3 Florent Leray 1 Mathieu Simon 1 Sorina Camarasu Pop 4 Pascal Girard 4 Roxana Ameli 5 Jean-Christophe Ferré 6, 1 Anne Kerbrat 1, 7 Thomas Tourdias 8 Frédéric Cervenansky 9 Tristan Glatard 10 Jeremy Beaumont 1 Senan Doyle 11 Florence Forbes 12 Jesse Knight 13 April Khademi 14 Amirreza Mahbod 15 Chunliang Wang 15 Richard Mckinley 16 Franca Wagner 16 John Muschelli 17 Elizabeth Sweeney 17 Eloy Roura 18 Xavier Llado 19 Michel Santos 20 Wellington Santos 20 Abel Silva-Filho 20 Xavier Tomas-Fernandez 21 Hélène Urien 22 Isabelle Bloch 22 Sergi Valverde 23 Mariano Cabezas 18 Francisco Javier Vera-Olmos 24 Norberto Malpica 24 Charles Guttmann 25 Sandra Vukusic 26 Gilles Edan 1 Michel Dojat 27 Martin Styner 28 Simon Warfield 21 François Cotton 2 Christian Barillot 1
1 VisAGeS - Vision, Action et Gestion d'informations en Santé
INSERM - Institut National de la Santé et de la Recherche Médicale : U1228, Inria Rennes – Bretagne Atlantique , IRISA-D5 - SIGNAUX ET IMAGES NUMÉRIQUES, ROBOTIQUE
9 Service Informatique et développements
CREATIS - Centre de Recherche en Acquisition et Traitement de l'Image pour la Santé
12 MISTIS - Modelling and Inference of Complex and Structured Stochastic Systems
Inria Grenoble - Rhône-Alpes, LJK - Laboratoire Jean Kuntzmann, INPG - Institut National Polytechnique de Grenoble
Abstract : We present a study of multiple sclerosis segmentation algorithms conducted at the international MICCAI 2016 challenge. This challenge was operated using a new open-science computing infrastructure. This allowed for the automatic and independent evaluation of a large range of algorithms in a fair and completely automatic manner. This computing infrastructure was used to evaluate thirteen methods of MS lesions segmentation, exploring a broad range of state-of-theart algorithms, against a high-quality database of 53 MS cases coming from four centers following a common definition of the acquisition protocol. Each case was annotated manually by an unprecedented number of seven different experts. Results of the challenge highlighted that automatic algorithms, including the recent machine learning methods (random forests, deep learning, …), are still trailing human expertise on both detection and delineation criteria. In addition, we demonstrate that computing a statistically robust consensus of the algorithms performs closer to human expertise on one score (segmentation) although still trailing on detection scores.
Document type :
Journal articles
Complete list of metadatas

Cited literature [46 references]  Display  Hide  Download

Contributor : Gilles Edan <>
Submitted on : Tuesday, January 28, 2020 - 11:01:25 AM
Last modification on : Wednesday, February 19, 2020 - 1:32:38 AM


Publication funded by an institution



Olivier Commowick, Audrey Istace, Michael Kain, Baptiste Laurent, Florent Leray, et al.. Objective Evaluation of Multiple Sclerosis Lesion Segmentation using a Data Management and Processing Infrastructure. Scientific Reports, Nature Publishing Group, 2018, 8 (1), pp.13650. ⟨10.1038/s41598-018-31911-7⟩. ⟨inserm-01847873v3⟩



Record views


Files downloads