M. Pugliatti, G. Rosati, H. Carton, T. Riise, J. Drulovic et al., The epidemiology of multiple sclerosis in Europe, European Journal of Neurology, vol.51, issue.2, pp.700-722, 2006.
DOI : 10.1093/heapol/16.3.326

C. H. Polman, S. C. Reingold, B. Banwell, M. Clanet, J. A. Cohen et al., Diagnostic criteria for multiple sclerosis: 2010 Revisions to the McDonald criteria, Diagnostic criteria for multiple sclerosis: 2010 Revisions to the McDonald criteria, pp.292-302, 2011.
DOI : 10.1002/ana.21464
URL : http://onlinelibrary.wiley.com/doi/10.1002/ana.22366/pdf

E. Leray, J. Yaouanq, E. Le-page, M. Coustans, D. Laplaud et al., Evidence for a two-stage disability progression in multiple sclerosis, Brain, vol.67, issue.5, pp.1900-1913, 2010.
DOI : 10.1212/01.wnl.0000234064.17156.03
URL : https://academic.oup.com/brain/article-pdf/133/7/1900/927470/awq076.pdf

D. Mortazavi, A. Z. Kouzani, and H. Soltanian-zadeh, Segmentation of multiple sclerosis lesions in MR images: a review, Neuroradiology, vol.12, issue.3, pp.299-320, 2012.
DOI : 10.1007/s00330-001-1170-8

X. Lladó, A. Oliver, M. Cabezas, J. Freixenet, J. C. Vilanova et al., Segmentation of multiple sclerosis lesions in brain MRI: A review of automated approaches, Information Sciences, vol.186, issue.1, pp.164-185, 2012.
DOI : 10.1016/j.ins.2011.10.011

D. García-lorenzo, S. Francis, S. Narayanan, D. L. Arnold, and D. L. Collins, Review of automatic segmentation methods of multiple sclerosis white matter lesions on conventional magnetic resonance imaging, Medical Image Analysis, vol.17, issue.1, pp.1-18, 2013.
DOI : 10.1016/j.media.2012.09.004

M. Styner, J. Lee, B. Chin, M. Chin, O. Commowick et al., 3D Segmentation in the Clinic: A Grand Challenge II: MS lesion segmentation

A. Carass, S. Roy, A. Jog, J. L. Cuzzocreo, E. Magrath et al., Longitudinal multiple sclerosis lesion segmentation: Resource and challenge, Longitudinal Multiple Sclerosis Lesion Segmentation: Resource & Challenge, pp.77-102, 2017.
DOI : 10.1016/j.neuroimage.2016.12.064
URL : https://hal.archives-ouvertes.fr/inserm-01480156

F. Cotton, S. Kremer, S. Hannoun, S. Vukusic, and V. Dousset, OFSEP, a nationwide cohort of people with multiple sclerosis: Consensus minimal MRI protocol, Journal of Neuroradiology, vol.42, issue.3, pp.133-140, 2015.
DOI : 10.1016/j.neurad.2014.12.001
URL : https://hal.archives-ouvertes.fr/hal-01378309

A. Akhondi-asl, L. Hoyte, M. E. Lockhart, and S. K. Warfield, A Logarithmic Opinion Pool Based STAPLE Algorithm for the Fusion of Segmentations With Associated Reliability Weights, IEEE Transactions on Medical Imaging, vol.33, issue.10, 1997.
DOI : 10.1109/TMI.2014.2329603

J. Beaumont, O. Commowick, and C. Barillot, Multiple sclerosis lesion segmentation using an automated multimodal graph cut, Proceedings of the 1st MICCAI Challenge on Multiple Sclerosis Lesions Segmentation Challenge Using a Data Management and Processing Infrastructure -MICCAI- MSSEG, pp.1-7, 2016.
URL : https://hal.archives-ouvertes.fr/inserm-01417378

D. García-lorenzo, J. Lecoeur, D. Arnold, D. L. Collins, and C. Barillot, Multiple Sclerosis Lesion Segmentation Using an Automatic Multimodal Graph Cuts, 12th International Conference on Medical Image Computing and Computer Assisted Intervention, pp.584-591, 2009.
DOI : 10.1007/978-3-642-04271-3_71

J. Beaumont, O. Commowick, and C. Barillot, Automatic Multiple Sclerosis lesion segmentation from Intensity-Normalized multi-channel MRI, Proceedings of the 1st MICCAI Challenge on Multiple Sclerosis Lesions Segmentation Challenge Using a Data Management and Processing Infrastructure -MICCAI-MSSEG, pp.8-15, 2016.
URL : https://hal.archives-ouvertes.fr/inserm-01424802

Y. Karpate, O. Commowick, and C. Barillot, Robust Detection of Multiple Sclerosis Lesions from Intensity-Normalized Multi-Channel MRI, in: SPIE Medical Imaging, 2015.
DOI : 10.1117/12.2082032
URL : http://www.hal.inserm.fr/inserm-01127692/file/Yogesh_camera_ready_SPIE.pdf

F. Forbes, S. Doyle, D. Garcia-lorenzo, C. Barillot, and M. Dojat, A weighted multi-sequence markov model for brain lesion segmentation, Proceedings of the Thirteenth International Conference on Arti cial Intelligence and Statistics (AISTATS), pp.225-232, 2010.
URL : https://hal.archives-ouvertes.fr/inserm-00723808

F. Forbes, S. Doyle, D. García-lorenzo, C. Barillot, and M. Dojat, Adaptive weighted fusion of multiple MR sequences for brain lesion segmentation, 2010 IEEE International Symposium on Biomedical Imaging: From Nano to Macro, pp.69-72, 2010.
DOI : 10.1109/ISBI.2010.5490413
URL : https://hal.archives-ouvertes.fr/inserm-00723807

J. Knight and A. Khademi, MS Lesion Segmentation Using FLAIR MRI Only, Proceedings of the 1st MICCAI Challenge on Multiple Sclerosis Lesions Segmentation Challenge Using a Data Management and Processing Infrastructure -MICCAI-MSSEG, pp.21-28, 2016.

A. Mahbod, M. Chowdhury, Ö. Smedby, and C. Wang, Automatic brain segmentation using artificial neural networks with shape context, Pattern Recognition Letters, vol.101, pp.74-79, 2018.
DOI : 10.1016/j.patrec.2017.11.016

A. Mahbod, C. Wang, and Ö. Smedby, Automatic multiple sclerosis lesion segmentation using hybrid artificial neural networks, Proceedings of the 1st MICCAI Challenge on Multiple Sclerosis Lesions Segmentation Challenge Using a Data Management and Processing Infrastructure -MICCAI- MSSEG, pp.29-36, 2016.

R. Mckinley, T. Gundersen, F. Wagner, A. Chan, R. Wiest et al., Nabla-net: a deep dag-like convolutional architecture for biomedical image segmentation: application to white-matter lesion segmentation in multiple sclerosis, Proceedings of the 1st MICCAI Challenge on Multiple Sclerosis Lesions Segmentation Challenge Using a Data Management and Processing Infrastructure -MICCAI-MSSEG, pp.37-43, 2016.

J. Muschelli, E. Sweeney, J. Maronge, and C. Crainiceanu, Prediction of MS Lesions using Random Forests, Proceedings of the 1st MICCAI Challenge on Multiple Sclerosis Lesions Segmentation Challenge Using a Data Management and Processing Infrastructure -MICCAI-MSSEG, pp.45-50, 2016.

M. Cabezas, A. Oliver, E. Roura, J. Freixenet, J. C. Vilanova et al., Automatic multiple sclerosis lesion detection in brain MRI by FLAIR thresholding, Computer Methods and Programs in Biomedicine, vol.115, issue.3, pp.147-161, 2014.
DOI : 10.1016/j.cmpb.2014.04.006

E. Roura, A. Oliver, M. Cabezas, S. Valverde, D. Pareto et al., A toolbox for multiple sclerosis lesion segmentation, Neuroradiology, vol.48, issue.1, pp.1031-1043, 2015.
DOI : 10.1016/j.neuroimage.2009.06.039

M. M. Santos, P. R. Diniz, A. G. Silva-filho, and W. P. Santos, Evaluation-Oriented Training via Surrogate Metrics for Multiple Sclerosis Segmentation, LNCS, vol.8, issue.5, pp.398-405, 2016.
DOI : 10.1016/j.nicl.2015.05.001

M. M. Santos, P. R. Diniz, A. G. Silva-filho, and W. P. Santos, Evaluation- Oriented Training Strategy on MS Segmentation Challenge, Proceedings of the 1st MICCAI Challenge on Multiple Sclerosis Lesions Segmentation Challenge Using a Data Management and Processing Infrastructure -MICCAI-MSSEG, pp.57-62, 2016.

X. Tomas-fernandez and S. K. Warfield, A Model of Population and Subject (MOPS) Intensities With Application to Multiple Sclerosis Lesion Segmentation, IEEE Transactions on Medical Imaging, vol.34, issue.6, pp.1349-1361, 2015.
DOI : 10.1109/TMI.2015.2393853
URL : http://europepmc.org/articles/pmc4506921?pdf=render

X. Tomas-fernandez and S. K. Warfield, MRI Robust Brain Tissue Segmentation with application to Multiple Sclerosis, Proceedings of the 1st MIC- CAI Challenge on Multiple Sclerosis Lesions Segmentation Challenge Using a Data Management and Processing Infrastructure -MICCAI-MSSEG, pp.63-67, 2016.

H. Urien, I. Buvat, N. Rougon, M. Soussan, and I. Bloch, Brain Lesion Detection in 3D PET Images Using Max-Trees and a New Spatial Context Criterion, International Symposium on Mathematical Morphology (ISMM), pp.455-466, 2017.
DOI : 10.1016/S0167-8655(03)00049-7

H. Urien, I. Buvat, N. Rougon, and I. Bloch, A 3D hierarchical multimodal detection and segmentation method for multiple sclerosis lesions in MRI, Proceedings of the 1st MICCAI Challenge on Multiple Sclerosis Lesions Segmentation Challenge Using a Data Management and Processing Infrastructure -MICCAI-MSSEG, pp.69-73, 2016.
URL : https://hal.archives-ouvertes.fr/inserm-01417465

S. Valverde, M. Cabezas, E. Roura, S. González-villà, D. Pareto et al., Improving automated multiple sclerosis lesion segmentation with a cascaded 3D convolutional neural network approach, NeuroImage, vol.155, pp.159-168, 2017.
DOI : 10.1016/j.neuroimage.2017.04.034

F. Vera-olmos, H. Melero, and N. Malpica, Random Forest for Multiple Sclerosis Lesion Segmentation, Proceedings of the 1st MICCAI Challenge on Multiple Sclerosis Lesions Segmentation Challenge Using a Data Management and Processing Infrastructure -MICCAI-MSSEG, pp.81-86, 2016.

M. Filippi, M. A. Horsfield, S. P. Morrissey, D. G. Macmanus, P. Rudge et al., Quantitative brain MRI lesion load predicts the course of clinically isolated syndromes suggestive of multiple sclerosis, Neurology, vol.44, issue.4, pp.635-635, 1994.
DOI : 10.1212/WNL.44.4.635

R. A. Rudick, J. Lee, J. Simon, and E. Fisher, Significance of T2 lesions in multiple sclerosis: A 13-year longitudinal study, Annals of Neurology, vol.112, issue.2, pp.236-242, 2006.
DOI : 10.1212/WNL.53.8.1698

A. S. Ribeiro, D. J. Nutt, and J. Mcgonigle, Which Metrics Should Be Used in Non-linear Registration Evaluation?, pp.388-395, 2015.
DOI : 10.1016/j.neuroimage.2008.09.016

P. Coupé, P. Yger, S. Prima, P. Hellier, C. Kervrann et al., An Optimized Blockwise Nonlocal Means Denoising Filter for 3-D Magnetic Resonance Images, IEEE Transactions on Medical Imaging, vol.27, issue.4, pp.425-441, 2008.
DOI : 10.1109/TMI.2007.906087

O. Commowick, N. Wiest-daesslé, and S. Prima, Block-matching strategies for rigid registration of multimodal medical images, 2012 9th IEEE International Symposium on Biomedical Imaging (ISBI), pp.700-703, 2012.
DOI : 10.1109/ISBI.2012.6235644
URL : https://hal.archives-ouvertes.fr/inserm-00681610

J. V. Manjón and P. Coupé, volBrain: An Online MRI Brain Volumetry System, Frontiers in Neuroinformatics, vol.13, issue.54, 2016.
DOI : 10.1109/42.363096

N. J. Tustison, B. B. Avants, P. A. Cook, Y. Zheng, A. Egan et al., N4ITK: Improved N3 Bias Correction, N4ITK: Improved N3 Bias Correction, pp.1310-1320, 2010.
DOI : 10.1109/TMI.2010.2046908

C. Barillot, E. Bannier, O. Commowick, I. Corouge, A. Baire et al., Shanoir: Applying the Software as a Service Distribution Model to Manage Brain Imaging Research Repositories, Frontiers in information and communication technolo- giesdoi:10

T. Glatard, C. Lartizien, B. Gibaud, R. F. Da-silva, G. Forestier et al., A Virtual Imaging Platform for Multi-Modality Medical Image Simulation, IEEE Transactions on Medical Imaging, vol.32, issue.1, pp.110-118, 2013.
DOI : 10.1109/TMI.2012.2220154
URL : https://hal.archives-ouvertes.fr/inserm-00762497

A. Y. Ng, M. I. Jordan, and Y. Weiss, On spectral clustering: Analysis and an algorithm, Advances in Neural Information Processing Systems 14, pp.849-856, 2001.

M. Calvo and J. Oller, AN EXPLICIT SOLUTION OF INFORMATION GEODESIC EQUATIONS FOR THE MULTIVARIATE NORMAL MODEL, Statistics & Risk Modeling, vol.9, issue.1-2
DOI : 10.1524/strm.1991.9.12.119

L. Dice, Measures of the Amount of Ecologic Association Between Species, Ecology, vol.26, issue.3, pp.297-302, 1945.
DOI : 10.2307/1932409

, Additional Information Authors of this article contributed to the article in the following manner

O. @bullet-jérémy-beaumont, C. Commowick, S. Barillot, M. Doyle, F. Dojat et al.,

H. Warfield, I. Urien, S. Bloch, M. Valverde, and . Cabezas, Francisco Javier Vera-Olmos and Norberto Malpica designed the challengers' repsective algorithms, participated to the challenge, participated in the writing and proof-reading of the evaluated teams description in particular, and of the proof-reading of the whole article

B. @bullet-michaël-kain, F. Laurent, M. Leray, S. Simon, P. Camarasu-pop et al., Christian Barillot and Michel Dojat participated in the setup of the platform and running of the experiments on the France Life Imaging platform, the writing and proof-reading of the pipeline processing description and results in particular, and of the proof-reading of the whole article

A. @bullet-olivier-commowick, F. Istace, B. Leray, R. Laurent, J. Améli et al., Gilles Edan and François Cotton participated in the constitution of the evaluation database (selection of the patients, expert guidance on the delineation of the lesions), in the analysis of results