Rules

Registration
Confidentiality
Viewing
Publishing

Team registration

  • Once you have successfully registered a team and faxed or emailed the signed agreement of data confidentiality to +31-10-7044722, you can download data and submit results.
  • Anonymous registration is not allowed; you must use your real name, affiliation and e-mail address to register.
  • The provided information will only be listed on the site if you choose to publish the results (see below).

Data confidentiality

  • CTA datasets and associated reference standard provided by the organizers may not be given or distributed under any circumstances to persons other than the person registered for the team and the people mentioned in the data confidentiality agreement.
  • The data of the evaluation framework may only be used for the evaluation of Carotid bifurcation Lumen segmentation and Stenosis Grading (CLS) Algorithm Evaluation framework.
  • When publishing obtained results, you have to cite the paper describing the evaluation framework. Citation details can be found below.

Viewing results

  • Results on the training data can be viewed any time after the uploaded results have been processed.
  • Testing results can only be inspected after confirmation by the workshop organizers. Confirmation for a submission can be requested once a week by sending an e-mail to the organizers.

Publishing results

  • Results of your submission will only be published on the website when they are confirmed and when a document (e.g. journal paper) describing the method is provided. If a commercial system is evaluated no method description is necessary, but a document containing the exact name and version number of the system being used and a description of the procedure that was followed to obtain the results (the settings were used, the user interaction was performed) has to be publicly available.
  • The organizers of the evaluation framework will check the method description before your results will be published on the website.
  • When publishing the obtained results, you have to cite the journal paper describing the evaluation framework:
    K. Hameeteman, M.A. Zuluaga, M. Freiman, L. Joskowicz, O Cuisenaire, L. Florez Valencia, M.A. Gulsun, K. Krissian, J. Mille, W.C.K. Wong, M. Orkisz, H. Tek, M. Hernandez Hoyos, F. Benmansour, A.C.S. Chung, S. Rozie, M.J. van Gils, L. van den Borne, J. Sosna, P. Berman, N. Cohen, P. Douek, I. Sánchez, M. Aissat, M. Schaap, C.T. Metz, G. P. Krestin, A van der Lugt, W.J. Niessen and T. van Walsum, Evaluation framework for carotid bifurcation lumen segmentation and stenosis grading, Medical Image Analysis, 8 2011, vol. 15, pp. 477-488.

    Download paper

    Bibtex : 
    @article{
author = {K. Hameeteman and M.A. Zuluaga and M. Freiman and L. Joskowicz 
and O Cuisenaire and L. Florez Valencia and M.A. Gulsun and K. Krissian 
and J. Mille and W.C.K. Wong and M. Orkisz and H. Tek and M. Hernandez Hoyos 
and F. Benmansour and A.C.S. Chung and S. Rozie and M.J. van Gils and 
L. van den Borne and J. Sosna and P. Berman and N. Cohen and P. Douek 
and I. S{\'a}nchez and M. Aissat and M. Schaap and C.T. Metz and 
G. P. Krestin and A van der Lugt and W.J. Niessen and T. van Walsum},
journal = {Medical Image Analysis},
year = {2011},
month = {8},
number = {4},
pages = {477-488},
title = {Evaluation framework for carotid bifurcation lumen segmentation 
and stenosis grading},
volume = {15},
abstract = {This paper describes an evaluation framework that allows a 
standardized and objective quantitative comparison of carotidartery lumen 
segmentation and stenosis grading algorithms. We describe the data repository 
comprising 56 multi-center,multi-vendor CTA datasets, their acquisition, 
the creation of the reference standard and the evaluation measures. 
This framework has been introduced at the MICCAI 2009 workshop 3D Segmentation 
in the Clinic: A Grand Challenge III,and we compare the results of eight teams 
that participated. These results show that automated segmentation of thevessel 
lumen is possible with a precision that is comparable to manual annotation. 
The framework is open for newsubmissions through the website 
http://cls2009.bigr.nl.},
}