AbstractCrater Detection Algorithms (CDAs) applications range from estimation of lunar/planetary surface age to autonomous landing on planets and asteroids and advanced statistical analyses. A large amount of work on CDAs has already been published. However, problems arise when evaluation results of some new CDA have to be compared with already published evaluation results. The problem is that different authors use different test-fields, different Ground-Truth (GT) catalogues, and even different methodologies for evaluation of their CDAs. Re-implementation of already published CDAs or its evaluation environment is a time-consuming and unpractical solution to this problem. In addition, implementation details are often insufficiently described in publications. As a result, there is a need in research community to develop a framework for objective evaluation of CDAs. A scientific question is how CDAs should be evaluated so that the results are easily and reliably comparable. In attempt to solve this issue we first analyzed previously published work on CDAs. In this paper, we propose a framework for solution of the problem of objective CDA evaluation. The framework includes: (1) a definition of the measure for differences between craters; (2) test-field topography based on the 1/64° MOLA data; (3) the GT catalogue wherein each of 17,582 craters is aligned with MOLA data and confirmed with catalogues by N.G. Barlow et al. and J.F. Rodionova et al.; (4) selection of methodology for training and testing; and (5) a Free-response Receiver Operating Characteristics (F-ROC) curves as a way to measure CDA performance. The handling of possible improvements of the framework in the future is additionally addressed as a part of discussion of results. Possible extensions with additional test-field subsystems based on visual images, data sets for other planets, evaluation methodologies for CDAs developed for different purposes than cataloguing of craters, are proposed as well. The goal of the proposed framework is to contribute to the research community by establishing guidelines for objective evaluation of CDAs.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Open framework for objective evaluation of crater detection algorithms with first test-field subsystem based on MOLA data


    Contributors:

    Published in:

    Publication date :

    2007-04-10


    Size :

    14 pages




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English





    Linking Automatic Incident Detection With Diversion Strategies Using MOLA

    Strong, T. / Harwood, N. / White, J. et al. | British Library Conference Proceedings | 2002


    El far de Formentera : (la Mola)

    Pérez de Arévalo, Javier ;Seoane, Kole | SLUB | 2001


    Using MOC and MOLA Data to Assess Landing Site Safety

    Anderson, F. S. / Golombek, M. P. | NTRS | 2001


    Strategic management of traffic in Kent using MOLA

    Still, P. B. / Harbord, B. J. / IEE; Road Transport Group | British Library Conference Proceedings | 1998