Performance testing and evaluation always plays an important role in the developmental process of a vehicle, which also applies to autonomous vehicles. The complex nature of an autonomous vehicle from architecture to functionality demands even more quality-and-quantity controlled testing and evaluation than ever before. Most of the existing testing methodologies are task-or-scenario based and can only support single or partial functional testing. These approaches may be helpful at the initial stage of autonomous vehicle development. However, as the integrated autonomous system gets mature, these approaches fall short of supporting comprehensive performance evaluation. This paper proposes a novel hierarchical and systematic testing and evaluation approach to bridge the above-mentioned gap. In this paper, firstly a three-dimensional evaluation model conforming to the functional architecture of autonomous vehicles was built, with each dimension representing one of the three key functional layers of autonomous vehicle including sensing & perception, decision-making ﹠ planning, control & execution. Each dimension has a set of metrics carefully defined with their weights fairly determined based on an entropy weights method. Then, considering environment effect on vehicle functions, we innovatively determine task-scenarios for testing the performance of each dimension. Besides, we design a hierarchical systematic testing method which could specially testing each function layer of autonomous vehicle. Fuzzy comprehensive and TOPSIS evaluation method was proposed to quantitatively evaluate the comprehensive performance of autonomous vehicles under defined task-scenarios. Finally, our methods are used to evaluate three candidate vehicles based on simulation scenario in PanoSim. Compared to traditional approach based on external task performance, the proposed approach can not only provide convincing results of the overall system performance but can also peek into each of the key functional layer and provide insights about their performance. Therefore, this approach provides better guidance for autonomous vehicle research.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    A Comprehensive Testing and Evaluation Approach for Autonomous Vehicles


    Additional title:

    Sae Technical Papers


    Contributors:
    Wang, Guojun (author) / Wang, Jinsong (author) / Yang, Shun (author) / Zhang, Sumin (author) / Deng, Weiwen (author)

    Conference:

    WCX World Congress Experience ; 2018



    Publication date :

    2018-04-03




    Type of media :

    Conference paper


    Type of material :

    Print


    Language :

    English




    A Comprehensive Testing and Evaluation Approach for Autonomous Vehicles

    Wang, Guojun / Deng, Weiwen / Zhang, Sumin et al. | British Library Conference Proceedings | 2018


    Intelligence Testing for Autonomous Vehicles: A New Approach

    Li, Li / Huang, Wu-Ling / Liu, Yuehu et al. | IEEE | 2016


    Testing predictions for autonomous vehicles

    FAIRFIELD NATHANIEL / FURMAN VADIM | European Patent Office | 2019

    Free access

    TESTING PREDICTIONS FOR AUTONOMOUS VEHICLES

    FAIRFIELD NATHANIEL / FURMAN VADIM | European Patent Office | 2018

    Free access

    Testing Predictions For Autonomous Vehicles

    FAIRFIELD NATHANIEL / FURMAN VADIM | European Patent Office | 2021

    Free access