Test Case Prioritization for Automated Driving Systems: A Cost-Cognizant,Version-Specific, Multi-Objective Black-Box Approach
Ladda ner
Typ
Examensarbete för masterexamen
Program
Publicerad
2019
Författare
KARLSSON, CHRISTOFFER
Modellbyggare
Tidskriftstitel
ISSN
Volymtitel
Utgivare
Sammanfattning
Testing and verification of self-driving cars play an important role in ensuring that
the vehicles work as intended and that the strict safety requirements are met. Automotive
companies are becoming increasingly agile and are adopting the principles
of continuous integration, where short feedback loops are essential. Test case prioritization
(TCP) techniques can help increase testing efficiency by promoting early
fault detection, which enables defects to be found and fixed earlier. This thesis is
done at Volvo Car Corporation (VCC), with the aim to investigate the applicability
of TCP techniques within the complex autonomous drive verification domain, and
to propose a TCP method that can be used in a continuous integration environment.
One challenge is that access to historical test data at VCC is limited, as continuous
integration has not been fully implemented yet. This is one of the reasons why VCC
has developed a system called Verification Data Model and Generator (VeriDaG),
which produces a simulated regression test history that is made to mimic the real
system in a realistic way. In this way, VCC strives to have a prioritization strategy
ready when continuous integration and test automation become fully implemented,
as it may otherwise be too late. The thesis is conducted as a case study involving
action research. A pre-study is conducted to form the basis of an experiment, in
which different TCP techniques are compared. While the results of the thesis indicate
that many existing TCP techniques are unsuitable within this domain, there
still exist some types of techniques that seem applicable. A proposed black-box
TCP method considering multiple factors is implemented and evaluated using the
Average Percentage of Faults Detected per Cost (APFDC) metric, which is used to
measure the efficiency of TCP techniques. It is black-box in the sense that it does
not require access to the source code of the system under test (SUT). Expert-review
evaluations are also conducted at the end of the thesis, to get further insights into
the usability of the proposed method. Combined evaluation of the proposed method
suggests that it can help improve testing efficiency at VCC and likely also at similar
companies.
Beskrivning
Ämne/nyckelord
testing , prioritization , TCP , RTO , value-based , software engineering , automotive , autonomous drive , cost-cognizant , version-specific , multi-objective , blackbox