NASA Logo

NTRS - NASA Technical Reports Server

Back to Results
Experimental Evaluation of Verification and Validation Tools on Martian Rover SoftwareWe report on a study to determine the maturity of different verification and validation technologies (V&V) on a representative example of NASA flight software. The study consisted of a controlled experiment where three technologies (static analysis, runtime analysis and model checking) were compared to traditional testing with respect to their ability to find seeded errors in a prototype Mars Rover. What makes this study unique is that it is the first (to the best of our knowledge) to do a controlled experiment to compare formal methods based tools to testing on a realistic industrial-size example where the emphasis was on collecting as much data on the performance of the tools and the participants as possible. The paper includes a description of the Rover code that was analyzed, the tools used as well as a detailed description of the experimental setup and the results. Due to the complexity of setting up the experiment, our results can not be generalized, but we believe it can still serve as a valuable point of reference for future studies of this kind. It did confirm the belief we had that advanced tools can outperform testing when trying to locate concurrency errors. Furthermore the results of the experiment inspired a novel framework for testing the next generation of the Rover.
Document ID
20040010327
Acquisition Source
Ames Research Center
Document Type
Preprint (Draft being sent to journal)
Authors
Brat, Guillaume (NASA Ames Research Center Moffett Field, CA, United States)
Giannakopoulou, Dimitra (NASA Ames Research Center Moffett Field, CA, United States)
Goldberg, Allen (NASA Ames Research Center Moffett Field, CA, United States)
Havelund, Klaus (NASA Ames Research Center Moffett Field, CA, United States)
Lowry, Mike (NASA Ames Research Center Moffett Field, CA, United States)
Pasareani, Corina (NASA Ames Research Center Moffett Field, CA, United States)
Venet, Arnaud (NASA Ames Research Center Moffett Field, CA, United States)
Visser, Willem (NASA Ames Research Center Moffett Field, CA, United States)
Washington, Rich (NASA Ames Research Center Moffett Field, CA, United States)
Date Acquired
September 7, 2013
Publication Date
January 1, 2003
Subject Category
Computer Programming And Software
Distribution Limits
Public
Copyright
Work of the US Gov. Public Use Permitted.

Related Records

There are no records associated with this record.
No Preview Available