NASA Logo

NTRS

NTRS - NASA Technical Reports Server

Back to Results
Topic Modeling of NASA Space System Problem Reports: Research in PracticeProblem reports at NASA are similar to bug reports: they capture defects found during test, post-launch operational anomalies, and document the investigation and corrective action of the issue. These artifacts are a rich source of lessons learned for NASA, but are expensive to analyze since problem reports are comprised primarily of natural language text. We apply topic modeling to a corpus of NASA problem reports to extract trends in testing and operational failures. We collected 16,669 problem reports from six NASA space flight missions and applied Latent Dirichlet Allocation topic modeling to the document corpus. We analyze the most popular topics within and across missions, and how popular topics changed over the lifetime of a mission. We find that hardware material and flight software issues are common during the integration and testing phase, while ground station software and equipment issues are more common during the operations phase. We identify a number of challenges in topic modeling for trend analysis: 1) that the process of selecting the topic modeling parameters lacks definitive guidance, 2) defining semantically-meaningful topic labels requires nontrivial effort and domain expertise, 3) topic models derived from the combined corpus of the six missions were biased toward the larger missions, and 4) topics must be semantically distinct as well as cohesive to be useful. Nonetheless,topic modeling can identify problem themes within missions and across mission lifetimes, providing useful feedback to engineers and project managers.
Document ID
20180001855
Acquisition Source
Goddard Space Flight Center
Document Type
Conference Paper
Authors
Layman, Lucas
(Fraunhofer USA, Inc. College Park, MD, United States)
Nikora, Allen P.
(Jet Propulsion Lab., California Inst. of Tech. Pasadena, CA, United States)
Meek, Joshua
(Fraunhofer USA, Inc. College Park, MD, United States)
Menzies, Tim
(North Carolina State Univ. Raleigh, NC, United States)
Date Acquired
March 9, 2018
Publication Date
May 14, 2016
Subject Category
Mathematical And Computer Sciences (General)
Report/Patent Number
GSFC-E-DAA-TN30783
Meeting Information
Meeting: International Conference on Mining Software Repositories
Location: Austin, TX
Country: United States
Start Date: May 14, 2016
End Date: May 15, 2016
Sponsors: Institute of Electrical and Electronics Engineers
Funding Number(s)
CONTRACT_GRANT: NNX11AP93G
CONTRACT_GRANT: NNN12AA01C
CONTRACT_GRANT: NNX15AC81G
Distribution Limits
Public
Copyright
Public Use Permitted.
Keywords
Data Mining
Defects
LDA
Natural Language Processing
Topic Modeling
No Preview Available