File(s) under permanent embargo
Automated Question Answering for Improved Understanding of Compliance Requirements: A Multi-Document Study
conference contribution
posted on 2023-02-22, 05:14 authored by S Abualhaija, Chetan AroraChetan Arora, A Sleimi, LC BriandSoftware systems are increasingly subject to regulatory compliance. Extracting compliance requirements from regulations is challenging. Ideally, locating compliance-related information in a regulation requires a joint effort from requirements engineers and legal experts, whose availability is limited. However, regulations are typically long documents spanning hundreds of pages, containing legal jargon, applying complicated natural language structures, and including cross-references, thus making their analysis effort-intensive. In this paper, we propose an automated question-answering (QA) approach that assists requirements engineers in finding the legal text passages relevant to compliance requirements. Our approach utilizes large-scale language models fine-tuned for QA, including BERT and three variants. We evaluate our approach on 107 question-answer pairs, manually curated by subject-matter experts, for four different European regulatory documents. Among these documents is the general data protection regulation (GDPR) - a major source for privacy-related requirements. Our empirical results show that, in $\approx 94$% of the cases, our approach finds the text passage containing the answer to a given question among the top five passages that our approach marks as most relevant. Further, our approach successfully demarcates, in the selected passage, the right answer with an average accuracy of $\approx$91%.
History
Volume
2022-AugustPagination
39-50Publisher DOI
Start date
2022-08-15End date
2022-08-19ISSN
1090-705XeISSN
2332-6441ISBN-13
9781665470001Title of proceedings
Proceedings of the IEEE International Conference on Requirements EngineeringEvent
2022 IEEE 30th International Requirements Engineering Conference (RE)Publisher
IEEEUsage metrics
Categories
No categories selectedKeywords
Licence
Exports
RefWorks
BibTeX
Ref. manager
Endnote
DataCite
NLM
DC