Please use this identifier to cite or link to this item:
http://hdl.handle.net/10397/89095
Title: | Multi-task learning for abstractive and extractive summarization | Authors: | Chen, Y Ma, Y Mao, X Li, Q |
Issue Date: | Mar-2019 | Source: | Data science and engineering, Mar. 2019, v. 4, no. 1, p. 14-23 | Abstract: | The abstractive method and extractive method are two main approaches for automatic document summarization. In this paper, to fully integrate the relatedness and advantages of both approaches, we propose a general unified framework for abstractive summarization which incorporates extractive summarization as an auxiliary task. In particular, our framework is composed of a shared hierarchical document encoder, a hierarchical attention mechanism-based decoder, and an extractor. We adopt multi-task learning method to train these two tasks jointly, which enables the shared encoder to better capture the semantics of the document. Moreover, as our main task is abstractive summarization, we constrain the attention learned in the abstractive task with the labels of the extractive task to strengthen the consistency between the two tasks. Experiments on the CNN/DailyMail dataset demonstrate that both the auxiliary task and the attention constraint contribute to improve the performance significantly, and our model is comparable to the state-of-the-art abstractive models. In addition, we cut half number of labels of the extractive task, pretrain the extractor, and jointly train the two tasks using the estimated sentence salience of the extractive task to constrain the attention of the abstractive task. The results do not decrease much compared with using full-labeled data of the auxiliary task. | Keywords: | Attention mechanism Automatic document summarization Multi-Task learning |
Publisher: | SpringerOpen | Journal: | Data science and engineering | ISSN: | 2364-1185 | EISSN: | 2364-1541 | DOI: | 10.1007/s41019-019-0087-7 | Rights: | © The Author(s) 2019 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The following publication Chen, Y., Ma, Y., Mao, X., & Li, Q. (2019). Multi-task learning for abstractive and extractive summarization. Data Science and Engineering, 4(1), 14-23 is available at https://dx.doi.org/10.1007/s41019-019-0087-7 |
Appears in Collections: | Journal/Magazine Article |
Files in This Item:
File | Description | Size | Format | |
---|---|---|---|---|
Chen2019_Article_Multi-TaskLearningForAbstracti.pdf | 1.63 MB | Adobe PDF | View/Open |
Page views
59
Last Week
0
0
Last month
Citations as of Apr 21, 2024
Downloads
18
Citations as of Apr 21, 2024
SCOPUSTM
Citations
37
Citations as of Apr 19, 2024
WEB OF SCIENCETM
Citations
26
Citations as of Apr 18, 2024
Google ScholarTM
Check
Altmetric
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.