A Text Summarization Model Guided by Key Information

doi:10.12068/j.issn.1005-3026.2023.09.005

Abstract

Abstract: Existing abstractive text summarization models lack attention to keyword information， which leads to the loss of key information in the input text. A keyword semantic information enhancement pointer-generator networks， named KSIE-PGN， is proposed. Firstly， the keyword selection model KSBERT is built to extract keywords. Secondly， a keyword-masked coverage mechanism based on the information of keywords is proposed. When using the coverage mechanism， the continuous attention to keywords in the decoding process is retained. Then， the KSIE-PGN model integrates keyword information in the decoding process including the keyword semantic vector and the keyword context vector. Therefore， the decoder can avoid losing the key information in the input text. The experimental results on the CNN/Daily Mail dataset show that the model can capture the key information in the input text well.

Key words: abstractive text summarization; pointer generator network; information of keywords; keyword-masked; coverage mechanism

CLC Number:

TP391

LIN Zhou， ZHOU Qi-feng. A Text Summarization Model Guided by Key Information[J]. Journal of Northeastern University(Natural Science), 2023, 44(9): 1251-1258.

References

[1]Zhang Y Z，Zincir-Heywood N，Milios E.World wide web site summarization［J］.Web Intelligence and Agent Systems，2004，2(1):39-53.
[2]Wang L，Cardie C.Domain-independent abstract generation for focused meeting summarization［C］//Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics.Sofia，2013:1395-1405.
[3]Wan X，Yang J，Xiao J.Towards an iterative reinforcement approach for simultaneous document summarization and keyword extraction［C］//Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics.Prague，2007:552-559.
[4]Gehrmann S，Deng Y，Rush A M.Bottom-up abstractive summarization［C］//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.Brussels，2018:4098-4109.
[5]Li H，Zhu J，Zhang J，et al.Keywords-guided abstractive sentence summarization［C］//Proceedings of the AAAI Conference on Artificial Intelligence.New York，2020:8196-8203.
[6]Li C，Xu W，Li S，et al.Guiding generation for abstractive text summarization based on key information guide network［C］//Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.New Orleans，2018:55-60.
[7]Wang Q，Ren J.Abstractive summarization with keyword and generated word attention［C］//2019 International Joint Conference on Neural Networks.Budapest，2019:1-8.
[8]Sutskever I，Vinyals O，Le Q V.Sequence to sequence learning with neural networks［C］//Proceedings of the 27th International Conference on Neural Information Processing Systems.Cambridge，2014:3104-3112.
[9]See A，Liu P J，Manning C D.Get to the point:summarization with pointer-generator networks ［C］ //Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics.Vancouver，2017:1073-1083.
[10]Tu Z，Lu Z，Liu Y，et al.Modeling coverage for neural machine translation［C］//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics.Berlin，2016:76-85.
[11]田脉.新闻文本关键词提取算法研究与实现［D］.武汉:中南财经政法大学，2019.(Tian Mai.Research and implementation of news text keyword extraction algorithm ［D］.Wuhan:Zhongnan University of Economics and Law，2019.)
[12]Gleich D F.PageRank beyond the Web［J］.SIAM Review，2015，57(3):321-363.
[13]Sanh V，Debut L，Chaumond J，et al.DistilBERT，a distilled version of BERT:smaller，faster，cheaper and lighter［EB/OL］.［2022-05-10］.arXiv preprint arXiv:1910.01108，2019.
[14]Guo S，Sanner S.Probabilistic latent maximal marginal relevance［C］//Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval.New York，2010:833-834.
[15]Kenton J D M W C，Toutanova L K.BERT:pre-training of deep bidirectional transformers for language understanding［C］//Proceedings of NAACL-HLT.Minneapolis，2019:4171-4186.
[16]Vaswani A，Shazeer N，Parmar N，et al.Attention is all you need［C］//Proceedings of the 31st International Conference on Neural Information Processing Systems.Long Beach，2017:6000-6010.
[17]Mao Y，Qu Y，Xie Y，et al.Multi-document summarization with maximal marginal relevance-guided reinforcement learning［C］// Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing.Punta Cana，2020:1737-1751.
[18]Erkan G，Radev D R.Lexrank:graph-based lexical centrality as salience in text summarization［J］.Journal of Artificial Intelligence Research，2004，22:457-479.
[19]Liu L，Lu Y，Yang M，et al.Generative adversarial network for abstractive text summarization［C］ //Proceedings of the AAAI Conference on Artificial Intelligence.New Orleans，2018:8109-8110.