基于关键信息指导的文本摘要模型

doi:10.12068/j.issn.1005-3026.2023.09.005

摘要/Abstract

摘要： 现有生成式文本摘要模型缺乏对关键词信息的关注，存在输入文本中关键信息丢失问题.因此，提出了一种基于关键词语义信息增强的指针生成网络(keyword semantic information enhancement pointer-generator networks， KSIE-PGN)模型.首先，构建了基于DistilBERT的关键词抽取模型(keywords selection method based on BERT， KSBERT).其次，提出了基于关键词掩码的覆盖机制，在使用覆盖机制时，保留解码过程中模型对关键词的持续关注.接着，KSIE-PGN模型在解码过程融合了多种关键词信息，包括关键词语义向量和关键词上下文向量，从而解决解码器丢失输入文本关键信息这一问题.在CNN/Daily Mail数据集上的实验结果表明KSIE-PGN模型能够较好地捕捉输入文本中的关键信息.

关键词: 生成式文本摘要；指针生成网络；关键词信息；关键词掩码；覆盖机制

Abstract: Existing abstractive text summarization models lack attention to keyword information， which leads to the loss of key information in the input text. A keyword semantic information enhancement pointer-generator networks， named KSIE-PGN， is proposed. Firstly， the keyword selection model KSBERT is built to extract keywords. Secondly， a keyword-masked coverage mechanism based on the information of keywords is proposed. When using the coverage mechanism， the continuous attention to keywords in the decoding process is retained. Then， the KSIE-PGN model integrates keyword information in the decoding process including the keyword semantic vector and the keyword context vector. Therefore， the decoder can avoid losing the key information in the input text. The experimental results on the CNN/Daily Mail dataset show that the model can capture the key information in the input text well.

Key words: abstractive text summarization; pointer generator network; information of keywords; keyword-masked; coverage mechanism

中图分类号:

TP391

林舟，周绮凤. 基于关键信息指导的文本摘要模型[J]. 东北大学学报（自然科学版）, 2023, 44(9): 1251-1258.

LIN Zhou， ZHOU Qi-feng. A Text Summarization Model Guided by Key Information[J]. Journal of Northeastern University(Natural Science), 2023, 44(9): 1251-1258.

参考文献

[1]Zhang Y Z，Zincir-Heywood N，Milios E.World wide web site summarization［J］.Web Intelligence and Agent Systems，2004，2(1):39-53.
[2]Wang L，Cardie C.Domain-independent abstract generation for focused meeting summarization［C］//Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics.Sofia，2013:1395-1405.
[3]Wan X，Yang J，Xiao J.Towards an iterative reinforcement approach for simultaneous document summarization and keyword extraction［C］//Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics.Prague，2007:552-559.
[4]Gehrmann S，Deng Y，Rush A M.Bottom-up abstractive summarization［C］//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.Brussels，2018:4098-4109.
[5]Li H，Zhu J，Zhang J，et al.Keywords-guided abstractive sentence summarization［C］//Proceedings of the AAAI Conference on Artificial Intelligence.New York，2020:8196-8203.
[6]Li C，Xu W，Li S，et al.Guiding generation for abstractive text summarization based on key information guide network［C］//Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.New Orleans，2018:55-60.
[7]Wang Q，Ren J.Abstractive summarization with keyword and generated word attention［C］//2019 International Joint Conference on Neural Networks.Budapest，2019:1-8.
[8]Sutskever I，Vinyals O，Le Q V.Sequence to sequence learning with neural networks［C］//Proceedings of the 27th International Conference on Neural Information Processing Systems.Cambridge，2014:3104-3112.
[9]See A，Liu P J，Manning C D.Get to the point:summarization with pointer-generator networks ［C］ //Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics.Vancouver，2017:1073-1083.
[10]Tu Z，Lu Z，Liu Y，et al.Modeling coverage for neural machine translation［C］//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics.Berlin，2016:76-85.
[11]田脉.新闻文本关键词提取算法研究与实现［D］.武汉:中南财经政法大学，2019.(Tian Mai.Research and implementation of news text keyword extraction algorithm ［D］.Wuhan:Zhongnan University of Economics and Law，2019.)
[12]Gleich D F.PageRank beyond the Web［J］.SIAM Review，2015，57(3):321-363.
[13]Sanh V，Debut L，Chaumond J，et al.DistilBERT，a distilled version of BERT:smaller，faster，cheaper and lighter［EB/OL］.［2022-05-10］.arXiv preprint arXiv:1910.01108，2019.
[14]Guo S，Sanner S.Probabilistic latent maximal marginal relevance［C］//Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval.New York，2010:833-834.
[15]Kenton J D M W C，Toutanova L K.BERT:pre-training of deep bidirectional transformers for language understanding［C］//Proceedings of NAACL-HLT.Minneapolis，2019:4171-4186.
[16]Vaswani A，Shazeer N，Parmar N，et al.Attention is all you need［C］//Proceedings of the 31st International Conference on Neural Information Processing Systems.Long Beach，2017:6000-6010.
[17]Mao Y，Qu Y，Xie Y，et al.Multi-document summarization with maximal marginal relevance-guided reinforcement learning［C］// Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing.Punta Cana，2020:1737-1751.
[18]Erkan G，Radev D R.Lexrank:graph-based lexical centrality as salience in text summarization［J］.Journal of Artificial Intelligence Research，2004，22:457-479.
[19]Liu L，Lu Y，Yang M，et al.Generative adversarial network for abstractive text summarization［C］ //Proceedings of the AAAI Conference on Artificial Intelligence.New Orleans，2018:8109-8110.