Google AI引入了ALBERT ,它是BERT 的精简版本,用于自监督学习上下文语言表示。主要改进是减少冗余并更有效地分配模型的容量。该方法提高了12个NLP任务的最新性能。
Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, Radu Soricut: ALBERT: ALiteBERTforSelf-supervised LearningofLanguageRepresentations.ICLR 2020.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova: BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. NAACL-HLT (1) 2019: 4171-4186
https://arxiv.org/abs/1810.04805
Tero Karras, Samuli Laine, Timo Aila: A Style-Based Generator Architecture for Generative Adversarial Networks. CVPR 2019: 4401-4410
Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, Timo Aila: Analyzing and Improving the Image Quality of StyleGAN. CoRR abs/1912.04958 (2019)
Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, Jaewoo Kang: BioBERT: a pre-trained biomedical language representation model for biomedical text mining. CoRR abs/1901.08746 (2019)
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, Veselin Stoyanov:RoBERTa: A Robustly Optimized BERT Pretraining Approach. CoRR abs/1907.11692 (2019)
Alejandro Barredo Arrieta, Natalia Díaz Rodríguez, Javier Del Ser, Adrien Bennetot, Siham Tabik, Alberto Barbado, Salvador García, Sergio Gil-Lopez, Daniel Molina, Richard Benjamins, Raja Chatila, Francisco Herrera:Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI. CoRR abs/1910.10045 (2019)
Sebastian Ruder发表了有关自然语言处理的神经迁移学习的论文
(https://ruder.io/thesis/)。
Ruder2019Neural, Neural Transfer Learning for Natural Language Processing, Ruder, Sebastian,2019,National University of Ireland, Galway
Devamanyu Hazarika, Soujanya Poria, Roger Zimmermann, Rada Mihalcea: Emotion Recognition in Conversations with Transfer Learning from Generative Conversation Modeling. CoRR abs/1910.04980 (2019)
Deepanway Ghosal, Navonil Majumder, Soujanya Poria, Niyati Chhaya, Alexander F. Gelbukh: DialogueGCN: A Graph Convolutional Neural Network for Emotion Recognition in Conversation. EMNLP/IJCNLP (1) 2019: 154-164
Google AI Quantum团队在《自然》杂志上发表了一篇论文(https://www.nature.com/articles/s41586-019-1666-5),他们声称自己开发了一种量子计算机,其速度比世界上最大的超级计算机还要快。在此处详细了解他们的实验。
Arute, F., Arya, K., Babbush, R. et al. Quantum supremacy using a programmable superconducting processor. Nature 574, 505–510 (2019) doi:10.1038/s41586-019-1666-5
Sandeep Subramanian, Raymond Li, Jonathan Pilault, Christopher J. Pal: OnExtractiveandAbstractiveNeuralDocumentSummarizationwithTransformerLanguageModels. CoRRabs/1909.03186 (2019)
Nelson Liu等人发表了一篇论文,讨论了预先训练的语境设定者(如BERT和ELMo)获取的语言知识的类型。
https://arxiv.org/abs/1903.08855
Nelson F. Liu, Matt Gardner, Yonatan Belinkov, Matthew E. Peters, Noah A. Smith: Linguistic Knowledge and Transferability of Contextual Representations. NAACL-HLT (1) 2019: 1073-1094
Zhilin Yang, Zihang Dai, Yiming Yang, Jaime G. Carbonell, Ruslan Salakhutdinov, Quoc V. Le: XLNet: Generalized Autoregressive Pretraining for Language Understanding. CoRR abs/1906.08237 (2019)
Dani Yogatama, Cyprien de Masson d'Autume, Jerome Connor, Tomás Kociský, Mike Chrzanowski, Lingpeng Kong, Angeliki Lazaridou, Wang Ling, Lei Yu, Chris Dyer, Phil Blunsom: Learning and Evaluating General Linguistic Intelligence. CoRR abs/1901.11373 (2019)
Liunian Harold Li, Mark Yatskar, Da Yin, Cho-Jui Hsieh, Kai-Wei Chang:VisualBERT: A Simple and Performant Baseline for Vision and Language. CoRR abs/1908.03557 (2019)
这项工作提供了一个详细的分析比较NLP转移学习方法和指导NLP的从业者。
https://arxiv.org/abs/1903.05987
Matthew E. Peters, Sebastian Ruder, Noah A. Smith: To Tune or Not to Tune? Adapting Pretrained Representations to Diverse Tasks.RepL4NLP@ACL 2019: 7-14