人工智能学术速递[8.30]

Update！H5支持摘要折叠，体验更佳！点击阅读原文访问arxivdaily.com，涵盖CS|物理|数学|经济|统计|金融|生物|电气领域，更有搜索、收藏等功能！

cs.AI人工智能，共计25篇

【1】 A Pedestrian Detection and Tracking Framework for Autonomous Cars: Efficient Fusion of Camera and LiDAR Data 标题：一种自动驾驶汽车行人检测与跟踪框架：摄像机与LiDAR数据的有效融合 链接：https://arxiv.org/abs/2108.12375

作者：Muhammad Mobaidul Islam,Abdullah Al Redwan Newaz,Ali Karimoddini 机构：Karimoddini are with theDepartment of Electrical and Computer Engineering, North Carolina A&TState University 摘要：提出了一种融合摄像机和激光雷达传感器数据的行人检测与跟踪新方法。为了应对与自主驾驶场景相关的挑战，提出了一个集成的跟踪和检测框架。检测阶段通过将激光雷达流转换为计算可处理的深度图像来执行，然后，开发深度神经网络来识别RGB和深度图像中的行人候选。为了提供准确的信息，通过使用卡尔曼滤波器融合多模态传感器信息，进一步增强了检测阶段。跟踪阶段是卡尔曼滤波预测和光流算法的组合，用于跟踪场景中的多个行人。我们在真实的公共驾驶数据集上评估我们的框架。实验结果表明，与仅使用基于图像的行人检测的基线方法相比，该方法实现了显著的性能改进。 摘要：This paper presents a novel method for pedestrian detection and tracking by fusing camera and LiDAR sensor data. To deal with the challenges associated with the autonomous driving scenarios, an integrated tracking and detection framework is proposed. The detection phase is performed by converting LiDAR streams to computationally tractable depth images, and then, a deep neural network is developed to identify pedestrian candidates both in RGB and depth images. To provide accurate information, the detection phase is further enhanced by fusing multi-modal sensor information using the Kalman filter. The tracking phase is a combination of the Kalman filter prediction and an optical flow algorithm to track multiple pedestrians in a scene. We evaluate our framework on a real public driving dataset. Experimental results demonstrate that the proposed method achieves significant performance improvement over a baseline method that solely uses image-based pedestrian detection.

【2】 DomiKnowS: A Library for Integration of Symbolic Domain Knowledge in Deep Learning 标题：DomiKnowS：深度学习中符号领域知识集成的库 链接：https://arxiv.org/abs/2108.12370

作者：Hossein Rajaby Faghihi,Quan Guo,Andrzej Uszok,Aliakbar Nafar,Elaheh Raisi,Parisa Kordjamshidi 机构：Michigan State University, Sichuan University, Florida Institute for Human and Machine Cognition 备注：Accepted at EMNLP 2021 demo track 摘要：我们演示了一个用于在深度学习体系结构中集成领域知识的库。使用该库，数据的结构通过图形声明以符号方式表示，输出或潜在变量的逻辑约束可以无缝地添加到深层模型中。领域知识可以明确定义，这不仅提高了模型在低数据区的性能和可推广性，还提高了模型的可解释性。介绍了符号和亚符号模型集成的几种方法；然而，在可以使用各种底层算法的情况下，没有库以通用方式促进此类集成的编程。我们的库旨在简化在训练和推理阶段进行集成的编程，同时将知识表示与学习算法分离。我们展示了各种NLP基准任务和其他任务。该框架在Github上公开提供(https://github.com/HLR/DomiKnowS). 摘要：We demonstrate a library for the integration of domain knowledge in deep learning architectures. Using this library, the structure of the data is expressed symbolically via graph declarations and the logical constraints over outputs or latent variables can be seamlessly added to the deep models. The domain knowledge can be defined explicitly, which improves the models' explainability in addition to the performance and generalizability in the low-data regime. Several approaches for such an integration of symbolic and sub-symbolic models have been introduced; however, there is no library to facilitate the programming for such an integration in a generic way while various underlying algorithms can be used. Our library aims to simplify programming for such an integration in both training and inference phases while separating the knowledge representation from learning algorithms. We showcase various NLP benchmark tasks and beyond. The framework is publicly available at Github(https://github.com/HLR/DomiKnowS).

【3】 Integrating Heuristics and Learning in a Computational Architecture for Cognitive Trading 标题：在认知交易的计算体系结构中集成启发式和学习 链接：https://arxiv.org/abs/2108.12333

作者：Remo Pareschi,Federico Zappone 机构：Stake Lab, University of Molise 备注：16 pages, with 5 figures; figure 5 groups 5 subfigures a, b, c, d. Currently under peer review for publication in volume to be published by Elgar on "AI and Behavioral Finance" 摘要：近年来，人工智能在图像分析、自然语言理解和战略游戏等领域的成功引起了金融界的兴趣。具体而言，对于人工代理（称为机器人交易者）的创建，人们有很高的期望和正在进行的工程项目，这些人工代理能够用经验丰富的人类交易者的技能操纵金融市场。撇开明显的经济影响不谈，这无疑是一个具有重大科学意义的领域，因为这样的真实环境对人工智能技术的使用构成了挑战。正是出于这个原因，我们必须意识到，能够在这样的水平上运行的人工智能体不仅指日可待，而且不会有简单的答案，而是各种技术和方法的共同作用才能使这项工作取得成功。在这篇文章的过程中，我们回顾了有效的机器人交易者设计中固有的问题以及相应的解决方案，考虑到将机器人交易的当前技术状态提升到下一个智能水平的总体目标，我们称之为认知交易。我们的方法的关键是将两个方法学和技术方向结合起来，尽管这两个方向都深深扎根于人工智能的学科领域，但迄今为止，它们已经走上了各自的道路：启发式和学习。 摘要：The successes of Artificial Intelligence in recent years in areas such as image analysis, natural language understanding and strategy games have sparked interest from the world of finance. Specifically, there are high expectations, and ongoing engineering projects, regarding the creation of artificial agents, known as robotic traders, capable of juggling the financial markets with the skill of experienced human traders. Obvious economic implications aside, this is certainly an area of great scientific interest, due to the challenges that such a real context poses to the use of AI techniques. Precisely for this reason, we must be aware that artificial agents capable of operating at such levels are not just round the corner, and that there will be no simple answers, but rather a concurrence of various technologies and methods to the success of the effort. In the course of this article, we review the issues inherent in the design of effective robotic traders as well as the consequently applicable solutions, having in view the general objective of bringing the current state of the art of robo-trading up to the next level of intelligence, which we refer to as Cognitive Trading. Key to our approach is the joining of two methodological and technological directions which, although both deeply rooted in the disciplinary field of artificial intelligence, have so far gone their separate ways: heuristics and learning.

【4】 SMT-Based Safety Verification of Data-Aware Processes under Ontologies (Extended Version) 标题：基于SMT的本体下数据感知过程的安全验证(扩展版) 链接：https://arxiv.org/abs/2108.12330

作者：Diego Calvanese,Alessandro Gianola,Andrea Mazzullo,Marco Montali 机构：KRDB Research Centre for Knowledge and Data, Free University of Bozen-Bolzano, Italy, Computing Science Department, Ume˚a University, Sweden 摘要：在验证数据感知过程（DAP）的背景下，考虑了一种基于可满足性模理论（SMT）的形式化方法来验证所谓以人为中心的系统的参数化安全特性。这种方法需要结合模型理论概念和基于后向可达性的算法技术。我们在这里介绍了这一领域中最受研究的模型之一的变体，即简单工件系统（SASs），其中，我们不管理数据库，而是在描述逻辑（DL）本体上进行操作，该本体以RDF（稍微扩展）表示。该DL具有合适的模型理论性质，允许我们定义基于DL的SASs，其后向可达性仍然可以应用，从而导致相应安全问题在PSPACE中的可判定性。 摘要：In the context of verification of data-aware processes (DAPs), a formal approach based on satisfiability modulo theories (SMT) has been considered to verify parameterised safety properties of so-called artifact-centric systems. This approach requires a combination of model-theoretic notions and algorithmic techniques based on backward reachability. We introduce here a variant of one of the most investigated models in this spectrum, namely simple artifact systems (SASs), where, instead of managing a database, we operate over a description logic (DL) ontology expressed in (a slight extension of) RDFS. This DL, enjoying suitable model-theoretic properties, allows us to define DL-based SASs to which backward reachability can still be applied, leading to decidability in PSPACE of the corresponding safety problems.

【5】 TE-YOLOF: Tiny and efficient YOLOF for blood cell detection 标题：TE-YOLOF：微小而高效的血细胞检测YOLOF 链接：https://arxiv.org/abs/2108.12313

作者：Fanxin Xu,Xiangkui Li,Hang Yang,Yali Wang,Wei Xiang 机构：College of Electronic and Information, Southwest Minzu University, West China Biomedical Big Data Center 摘要：显微图像中的血细胞检测是医学图像处理研究的一个重要分支。由于基于人工检查血细胞的疾病检测耗时且充满误差，因此使用具有深度卷积神经网络的目标检测器检测血细胞可以被视为一种可行的解决方案。在这项工作中，提出了一种基于YOLOF的目标检测器，用于检测红细胞、白细胞和血小板等血细胞目标。这种物体检测器称为TE-YOLOF，小巧高效，是一种利用扩展编码器从单级特征地图中提取信息的单级检测器。为了提高效率和灵活性，EfficientNet卷积神经网络被用作目标检测器的主干。此外，为了提高网络性能和最小化网络参数，采用了深度可分离卷积。此外，采用Mish激活函数来提高精度。在BCCD数据集上的大量实验证明了该模型的有效性，它比现有的其他血细胞检测研究更有效。 摘要：Blood cell detection in microscopic images is an essential branch of medical image processing research. Since disease detection based on manual checking of blood cells is time-consuming and full of errors, testing of blood cells using object detectors with Deep Convolutional Neural Network can be regarded as a feasible solution. In this work, an object detector based on YOLOF has been proposed to detect blood cell objects such as red blood cells, white blood cells and platelets. This object detector is called TE-YOLOF, Tiny and Efficient YOLOF, and it is a One-Stage detector using dilated encoder to extract information from single-level feature maps. For increasing efficiency and flexibility, the EfficientNet Convolutional Neural Network is utilized as the backbone for the proposed object detector. Furthermore, the Depthwise Separable Convolution is applied to enhance the performance and minimize the parameters of the network. In addition, the Mish activation function is employed to increase the precision. Extensive experiments on the BCCD dataset prove the effectiveness of the proposed model, which is more efficient than other existing studies for blood cell detection.

【6】 A Framework for Supervised Heterogeneous Transfer Learning using Dynamic Distribution Adaptation and Manifold Regularization 标题：基于动态分布自适应和流形正则化的有监督异构迁移学习框架 链接：https://arxiv.org/abs/2108.12293

作者：Md Geaur Rahman,Md Zahidul Islam 机构：School of Computing, Mathematics and Engineering, Charles Sturt University, Australia 备注：34 pages, 10 figures 摘要：迁移学习的目的是通过从源领域转移知识来学习目标领域的分类器。然而，由于两个主要问题：特征差异和分布差异，迁移学习在实践中可能是一个非常困难的问题。在本文中，我们提出了一个称为TLF的框架，该框架通过从具有多个标记记录的源域转移知识，为只有少量标记训练记录的目标域构建分类器。虽然现有的方法通常只关注一个问题，而将另一个问题留给下一步的工作，但TLF能够同时处理这两个问题。在TLF中，我们通过识别作为连接域的枢轴的共享标签分布来缓解特征差异。我们通过同时优化结构风险函数、域之间的联合分布和边际分布下的流形一致性来处理分布分歧。此外，对于流形一致性，我们通过识别记录的k个最近邻来利用其固有属性，其中k的值在TLF中自动确定。此外，由于不需要负迁移，我们只考虑在知识转移过程中属于源枢轴的源记录。我们在七个公开的自然数据集上评估TLF，并将TLF的性能与十一种最先进技术的性能进行比较。我们还评估了TLF在一些具有挑战性的情况下的有效性。我们的实验结果，包括统计符号检验和Nemenyi检验分析，表明所提出的框架明显优于最先进的技术。 摘要：Transfer learning aims to learn classifiers for a target domain by transferring knowledge from a source domain. However, due to two main issues: feature discrepancy and distribution divergence, transfer learning can be a very difficult problem in practice. In this paper, we present a framework called TLF that builds a classifier for the target domain having only few labeled training records by transferring knowledge from the source domain having many labeled records. While existing methods often focus on one issue and leave the other one for the further work, TLF is capable of handling both issues simultaneously. In TLF, we alleviate feature discrepancy by identifying shared label distributions that act as the pivots to bridge the domains. We handle distribution divergence by simultaneously optimizing the structural risk functional, joint distributions between domains, and the manifold consistency underlying marginal distributions. Moreover, for the manifold consistency we exploit its intrinsic properties by identifying k nearest neighbors of a record, where the value of k is determined automatically in TLF. Furthermore, since negative transfer is not desired, we consider only the source records that are belonging to the source pivots during the knowledge transfer. We evaluate TLF on seven publicly available natural datasets and compare the performance of TLF against the performance of eleven state-of-the-art techniques. We also evaluate the effectiveness of TLF in some challenging situations. Our experimental results, including statistical sign test and Nemenyi test analyses, indicate a clear superiority of the proposed framework over the state-of-the-art techniques.

【7】 Music Composition with Deep Learning: A Review 标题：基于深度学习的音乐创作述评 链接：https://arxiv.org/abs/2108.12290

作者：Carlos Hernandez-Olivan,Jose R. Beltran 机构：Department of Engineering and Communications, Calle María de Luna, Universidad de Zaragoza 摘要：创作复杂的艺术作品，如音乐作品，需要展现真正的创造力，这取决于与音乐语言层次相关的各种因素。音乐生成一直面临着算法方法的挑战，最近，深度学习模型正被用于计算机视觉等其他领域。在这篇论文中，我们想把基于人工智能的音乐作曲模型与人类音乐作曲和创作过程之间存在的关系放到上下文中去。我们概述了最近的音乐创作深度学习模型，并从理论角度将这些模型与音乐创作过程进行了比较。我们试图通过分析当前深度学习模型生成具有创造力的音乐的能力，或人工智能与人类作曲过程之间的相似性，来回答一些与此任务最相关的开放性问题。 摘要：Generating a complex work of art such as a musical composition requires exhibiting true creativity that depends on a variety of factors that are related to the hierarchy of musical language. Music generation have been faced with Algorithmic methods and recently, with Deep Learning models that are being used in other fields such as Computer Vision. In this paper we want to put into context the existing relationships between AI-based music composition models and human musical composition and creativity processes. We give an overview of the recent Deep Learning models for music composition and we compare these models to the music composition process from a theoretical point of view. We have tried to answer some of the most relevant open questions for this task by analyzing the ability of current Deep Learning models to generate music with creativity or the similarity between AI and human composition processes, among others.

【8】 The Devil is in the Detail: Simple Tricks Improve Systematic Generalization of Transformers 标题：魔鬼在细节中：简单的技巧改善了Transformer的系统化 链接：https://arxiv.org/abs/2108.12284

作者：Róbert Csordás,Kazuki Irie,Jürgen Schmidhuber 机构：The Swiss AI Lab IDSIA, USI & SUPSI, Lugano, Switzerland 备注：Accepted to EMNLP 2021 摘要：最近，人们提出了许多数据集来测试神经网络的系统泛化能力。伴随基线转换器通常使用标准任务中的默认超参数进行训练，结果显示会出现严重故障。在这里，我们证明了通过重新审视模型配置（如嵌入的缩放、早期停止、相对位置嵌入和通用变换器变体），我们可以在系统泛化方面显著提高变换器的性能。我们报告了五种流行数据集的改进：扫描、CFQ、PCFG、COGS和数学数据集。我们的模型将PCFG生产率分割的准确率从50%提高到85%，COGS的准确率从35%提高到81%。扫描时，相对位置嵌入在很大程度上缓解了EOS决策问题（Newman et al.，2020），在长度分割上产生100%的准确性，截止值为26。重要的是，这些模型之间的性能差异通常在IID数据分割上看不到。这就需要适当的泛化验证集来开发系统泛化的神经网络。我们公开发布代码以复制我们的结果。 摘要：Recently, many datasets have been proposed to test the systematic generalization ability of neural networks. The companion baseline Transformers, typically trained with default hyper-parameters from standard tasks, are shown to fail dramatically. Here we demonstrate that by revisiting model configurations as basic as scaling of embeddings, early stopping, relative positional embedding, and Universal Transformer variants, we can drastically improve the performance of Transformers on systematic generalization. We report improvements on five popular datasets: SCAN, CFQ, PCFG, COGS, and Mathematics dataset. Our models improve accuracy from 50% to 85% on the PCFG productivity split, and from 35% to 81% on COGS. On SCAN, relative positional embedding largely mitigates the EOS decision problem (Newman et al., 2020), yielding 100% accuracy on the length split with a cutoff at 26. Importantly, performance differences between these models are typically invisible on the IID data split. This calls for proper generalization validation sets for developing neural networks that generalize systematically. We publicly release the code to reproduce our results.

【9】 Lifelong Infinite Mixture Model Based on Knowledge-Driven Dirichlet Process 标题：基于知识驱动Dirichlet过程的终身无限混合模型 链接：https://arxiv.org/abs/2108.12278

作者：Fei Ye,Adrian G. Bors 机构：Department of Computer Science, University of York, York YO,GH, UK 备注：Accepted by International Conference on Computer Vision (ICCV 2021) 摘要：最近在终身学习方面的研究工作提出了一种混合模式，以适应越来越多的任务。所提出的方法在克服灾难性遗忘方面显示了良好的效果。然而，这些成功模式背后的理论仍然没有得到很好的理解。在本文中，我们通过基于模型生成的数据的概率表示与目标数据集对应的概率表示之间的差异距离来推导风险边界，从而对终身学习模型进行理论分析。受理论分析的启发，我们引入了一种新的终身学习方法，即终身无限混合（LIMix）模型，该模型可以自动扩展其网络结构或选择适当的组件来调整其参数以学习新任务，同时保留其先前学习的信息。我们建议通过Dirichlet过程，通过使用门控机制来合并知识，门控机制计算先前学习并存储在每个组件中的知识与新数据集之间的依赖关系。此外，我们还训练了一个紧凑的学生模型，该模型可以随着时间的推移积累跨域表示并进行快速推断。该守则可于https://github.com/dtuzi123/Lifelong-infinite-mixture-model. 摘要：Recent research efforts in lifelong learning propose to grow a mixture of models to adapt to an increasing number of tasks. The proposed methodology shows promising results in overcoming catastrophic forgetting. However, the theory behind these successful models is still not well understood. In this paper, we perform the theoretical analysis for lifelong learning models by deriving the risk bounds based on the discrepancy distance between the probabilistic representation of data generated by the model and that corresponding to the target dataset. Inspired by the theoretical analysis, we introduce a new lifelong learning approach, namely the Lifelong Infinite Mixture (LIMix) model, which can automatically expand its network architectures or choose an appropriate component to adapt its parameters for learning a new task, while preserving its previously learnt information. We propose to incorporate the knowledge by means of Dirichlet processes by using a gating mechanism which computes the dependence between the knowledge learnt previously and stored in each component, and a new set of data. Besides, we train a compact Student model which can accumulate cross-domain representations over time and make quick inferences. The code is available at https://github.com/dtuzi123/Lifelong-infinite-mixture-model.

【10】 End-To-End Anomaly Detection for Identifying Malicious Cyber Behavior through NLP-Based Log Embeddings 标题：基于NLP的日志嵌入识别恶意网络行为的端到端异常检测 链接：https://arxiv.org/abs/2108.12276

作者：Andrew Golczynski,John A. Emanuello 机构：Laboratory for Advanced Cybersecurity Research, National Security Agency 备注：Presented at 1st International Workshop on Adaptive Cyber Defense, 2021 (arXiv:2108.08476) 摘要：基于规则的入侵检测系统（入侵检测系统）正被更健壮的神经网络入侵检测系统所取代，在网络安全领域显示出巨大的潜力。然而，这些ML方法仍然依赖于特定的特征工程技术，缺乏以与发现异常网络活动完全相关的方式对输入进行矢量化的能力。我们提出了一个具有NLP启发的组件的深度端到端框架，用于识别企业计算机网络上的潜在恶意行为。我们还在最近发布的DARPA OpTC数据集上展示了该技术的有效性。 摘要：Rule-based IDS (intrusion detection systems) are being replaced by more robust neural IDS, which demonstrate great potential in the field of Cybersecurity. However, these ML approaches continue to rely on ad-hoc feature engineering techniques, which lack the capacity to vectorize inputs in ways that are fully relevant to the discovery of anomalous cyber activity. We propose a deep end-to-end framework with NLP-inspired components for identifying potentially malicious behaviors on enterprise computer networks. We also demonstrate the efficacy of this technique on the recently released DARPA OpTC data set.

【11】 Deep learning models are not robust against noise in clinical text 标题：深度学习模型对临床文本中的噪声不是很健壮 链接：https://arxiv.org/abs/2108.12242

作者：Milad Moradi,Kathrin Blagec,Matthias Samwald 机构：Institute for Artificial Intelligence and Decision Support, Center for Medical Statistics, Informatics, and, Intelligent Systems, Medical University of Vienna, Vienna, Austria 摘要：人工智能（AI）系统由于能够学习需要人类智能和专家知识的复杂任务，在医学领域吸引着越来越多的兴趣。利用高性能自然语言处理（NLP）模型的人工智能系统在各种临床文本处理基准上取得了最先进的结果。在某些任务上，它们甚至超过了人类的准确性。然而，此类人工智能系统的性能评估仅限于对经过策划和清洁的基准数据集的准确度测量，这些数据集可能无法正确反映这些系统在实际情况下的运行能力。为了应对这一挑战，我们引入并实现了各种各样的扰动方法，模拟临床文本数据中不同类型的噪声和可变性。虽然这些扰动方法产生的噪声样本通常可以被人类理解，但它们可能会导致人工智能系统做出错误的决策。在几个临床文本处理任务上进行了广泛的实验，我们评估了高性能NLP模型对各种类型的字符级和单词级噪声的鲁棒性。结果表明，当输入含有少量噪声时，NLP模型的性能下降。这项研究是暴露临床文本处理系统中使用的人工智能模型漏洞的重要一步。所提出的扰动方法可用于性能评估测试，以评估在真实环境中，临床NLP模型在噪声数据上的鲁棒性。 摘要：Artificial Intelligence (AI) systems are attracting increasing interest in the medical domain due to their ability to learn complicated tasks that require human intelligence and expert knowledge. AI systems that utilize high-performance Natural Language Processing (NLP) models have achieved state-of-the-art results on a wide variety of clinical text processing benchmarks. They have even outperformed human accuracy on some tasks. However, performance evaluation of such AI systems have been limited to accuracy measures on curated and clean benchmark datasets that may not properly reflect how robustly these systems can operate in real-world situations. In order to address this challenge, we introduce and implement a wide variety of perturbation methods that simulate different types of noise and variability in clinical text data. While noisy samples produced by these perturbation methods can often be understood by humans, they may cause AI systems to make erroneous decisions. Conducting extensive experiments on several clinical text processing tasks, we evaluated the robustness of high-performance NLP models against various types of character-level and word-level noise. The results revealed that the NLP models performance degrades when the input contains small amounts of noise. This study is a significant step towards exposing vulnerabilities of AI models utilized in clinical text processing systems. The proposed perturbation methods can be used in performance evaluation tests to assess how robustly clinical NLP models can operate on noisy data, in real-world settings.

【12】 Geometric Models for (Temporally) Attributed Description Logics 标题：(时间)属性描述逻辑的几何模型 链接：https://arxiv.org/abs/2108.12239

作者：Camille Bourgaux,Ana Ozaki,Jeff Z. Pan 机构：DI ENS, ENS, CNRS, PSL University & Inria, Paris, France, University of Bergen, Norway, University of Edinburgh, United Kingdom 备注：Long version of a DL 2021 paper. 24 pages 摘要：为了寻找能够获取本体知识的知识图嵌入，最近引入了存在规则的几何模型。已经证明，凸几何区域捕获所谓的准链式规则。属性描述逻辑（DL）的定义是为了弥合DL语言和知识图之间的鸿沟，知识图的事实往往带有各种注释，在推理时可能需要考虑这些注释。特别是，时间属性的DLs由特定属性丰富，这些属性的语义允许一些时间推理。考虑到几何模型和（暂时）属性化DLs是为知识图设计的有前途的工具，本文研究了它们的兼容性，重点是DL-Lite族Horn方言的属性化版本。首先，我们将几何模型的定义应用于属性DLs，并证明每个可满足本体都有一个凸几何模型。我们的第二个贡献是研究时间属性的影响。我们证明了时间属性的DL通常可能没有凸几何模型，但我们可以通过对时间属性的使用施加一些限制来恢复几何可满足性。 摘要：In the search for knowledge graph embeddings that could capture ontological knowledge, geometric models of existential rules have been recently introduced. It has been shown that convex geometric regions capture the so-called quasi-chained rules. Attributed description logics (DL) have been defined to bridge the gap between DL languages and knowledge graphs, whose facts often come with various kinds of annotations that may need to be taken into account for reasoning. In particular, temporally attributed DLs are enriched by specific attributes whose semantics allows for some temporal reasoning. Considering that geometric models and (temporally) attributed DLs are promising tools designed for knowledge graphs, this paper investigates their compatibility, focusing on the attributed version of a Horn dialect of the DL-Lite family. We first adapt the definition of geometric models to attributed DLs and show that every satisfiable ontology has a convex geometric model. Our second contribution is a study of the impact of temporal attributes. We show that a temporally attributed DL may not have a convex geometric model in general but we can recover geometric satisfiability by imposing some restrictions on the use of the temporal attributes.

【13】 Evaluating the Robustness of Neural Language Models to Input Perturbations 标题：评估神经语言模型对输入扰动的鲁棒性 链接：https://arxiv.org/abs/2108.12237

作者：Milad Moradi,Matthias Samwald 机构：Institute for Artificial Intelligence, Medical University of Vienna, Austria 备注：Accepted by EMNLP 2021 摘要：高性能神经语言模型已经在广泛的自然语言处理（NLP）任务中获得了最新的结果。然而，当应用于嘈杂的真实数据时，通用基准数据集的结果通常不能反映模型的可靠性和鲁棒性。在这项研究中，我们设计并实现了各种类型的字符级和单词级扰动方法，以模拟输入文本可能有轻微噪声或与NLP系统训练的数据分布不同的真实场景。通过对不同NLP任务的综合实验，我们研究了高性能语言模型（如BERT、XLNet、RoBERTa和ELMo）处理不同类型输入扰动的能力。结果表明，语言模型对输入扰动非常敏感，即使引入微小的变化，其性能也会下降。我们强调模型需要进一步改进，当前的基准没有很好地反映模型的稳健性。我们认为，对扰动输入的评估应该常规地补充广泛使用的基准，以便更现实地理解NLP系统的鲁棒性。 摘要：High-performance neural language models have obtained state-of-the-art results on a wide range of Natural Language Processing (NLP) tasks. However, results for common benchmark datasets often do not reflect model reliability and robustness when applied to noisy, real-world data. In this study, we design and implement various types of character-level and word-level perturbation methods to simulate realistic scenarios in which input texts may be slightly noisy or different from the data distribution on which NLP systems were trained. Conducting comprehensive experiments on different NLP tasks, we investigate the ability of high-performance language models such as BERT, XLNet, RoBERTa, and ELMo in handling different types of input perturbations. The results suggest that language models are sensitive to input perturbations and their performance can decrease even when small changes are introduced. We highlight that models need to be further improved and that current benchmarks are not reflecting model robustness well. We argue that evaluations on perturbed inputs should routinely complement widely-used benchmarks in order to yield a more realistic understanding of NLP systems robustness.

【14】 GLocal-K: Global and Local Kernels for Recommender Systems 标题：GLOCAL-K：面向推荐系统的全局和局部内核 链接：https://arxiv.org/abs/2108.12184

作者：Soyeon Caren Han,Taejun Lim,Siqu Long,Bernd Burgstaller,Josiah Poon 机构：The University of Sydney, Australia, Yonsei University, Republic of Korea 备注：Accepted by CIKM 2021 摘要：推荐系统通常在高维稀疏用户项矩阵上运行。矩阵完成是一项非常具有挑战性的任务，要根据数百万其他用户看到的数千个项目的一小部分来预测一个人的兴趣。我们提出了一个基于全局局部核的矩阵完成框架GLocal-K，该框架旨在将高维稀疏用户项矩阵项推广并表示到低维空间中，其中包含少量重要特征。我们的GLocal-K可分为两个主要阶段。首先，我们使用局部核化权重矩阵预训练一个自动编码器，该编码器使用2d RBF核将数据从一个空间转换到特征空间。然后，使用基于卷积的全局核生成的评分矩阵对预先训练的自动编码器进行微调，该全局核捕获每个项目的特征。我们在极低资源设置下应用GLocal-K模型，该设置仅包括一个用户项目评级矩阵，没有附带信息。我们的模型在三个协作过滤基准上的性能优于最先进的基线：ML-100K、ML-1M和豆瓣。 摘要：Recommender systems typically operate on high-dimensional sparse user-item matrices. Matrix completion is a very challenging task to predict one's interest based on millions of other users having each seen a small subset of thousands of items. We propose a Global-Local Kernel-based matrix completion framework, named GLocal-K, that aims to generalise and represent a high-dimensional sparse user-item matrix entry into a low dimensional space with a small number of important features. Our GLocal-K can be divided into two major stages. First, we pre-train an auto encoder with the local kernelised weight matrix, which transforms the data from one space into the feature space by using a 2d-RBF kernel. Then, the pre-trained auto encoder is fine-tuned with the rating matrix, produced by a convolution-based global kernel, which captures the characteristics of each item. We apply our GLocal-K model under the extreme low-resource setting, which includes only a user-item rating matrix, with no side information. Our model outperforms the state-of-the-art baselines on three collaborative filtering benchmarks: ML-100K, ML-1M, and Douban.

【15】 Cleaning Inconsistent Data in Temporal DL-Lite Under Best Repair Semantics 标题：最佳修复语义下时态DL-Lite中不一致数据的清理 链接：https://arxiv.org/abs/2108.12149

作者：Mourad Ouziri,Sabiha Tahrat,Salima Benbernou,Mourad Ouzirri 机构：LIPADE, Université de Paris, France 摘要：本文讨论了时态描述逻辑（TDL）知识库中不一致数据的处理问题。考虑到知识库的数据部分是不一致性的来源，我们提出了一种ABox修复方法。这是在TDL知识库中处理修复的第一项工作。为此，我们的目标有两个：1）检测时间不一致性，2）提出数据时间修复。对于不一致性检测，我们提出了一种从TDL到DL的简化方法，该方法允许为TDL概念的可满足性提供一个紧NP完全上界，并使用高度优化的DL推理器，该推理器可以带来精确的解释（不一致数据断言集）。然后，根据得到的解释，我们提出了一种基于允许的刚性谓词和断言的时间顺序自动计算时间设置中最佳修复的方法。 摘要：In this paper, we address the problem of handling inconsistent data in Temporal Description Logic (TDL) knowledge bases. Considering the data part of the Knowledge Base as the source of inconsistency over time, we propose an ABox repair approach. This is the first work handling the repair in TDL Knowledge bases. To do so, our goal is twofold: 1) detect temporal inconsistencies and 2) propose a data temporal reparation. For the inconsistency detection, we propose a reduction approach from TDL to DL which allows to provide a tight NP-complete upper bound for TDL concept satisfiability and to use highly optimised DL reasoners that can bring precise explanation (the set of inconsistent data assertions). Thereafter, from the obtained explanation, we propose a method for automatically computing the best repair in the temporal setting based on the allowed rigid predicates and the time order of assertions.

【16】 Lyra: A Benchmark for Turducken-Style Code Generation 标题：Lyra：Turducken风格代码生成的基准 链接：https://arxiv.org/abs/2108.12144

作者：Qingyuan Liang,Zeyu Sun,Qihao Zhu,Wenjie Zhang,Lian Yu,Yingfei Xiong,Lu Zhang 机构：School of Software & Microelectronics, Peking University, P. R. China, Key Laboratory of High Confidence Software Technologies (Peking University), MoE;, Software Institute, Peking University, P. R. China 备注：9 pages, 4 figures 摘要：代码生成对于减少手动软件开发工作至关重要。最近，神经技术被用来自动生成源代码。虽然这些方法很有前途，但它们是在用单一编程语言生成代码的任务上进行评估的。然而，在实际开发中，一种编程语言常常嵌入到另一种编程语言中。例如，SQL语句通常作为字符串嵌入在基本编程语言（如Python和Java）中，JavaScript程序通常嵌入在服务器端编程语言（如PHP、Java和Python）中。我们称之为turducken风格的编程。在本文中，我们定义了一个新的代码生成任务：给定一个自然语言注释，该任务旨在生成一个带有嵌入式语言的基础语言程序。据我们所知，这是第一个turducken风格的代码生成任务。对于这项任务，我们将介绍Lyra：一个Python中的数据集，其中包含嵌入式SQL。该数据集包含来自实际使用项目的2000个仔细注释的数据库操作程序。每个节目都配有中文和英文评论。在我们的实验中，我们采用了最先进的技术Transformer作为基线。在最佳设置下，Transformer使用中文和英文注释分别达到0.5%和1.5%的AST精确匹配精度。因此，我们认为Lyra为代码生成提供了新的挑战。 摘要：Code generation is crucial to reduce manual software development efforts. Recently, neural techniques have been used to generate source code automatically. While promising, these approaches are evaluated on tasks for generating code in single programming languages. However, in actual development, one programming language is often embedded in another. For example, SQL statements are often embedded as strings in base programming languages such as Python and Java, and JavaScript programs are often embedded in sever-side programming languages, such as PHP, Java, and Python. We call this a turducken-style programming. In this paper, we define a new code generation task: given a natural language comment, this task aims to generate a program in a base language with an embedded language. To our knowledge, this is the first turducken-style code generation task. For this task, we present Lyra: a dataset in Python with embedded SQL. This dataset contains 2,000 carefully annotated database manipulation programs from real usage projects. Each program is paired with both a Chinese comment and an English comment. In our experiment, we adopted Transformer, a state-of-the-art technique, as the baseline. In the best setting, Transformer achieves 0.5% and 1.5% AST exact matching accuracy using Chinese and English comments, respectively. Therefore, we believe that Lyra provides a new challenge for code generation.

【17】 WAD: A Deep Reinforcement Learning Agent for Urban Autonomous Driving 标题：WAD：一种面向城市自动驾驶的深度强化学习Agent 链接：https://arxiv.org/abs/2108.12134

作者：Arjit Sharma,Sahil Sharma 机构：Liverpool John Moores University, Liverpool, United Kingdom, Thapar Institute of Engineering and Technology, Patiala, India 备注：10 pages, 8 figures, and 4 tables 摘要：城市自主驾驶是一个开放且具有挑战性的问题，因为决策系统必须考虑多个动态因素，如多智能体交互、不同的场景感知、复杂的道路几何以及其他很少发生的真实世界事件。另一方面，通过深度强化学习（DRL）技术，代理学习了许多复杂的策略。他们甚至在各种Atari游戏和Deepmind的AlphaGo中取得了超人水平的表演。然而，目前的DRL技术不能很好地推广到复杂的城市驾驶场景中。本文介绍了用于端到端城市自主驾驶的DRL驱动的手表和驾驶（WAD）代理。受最新进展的推动，该研究旨在检测卡拉高维空间中的重要对象/状态，并从中提取潜在状态。进一步，基于TD3和SAC方法将潜在状态信息传递给WAD代理，以学习最优驾驶策略。我们利用较少资源的新方法、对不同驾驶任务的逐步学习、硬插曲终止策略和奖励机制，使我们的代理在最初的CARLA基准测试中所有驾驶任务的成功率达到100%，并在进一步复杂的NoCrash基准测试中创下82%的新记录，在NoCrash基准上超过最先进模型30%以上。 摘要：Urban autonomous driving is an open and challenging problem to solve as the decision-making system has to account for several dynamic factors like multi-agent interactions, diverse scene perceptions, complex road geometries, and other rarely occurring real-world events. On the other side, with deep reinforcement learning (DRL) techniques, agents have learned many complex policies. They have even achieved super-human-level performances in various Atari Games and Deepmind's AlphaGo. However, current DRL techniques do not generalize well on complex urban driving scenarios. This paper introduces the DRL driven Watch and Drive (WAD) agent for end-to-end urban autonomous driving. Motivated by recent advancements, the study aims to detect important objects/states in high dimensional spaces of CARLA and extract the latent state from them. Further, passing on the latent state information to WAD agents based on TD3 and SAC methods to learn the optimal driving policy. Our novel approach utilizing fewer resources, step-by-step learning of different driving tasks, hard episode termination policy, and reward mechanism has led our agents to achieve a 100% success rate on all driving tasks in the original CARLA benchmark and set a new record of 82% on further complex NoCrash benchmark, outperforming the state-of-the-art model by more than +30% on NoCrash benchmark.

【18】 Task-aware Warping Factors in Mask-based Speech Enhancement 标题：基于掩码的语音增强中的任务感知翘曲因子 链接：https://arxiv.org/abs/2108.12128

作者：Qiongqiong Wang,Kong Aik Lee,Takafumi Koshinaka,Koji Okabe,Hitoshi Yamamoto 机构：Biometrics Research Laboratories, NEC Corporation, Japan 备注：EUSIPCO 2021 (the 29th European Signal Processing Conference) 摘要：本文提出在基于掩模的语音增强（SE）中使用两种任务感知扭曲因子。一个控制训练阶段语音维持和噪声消除之间的平衡，另一个控制测试阶段特定下游任务的SE功率。我们的目的是缓解这样一个问题，即经过训练以提高语音质量的SE系统通常无法改善其他下游任务，例如自动说话人验证（ASV）和自动语音识别（ASR），因为它们不共享相同的对象。将所提出的双翘曲因子方法应用于任何基于掩码的SE方法都很容易，并且它允许单个SE系统处理多个任务，而无需依赖于任务的训练。我们提出的方法的有效性已在SITW数据集上得到证实，该数据集用于ASV评估，LibriSpeech数据集用于ASR和0-20dB的语音质量评估。我们表明，不同的翘曲值对于单个SE来说是必要的，以实现三个任务的最佳性能w.r.t。通过使用任务相关的扭曲因子，在0dB语音上，语音质量提高了84.7%的PESQ，ASV的EER降低了22.4%，ASR的WER降低了52.2%。任务相关翘曲因子的有效性也在ASV的VoxCeleb-1测试集和ASV和质量评估的LibriSpeech-dev清洁集上进行了交叉验证。该方法效率高，易于实际应用。 摘要：This paper proposes the use of two task-aware warping factors in mask-based speech enhancement (SE). One controls the balance between speech-maintenance and noise-removal in training phases, while the other controls SE power applied to specific downstream tasks in testing phases. Our intention is to alleviate the problem that SE systems trained to improve speech quality often fail to improve other downstream tasks, such as automatic speaker verification (ASV) and automatic speech recognition (ASR), because they do not share the same objects. It is easy to apply the proposed dual-warping factors approach to any mask-based SE method, and it allows a single SE system to handle multiple tasks without task-dependent training. The effectiveness of our proposed approach has been confirmed on the SITW dataset for ASV evaluation and the LibriSpeech dataset for ASR and speech quality evaluations of 0-20dB. We show that different warping values are necessary for a single SE to achieve optimal performance w.r.t. the three tasks. With the use of task-dependent warping factors, speech quality was improved by an 84.7% PESQ increase, ASV had a 22.4% EER reduction, and ASR had a 52.2% WER reduction, on 0dB speech. The effectiveness of the task-dependent warping factors were also cross-validated on VoxCeleb-1 test set for ASV and LibriSpeech dev-clean set for ASV and quality evaluations. The proposed method is highly effective and easy to apply in practice.

【19】 Learning to Give Checkable Answers with Prover-Verifier Games 标题：用验证者-验证者博弈学习给出可检查的答案 链接：https://arxiv.org/abs/2108.12099

作者：Cem Anil,Guodong Zhang,Yuhuai Wu,Roger Grosse 摘要：我们知道何时信任机器学习系统做出的决策的能力没有跟上其性能的惊人提高，限制了其在高风险领域的适用性。我们引入了Prover-Verifier博弈（PVGs），这是一个博弈论框架，鼓励学习代理以可验证的方式解决决策问题。PVG由两个相互竞争的学习者组成：一个可信的验证者网络试图选择正确的答案，另一个更强大但不可信的验证者网络试图说服验证者接受特定的答案，而不管其正确性如何。目标是从这个游戏中产生一个可靠的证明协议。我们分析了框架的各种变体，包括同时博弈和连续博弈，并将空间缩小到可证明具有期望均衡的博弈子集。我们为两个算法任务开发了PVG的实例，并表明在实践中，验证者学习了一个健壮的决策规则，该规则能够从不可信的验证者那里接收有用和可靠的信息。重要的是，即使验证者被冻结，并且验证者的消息被直接优化以说服验证者，该协议仍然有效。 摘要：Our ability to know when to trust the decisions made by machine learning systems has not kept up with the staggering improvements in their performance, limiting their applicability in high-stakes domains. We introduce Prover-Verifier Games (PVGs), a game-theoretic framework to encourage learning agents to solve decision problems in a verifiable manner. The PVG consists of two learners with competing objectives: a trusted verifier network tries to choose the correct answer, and a more powerful but untrusted prover network attempts to persuade the verifier of a particular answer, regardless of its correctness. The goal is for a reliable justification protocol to emerge from this game. We analyze variants of the framework, including simultaneous and sequential games, and narrow the space down to a subset of games which provably have the desired equilibria. We develop instantiations of the PVG for two algorithmic tasks, and show that in practice, the verifier learns a robust decision rule that is able to receive useful and reliable information from an untrusted prover. Importantly, the protocol still works even when the verifier is frozen and the prover's messages are directly optimized to convince the verifier.

【20】 Harms of Gender Exclusivity and Challenges in Non-Binary Representation in Language Technologies 标题：语言技术中性别排他性的危害和非二进制表示的挑战 链接：https://arxiv.org/abs/2108.12084

作者：Sunipa Dev,Masoud Monajatipoor,Anaelia Ovalle,Arjun Subramonian,Jeff M Phillips,Kai-Wei Chang 机构：sheher, UCLA, hehim, theyheshe, theythem, University of Utah 备注：None 摘要：性别问题在语言任务和研究语言模型传播的陈规定型观念时得到广泛讨论。然而，目前的讨论主要将性别视为二元性，这可能会造成伤害，如非二元性身份的周期性抹杀。这些危害是由模型和数据集偏见造成的，这些偏见是社会中对非二元性别不认识和缺乏理解的后果。在本文中，我们解释了性别和语言的复杂性，并调查了非二元性的人，以了解在英语语言技术中将性别视为二元性所带来的危害。我们还详细介绍了当前的语言表征（如手套、BERT）是如何捕捉和延续这些危害和相关挑战的，这些危害和挑战需要得到承认和解决，以便表征公平地编码性别信息。 摘要：Gender is widely discussed in the context of language tasks and when examining the stereotypes propagated by language models. However, current discussions primarily treat gender as binary, which can perpetuate harms such as the cyclical erasure of non-binary gender identities. These harms are driven by model and dataset biases, which are consequences of the non-recognition and lack of understanding of non-binary genders in society. In this paper, we explain the complexity of gender and language around it, and survey non-binary persons to understand harms associated with the treatment of gender as binary in English language technologies. We also detail how current language representations (e.g., GloVe, BERT) capture and perpetuate these harms and related challenges that need to be acknowledged and addressed for representations to equitably encode gender information.

【21】 Continual learning under domain transfer with sparse synaptic bursting 标题：稀疏突触爆发域转移下的连续学习 链接：https://arxiv.org/abs/2108.12056

作者：Shawn L. Beaulieu,Jeff Clune,Nick Cheney 机构：Systems Center; fDepartment of Computer Science, University of British Columbia: Vancouver, BC, Canada 摘要：现有的机器是功能特定的工具，易于预测和控制。未来的机器在易变性、弹性和自主性方面可能更接近生物系统。但首先，他们必须能够学习和保留新信息，而不必反复接触。过去设计这类系统的努力都是在应用环境受限的情况下，利用特定于任务的模块来构建或调节人工神经网络。这还不能在不破坏现有知识的情况下对以前看不见的长序列数据进行持续学习：这是一个被称为灾难性遗忘的问题。在本文中，我们介绍了一个系统，该系统可以在以前看不到的数据集（ImageNet，CIFAR-100）上顺序学习，并且随着时间的推移几乎不会忘记。这是通过使用由第二个前馈神经网络生成的自上而下调制，在输入的基础上调节卷积神经网络中权重的活动来实现的。我们发现，我们的方法在域转移下不断学习，在任务之间循环的权重中有稀疏的活动突发，而不是通过维护特定于任务的模块。研究发现，稀疏的突触爆破可以平衡活动的增强和减弱，从而有助于适应新的输入，而不会破坏先前获得的功能。这种行为出现在先前的元学习阶段，在此阶段，受调节的突触从一致抑制的初始状态选择性地去抑制或生长。 摘要：Existing machines are functionally specific tools that were made for easy prediction and control. Tomorrow's machines may be closer to biological systems in their mutability, resilience, and autonomy. But first they must be capable of learning, and retaining, new information without repeated exposure to it. Past efforts to engineer such systems have sought to build or regulate artificial neural networks using task-specific modules with constrained circumstances of application. This has not yet enabled continual learning over long sequences of previously unseen data without corrupting existing knowledge: a problem known as catastrophic forgetting. In this paper, we introduce a system that can learn sequentially over previously unseen datasets (ImageNet, CIFAR-100) with little forgetting over time. This is accomplished by regulating the activity of weights in a convolutional neural network on the basis of inputs using top-down modulation generated by a second feed-forward neural network. We find that our method learns continually under domain transfer with sparse bursts of activity in weights that are recycled across tasks, rather than by maintaining task-specific modules. Sparse synaptic bursting is found to balance enhanced and diminished activity in a way that facilitates adaptation to new inputs without corrupting previously acquired functions. This behavior emerges during a prior meta-learning phase in which regulated synapses are selectively disinhibited, or grown, from an initial state of uniform suppression.

【22】 Semantic-based Self-Critical Training For Question Generation 标题：基于语义的自我批判性问题生成训练 链接：https://arxiv.org/abs/2108.12026

作者：Loïc,Kwate Dassi 机构：Ensimag, Grenoble, France 备注：5 pages, 1 figure 摘要：在这项工作中，我们提出了一个完全基于Transformer的强化学习生成器评估器架构，用于神经问题生成。问题生成是在给定上下文和答案的情况下生成问题的任务。为了提高生成问题的质量，我们在生成器evaluator体系结构中提出了一种基于语义的自关键训练布局，它超越了典型的最大似然训练。仅基于N-gram重叠的语言建模的评价度量不考虑引用字符串和候选字符串之间的语义关系。为了改进评估步骤，我们使用BLEU评估了我们的模型中的n-gram重叠，并在语义上使用BERTScore和NUBIA，这是一种用于文本生成的最新评估指标。问题生成可用于许多下游应用程序，包括扩展问答数据集、对话系统和教育评估系统。 摘要：We present in this work a fully Transformer-based reinforcement learning generator-evaluator architecture for neural question generation. Question generation is a task that consists in generating questions given a context and answer. To improve the quality of the generated question, we came up with a semantic-based self-critical training layout in generator-evaluator architecture, which goes beyond typical maximum likelihood training. Evaluation metrics for language modeling only based on n-gram overlapping do not consider semantic relations between reference and candidate strings. To improve the evaluation step, we assess our model for both n-gram overlap using BLEU and semantically using BERTScore and NUBIA, a novel state-of-the-art evaluation metric for text generation. Question generation could be used in many downstream applications, including in extending question answering datasets, conversational systems, and educational assessment systems.

【23】 Understanding the Logit Distributions of Adversarially-Trained Deep Neural Networks 标题：理解对抗性训练的深度神经网络的Logit分布 链接：https://arxiv.org/abs/2108.12001

作者：Landan Seguin,Anthony Ndirango,Neeli Mishra,SueYeon Chung,Tyler Lee 机构：Intel Labs, Columbia University 备注：29 pages (13 main, 16 supplemental), 22 figures (5 main, 17 supplemental) 摘要：对抗性防御训练深层神经网络对来自对抗性攻击的输入扰动保持不变。几乎所有的防御策略都是通过对抗性训练来实现这种不变性的，即在对抗性干扰下对输入进行训练。尽管对抗性训练在缓解对抗性攻击方面取得了成功，但对抗性训练（at）模型和标准模型之间的行为差异仍不清楚。受最近通过提取AT模型对无输入扰动的学习鲁棒性进行研究的启发，我们通过分析AT模型中Logit的分布来探索对抗性训练中学习的内容。我们确定了学习对抗鲁棒性所必需的三个logit特征。首先，我们为以下发现提供了理论依据：对抗性训练缩小了logit分布的两个重要特征：AT模型的最大logit值和“logit差距”（logit最大值和下一个最大值之间的差异）平均较低。其次，我们证明了AT和标准模型在高置信度和低置信度样本上存在显著差异，然后通过可视化置信度差异最大的样本来说明明显的质量差异。最后，我们发现关于错误类的学习信息对于学习稳健性至关重要，方法是在蒸馏过程中操纵非最大logit信息并测量对学生稳健性的影响。我们的结果表明，在没有输入扰动的情况下学习一些对抗性稳健性需要一个模型来学习特定的样本信心和遵循复杂分布的错误类顺序。 摘要：Adversarial defenses train deep neural networks to be invariant to the input perturbations from adversarial attacks. Almost all defense strategies achieve this invariance through adversarial training i.e. training on inputs with adversarial perturbations. Although adversarial training is successful at mitigating adversarial attacks, the behavioral differences between adversarially-trained (AT) models and standard models are still poorly understood. Motivated by a recent study on learning robustness without input perturbations by distilling an AT model, we explore what is learned during adversarial training by analyzing the distribution of logits in AT models. We identify three logit characteristics essential to learning adversarial robustness. First, we provide a theoretical justification for the finding that adversarial training shrinks two important characteristics of the logit distribution: the max logit values and the "logit gaps" (difference between the logit max and next largest values) are on average lower for AT models. Second, we show that AT and standard models differ significantly on which samples are high or low confidence, then illustrate clear qualitative differences by visualizing samples with the largest confidence difference. Finally, we find learning information about incorrect classes to be essential to learning robustness by manipulating the non-max logit information during distillation and measuring the impact on the student's robustness. Our results indicate that learning some adversarial robustness without input perturbations requires a model to learn specific sample-wise confidences and incorrect class orderings that follow complex distributions.

【24】 A New Sentence Ordering Method Using BERT Pretrained Model 标题：一种基于BERT预训练模型的句子排序新方法 链接：https://arxiv.org/abs/2108.11994

作者：Melika Golestani,Seyedeh Zahra Razavi,Heshaam Faili 机构：department of ECE, University of Tehran, Tehran, Iran, department of Computer Science, University of Rochester, Rochester, USA 备注：7 pages, 4 figures, 2020 11th International Conference on Information and Knowledge Technology (IKT) 摘要：构建具有自然语言理解能力（NLU）的系统是人工智能最古老的领域之一。NLU的一个重要组成部分是检测文本中包含的事件的逻辑顺序。提出了句子排序任务来学习事件的连续性，并将其应用于人工智能任务中。以往采用统计方法的工作性能较差，而基于神经网络的方法迫切需要用于模型学习的大型语料库。在本文中，我们提出了一种句子排序的方法，它不需要训练阶段，也不需要大量的学习语料。为此，我们使用BERT预训练模型生成句子嵌入，并使用余弦相似度得分度量句子相似度。我们建议将该分数作为连续事件连贯性水平的指标。最后，我们通过暴力搜索对句子进行排序，以最大限度地提高顺序句子的整体相似性。我们提出的方法在ROCStories上的表现优于其他基线，ROCStories是一个由5句话的人造故事组成的语料库。特别是在没有大型语料库的情况下，该方法比基于神经网络的方法更有效。这种方法的其他优点之一是它的可解释性和不需要语言知识。 摘要：Building systems with capability of natural language understanding (NLU) has been one of the oldest areas of AI. An essential component of NLU is to detect logical succession of events contained in a text. The task of sentence ordering is proposed to learn succession of events with applications in AI tasks. The performance of previous works employing statistical methods is poor, while the neural networks-based approaches are in serious need of large corpora for model learning. In this paper, we propose a method for sentence ordering which does not need a training phase and consequently a large corpus for learning. To this end, we generate sentence embedding using BERT pre-trained model and measure sentence similarity using cosine similarity score. We suggest this score as an indicator of sequential events' level of coherence. We finally sort the sentences through brute-force search to maximize overall similarities of the sequenced sentences. Our proposed method outperformed other baselines on ROCStories, a corpus of 5-sentence human-made stories. The method is specifically more efficient than neural network-based methods when no huge corpus is available. Among other advantages of this method are its interpretability and needlessness to linguistic knowledge.

【25】 Cascading Neural Network Methodology for Artificial Intelligence-Assisted Radiographic Detection and Classification of Lead-Less Implanted Electronic Devices within the Chest 标题：级联神经网络胸腔内无铅植入式电子设备的人工智能射线检测与分类 链接：https://arxiv.org/abs/2108.11954

作者：Mutlu Demirer,Richard D. White,Vikash Gupta,Ronnie A. Sebro,Barbaros S. Erdal 机构：Barbaros Selnur Erdal, DDS, MS, PhD, Center for Augmented Intelligence in Imaging-Department of Radiology, Jacksonville, FL, Corresponding Author:, Technical Director - Center for Augmented Intelligence in Imaging, San Pablo Road, Jacksonville FL , Office: ,-,- 备注：22 pages, 3 figures 摘要：背景与目的：胸部X射线（CXR）用于无铅植入电子设备（LLIED）的MRI前安全筛查，在正面视图上容易被忽视或错误识别（通常仅获得）是常见的。尽管大多数LLIED类型是“MRI条件的”：1.一些是严格条件的；2.不同的条件类型有特定的患者或设备管理要求；3.特定类型为“MRI不安全”。这项工作的重点是开发CXR解释辅助人工智能（AI）方法：1.100%检测LLIED的存在/位置；2.LLIED分型分类高。材料与方法：数据挖掘（1993年3月-2021年2月）产生了人工智能模型开发人群（1100名患者/4871张图像），创建了4924个用于训练、验证和测试的感兴趣区域（ROI）（具有图像质量分级）。为了开发级联神经网络（通过更快的R-CNN进行检测，通过Inception V3进行分类），“地面真相”CXR注释（根据LLIED进行ROI标记）以及推理显示（生成的边界框（GBB）），依赖于基于GPU的图形用户界面。结果：为了实现100%的LLIED检测，模型1需要将概率阈值降低到0.00002，从而增加每个LLIED相关ROI的GBBs。针对所有LLIED检测后的LLIED类型分类，模型2多分类以达到高性能，同时减少误报GBB。尽管有24%的ROI图像质量不理想，但分类正确率为98.9%，9种LLIED类型的AUC分别为1.00（8）和0.92（1）。对于所有错误分类案例：1.无一例涉及严格条件或不安全的LLIED；大多数是由于图像不理想造成的。结论：本项目成功开发了一种与LLIED相关的AI方法，支持：1.100%检测；2.典型的100%类型分类。 摘要：Background & Purpose: Chest X-Ray (CXR) use in pre-MRI safety screening for Lead-Less Implanted Electronic Devices (LLIEDs), easily overlooked or misidentified on a frontal view (often only acquired), is common. Although most LLIED types are "MRI conditional": 1. Some are stringently conditional; 2. Different conditional types have specific patient- or device- management requirements; and 3. Particular types are "MRI unsafe". This work focused on developing CXR interpretation-assisting Artificial Intelligence (AI) methodology with: 1. 100% detection for LLIED presence/location; and 2. High classification in LLIED typing. Materials & Methods: Data-mining (03/1993-02/2021) produced an AI Model Development Population (1,100 patients/4,871 images) creating 4,924 LLIED Region-Of-Interests (ROIs) (with image-quality grading) used in Training, Validation, and Testing. For developing the cascading neural network (detection via Faster R-CNN and classification via Inception V3), "ground-truth" CXR annotation (ROI labeling per LLIED), as well as inference display (as Generated Bounding Boxes (GBBs)), relied on a GPU-based graphical user interface. Results: To achieve 100% LLIED detection, probability threshold reduction to 0.00002 was required by Model 1, resulting in increasing GBBs per LLIED-related ROI. Targeting LLIED-type classification following detection of all LLIEDs, Model 2 multi-classified to reach high-performance while decreasing falsely positive GBBs. Despite 24% suboptimal ROI image quality, classification was correct in 98.9% and AUCs for the 9 LLIED-types were 1.00 for 8 and 0.92 for 1. For all misclassification cases: 1. None involved stringently conditional or unsafe LLIEDs; and 2. Most were attributable to suboptimal images. Conclusion: This project successfully developed a LLIED-related AI methodology supporting: 1. 100% detection; and 2. Typically 100% type classification.

机器翻译，仅供参考