人工智能学术速递[7.8]

cs.AI人工智能，共计39篇

【1】 Mitigating Performance Saturation in Neural Marked Point Processes: Architectures and Loss Functions 标题：减轻神经标记点过程的性能饱和：结构和损失函数

作者：Tianbo Li,Tianze Luo,Yiping Ke,Sinno Jialin Pan 机构：Sea AI Lab, Singapore, Nanyang Technological University 备注：9 pages, 4 figures, accepted by KDD-21 research track. The source code is available at this https URL Hawkes-Processes-GCHP 链接：https://arxiv.org/abs/2107.03354 摘要：属性化事件序列在实践中经常遇到。最近的一个研究方向是将神经网络与统计模型——标记点过程相结合，标记点过程是处理属性事件序列的传统工具。神经标记点过程具有很好的概率模型解释能力和神经网络的表示能力。然而，我们发现神经标记点过程的性能并不总是随着网络结构的复杂化和大型化而提高，这就是我们所说的性能饱和现象。这是由于神经标记点过程的泛化误差同时由网络的表示能力和模型规格决定的。因此，我们可以得出两个主要结论：第一，在某些情况下，简单的网络结构并不比复杂的网络结构差；其次，使用适当的概率假设与提高网络的复杂性同等重要，甚至更重要。基于这一观察，我们提出了一种简单的基于图的网络结构GCHP，它只使用图卷积层，因此可以很容易地被并行机制加速。我们直接考虑到达时间的分布，而不是对条件强度函数施加特定假设，并提出使用似然比损失与矩匹配机制进行优化和模型选择。实验结果表明，GCHP能显著减少训练时间，而在间隔时间概率假设下的似然比损失能显著提高模型性能。 摘要：Attributed event sequences are commonly encountered in practice. A recent research line focuses on incorporating neural networks with the statistical model -- marked point processes, which is the conventional tool for dealing with attributed event sequences. Neural marked point processes possess good interpretability of probabilistic models as well as the representational power of neural networks. However, we find that performance of neural marked point processes is not always increasing as the network architecture becomes more complicated and larger, which is what we call the performance saturation phenomenon. This is due to the fact that the generalization error of neural marked point processes is determined by both the network representational ability and the model specification at the same time. Therefore we can draw two major conclusions: first, simple network structures can perform no worse than complicated ones for some cases; second, using a proper probabilistic assumption is as equally, if not more, important as improving the complexity of the network. Based on this observation, we propose a simple graph-based network structure called GCHP, which utilizes only graph convolutional layers, thus it can be easily accelerated by the parallel mechanism. We directly consider the distribution of interarrival times instead of imposing a specific assumption on the conditional intensity function, and propose to use a likelihood ratio loss with a moment matching mechanism for optimization and model selection. Experimental results show that GCHP can significantly reduce training time and the likelihood ratio loss with interarrival time probability assumptions can greatly improve the model performance.

【2】 Regularization-based Continual Learning for Fault Prediction in Lithium-Ion Batteries 标题：基于正则化的连续学习在锂离子电池故障预测中的应用

作者：Benjamin Maschler,Sophia Tatiyosyan,Michael Weyrich 机构：a University of Stuttgart, Institute of Industrial Automation and Software Engineering, Pfaffenwaldring , Stuttgart, Germany 备注：6 pages, 5 figures, 4 tables. Accepted at CIRP ICME 2021. arXiv admin note: text overlap with arXiv:2101.00509 链接：https://arxiv.org/abs/2107.03336 摘要：近年来，锂离子电池的使用已大大扩展到许多工业部门的产品，如汽车、电动工具或医疗设备。因此，对电池故障的早期预测和强有力的理解可以大大提高这些领域的产品质量。虽然目前的数据驱动故障预测方法在其训练的确切过程中提供了良好的结果，但它们往往缺乏灵活适应变化的能力，例如在操作或环境参数方面。持续的学习保证了这样的灵活性，允许以前学习的知识自动适应新的任务。因此，本文从一组正则化策略中讨论了不同的持续学习方法，并基于一个真实的电池磨损数据集对这些方法进行了实现、评估和比较。在线弹性权重整合提供了最好的结果，但是，与所有被检查的方法一样，它的性能似乎强烈依赖于任务特征和任务序列。 摘要：In recent years, the use of lithium-ion batteries has greatly expanded into products from many industrial sectors, e.g. cars, power tools or medical devices. An early prediction and robust understanding of battery faults could therefore greatly increase product quality in those fields. While current approaches for data-driven fault prediction provide good results on the exact processes they were trained on, they often lack the ability to flexibly adapt to changes, e.g. in operational or environmental parameters. Continual learning promises such flexibility, allowing for an automatic adaption of previously learnt knowledge to new tasks. Therefore, this article discusses different continual learning approaches from the group of regularization strategies, which are implemented, evaluated and compared based on a real battery wear dataset. Online elastic weight consolidation delivers the best results, but, as with all examined approaches, its performance appears to be strongly dependent on task characteristics and task sequence.

【3】 Enhancing an Intelligent Digital Twin with a Self-organized Reconfiguration Management based on Adaptive Process Models 标题：基于自适应过程模型的自组织重构管理增强智能数字双胞胎

作者：Timo Müller,Benjamin Lindemann,Tobias Jung,Nasser Jazdi,Michael Weyrich 机构：Institute of Industrial Automation and Software Engineering, University of Stuttgart, Pfaffenwaldring , Stuttgart, Germany 备注：6 pages, 2 figures. Submitted to 54th CIRP Conference on Manufacturing Systems 2021 链接：https://arxiv.org/abs/2107.03324 摘要：产品生命周期的缩短和生产个性化程度的提高，使得工业自动化系统领域的重构需求不断增加，未来工业自动化系统将以网络物理生产系统为主。然而，在不断变化的系统中，并不是几乎无限状态空间的所有配置方案都被完全理解。因此，某些配置可能导致工艺不稳定、质量降低或机器故障。因此，本文提出了一种基于自适应过程模型的自组织重构管理方法来增强智能数字孪生子系统的性能，以便更全面地找到优化配置。 摘要：Shorter product life cycles and increasing individualization of production leads to an increased reconfiguration demand in the domain of industrial automation systems, which will be dominated by cyber-physical production systems in the future. In constantly changing systems, however, not all configuration alternatives of the almost infinite state space are fully understood. Thus, certain configurations can lead to process instability, a reduction in quality or machine failures. Therefore, this paper presents an approach that enhances an intelligent Digital Twin with a self-organized reconfiguration management based on adaptive process models in order to find optimized configurations more comprehensively.

【4】 Modelling Players in Mobile Puzzle Games 标题：手机益智游戏中的玩家建模

作者：Jeppe Theiss Kristensen,Arturo Valdivia,Paolo Burelli 机构：IT University of CopenhagenTactile Games, Copenhagen, Denmark 备注：Conference on Games 2021 conference paper 链接：https://arxiv.org/abs/2107.03305 摘要：成功和准确的难度建模是玩家体验操作化的一个基本组成部分，因为难度是内容设计和改编最重要和最常用的信号之一。在具有中间里程碑的游戏中，例如可完成区域或水平，难度通常由完成概率或完成率来定义；然而，这种操作是有限的，因为它不能描述玩家在该区域内的行为。在这项研究工作中，我们形式化了益智游戏的水平难度模型，它超越了经典的成功概率。我们通过使用参数统计模型描述游戏级别内执行的动作的分布来实现这一点，从而创建更丰富的难度描述。以《百合园》为例，通过触觉游戏对模型进行了拟合和评价，评价结果表明该模型能够描述和解释绝大多数水平的难度。 摘要：Successful and accurate modelling of level difficulty is a fundamental component of the operationalisation of player experience as difficulty is one of the most important and commonly used signals for content design and adaptation. In games that feature intermediate milestones, such as completable areas or levels, difficulty is often defined by the probability of completion or completion rate; however, this operationalisation is limited in that it does not describe the behaviour of the player within the area. In this research work, we formalise a model of level difficulty for puzzle games that goes beyond the classical probability of success. We accomplish this by describing the distribution of actions performed within a game level using a parametric statistical model thus creating a richer descriptor of difficulty. The model is fitted and evaluated on a dataset collected from the game Lily's Garden by Tactile Games, and the results of the evaluation show that the it is able to describe and explain difficulty in a vast majority of the levels.

【5】 Trans4E: Link Prediction on Scholarly Knowledge Graphs 标题：Trans4E：学术知识图上的链接预测

作者：Mojtaba Nayyeri,Gokce Muge Cil,Sahar Vahdati,Francesco Osborne,Mahfuzur Rahman,Simone Angioni,Angelo Salatino,Diego Reforgiato Recupero,Nadezhda Vassilyeva,Enrico Motta,Jens Lehmann 机构：SDA Research Group, University of Bonn (Germany), Institute for Applied Informatics (InfAI), Fraunhofer IAIS, Dresden (Germany), Knowledge Media Institute, The Open University, Milton Keynes (UK) 链接：https://arxiv.org/abs/2107.03297 摘要：知识图的不完全性是影响人工智能服务质量的关键问题。在学术领域，描述研究出版物的KG通常缺乏重要信息，妨碍了我们分析和预测研究动态的能力。近年来，基于知识图嵌入模型的链路预测方法成为解决这一问题的急救手段。在这项工作中，我们提出了一个新的嵌入模型Trans4E，它特别适合于包含N到M关系和N$\gg$M的KGs。这对于将大量实体（例如，研究文章、专利、人员）按照相对较小的类别进行分类的KG来说是典型的。Trans4E被应用于两个大规模的知识图，学术/工业动态（AIDA）和微软学术图（MAG），以完成有关研究领域（如“神经网络”、“机器学习”、“人工智能”）和附属类型（如“教育”、“公司”、“政府”）的信息，提高结果数据的范围和准确性。我们根据AIDA、MAG和其他四个基准（FB15k、FB15k-237、WN18和WN18RR）的替代解决方案评估了我们的方法。Trans4E模型在低嵌入维数下的性能优于其他模型，在高嵌入维数下具有竞争性。 摘要：The incompleteness of Knowledge Graphs (KGs) is a crucial issue affecting the quality of AI-based services. In the scholarly domain, KGs describing research publications typically lack important information, hindering our ability to analyse and predict research dynamics. In recent years, link prediction approaches based on Knowledge Graph Embedding models became the first aid for this issue. In this work, we present Trans4E, a novel embedding model that is particularly fit for KGs which include N to M relations with N$\gg$M. This is typical for KGs that categorize a large number of entities (e.g., research articles, patents, persons) according to a relatively small set of categories. Trans4E was applied on two large-scale knowledge graphs, the Academia/Industry DynAmics (AIDA) and Microsoft Academic Graph (MAG), for completing the information about Fields of Study (e.g., 'neural networks', 'machine learning', 'artificial intelligence'), and affiliation types (e.g., 'education', 'company', 'government'), improving the scope and accuracy of the resulting data. We evaluated our approach against alternative solutions on AIDA, MAG, and four other benchmarks (FB15k, FB15k-237, WN18, and WN18RR). Trans4E outperforms the other models when using low embedding dimensions and obtains competitive results in high dimensions.

【6】 Attribute reduction and rule acquisition of formal decision context based on two new kinds of decision rules 标题：基于两种新型决策规则的形式决策背景属性约简与规则获取

作者：Qian Hu,Keyun Qin 机构：School of Computing and Artificial Intelligence, Southwest Jiaotong University, Chengdu, Sichuan, China, School of Mathematics 备注：20 pages, 3figures 链接：https://arxiv.org/abs/2107.03288 摘要：本文主要研究了基于两种新的决策规则I-决策规则和II-决策规则的形式化决策上下文的规则获取和属性约简。这些规则的前提是面向对象的概念，结论分别是形式概念和面向属性的概念。给出了I-决策规则和II-决策规则的规则获取算法。对这些算法与现有算法进行了比较分析，结果表明本文提出的算法性能良好。利用区分矩阵给出了保留I-决策规则和II-决策规则的属性约简方法。 摘要：This paper mainly studies the rule acquisition and attribute reduction for formal decision context based on two new kinds of decision rules, namely I-decision rules and II-decision rules. The premises of these rules are object-oriented concepts, and the conclusions are formal concept and property-oriented concept respectively. The rule acquisition algorithms for I-decision rules and II-decision rules are presented. Some comparative analysis of these algorithms with the existing algorithms are examined which shows that the algorithms presented in this study behave well. The attribute reduction approaches to preserve I-decision rules and II-decision rules are presented by using discernibility matrix.

【7】 Strategy Complexity of Mean Payoff, Total Payoff and Point Payoff Objectives in Countable MDPs 标题：可数MDP的平均支付、总支付和点支付目标的策略复杂性

作者：Richard Mayr,Eric Munday 机构：University of Edinburgh, UK 备注：Full version of a conference paper at CONCUR 2021. 41 pages 链接：https://arxiv.org/abs/2107.03287 摘要：研究了具有实值转移报酬的可数无限马尔可夫决策过程。每一次无限跑都会产生如下的回报序列：1.点回报（直接看到的过渡回报序列），2.总回报（到目前为止所有回报总和的序列），3.平均回报。对于每种支付类型，目标是最大化$\liminf$非负的概率。我们建立了这些目标的策略复杂性的完整图景，即$\varepsilon$-optimal（resp。最佳）策略。有些情况下可以用无记忆的确定性策略取胜，而另一些情况则需要一个步进计数器、一个奖励计数器或两者兼而有之。 摘要：We study countably infinite Markov decision processes (MDPs) with real-valued transition rewards. Every infinite run induces the following sequences of payoffs: 1. Point payoff (the sequence of directly seen transition rewards), 2. Total payoff (the sequence of the sums of all rewards so far), and 3. Mean payoff. For each payoff type, the objective is to maximize the probability that the $\liminf$ is non-negative. We establish the complete picture of the strategy complexity of these objectives, i.e., how much memory is necessary and sufficient for $\varepsilon$-optimal (resp. optimal) strategies. Some cases can be won with memoryless deterministic strategies, while others require a step counter, a reward counter, or both.

【8】 Introducing the structural bases of typicality effects in deep learning 标题：浅谈深度学习中典型性效应的结构基础

作者：Omar Vidal Pino,Erickson Rangel Nascimento,Mario Fernando Montenegro Campos 机构：Computer Vision and Robotics Laboratory, Computer Science Department, Universidade Federal de Minas Gerais, Belo Horizonte,-, Brazil 备注：14 pages (12 + 2 reference); 13 Figures and 2 Tables. arXiv admin note: text overlap with arXiv:1906.03365 链接：https://arxiv.org/abs/2107.03279 摘要：在本文中，我们假设自然语义范畴的典型程度的影响可以通过深度学习模型学习的人工范畴的结构来产生。基于人类表示自然语义范畴的方法，在原型理论的基础上，提出了一种新的计算原型模型（CPM）来表示语义范畴的内部结构。与其他原型学习方法不同的是，我们的数学框架提出了第一种方法来提供深层神经网络对抽象语义概念的建模能力，例如类别中心语义、对象图像的典型程度和家族相似关系。在图像分类、全局语义描述和迁移学习等图像语义处理任务中，我们提出了几种基于典型性概念的方法来评价我们的CPM模型。我们在不同图像数据集（如ImageNet和Coco）上的实验表明，我们的方法可能是一个可接受的命题，目的是赋予机器更大的抽象能力来表示对象类别的语义。 摘要：In this paper, we hypothesize that the effects of the degree of typicality in natural semantic categories can be generated based on the structure of artificial categories learned with deep learning models. Motivated by the human approach to representing natural semantic categories and based on the Prototype Theory foundations, we propose a novel Computational Prototype Model (CPM) to represent the internal structure of semantic categories. Unlike other prototype learning approaches, our mathematical framework proposes a first approach to provide deep neural networks with the ability to model abstract semantic concepts such as category central semantic meaning, typicality degree of an object's image, and family resemblance relationship. We proposed several methodologies based on the typicality's concept to evaluate our CPM-model in image semantic processing tasks such as image classification, a global semantic description, and transfer learning. Our experiments on different image datasets, such as ImageNet and Coco, showed that our approach might be an admissible proposition in the effort to endow machines with greater power of abstraction for the semantic representation of objects' categories.

【9】 Contrastive Explanations for Argumentation-Based Conclusions 标题：论辩结论的对比解释

作者：AnneMarie Borg,Floris Bex 机构：Department of Information and Computing Sciences, Utrecht University, Department of Law, Technology, Markets and Society, Tilburg University 链接：https://arxiv.org/abs/2107.03265 摘要：在本文中，我们讨论了形式论证的对比解释，即在各种基于扩展的语义下，为什么某个论证（事实）可以被接受，而另一个论证（箔片）不能被接受的问题。最近关于论证结论解释的工作主要集中在为（不）接受论据提供最低限度的解释。然而，仍然缺乏的是一个正确的基于论证的对比解释解释。我们展示了在何种条件下，抽象论证和结构化论证中的对比解释是有意义的，以及论证是如何使隐含的箔显化的。 摘要：In this paper we discuss contrastive explanations for formal argumentation - the question why a certain argument (the fact) can be accepted, whilst another argument (the foil) cannot be accepted under various extension-based semantics. The recent work on explanations for argumentation-based conclusions has mostly focused on providing minimal explanations for the (non-)acceptance of arguments. What is still lacking, however, is a proper argumentation-based interpretation of contrastive explanations. We show under which conditions contrastive explanations in abstract and structured argumentation are meaningful, and how argumentation allows us to make implicit foils explicit.

【10】 Graphing else matters: exploiting aspect opinions and ratings in explainable graph-based recommendations 标题：用图表表示其他问题：在可解释的基于图表的推荐中利用方面意见和评级

作者：Iván Cantador,Andrés Carvallo,Fernando Diez,Denis Parra 机构： Universidad Autónoma de Madrid, Pontificia Universidad Católica de Chile 链接：https://arxiv.org/abs/2107.03226 摘要：随着神经网络嵌入技术的成功，人们对利用知识图进行各种机器学习和信息检索产生了新的兴趣。特别是，目前基于图嵌入的推荐方法已经显示出了最先进的性能。这些方法通常对潜在的评级模式和内容特征进行编码。与以往的工作不同，本文提出利用从图中提取的嵌入信息，将来自评分的信息和文本评论中表达的基于方面的观点结合起来。然后，我们在6个域上采用并评估了最先进的图嵌入技术，这些技术比Amazon和Yelp评论生成的图具有更好的性能。我们的方法的优点是提供解释，利用用户对推荐项目给出的基于方面的意见。此外，我们还提供了在可视化仪表板中使用方面观点作为解释的建议的适用性示例，该仪表板允许获得从输入图的嵌入中获得的类似用户最喜欢和最不喜欢的方面的信息。 摘要：The success of neural network embeddings has entailed a renewed interest in using knowledge graphs for a wide variety of machine learning and information retrieval tasks. In particular, current recommendation methods based on graph embeddings have shown state-of-the-art performance. These methods commonly encode latent rating patterns and content features. Different from previous work, in this paper, we propose to exploit embeddings extracted from graphs that combine information from ratings and aspect-based opinions expressed in textual reviews. We then adapt and evaluate state-of-the-art graph embedding techniques over graphs generated from Amazon and Yelp reviews on six domains, outperforming baseline recommenders. Our approach has the advantage of providing explanations which leverage aspect-based opinions given by users about recommended items. Furthermore, we also provide examples of the applicability of recommendations utilizing aspect opinions as explanations in a visualization dashboard, which allows obtaining information about the most and least liked aspects of similar users obtained from the embeddings of an input graph.

【11】 Hierarchical Semantic Segmentation using Psychometric Learning 标题：基于心理测量学习的层次化语义分割

作者：Lu Yin,Vlado Menkovski,Shiwei Liu,Mykola Pechenizkiy 机构：Eindhoven University of Technology, Eindhoven , MB, Netherlands 备注：17 pages, 12 figures 链接：https://arxiv.org/abs/2107.03212 摘要：语义图像分割的目标是为部分图像数据赋予意义。机器学习方法，特别是有监督的学习方法，广泛应用于各种任务的语义分割。有监督学习方法的主要挑战之一是表达和收集专家们对图像数据中存在的意义的丰富知识。为此，通常会指定一组固定的标签，专家的任务是用给定的标签注释图像中的像素、面片或片段。然而，通常情况下，类集合并不能完全捕获图像中丰富的语义信息。例如，在诸如组织学图像之类的医学成像中，可以根据病理学家的专业知识对细胞的不同部分进行分组和再分组。为了实现图像中概念的精确语义表示，我们需要获得注释者的全部知识深度。在这项工作中，我们提出了一种新的方法来收集分割注释的专家基于心理测试。我们的方法由心理测试程序、主动查询选择、查询增强和深度度量学习模型组成，以实现一个补丁级的图像嵌入，从而实现图像的语义分割。通过对合成图像、航空图像和组织学图像的评价，说明了该方法的优点。 摘要：Assigning meaning to parts of image data is the goal of semantic image segmentation. Machine learning methods, specifically supervised learning is commonly used in a variety of tasks formulated as semantic segmentation. One of the major challenges in the supervised learning approaches is expressing and collecting the rich knowledge that experts have with respect to the meaning present in the image data. Towards this, typically a fixed set of labels is specified and experts are tasked with annotating the pixels, patches or segments in the images with the given labels. In general, however, the set of classes does not fully capture the rich semantic information present in the images. For example, in medical imaging such as histology images, the different parts of cells could be grouped and sub-grouped based on the expertise of the pathologist. To achieve such a precise semantic representation of the concepts in the image, we need access to the full depth of knowledge of the annotator. In this work, we develop a novel approach to collect segmentation annotations from experts based on psychometric testing. Our method consists of the psychometric testing procedure, active query selection, query enhancement, and a deep metric learning model to achieve a patch-level image embedding that allows for semantic segmentation of images. We show the merits of our method with evaluation on the synthetically generated image, aerial image and histology image.

【12】 Nested Counterfactual Identification from Arbitrary Surrogate Experiments 标题：任意代理实验中的嵌套式反事实鉴定

作者：Juan D Correa,Sanghack Lee,Elias Bareinboim 机构：Seoul National University, Columbia University 链接：https://arxiv.org/abs/2107.03190 摘要：因果关系阶梯描述了代理人可能感兴趣的三种性质不同的活动类型，即看（观察）、做（干预）和想象（反事实）（Pearl和Mackenzie，2018）。因果层次结构带来的推理挑战是，数据是由观察或干预系统的代理收集的（第1层和第2层），而它的目标可能是了解如果它采取不同的行动过程会发生什么，与实际结果相反（第3层）。虽然人们对允许从观察到干预进行跨层推断的条件有着坚实的理解，但在针对反事实量时，结果却有点少。在本文中，我们研究从观察和实验的任意组合中识别嵌套反事实。具体地说，基于嵌套反实数的一个更明确的定义，我们证明了反实数不可测定理（CUT），它允许我们将任意嵌套的反实数映射到非嵌套的反实数。例如，调解和公平性分析中的应用通常会引发直接、间接和虚假效果的概念，这自然需要嵌套。其次，我们从观测分布和实验分布的任意组合引入了反事实识别的充要图形条件。最后，我们提出了一个有效且完整的识别嵌套反事实的算法；算法返回查询表达式失败意味着它不可识别。 摘要：The Ladder of Causation describes three qualitatively different types of activities an agent may be interested in engaging in, namely, seeing (observational), doing (interventional), and imagining (counterfactual) (Pearl and Mackenzie, 2018). The inferential challenge imposed by the causal hierarchy is that data is collected by an agent observing or intervening in a system (layers 1 and 2), while its goal may be to understand what would have happened had it taken a different course of action, contrary to what factually ended up happening (layer 3). While there exists a solid understanding of the conditions under which cross-layer inferences are allowed from observations to interventions, the results are somewhat scarcer when targeting counterfactual quantities. In this paper, we study the identification of nested counterfactuals from an arbitrary combination of observations and experiments. Specifically, building on a more explicit definition of nested counterfactuals, we prove the counterfactual unnesting theorem (CUT), which allows one to map arbitrary nested counterfactuals to unnested ones. For instance, applications in mediation and fairness analysis usually evoke notions of direct, indirect, and spurious effects, which naturally require nesting. Second, we introduce a sufficient and necessary graphical condition for counterfactual identification from an arbitrary combination of observational and experimental distributions. Lastly, we develop an efficient and complete algorithm for identifying nested counterfactuals; failure of the algorithm returning an expression for a query implies it is not identifiable.

【13】 Intensity Prediction of Tropical Cyclones using Long Short-Term Memory Network 标题：利用长短期记忆网络预报热带气旋强度

作者：Koushik Biswas,Sandeep Kumar,Ashish Kumar Pandey 机构：Department of Computer Science, IIIT Delhi, New Delhi, India,., &, Shaheed Bhagat Singh College, University of Delhi, Department of Mathematics, IIIT Delhi 备注：10 pages 链接：https://arxiv.org/abs/2107.03187 摘要：热带气旋的强度是多种多样的，如果强度足够大，就会造成巨大的生命和财产损失。因此，及时预报热带气旋的强度具有十分重要的意义。提出了一种新的基于叠加双向长短时记忆网络（BiLSTM）的模式结构，以最大地表持续风速（MSWS）预测热带气旋的强度。该模型能很好地预测城市固体废弃物，预测精度很高。我们将该模型应用于1982年至2018年北印度洋热带气旋，并在最近的两个热带气旋Fani和Vayu上检验了其性能。该模型预测未来3、12、24、36、48、60和72小时的MSW（以节为单位），平均绝对误差分别为1.52、3.66、5.88、7.42、8.96、10.15和11.92。 摘要：Tropical cyclones can be of varied intensity and cause a huge loss of lives and property if the intensity is high enough. Therefore, the prediction of the intensity of tropical cyclones advance in time is of utmost importance. We propose a novel stacked bidirectional long short-term memory network (BiLSTM) based model architecture to predict the intensity of a tropical cyclone in terms of Maximum surface sustained wind speed (MSWS). The proposed model can predict MSWS well advance in time (up to 72 h) with very high accuracy. We have applied the model on tropical cyclones in the North Indian Ocean from 1982 to 2018 and checked its performance on two recent tropical cyclones, namely, Fani and Vayu. The model predicts MSWS (in knots) for the next 3, 12, 24, 36, 48, 60, and 72 hours with a mean absolute error of 1.52, 3.66, 5.88, 7.42, 8.96, 10.15, and 11.92, respectively.

【14】 Urban Tree Species Classification Using Aerial Imagery 标题：基于航空影像的城市树种分类

作者：Emily Waters,Mahdi Maktabdar Oghaz,Lakshmi Babu Saheer 机构：AngliaRuskinUniversity 备注：International Conference on Machine Learning (ICML 2021), Workshop on Tackling Climate Change with Machine Learning 链接：https://arxiv.org/abs/2107.03182 摘要：城市树木有助于调节温度，减少能源消耗，改善城市空气质量，降低风速，减轻城市热岛效应。城市树木在减缓气候变化和全球变暖方面也发挥着关键作用，它捕获和储存大气中的二氧化碳，而二氧化碳是温室气体的最大贡献者。利用航空图像进行树木自动检测和物种分类是可持续森林和城市树木管理的有力工具。因此，本研究首先提供了一个利用Google-Map航空影像生成城市树木标签数据集的管道，然后研究了VGG和ResNet等先进的深度卷积神经网络模型在不同参数下如何处理城市树木航空影像的分类问题。实验结果表明，我们的最佳模型在6个树种上的平均准确率达到60%。 摘要：Urban trees help regulate temperature, reduce energy consumption, improve urban air quality, reduce wind speeds, and mitigating the urban heat island effect. Urban trees also play a key role in climate change mitigation and global warming by capturing and storing atmospheric carbon-dioxide which is the largest contributor to greenhouse gases. Automated tree detection and species classification using aerial imagery can be a powerful tool for sustainable forest and urban tree management. Hence, This study first offers a pipeline for generating labelled dataset of urban trees using Google Map's aerial images and then investigates how state of the art deep Convolutional Neural Network models such as VGG and ResNet handle the classification problem of urban tree aerial images under different parameters. Experimental results show our best model achieves an average accuracy of 60% over 6 tree species.

【15】 Levels of explainable artificial intelligence for human-aligned conversational explanations 标题：可解释的人工智能水平，用于人类对齐的会话解释

作者：Richard Dazeley,Peter Vamplew,Cameron Foale,Charlotte Young,Sunil Aryal,Francisco Cruz 机构：School of Information Technology, Deakin University, Locked Bag , Geelong, Victoria , Australia, School of Engineering, Information Technology and Physical Sciences, Federation University, Ballarat, Victoria , Australia, A R T I C L E I N F O 备注：None 链接：https://arxiv.org/abs/2107.03178 摘要：近几年来，可解释人工智能（XAI）和可解释机器学习（IML）的研究发展迅速。推动这一增长的因素包括最近的立法改革、工业和政府投资的增加，以及公众日益关注的问题。人们每天都受到自主决策的影响，公众需要了解决策过程以接受结果。然而，XAI/IML的绝大多数应用都侧重于提供低层次的“狭义”解释，说明个人决策是如何基于特定数据做出的。虽然这些解释很重要，但很少能提供对代理人的见解：信念和动机；其他（人类、动物或人工智能）主体意图的假设；解读外部文化期待；或者，用于生成自己的解释的过程。然而，我们认为，所有这些因素对于提供人们接受和信任人工智能决策所需的解释深度至关重要。本文旨在定义解释的层次，并描述如何整合这些层次来建立一个符合人类的会话解释系统。在此过程中，本文将调查当前的方法，并讨论通过广泛可解释人工智能（Broad XAI）实现这些层次的不同技术的集成，从而走向高层次的“强”解释。 摘要：Over the last few years there has been rapid research growth into eXplainable Artificial Intelligence (XAI) and the closely aligned Interpretable Machine Learning (IML). Drivers for this growth include recent legislative changes and increased investments by industry and governments, along with increased concern from the general public. People are affected by autonomous decisions every day and the public need to understand the decision-making process to accept the outcomes. However, the vast majority of the applications of XAI/IML are focused on providing low-level `narrow' explanations of how an individual decision was reached based on a particular datum. While important, these explanations rarely provide insights into an agent's: beliefs and motivations; hypotheses of other (human, animal or AI) agents' intentions; interpretation of external cultural expectations; or, processes used to generate its own explanation. Yet all of these factors, we propose, are essential to providing the explanatory depth that people require to accept and trust the AI's decision-making. This paper aims to define levels of explanation and describe how they can be integrated to create a human-aligned conversational explanation system. In so doing, this paper will survey current approaches and discuss the integration of different technologies to achieve these levels with Broad eXplainable Artificial Intelligence (Broad-XAI), and thereby move towards high-level `strong' explanations.

【16】 A Survey on Data Augmentation for Text Classification 标题：面向文本分类的数据增强技术综述

作者：Markus Bayer,Marc-André Kaufhold,Christian Reuter 机构： Technical University of Darmstadt 备注：35 pages, 6 figures, 8 tables 链接：https://arxiv.org/abs/2107.03158 摘要：数据增广是通过变换人工生成机器学习训练数据，是机器学习学科中一个广泛研究的领域。虽然它有助于提高模型的泛化能力，但它也可以解决许多其他挑战和问题，从克服有限的训练数据量过度规范化目标到限制用于保护隐私的数据量。基于对数据扩充（C1）的目标和应用的精确描述和现有著作的分类法（C2），本调查关注文本分类的数据扩充方法，旨在为研究者和实践者提供一个简明而全面的概述（C3）。根据分类法，我们将100多种方法分为12个不同的组，并提供最新的参考资料，阐明哪些方法是非常有前途的（C4）。最后，给出了可能构成未来工作基石的研究前景（C5）。 摘要：Data augmentation, the artificial creation of training data for machine learning by transformations, is a widely studied research field across machine learning disciplines. While it is useful for increasing the generalization capabilities of a model, it can also address many other challenges and problems, from overcoming a limited amount of training data over regularizing the objective to limiting the amount data used to protect privacy. Based on a precise description of the goals and applications of data augmentation (C1) and a taxonomy for existing works (C2), this survey is concerned with data augmentation methods for textual classification and aims to achieve a concise and comprehensive overview for researchers and practitioners (C3). Derived from the taxonomy, we divided more than 100 methods into 12 different groupings and provide state-of-the-art references expounding which methods are highly promising (C4). Finally, research perspectives that may constitute a building block for future work are given (C5).

【17】 R2F: A Remote Retraining Framework for AIoT Processors with Computing Errors 标题：R2F：一种有计算错误的AIoT处理器远程再训练框架

作者：Dawen Xu,Meng He,Cheng Liu,Ying Wang,Long Cheng,Huawei Li,Xiaowei Li,Kwang-Ting Cheng 链接：https://arxiv.org/abs/2107.03096 摘要：采用较新技术节点制造的AIoT处理器，由于晶体管尺寸的缩小和电源的降低，软误差不断上升。AIoT处理器上的软错误，特别是大规模计算的深度学习加速器（dla）可能会导致大量的计算错误。这些计算错误很难通过对服务器中的cpu和gpu等通用处理器的常规训练来捕获。将离线训练的神经网络模型直接应用于有误差的边缘加速器，会造成较大的预测精度损失。为了解决这个问题，我们提出了一个远程再训练框架（R2F）来解决有计算错误的远程AIoT处理器。该算法在训练环中引入了带有软错误的远程AIoT处理器，使得现场计算错误可以通过服务器上的应用数据进行学习，再训练后的模型能够抵抗软错误。同时，我们提出了一个优化的部分TMR策略来加强再训练。根据我们的实验，R2F可以在模型精度和性能损失之间进行弹性设计权衡。在高错误率下，前5位模型精度可提高1.93%-13.73%，性能损失为0%-200%。此外，我们注意到重训练需要大量的数据传输，甚至控制了训练时间，并提出了一种稀疏增量压缩的数据传输优化方法，与直接的远程重训练相比，平均减少了38%-88%的重训练时间，精度损失可以忽略不计。 摘要：AIoT processors fabricated with newer technology nodes suffer rising soft errors due to the shrinking transistor sizes and lower power supply. Soft errors on the AIoT processors particularly the deep learning accelerators (DLAs) with massive computing may cause substantial computing errors. These computing errors are difficult to be captured by the conventional training on general purposed processors like CPUs and GPUs in a server. Applying the offline trained neural network models to the edge accelerators with errors directly may lead to considerable prediction accuracy loss. To address the problem, we propose a remote retraining framework (R2F) for remote AIoT processors with computing errors. It takes the remote AIoT processor with soft errors in the training loop such that the on-site computing errors can be learned with the application data on the server and the retrained models can be resilient to the soft errors. Meanwhile, we propose an optimized partial TMR strategy to enhance the retraining. According to our experiments, R2F enables elastic design trade-offs between the model accuracy and the performance penalty. The top-5 model accuracy can be improved by 1.93%-13.73% with 0%-200% performance penalty at high fault error rate. In addition, we notice that the retraining requires massive data transmission and even dominates the training time, and propose a sparse increment compression approach for the data transmission optimization, which reduces the retraining time by 38%-88% on average with negligible accuracy loss over a straightforward remote retraining.

【18】 RISAN: Robust Instance Specific Abstention Network 标题：Risan：健壮的实例特定弃权网络

作者：Bhavya Kalra,Kulin Shah,Naresh Manwani 机构：Machine Learning Lab, International Institute of Technology, Hyderabad, India, Microsoft Research, Bangalore, India 链接：https://arxiv.org/abs/2107.03090 摘要：在本文中，我们提出了学习特定实例的弃权（拒绝选项）二进制分类器的深层结构。所提出的方法使用Kulin Shah和Naresh Manwani在《拒绝选项分类器的在线主动学习》，AAAI，2020）中描述的双S形损失函数作为性能度量。我们证明了双乙状结肠损失是分类校正的。我们还证明了0-d-1损失的超额风险是双s形损失超额风险的上界。我们推导了所提出的拒绝选项分类器结构的泛化误差界。为了验证所提出的方法的有效性，我们用几个真实的数据集进行了实验。我们观察到，所提出的方法不仅性能相当于国家的最先进的方法，它还对标签噪声的鲁棒性。我们还提供了可视化，以观察网络学习到的与弃权决定相对应的重要特征。 摘要：In this paper, we propose deep architectures for learning instance specific abstain (reject option) binary classifiers. The proposed approach uses double sigmoid loss function as described by Kulin Shah and Naresh Manwani in ("Online Active Learning of Reject Option Classifiers", AAAI, 2020), as a performance measure. We show that the double sigmoid loss is classification calibrated. We also show that the excess risk of 0-d-1 loss is upper bounded by the excess risk of double sigmoid loss. We derive the generalization error bounds for the proposed architecture for reject option classifiers. To show the effectiveness of the proposed approach, we experiment with several real world datasets. We observe that the proposed approach not only performs comparable to the state-of-the-art approaches, it is also robust against label noise. We also provide visualizations to observe the important features learned by the network corresponding to the abstaining decision.

【19】 WeClick: Weakly-Supervised Video Semantic Segmentation with Click Annotations 标题：WeClick：带点击标注的弱监督视频语义分割

作者：Peidong Liu,Zibin He,Xiyu Yan,Yong Jiang,Shutao Xia,Feng Zheng,Maowei Hu 机构：Tsinghua Shenzhen International Graduate School, Tsinghua University, PCL Research Center of Networks and Communications, Peng Cheng Laboratory, Department of Computer Science and Engineering, Southern University of Science and Technology 备注：Accepted by ACM MM2021 链接：https://arxiv.org/abs/2107.03088 摘要：与单调乏味的每像素遮罩注释相比，通过单击注释数据要容易得多，一张图像只需几秒钟。然而，应用点击学习视频语义分割模型的研究还不多见。在这项工作中，我们提出了一个有效的弱监督视频语义分割管道，称为WeClick，通过只需一次单击就可以分割语义类的一个实例，从而节省了费力的注释工作。由于点击无法捕捉到详细的语义信息，直接使用点击标签进行训练会导致分割预测效果不佳。为了缓解这一问题，我们设计了一种新的记忆流知识提取策略，利用大量未标记视频帧中的时间信息（称为记忆流），通过估计运动将相邻预测提取到目标帧中。此外，本文还采用了香草知识提取的方法进行模型压缩。在这种情况下，WeClick在训练阶段学习具有低成本click注释的紧凑视频语义分割模型，而在推理阶段获得实时、准确的模型。在Cityscapes和Camvid上的实验结果表明，WeClick的性能优于现有的方法，比基线提高了10.24%mIoU，实现了实时执行。 摘要：Compared with tedious per-pixel mask annotating, it is much easier to annotate data by clicks, which costs only several seconds for an image. However, applying clicks to learn video semantic segmentation model has not been explored before. In this work, we propose an effective weakly-supervised video semantic segmentation pipeline with click annotations, called WeClick, for saving laborious annotating effort by segmenting an instance of the semantic class with only a single click. Since detailed semantic information is not captured by clicks, directly training with click labels leads to poor segmentation predictions. To mitigate this problem, we design a novel memory flow knowledge distillation strategy to exploit temporal information (named memory flow) in abundant unlabeled video frames, by distilling the neighboring predictions to the target frame via estimated motion. Moreover, we adopt vanilla knowledge distillation for model compression. In this case, WeClick learns compact video semantic segmentation models with the low-cost click annotations during the training phase yet achieves real-time and accurate models during the inference period. Experimental results on Cityscapes and Camvid show that WeClick outperforms the state-of-the-art methods, increases performance by 10.24% mIoU than baseline, and achieves real-time execution.

【20】 Android Security using NLP Techniques: A Review 标题：基于NLP技术的Android安全研究综述

作者：Sevil Sen,Burcu Can 机构：WISE Lab., Dept. of Computer Engineering, Hacettepe University, Ankara, TURKEY, Research Institute of Information and Language Processing, University of Wolverhampton, Wolverhampton, UK 链接：https://arxiv.org/abs/2107.03072 摘要：Android是攻击者针对性最强的平台之一。在攻击者不断改进技术的同时，基于静态和动态分析的传统解决方案也在不断发展。除了应用程序代码之外，Android应用程序还有一些元数据，可以用于应用程序的安全分析。与传统的应用程序分发机制不同，Android应用程序在移动市场集中分发。因此，除了应用程序包，这些市场还包含应用程序开发者和应用程序用户提供的应用程序信息。这些有用的文本数据的可用性，以及用于处理和理解文本数据的自然语言处理（NLP）的进步，鼓励了研究人员研究NLP技术在Android安全中的应用。特别是，基于NLP的安全解决方案在过去5年中加速发展，并被证明是有用的。本研究回顾了这些建议，旨在通过介绍这一领域的最新进展，探索未来研究的可能方向。我们主要研究基于NLP的四类解决方案：行为保真度描述、描述生成、隐私和恶意软件检测。 摘要：Android is among the most targeted platform by attackers. While attackers are improving their techniques, traditional solutions based on static and dynamic analysis have been also evolving. In addition to the application code, Android applications have some metadata that could be useful for security analysis of applications. Unlike traditional application distribution mechanisms, Android applications are distributed centrally in mobile markets. Therefore, beside application packages, such markets contain app information provided by app developers and app users. The availability of such useful textual data together with the advancement in Natural Language Processing (NLP) that is used to process and understand textual data has encouraged researchers to investigate the use of NLP techniques in Android security. Especially, security solutions based on NLP have accelerated in the last 5 years and proven to be useful. This study reviews these proposals and aim to explore possible research directions for future studies by presenting state-of-the-art in this domain. We mainly focus on NLP-based solutions under four categories: description-to-behaviour fidelity, description generation, privacy and malware detection.

【21】 EchoEA: Echo Information between Entities and Relations for Entity Alignment 标题：EchoEA：实体之间的回声信息和实体对齐的关系

作者：Xueyuan Lin,Haihong E,Wenyu Song,Haoran Luo 机构：Department of Computer Science, Beijing University of Posts and Telecommunications, Beijing, China 链接：https://arxiv.org/abs/2107.03054 摘要：实体对齐（EA）是从不同的知识图（KG）中发现现实世界中引用同一对象的实体。它在自动整合来自多个来源的KG中起着重要的作用。现有的基于图神经网络（GNNs）的知识图嵌入（KGE）方法都取得了很好的效果，单向地增强了实体的关系表示。此外，越来越多的方法采用半监督的方法来要求更多的训练数据。然而，这些方法仍然存在两个挑战：（1）互动不足：实体和关系之间的互动没有得到充分利用(2） 低质量引导：生成的半监督数据质量低。在本文中，我们提出了一个新的框架，回音实体对齐（echoa），它利用自我注意机制将实体信息传播到关系中，并回音到实体。关系表示是从实体表示中动态计算出来的。对称地，从关系表示中动态计算出下一个实体表示，这显示了足够的交互作用。此外，我们提出了属性组合双向全局滤波策略（ABGS）来改善自举，减少虚假样本，生成高质量的训练数据。在三个真实的跨语言数据集上的实验结果稳定在96%左右hits@1平均而言，我们的方法不仅显著优于最新的方法，而且对于现有的KGE方法具有普遍性和可移植性。 摘要：Entity alignment (EA) is to discover entities referring to the same object in the real world from different knowledge graphs (KGs). It plays an important role in automatically integrating KGs from multiple sources. Existing knowledge graph embedding (KGE) methods based on Graph Neural Networks (GNNs) have achieved promising results, which enhance entity representation with relation information unidirectionally. Besides, more and more methods introduce semi-supervision to ask for more labeled training data. However, two challenges still exist in these methods: (1) Insufficient interaction: The interaction between entities and relations is insufficiently utilized. (2) Low-quality bootstrapping: The generated semi-supervised data is of low quality. In this paper, we propose a novel framework, Echo Entity Alignment (EchoEA), which leverages self-attention mechanism to spread entity information to relations and echo back to entities. The relation representation is dynamically computed from entity representation. Symmetrically, the next entity representation is dynamically calculated from relation representation, which shows sufficient interaction. Furthermore, we propose attribute-combined bi-directional global-filtered strategy (ABGS) to improve bootstrapping, reduce false samples and generate high-quality training data. The experimental results on three real-world cross-lingual datasets are stable at around 96\% at hits@1 on average, showing that our approach not only significantly outperforms the state-of-the-art methods, but also is universal and transferable for existing KGE methods.

【22】 A convolutional neural network for teeth margin detection on 3-dimensional dental meshes 标题：基于卷积神经网络的三维齿面边缘检测

作者：Hu Chen,Hong Li,Bifu Hu,Kenan Ma,Yuchun Sun 机构：Center of Digital Dentistry, Department of Prosthodontics, National Engineering Laboratory for Digital and material, technology of stomatology, Research Center of Engineering and Technology for Digital Dentistry, Peking University 备注：11 pages, 4 figures 链接：https://arxiv.org/abs/2107.03030 摘要：提出了一种用于三维牙齿网格顶点分类的卷积神经网络，并将其用于牙齿边缘检测。构造扩展层，收集相邻顶点特征的统计值，利用卷积神经网络计算每个顶点的新特征。提出了一种端到端的神经网络，以顶点特征（包括坐标、曲率和距离）为输入，输出每个顶点分类标签。利用1156个齿科网格设计和训练了具有不同扩展层参数的网络结构和无扩展层的基线网络。在145个牙齿网格上验证了该方法的准确性、召回率和准确度，并对最佳网络结构进行了评价，最后在另外144个牙齿网格上进行了测试。所有扩展层的网络性能均优于基线，最佳网络在验证数据集和测试数据集上的精度均达到0.877。 摘要：We proposed a convolutional neural network for vertex classification on 3-dimensional dental meshes, and used it to detect teeth margins. An expanding layer was constructed to collect statistic values of neighbor vertex features and compute new features for each vertex with convolutional neural networks. An end-to-end neural network was proposed to take vertex features, including coordinates, curvatures and distance, as input and output each vertex classification label. Several network structures with different parameters of expanding layers and a base line network without expanding layers were designed and trained by 1156 dental meshes. The accuracy, recall and precision were validated on 145 dental meshes to rate the best network structures, which were finally tested on another 144 dental meshes. All networks with our expanding layers performed better than baseline, and the best one achieved an accuracy of 0.877 both on validation dataset and test dataset.

【23】 SelfCF: A Simple Framework for Self-supervised Collaborative Filtering 标题：SelfCF：一种简单的自监督协同过滤框架

作者：Xin Zhou,Aixin Sun,Yong Liu,Jie Zhang,Chunyan Miao 机构： Miao are with the Schoolof Computer Science and Engineering 链接：https://arxiv.org/abs/2107.03019 摘要：协同过滤（CF）被广泛用于从观察到的交互中学习用户或项目的潜在信息表示。现有的基于CF的方法通常采用负抽样来区分不同的项目。即，将观察到的用户项对视为正实例；未观察到的对被视为负实例，并在定义的分布下进行抽样训练。在大数据集上使用负采样的训练在计算上是昂贵的。此外，负项应在定义的分布下仔细取样，以避免在训练数据集中选择观察到的正项。不可避免地，从训练数据集中抽取的一些负项在测试集中可能是正的。近年来，自监督学习（SSL）已成为一种学习无负样本模型的有力工具。在本文中，我们提出了一个自我监督的协同过滤框架（SelfCF），这是专为具有隐含反馈的推荐者场景而设计的。SelfCF的主要思想是增加骨干网生成的输出嵌入，因为增加用户/项目id的原始输入是不可行的。我们提出并研究了三种输出扰动技术，可应用于不同类型的主干网，包括传统的CF模型和基于图的模型。通过将两个流行的推荐模型封装到该框架中，我们在三个数据集上的实验表明，该框架的最佳性能与监督推荐模型相当或更好。我们还发现，与另一个自监督框架相比，SelfCF平均可以提高8.93%的性能。源代码位于：https://github.com/enoche/SelfCF. 摘要：Collaborative filtering (CF) is widely used to learn an informative latent representation of a user or item from observed interactions. Existing CF-based methods commonly adopt negative sampling to discriminate different items. That is, observed user-item pairs are treated as positive instances; unobserved pairs are considered as negative instances and are sampled under a defined distribution for training. Training with negative sampling on large datasets is computationally expensive. Further, negative items should be carefully sampled under the defined distribution, in order to avoid selecting an observed positive item in the training dataset. Unavoidably, some negative items sampled from the training dataset could be positive in the test set. Recently, self-supervised learning (SSL) has emerged as a powerful tool to learn a model without negative samples. In this paper, we propose a self-supervised collaborative filtering framework (SelfCF), that is specially designed for recommender scenario with implicit feedback. The main idea of SelfCF is to augment the output embeddings generated by backbone networks, because it is infeasible to augment raw input of user/item ids. We propose and study three output perturbation techniques that can be applied to different types of backbone networks including both traditional CF models and graph-based models. By encapsulating two popular recommendation models into the framework, our experiments on three datasets show that the best performance of our framework is comparable or better than the supervised counterpart. We also show that SelfCF can boost up the performance by up to 8.93\% on average, compared with another self-supervised framework as the baseline. Source codes are available at: https://github.com/enoche/SelfCF.

【24】 Exact Learning Augmented Naive Bayes Classifier 标题：精确学习增广朴素贝叶斯分类器

作者：Shouta Sugahara,Maomi Ueno 机构：Graduate school of Informatics and Engineering, The University of Electro-Communications, -,-, Chofugaoka, Chofu-shi, Tokyo, Japan, Editor: 备注：29 pages 链接：https://arxiv.org/abs/2107.03018 摘要：以往的研究表明，在给定特征变量的情况下，通过最大化一类变量的条件对数似然（CLL）得到的贝叶斯网络（BNs）的分类精度高于通过最大化边缘似然（ML）得到的分类精度。然而，在早期的研究中，这两个分数的表现之间的差异可能是由于他们使用的是近似的学习算法，而不是精确的学习算法。本文比较了用CLL近似学习和用ML精确学习的BNs分类精度，结果表明，对于大数据，最大化ML得到的BNs分类精度高于最大化CLL得到的BNs分类精度。然而，研究结果也显示，当样本量较小且类别变数有多个父变数时，使用ML的精确学习BNs的分类准确率要比其他方法差得多。为了解决这一问题，我们提出了一种精确学习的增广朴素贝叶斯分类器（ANB），它保证了类变量没有父变量。该方法保证了在精确学习的BN之后渐近估计同一类。对比实验表明，该方法具有良好的性能。 摘要：Earlier studies have shown that classification accuracies of Bayesian networks (BNs) obtained by maximizing the conditional log likelihood (CLL) of a class variable, given the feature variables, were higher than those obtained by maximizing the marginal likelihood (ML). However, differences between the performances of the two scores in the earlier studies may be attributed to the fact that they used approximate learning algorithms, not exact ones. This paper compares the classification accuracies of BNs with approximate learning using CLL to those with exact learning using ML. The results demonstrate that the classification accuracies of BNs obtained by maximizing the ML are higher than those obtained by maximizing the CLL for large data. However, the results also demonstrate that the classification accuracies of exact learning BNs using the ML are much worse than those of other methods when the sample size is small and the class variable has numerous parents. To resolve the problem, we propose an exact learning augmented naive Bayes classifier (ANB), which ensures a class variable with no parents. The proposed method is guaranteed to asymptotically estimate the identical class posterior to that of the exactly learned BN. Comparison experiments demonstrated the superior performance of the proposed method.

【25】 Evaluating the progress of Deep Reinforcement Learning in the real world: aligning domain-agnostic and domain-specific research 标题：评估真实世界中深度强化学习的进展：调整领域不可知性和领域特异性研究

作者：Juan Jose Garau-Luis,Edward Crawley,Bruce Cameron 机构： Massachusetts Institute ofTechnology 链接：https://arxiv.org/abs/2107.03015 摘要：深度强化学习（DRL）被认为是一个潜在的框架，以改善许多现实世界的自治系统；它引起了多个不同领域的关注。然而，在现实世界中的成功部署是大多数DRL模型仍然需要通过的一个考验。在这项工作中，我们通过回顾和评估来自领域不可知和领域特定社区的研究成果来关注这个问题。一方面，我们对DRL挑战进行了全面总结，并总结了缓解这些挑战的不同建议；这有助于确定领域不可知研究的五个缺口。另一方面，从特定领域的角度，我们讨论了不同的成功案例，并论证了为什么其他模型可能无法部署。最后，我们将讨论如何推进这两个角度的会计工作。 摘要：Deep Reinforcement Learning (DRL) is considered a potential framework to improve many real-world autonomous systems; it has attracted the attention of multiple and diverse fields. Nevertheless, the successful deployment in the real world is a test most of DRL models still need to pass. In this work we focus on this issue by reviewing and evaluating the research efforts from both domain-agnostic and domain-specific communities. On one hand, we offer a comprehensive summary of DRL challenges and summarize the different proposals to mitigate them; this helps identifying five gaps of domain-agnostic research. On the other hand, from the domain-specific perspective, we discuss different success stories and argue why other models might fail to be deployed. Finally, we take up on ways to move forward accounting for both perspectives.

【26】 Structured Denoising Diffusion Models in Discrete State-Spaces 标题：离散状态空间中的结构化去噪扩散模型

作者：Jacob Austin,Daniel Johnson,Jonathan Ho,Danny Tarlow,Rianne van den Berg 机构：Google Research, Brain Team 备注：10 pages plus references and appendices. First two authors contributed equally 链接：https://arxiv.org/abs/2107.03006 摘要：去噪扩散概率模型（DDPM）（Ho et al.2020）在连续状态空间中的图像和波形生成方面显示了令人印象深刻的结果。在这里，我们介绍了离散去噪扩散概率模型（D3PM），离散数据的类扩散生成模型，通过超越具有统一转移概率的腐败过程，推广了Hoogeboom等人2021年的多项式扩散模型。这包括连续空间中模仿高斯核的转移矩阵、嵌入空间中基于最近邻的矩阵以及引入吸收态的矩阵的损坏。第三种方法使我们能够在扩散模型和基于自回归和掩模的生成模型之间建立联系。我们证明了转换矩阵的选择是一个重要的设计决策，它可以改善图像和文本领域的结果。我们还引入了一个新的损失函数，它结合了变分下界和辅助交叉熵损失。对于文本，该模型类在字符级文本生成方面取得了很好的效果，同时可以扩展到LM1B上的大型词汇表。在CIFAR-10图像数据集上，我们的模型接近样本质量，超过了连续空间DDPM模型的对数似然。 摘要：Denoising diffusion probabilistic models (DDPMs) (Ho et al. 2020) have shown impressive results on image and waveform generation in continuous state spaces. Here, we introduce Discrete Denoising Diffusion Probabilistic Models (D3PMs), diffusion-like generative models for discrete data that generalize the multinomial diffusion model of Hoogeboom et al. 2021, by going beyond corruption processes with uniform transition probabilities. This includes corruption with transition matrices that mimic Gaussian kernels in continuous space, matrices based on nearest neighbors in embedding space, and matrices that introduce absorbing states. The third allows us to draw a connection between diffusion models and autoregressive and mask-based generative models. We show that the choice of transition matrix is an important design decision that leads to improved results in image and text domains. We also introduce a new loss function that combines the variational lower bound with an auxiliary cross entropy loss. For text, this model class achieves strong results on character-level text generation while scaling to large vocabularies on LM1B. On the image dataset CIFAR-10, our models approach the sample quality and exceed the log-likelihood of the continuous-space DDPM model.

【27】 Harnessing Heterogeneity: Learning from Decomposed Feedback in Bayesian Modeling 标题：利用异构性：从贝叶斯建模中分解的反馈中学习

作者：Kai Wang,Bryan Wilder,Sze-chuan Suen,Bistra Dilkina,Milind Tambe 机构：Harvard University, USA, University of Southern California, USA 链接：https://arxiv.org/abs/2107.03003 摘要：学习和优化一个由多个子组件组成的复杂系统，其中这些组件可以是代理或自主传感器，这引起了人们极大的兴趣。在这方面的丰富文献中，基于agent和特定领域的仿真可以捕获复杂的动力学和子组交互，但是在这样的仿真上进行优化在计算和算法上都具有挑战性。贝叶斯方法，如高斯过程（GPs），可以用来学习一个计算上易于处理的近似基础动力学，但通常忽略了有关复杂系统中子群的详细信息。我们试图通过提出分解反馈的思想来找到两个世界中最好的一个，它捕获了基于组的异质性和动态性。我们引入了一种新的分解GP回归方法来结合子组分解反馈。与以前的方法相比，我们的修正回归具有更低的方差，因此后验概率更准确；它还允许我们引入一个分解GP-UCB优化算法，利用子组反馈。该方法的贝叶斯性质使得优化算法在理论上具有收敛性和无遗憾性。为了证明这项工作的广泛适用性，我们在两个不同的社会问题上执行了我们的算法：异质人群中的传染病控制和分布式天气传感器的分配。实验结果表明，与现有方法相比，新方法有了显著的改进。 摘要：There is significant interest in learning and optimizing a complex system composed of multiple sub-components, where these components may be agents or autonomous sensors. Among the rich literature on this topic, agent-based and domain-specific simulations can capture complex dynamics and subgroup interaction, but optimizing over such simulations can be computationally and algorithmically challenging. Bayesian approaches, such as Gaussian processes (GPs), can be used to learn a computationally tractable approximation to the underlying dynamics but typically neglect the detailed information about subgroups in the complicated system. We attempt to find the best of both worlds by proposing the idea of decomposed feedback, which captures group-based heterogeneity and dynamics. We introduce a novel decomposed GP regression to incorporate the subgroup decomposed feedback. Our modified regression has provably lower variance -- and thus a more accurate posterior -- compared to previous approaches; it also allows us to introduce a decomposed GP-UCB optimization algorithm that leverages subgroup feedback. The Bayesian nature of our method makes the optimization algorithm trackable with a theoretical guarantee on convergence and no-regret property. To demonstrate the wide applicability of this work, we execute our algorithm on two disparate social problems: infectious disease control in a heterogeneous population and allocation of distributed weather sensors. Experimental results show that our new method provides significant improvement compared to the state-of-the-art.

【28】 Keiki: Towards Realistic Danmaku Generation via Sequential GANs 标题：Keiki：通过顺序Gans走向现实的Danmaku一代

作者：Ziqi Wang,Jialin Liu,Georgios N. Yannakakis 机构：∗ Research Institute of Trustworthy Autonomous System, Southern University of Science and Technology, †Guangdong Provincial Key Laboratory of Brain-inspired Intelligent Computation, Department of Computer Science and Engineering, Shenzhen, China 备注：This paper is accepted by the 2021 IEEE Conference on Games 链接：https://arxiv.org/abs/2107.02991 摘要：基于搜索的程序性内容生成方法最近被引入到子弹地狱游戏的自主创建中。然而，基于搜索的方法很难显式地对danmakus（子弹地狱射击实体）的模式进行建模，而且生成的级别通常看起来不现实。在本文中，我们提出了一个新的子弹地狱游戏平台Keiki，它允许将danmakus表示为一个参数序列，而这个参数序列又可以模拟danmakus的序列行为。我们采用了三种类型的生成性对抗网络（GANs），并测试了Keiki的三个指标，旨在量化生成的danmakus的质量。时间序列GAN和周期空间GAN在所采用的评价指标、它们与人类设计的danmakus的偏差以及生成的danmakus的多样性方面表现出不同的竞争性能。初步的实验研究展示了时间序列GANs在游戏连续内容生成中的潜力。 摘要：Search-based procedural content generation methods have recently been introduced for the autonomous creation of bullet hell games. Search-based methods, however, can hardly model patterns of danmakus -- the bullet hell shooting entity -- explicitly and the resulting levels often look non-realistic. In this paper, we present a novel bullet hell game platform named Keiki, which allows the representation of danmakus as a parametric sequence which, in turn, can model the sequential behaviours of danmakus. We employ three types of generative adversarial networks (GANs) and test Keiki across three metrics designed to quantify the quality of the generated danmakus. The time-series GAN and periodic spatial GAN show different yet competitive performance in terms of the evaluation metrics adopted, their deviation from human-designed danmakus, and the diversity of generated danmakus. The preliminary experimental studies presented here showcase that potential of time-series GANs for sequential content generation in games.

【29】 SpectralFormer: Rethinking Hyperspectral Image Classification with Transformers 标题：SpectralFormer：用Transformer重新思考高光谱图像分类

作者：Danfeng Hong,Zhu Han,Jing Yao,Lianru Gao,Bing Zhang,Antonio Plaza,Jocelyn Chanussot 机构： Gao are with the Key Laboratory of Digital Earth Science 链接：https://arxiv.org/abs/2107.02988 摘要：高光谱（HS）图像的特点是近似连续的光谱信息，通过捕捉细微的光谱差异，实现对物质的精细识别。卷积神经网络（CNNs）具有良好的局部上下文建模能力，已被证明是HS图像分类中一种强大的特征提取工具。然而，由于CNNs固有的网络主干的限制，CNNs不能很好地挖掘和表示光谱特征的序列属性。为了解决这个问题，我们重新考虑了HS图像分类的顺序角度与Transformer，并提出了一种新的骨干网络称为{SpectralFormer}。除了经典变换中的带式表示外，SpectralFormer还能够从HS图像的相邻带中学习光谱局部序列信息，产生分组谱嵌入。更重要的是，为了减少在逐层传播过程中丢失有价值信息的可能性，我们设计了一种跨层跳转连接，通过自适应学习跨层融合“软”残差，从浅层到深层传递类记忆成分。值得注意的是，所提出的频谱变换器是一个高度灵活的主干网络，它可以同时适用于像素和贴片输入。通过大量的实验，我们评估了所提出的频谱变换器在三个HS数据集上的分类性能，显示了它比经典Transformer的优越性，并且与最先进的骨干网络相比，取得了显著的改进。此工作的代码将在\url上提供{https://sites.google.com/view/danfeng-hong}为了再现性。 摘要：Hyperspectral (HS) images are characterized by approximately contiguous spectral information, enabling the fine identification of materials by capturing subtle spectral discrepancies. Owing to their excellent locally contextual modeling ability, convolutional neural networks (CNNs) have been proven to be a powerful feature extractor in HS image classification. However, CNNs fail to mine and represent the sequence attributes of spectral signatures well due to the limitations of their inherent network backbone. To solve this issue, we rethink HS image classification from a sequential perspective with transformers, and propose a novel backbone network called \ul{SpectralFormer}. Beyond band-wise representations in classic transformers, SpectralFormer is capable of learning spectrally local sequence information from neighboring bands of HS images, yielding group-wise spectral embeddings. More significantly, to reduce the possibility of losing valuable information in the layer-wise propagation process, we devise a cross-layer skip connection to convey memory-like components from shallow to deep layers by adaptively learning to fuse "soft" residuals across layers. It is worth noting that the proposed SpectralFormer is a highly flexible backbone network, which can be applicable to both pixel- and patch-wise inputs. We evaluate the classification performance of the proposed SpectralFormer on three HS datasets by conducting extensive experiments, showing the superiority over classic transformers and achieving a significant improvement in comparison with state-of-the-art backbone networks. The codes of this work will be available at \url{https://sites.google.com/view/danfeng-hong} for the sake of reproducibility.

【30】 RoboCup@Home Education 2020 Best Performance: RoboBreizh, a modular approach 标题：RoboCup@Home Education 2020最佳表现：RoboBreizh，模块化方法

作者：Antoine Dizet,Cédric Le Bono,Amélie Legeleux,Maëlic neau,Cédric Buche 链接：https://arxiv.org/abs/2107.02978 摘要：每年Robocup@Home比赛挑战团队和机器人的能力。2020年RoboCup@Home教育挑战赛是在网上组织的，改变了通常的竞赛规则。在本文中，我们介绍了最新的发展，导致RoboBreizh队赢得比赛。这些发展包括几个相互连接的模块，使胡椒机器人能够理解、行动并适应当地环境。最新的可用技术已用于导航和对话。第一个贡献包括结合目标检测和姿态估计技术来检测用户的意图。第二个贡献是通过演示学习来轻松学习新动作，从而提高胡椒机器人的技能。该提案荣获2020年度最佳绩效奖RoboCup@Home教育挑战。 摘要：Every year, the Robocup@Home competition challenges teams and robots' abilities. In 2020, the RoboCup@Home Education challenge was organized online, altering the usual competition rules. In this paper, we present the latest developments that lead the RoboBreizh team to win the contest. These developments include several modules linked to each other allowing the Pepper robot to understand, act and adapt itself to a local environment. Up-to-date available technologies have been used for navigation and dialogue. First contribution includes combining object detection and pose estimation techniques to detect user's intention. Second contribution involves using Learning by Demonstrations to easily learn new movements that improve the Pepper robot's skills. This proposal won the best performance award of the 2020 RoboCup@Home Education challenge.

【31】 Neural Natural Language Processing for Unstructured Data in Electronic Health Records: a Review 标题：电子病历中非结构化数据的神经自然语言处理研究进展

作者：Irene Li,Jessica Pan,Jeremy Goldwasser,Neha Verma,Wai Pan Wong,Muhammed Yavuz Nuzumlalı,Benjamin Rosand,Yixin Li,Matthew Zhang,David Chang,R. Andrew Taylor,Harlan M. Krumholz,Dragomir Radev 机构：Yale University, New Haven CT 备注：33 pages, 11 figures 链接：https://arxiv.org/abs/2107.02975 摘要：电子健康记录（ehr）是患者医疗事件和观察的数字集合，在医学中无处不在，对医疗服务、运营和研究至关重要。尽管电子病历起着核心作用，但众所周知，它很难自动处理。EHRs中存储的信息中，有一半以上是非结构化文本（例如，提供商说明、运营报告）的形式，基本上还没有开发出来供二次使用。然而，最近，新的神经网络和自然语言处理（NLP）的深度学习方法取得了长足的进步，在许多任务上都优于传统的统计和基于规则的系统。在这篇综述文章中，我们总结了目前用于EHR的神经NLP方法。我们专注于广泛的任务，即分类和预测、单词嵌入、抽取、生成，以及其他主题，如问答、表型、知识图、医学对话、多语言性、可解释性等。 摘要：Electronic health records (EHRs), digital collections of patient healthcare events and observations, are ubiquitous in medicine and critical to healthcare delivery, operations, and research. Despite this central role, EHRs are notoriously difficult to process automatically. Well over half of the information stored within EHRs is in the form of unstructured text (e.g. provider notes, operation reports) and remains largely untapped for secondary use. Recently, however, newer neural network and deep learning approaches to Natural Language Processing (NLP) have made considerable advances, outperforming traditional statistical and rule-based systems on a variety of tasks. In this survey paper, we summarize current neural NLP methods for EHR applications. We focus on a broad scope of tasks, namely, classification and prediction, word embeddings, extraction, generation, and other topics such as question answering, phenotyping, knowledge graphs, medical dialogue, multilinguality, interpretability, etc.

【32】 RAM-VO: Less is more in Visual Odometry 标题：RAM-VO：视觉里程计中的少即是多

作者：Iury Cleveston,Esther L. Colombini 机构：Laboratory of Robotics and Cognitive Systems (LaRoCS), Institute of Computing, University of Campinas, Campinas, S˜ao Paulo, Brazil 链接：https://arxiv.org/abs/2107.02974 摘要：建造能够在没有人监督的情况下运行的车辆需要确定代理人的姿势。视觉里程计（VO）算法仅利用输入图像的视觉变化来估计自我运动。最新的VO方法广泛使用卷积神经网络（CNN）来实现深度学习，这在处理高分辨率图像时增加了大量的成本。此外，在VO任务中，输入数据越多并不意味着预测效果越好；相反，架构可能会过滤掉无用的信息。因此，实现计算效率高、轻量级的体系结构至关重要。在这项工作中，我们提出了RAM-VO，一个扩展的经常性注意模型（RAM）的视觉里程计任务。RAM-VO改进了信息的视觉和时间表示，实现了近端策略优化（PPO）算法来学习鲁棒策略。结果表明，RAM-VO可以用大约300万个参数对单目输入图像进行6个自由度的回归。此外，在KITTI数据集上的实验表明，RAM-VO只使用了5.7%的可用视觉信息就获得了具有竞争力的结果。 摘要：Building vehicles capable of operating without human supervision requires the determination of the agent's pose. Visual Odometry (VO) algorithms estimate the egomotion using only visual changes from the input images. The most recent VO methods implement deep-learning techniques using convolutional neural networks (CNN) extensively, which add a substantial cost when dealing with high-resolution images. Furthermore, in VO tasks, more input data does not mean a better prediction; on the contrary, the architecture may filter out useless information. Therefore, the implementation of computationally efficient and lightweight architectures is essential. In this work, we propose the RAM-VO, an extension of the Recurrent Attention Model (RAM) for visual odometry tasks. RAM-VO improves the visual and temporal representation of information and implements the Proximal Policy Optimization (PPO) algorithm to learn robust policies. The results indicate that RAM-VO can perform regressions with six degrees of freedom from monocular input images using approximately 3 million parameters. In addition, experiments on the KITTI dataset demonstrate that RAM-VO achieves competitive results using only 5.7% of the available visual information.

【33】 Quadruped Locomotion on Non-Rigid Terrain using Reinforcement Learning 标题：基于强化学习的非刚性地形四足行走

作者：Taehei Kim,Sung-Hee Lee 机构： Our experiments show that 1Graduate School of Cultural Technology 链接：https://arxiv.org/abs/2107.02955 摘要：腿部机器人需要能够在不同的地形条件下行走。在本文中，我们提出了一个新的强化学习框架，学习运动的非刚性动态地形。具体来说，我们的框架可以在平坦的弹性地形上产生四足动物的运动，该地形由机器人脚推动时被动上下移动的瓷砖矩阵组成。一个55厘米长的训练机器人可以在下沉5厘米的地形上行走。我们提出了一套观察和奖励条款，使这一运动；我们发现在观测中加入末端效应器历史和末端效应器速度项是至关重要的。通过对不同地形条件下的机器人进行训练，验证了该方法的有效性。 摘要：Legged robots need to be capable of walking on diverse terrain conditions. In this paper, we present a novel reinforcement learning framework for learning locomotion on non-rigid dynamic terrains. Specifically, our framework can generate quadruped locomotion on flat elastic terrain that consists of a matrix of tiles moving up and down passively when pushed by the robot's feet. A trained robot with 55cm base length can walk on terrain that can sink up to 5cm. We propose a set of observation and reward terms that enable this locomotion; in which we found that it is crucial to include the end-effector history and end-effector velocity terms into observation. We show the effectiveness of our method by training the robot with various terrain conditions.

【34】 Scalable Teacher Forcing Network for Semi-Supervised Large Scale Data Streams 标题：一种可扩展的半监督大规模数据流教师强迫网络

作者：Mahardhika Pratama,Choiru Za'in,Edwin Lughofer,Eric Pardede,Dwi A. P. Rahayu 机构：A.P. Rahayue, School of Computer Science and Engineering, Nanyang Technological University, Singapore, Monash University, Australia, Department of Knowledge-Based Mathematical Systems, Johannes Kepler University, Linz, Austria 备注：None 链接：https://arxiv.org/abs/2107.02943 摘要：大规模数据流问题是指在传统的计算平台下无法以可伸缩的方式处理的高速信息流。这个问题也带来了昂贵的标签成本，使得部署完全监督算法变得不可行。另一方面，由于大多数工作是在传统的单节点计算环境中设计的，同时也是完全监督的，因此半监督大规模数据流的问题在文献中很少被探讨。本文提出了一种弱监督的可扩展教师强制网络（WeScatterNet）来同时处理标记样本和大规模数据流的不足。WeScatterNet是在apachespark分布式计算平台下构建的，采用无数据模型融合策略对并行计算后的模型进行压缩。它采用开放的网络结构来解决全局和局部漂移问题，同时集成了数据增强、注释和自动校正（$DA^3$）方法来处理部分标记的数据流。在6个大规模数据流问题中，仅以$25\%$的标签比例对WeScatterNet的性能进行了数值评估。即使与100\%$标签比例的完全监督学习者相比，它也显示出很强的竞争力。 摘要：The large-scale data stream problem refers to high-speed information flow which cannot be processed in scalable manner under a traditional computing platform. This problem also imposes expensive labelling cost making the deployment of fully supervised algorithms unfeasible. On the other hand, the problem of semi-supervised large-scale data streams is little explored in the literature because most works are designed in the traditional single-node computing environments while also being fully supervised approaches. This paper offers Weakly Supervised Scalable Teacher Forcing Network (WeScatterNet) to cope with the scarcity of labelled samples and the large-scale data streams simultaneously. WeScatterNet is crafted under distributed computing platform of Apache Spark with a data-free model fusion strategy for model compression after parallel computing stage. It features an open network structure to address the global and local drift problems while integrating a data augmentation, annotation and auto-correction ($DA^3$) method for handling partially labelled data streams. The performance of WeScatterNet is numerically evaluated in the six large-scale data stream problems with only $25\%$ label proportions. It shows highly competitive performance even if compared with fully supervised learners with $100\%$ label proportions.

【35】 Supervised Bayesian Specification Inference from Demonstrations 标题：基于示例的有监督贝叶斯规范推理

作者：Ankit Shah,Pritish Kamath,Shen Li,Patrick Craven,Kevin Landers,Kevin Oden,Julie Shah 机构：Massachusetts Institute of Technology 链接：https://arxiv.org/abs/2107.02912 摘要：当观察任务演示时，人类学徒能够在获得实际执行任务的专业知识之前，识别给定任务是否正确执行。先前关于从示范中学习（LfD）的研究未能抓住任务执行的可接受性这一概念；同时，时态逻辑为任务规范的表达提供了一种灵活的语言。受此启发，我们提出了贝叶斯规范推理，一种将任务规范作为时序逻辑公式进行推理的概率模型。我们结合了概率规划的方法来定义我们的先验，以及一个独立于领域的似然函数来实现基于抽样的推理。我们证明了我们的模型用于推断规范的有效性，在合成域和实际表格设置任务中，推断规范和基本事实之间的相似度超过90%。 摘要：When observing task demonstrations, human apprentices are able to identify whether a given task is executed correctly long before they gain expertise in actually performing that task. Prior research into learning from demonstrations (LfD) has failed to capture this notion of the acceptability of a task's execution; meanwhile, temporal logics provide a flexible language for expressing task specifications. Inspired by this, we present Bayesian specification inference, a probabilistic model for inferring task specification as a temporal logic formula. We incorporate methods from probabilistic programming to define our priors, along with a domain-independent likelihood function to enable sampling-based inference. We demonstrate the efficacy of our model for inferring specifications, with over 90% similarity observed between the inferred specification and the ground truth, both within a synthetic domain and during a real-world table setting task.

【36】 Question Answering over Knowledge Graphs with Neural Machine Translation and Entity Linking 标题：具有神经机器翻译和实体链接的知识图问答

作者：Daniel Diomedi,Aidan Hogan 机构：DCC, Universidad de Chile; IMFD 链接：https://arxiv.org/abs/2107.02865 摘要：知识图问答（KGQA）的目标是在知识图上寻找自然语言问题的答案。最近的KGQA方法采用神经机器翻译（NMT）的方法，其中自然语言问题被翻译成结构化查询语言。然而，NMT面临着词汇表外的问题，即在训练过程中可能看不到问题中的术语，从而妨碍了它们的翻译。对于大型知识图所描述的数百万实体来说，这个问题尤其成问题。我们更倾向于提出一种KGQA方法，将实体的处理委托给实体链接（EL）系统。然后使用NMT创建一个查询模板，其中占位符由EL阶段中标识的实体填充。槽填充用于决定哪个实体填充哪个占位符。在Wikidata上进行的QA实验表明，我们的方法优于纯NMT：虽然在训练过程中仍然强烈依赖于看到过相似的查询模板，但是与实体相关的错误大大减少。 摘要：The goal of Question Answering over Knowledge Graphs (KGQA) is to find answers for natural language questions over a knowledge graph. Recent KGQA approaches adopt a neural machine translation (NMT) approach, where the natural language question is translated into a structured query language. However, NMT suffers from the out-of-vocabulary problem, where terms in a question may not have been seen during training, impeding their translation. This issue is particularly problematic for the millions of entities that large knowledge graphs describe. We rather propose a KGQA approach that delegates the processing of entities to entity linking (EL) systems. NMT is then used to create a query template with placeholders that are filled by entities identified in an EL phase. Slot filling is used to decide which entity fills which placeholder. Experiments for QA over Wikidata show that our approach outperforms pure NMT: while there remains a strong dependence on having seen similar query templates during training, errors relating to entities are greatly reduced.

【37】 Immuno-mimetic Deep Neural Networks (Immuno-Net) 标题：免疫仿生深度神经网络(Immuno-Net)

作者：Ren Wang,Tianqi Chen,Stephen Lindsly,Cooper Stansbury,Indika Rajapakse,Alfred Hero 机构： 1University of Michigan 链接：https://arxiv.org/abs/2107.02842 摘要：仿生学在人工神经网络的进化中起着关键的作用。到目前为止，电子隐喻已经被神经科学和认知心理学的概念所支配。在本文中，我们介绍了一种不同类型的仿生模型，一种借用免疫系统的概念，用于设计健壮的深层神经网络。这种免疫模拟模型为深层神经网络对抗对抗性攻击提供了一个新的计算生物学框架。在这个免疫网络框架中，我们定义了一个健壮的适应性免疫启发学习系统（Immuno-Net-RAILS），它在硅片中模拟了B细胞的适应性生物学机制，用于保护哺乳动物宿主免受致病性攻击。当应用于在基准数据集上的图像分类任务时，我们证明了免疫网络RAILS在基线方法（DkNN鲁棒CNN）的对抗性精度方面提高了12.5%，而在干净数据上没有明显的精度损失。 摘要：Biomimetics has played a key role in the evolution of artificial neural networks. Thus far, in silico metaphors have been dominated by concepts from neuroscience and cognitive psychology. In this paper we introduce a different type of biomimetic model, one that borrows concepts from the immune system, for designing robust deep neural networks. This immuno-mimetic model leads to a new computational biology framework for robustification of deep neural networks against adversarial attacks. Within this Immuno-Net framework we define a robust adaptive immune-inspired learning system (Immuno-Net RAILS) that emulates, in silico, the adaptive biological mechanisms of B-cells that are used to defend a mammalian host against pathogenic attacks. When applied to image classification tasks on benchmark datasets, we demonstrate that Immuno-net RAILS results in improvement of as much as 12.5% in adversarial accuracy of a baseline method, the DkNN-robustified CNN, without appreciable loss of accuracy on clean data.

【38】 Neural Contextual Bandits without Regret 标题：无怨无悔的神经情境性强盗

作者：Parnian Kassraie,Andreas Krause 机构：ETH Zurich 备注：37 pages, 6 figures 链接：https://arxiv.org/abs/2107.03144 摘要：上下文盗贼是一个丰富的模型，为顺序决策给定的边信息，具有重要的应用，如在推荐系统。提出了一种利用神经网络逼近未知奖赏函数的新算法。我们解决了在这种情况下证明一般上下文序列的次线性遗憾界的公开问题，同时考虑了完全连通网络和卷积网络。为此，我们首先分析了一种基于神经切线核（NTK）的核化bandit优化算法NTK-UCB，并以NTK最大信息增益$\gamma\u T$这一反映学习困难的复杂度参数来界定其遗憾。我们对NTK的$\gamma\u T$的边界可能有独立的兴趣。然后介绍了基于神经网络的算法NN-UCB，并证明了该算法能很好地跟踪NTK-UCB算法。在关于奖励函数的广泛的非参数假设下，我们的方法在$\tilde{\mathcal{O}（T^{-1/2d}）$率下收敛到最优策略，其中$d$是上下文的维度。 摘要：Contextual bandits are a rich model for sequential decision making given side information, with important applications, e.g., in recommender systems. We propose novel algorithms for contextual bandits harnessing neural networks to approximate the unknown reward function. We resolve the open problem of proving sublinear regret bounds in this setting for general context sequences, considering both fully-connected and convolutional networks. To this end, we first analyze NTK-UCB, a kernelized bandit optimization algorithm employing the Neural Tangent Kernel (NTK), and bound its regret in terms of the NTK maximum information gain $\gamma_T$, a complexity parameter capturing the difficulty of learning. Our bounds on $\gamma_T$ for the NTK may be of independent interest. We then introduce our neural network based algorithm NN-UCB, and show that its regret closely tracks that of NTK-UCB. Under broad non-parametric assumptions about the reward function, our approach converges to the optimal policy at a $\tilde{\mathcal{O}}(T^{-1/2d})$ rate, where $d$ is the dimension of the context.

【39】 Identification and validation of Triamcinolone and Gallopamil as treatments for early COVID-19 via an in silico repurposing pipeline 标题：通过硅胶改用管道鉴定和验证曲安奈德和加洛帕米治疗早期冠状病毒的有效性

作者：Méabh MacMahon,Woochang Hwang,Soorin Yim,Eoghan MacMahon,Alexandre Abraham,Justin Barton,Mukunthan Tharmakulasingam,Paul Bilokon,Vasanthi Priyadarshini Gaddi,Namshik Han 机构：Affiliations, a. Milner Therapeutics Institute, University of Cambridge, Cambridge, UK, b. Centre for Therapeutics Discovery, LifeArc, Stevenage, UK, c. Department of Bio and Brain Engineering, KAIST, Daejeon, Republic of Korea 备注：32 pages, 4 figures 链接：https://arxiv.org/abs/2107.02905 摘要：SARS-CoV-2是COVID-19的致病病毒，它继续在全球范围内引起大流行。治疗轻度和重度COVID-19仍然需要药物。药物再利用提供了一个机会，比开发新的治疗方法更快地部署COVID-19药物。一些现有的药物在临床试验中显示了治疗COVID-19的前景。这项硅内研究利用与临床试验药物的结构相似性来确定两种可能应用于治疗早期COVID-19的药物。我们应用硅内验证来提出两者可能的作用机制。曲安奈德是一种结构类似地塞米松的皮质类固醇。加洛帕米是一种钙通道阻滞剂，结构与维拉帕米相似。我们认为这两种药物都可以用于治疗早期COVID-19感染，因为它们在SARS-CoV-2诱导的蛋白-蛋白相互作用网络中的靶点与早期感染活跃的激酶以及与COVID-19传播相关的APOA1蛋白接近。 摘要：SARS-CoV-2, the causative virus of COVID-19 continues to cause an ongoing global pandemic. Therapeutics are still needed to treat mild and severe COVID-19. Drug repurposing provides an opportunity to deploy drugs for COVID-19 more rapidly than developing novel therapeutics. Some existing drugs have shown promise for treating COVID-19 in clinical trials. This in silico study uses structural similarity to clinical trial drugs to identify two drugs with potential applications to treat early COVID-19. We apply in silico validation to suggest a possible mechanism of action for both. Triamcinolone is a corticosteroid structurally similar to Dexamethasone. Gallopamil is a calcium channel blocker structurally similar to Verapamil. We propose that both these drugs could be useful to treat early COVID-19 infection due to the proximity of their targets within a SARS-CoV-2-induced protein-protein interaction network to kinases active in early infection, and the APOA1 protein which is linked to the spread of COVID-19.