真实大脑中的神经元是极其复杂的计算单元。除此之外，它们还负责将输入的电化学向量转化为输出的动作电位，更新中间突触的强度，调节自身的内部状态，并调节附近其他神经元的行为。有人可能会说，这些细胞是唯一显示出真正智慧的东西。因此，奇怪的是，机器学习社区这么长时间以来一直依赖于这样一种假设，即这种复杂性可以简化为简单的求和和火运算。我们问，在人工系统中大幅增加单个神经元的计算能力是否会有一些好处？为了回答这个问题，我们引入了深度人工神经元(DANs)，它们本身被实现为深度神经网络。从概念上来说，我们在传统神经网络的每个节点中嵌入DANs，并在多个突触位点连接这些神经元，从而矢量化成对细胞之间的连接。我们证明了元学习单个参数向量是可能的，我们称之为神经元表型，由网络中的所有DAN共享，这有助于部署期间的元目标。在这里，我们将持续学习作为我们的元目标，我们表明，一个合适的神经元表型可以赋予一个单一的网络以与生俱来的能力，以最小的遗忘更新其突触，使用标准的反向传播，没有经验重放，也没有单独的唤醒/睡眠阶段。我们在顺序非线性回归任务中证明了这种能力。

原文题目：Continual Learning with Deep Artificial Neurons

原文：Neurons in real brains are enormously complex computational units. Among other things, they’re responsible for transforming inbound electro-chemical vectors into outbound action potentials, updating the strengths of intermediate synapses, regulating their own internal states, and modulating the behavior of other nearby neurons. One could argue that these cells are the only things exhibiting any semblance of real intelligence. It is odd, therefore, that the machine learning community has, for so long, relied upon the assumption that this complexity can be reduced to a simple sum and fire operation. We ask, might there be some benefit to substantially increasing the computational power of individual neurons in artificial systems? To answer this question, we introduce Deep Artificial Neurons (DANs), which are themselves realized as deep neural networks. Conceptually, we embed DANs inside each node of a traditional neural network, and we connect these neurons at multiple synaptic sites, thereby vectorizing the connections between pairs of cells. We demonstrate that it is possible to meta-learn a single parameter vector, which we dub a neuronal phenotype, shared by all DANs in the network, which facilitates a meta-objective during deployment. Here, we isolate continual learning as our meta-objective, and we show that a suitable neuronal phenotype can endow a single network with an innate ability to update its synapses with minimal forgetting, using standard backpropagation, without experience replay, nor separate wake/sleep phases. We demonstrate this ability on sequential non-linear regression tasks.

原文作者：Blake Camp, Jaya Krishna Mandivarapu, Rolando Estrada

原文地址:https://arxiv.org/abs/2011.07035