Neuroscience has focused on the detailed implementation of computation, studying neural codes, dynamics and circuits. In machine learning, however, artificial neural networks tend to eschew precisely designed codes, dynamics or circuits in favor of brute force optimization of a cost function, often using simple and relatively uniform initial architectures.
神经科学和人工神经网络都关注神经元/人工神经元的编码、活性和神经网络
Two recent developments have emerged within machine learning that create an opportunity to connect these seemingly divergent perspectives.
两方面现在出现融合的现象
First,structured architectures are used, including dedicated systems for attention, recursion and various forms of short- and long-term memory storage.
出现架构方面的结构:注意力模型、LSTM
Second, cost functions and training procedures have become more complex and are varied across layers and over time. Here we think about the brain in terms of these ideas.
损失函数和训练过程变得更加复杂
We hypothesize that (1) the brain optimizes cost functions, (2) the cost functions are diverse and differ across brain locations and over development, and (3) optimization operates within a pre-structured architecture matched to the computational problems posed by behavior.
假设:1 大脑也优化损失函数 2 损失函数在大脑不同区域和时期都不同 3 优化操作是在针对行为计算问题的预定结构上实现的
In support of these hypotheses, we argue that a range of implementations of credit assignment through multiple layers of neurons are compatible with our current knowledge of neural circuitry, and that the brain’s specialized systems can be interpreted as enabling efficient optimization for specific problem classes.
大脑可以解释为是针对特定问题领域的高效优化器
Such a heterogeneously optimized system, enabled by a series of interacting cost functions, serves to make learning data-efficient and precisely targeted to the needs of the organism. We suggest directions by which neuroscience could seek to refine and test these hypotheses.
内容目录如下:
Putative differences between conventional and brain-like neural network designs. (A) In conventional deep learning, supervised training is based on externally-supplied, labeled data. (B) In the brain, supervised training of networks can still occur via ...
2.3.2. Learning in the cortical sheet
2.3.3. One-shot learning
Human learning is often one-shot
Human learning is often active and deliberate
多感知器官信息互为监督进行监督学习。
4.2.2. Buffers
4.2.3. Discrete gating of information flow between buffers
4.3.2. Hierarchical control
Importantly, many of the control problems we appear to be solving are hierarchical.
Spatial planning requires solving shortest-path problems subject to constraints.
Language and reasoning appear to present a problem for neural networks (Minsky, 1991; Marcus, 2001; Hadley, 2009):
语言单词和底层物理等认知的绑定
Fixed, static hierarchies (e.g., the hierarchical organization of cortical areas Felleman and Van Essen, 1991) only take us so far:
Humans excel at stitching together sub-actions to form larger actions (Verwey, 1996; Acuna et al., 2014; Sejnowski and Poizner, 2014). Structured, serial, hierarchical probabilistic programs
原文 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5021692/#fn0037 请阅读原文
本文由zdx3578推荐。