ICLR 2017 特稿：谷歌和 Facebook 亮眼表现盘点
【新智元导读】备受瞩目的深度学习&机器学习顶会 ICLR 2017 本周召开。新智元此前全面报道了 ICLR 的最佳论文及议程：ICLR 2017 最佳论文出炉，机器自主编程再称雄。这次我们将介绍谷歌和 Facebook 两家顶级实验室在 ICLR 的表现
本周 24 日到 26 日，素有深度学习届顶会“无冕之王”之称的第五届国际学习表征会议（ICLR 2017）在法国土伦举行。
ICLR 关注如何学习对于机器学习重要有用的数据，虽然今年只是第 5 届，却已经是业内顶尖的机器学习会议。ICLR 汇集了顶级的人工智能和机器学习专家，讨论如何最有效地学习对视觉、语音、音频和自然语言处理等应用领域有意义和有用的数据表征。
本届 ICLR 一共收到了大约 500 篇论文。与上届的最大不同在于，本届 ICLR 论文评审采用了 OpenReview 机制，也即全部公开透明，评论者可以匿名。在 ICLR 官网，不仅将被接收的论文列了出来（按口头汇报、海报展示和研讨会讨论 3 种），被拒绝的论文也全部公布。具体的评审、讨论，以及作者回复全都可以在 OpenReview 上查看：https://openreview.net/group?id=ICLR.cc/2017/conference
这样做的目标是提高整体审核流程的质量。组委会介绍，通过使用 OpenReview，作者可以随时更新他们的论文回复评论。此外，社区中的任何人都可以对提交内容进行评论，审核人员可以利用公众讨论来提高他们对论文的理解和评价。
ICLR 创始人 Yoshua Bengio 和 Yann LeCun 担任 General Chair。Facebook 的 Marc' Aurelio Ranzato 是 Senior Program Chair。谷歌 Tara Sainath、Google DeepMind Oriol Vinyals，以及 Google Brain 蒙特利尔分部负责人 Hugo Larochelle 担任 Program Chair。
最佳论文是由组委会决定——当然，其中一篇最佳论文也带来了一定的争议。不过，即使被拒绝的论文，如果想法创新，也会受邀在研讨会 Track 做讨论。
ICLR 会场盛况。图片来源：Twitter 用户 Abtin Setyani
谷歌在本届 ICLR：50 多人参会，2 篇最佳论文，白金赞助商
一如既往，Google Research 官方博客介绍了 Google 团队在 ICLR 的活动，而作者正是刚刚从 OpenAI 回到 Google Brain 的 Ian Goodfellow。据悉，Google 在本届 ICLR 参加的活动包括会议和研讨会，有特邀的讲者发言，还有关于深度学习、指标学习（metric learning）、核学习（kernel learning）、组合模型（compositional model）、非线性结构预测（non-linear structured prediction）和非凸优化问题优化（non-convex optimization）的一些最新研究的口头报告和海报展示。
Ian Goodfellow 介绍，在神经网络和深度学习领域尖端技术创新的前沿，Google 专注于理论和应用，开发学习方法理解并推广上述技术。作为 ICLR 2017 的白金赞助商，今年 Google 有超过 50 名研究人员（许多来自 Google Brain 团队和 Google Research 欧洲团队）出席，提交了多篇论文并被接收，包括口头汇报（三篇最佳论文中的两篇）和海报展示。
Google 也组织并参与了多个研讨会。其中，George Dahl, Slav Petrov, Vikas Sindhwani 担任区域主席（Area Chair）；Hugo Larochelle 和 Tara Sainath 是担任了程序主席（Program Chair）。
以下就是 Google 在这次 ICLR 上的论文，* 表示作者在 Google 工作/实习期间参与完成† 表示作者在 OpenAI 时完成：
- Contrbuted Talks
【最佳论文】理解深度学习需要重新思考泛化
Understanding Deep Learning Requires Rethinking Generalization
Chiyuan Zhang*, Samy Bengio, Moritz Hardt, Benjamin Recht*, Oriol Vinyals
最佳论文《理解深度学习需要重新思考泛化》的一作 Chiyuan Zhang 演讲，这篇论文是他在 Google 实习时完成的工作。Chiyuan Zhang 用这张PPT 展示了他眼中的一些视觉卷积神经网络。图片来源：Twitter 用户 Edward Grefenstette
Zhang 的PPT 都十分生动形象，赢得了现场很多好评。图片来源：Twitter 用户 Douglas Gray
【最佳论文】使用私密训练数据的半监督知识迁移深度学习
Semi-Supervised Knowledge Transfer for Deep Learning from Private Training Data
Nicolas Papernot*, Martín Abadi, Úlfar Erlingsson, Ian Goodfellow, Kunal Talwar
Ian Goodfellow 和他的合著者在现场免费发放他们获最佳论文的 FATE-G（Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data）T 恤
Q-Prop: Sample-Efficient Policy Gradient with An Off-Policy Critic
Shixiang (Shane) Gu*, Timothy Lillicrap, Zoubin Ghahramani, Richard E.Turner, Sergey Levine
Neural Architecture Search with Reinforcement Learning
Barret Zoph, Quoc Le
- 海报展示
Adversarial Machine Learning at Scale
Alexey Kurakin, Ian J. Goodfellow†, Samy Bengio
Capacity and Trainability in Recurrent Neural Networks
Jasmine Collins, Jascha Sohl-Dickstein, David Sussillo
Improving Policy Gradient by Exploring Under-Appreciated Rewards
Ofir Nachum, Mohammad Norouzi, Dale Schuurmans
Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, Jeff Dean
Unrolled Generative Adversarial Networks
Luke Metz, Ben Poole*, David Pfau, Jascha Sohl-Dickstein
Categorical Reparameterization with Gumbel-Softmax
Eric Jang, Shixiang (Shane) Gu*, Ben Poole*
Decomposing Motion and Content for Natural Video Sequence Prediction
Ruben Villegas, Jimei Yang, Seunghoon Hong, Xunyu Lin, Honglak Lee
Density Estimation Using Real NVP
Laurent Dinh*, Jascha Sohl-Dickstein, Samy Bengio
Latent Sequence Decompositions
William Chan*, Yu Zhang*, Quoc Le, Navdeep Jaitly*
Learning a Natural Language Interface with Neural Programmer
Arvind Neelakantan*, Quoc V. Le, Martín Abadi, Andrew McCallum*, Dario
Amodei*
Deep Information Propagation
Samuel Schoenholz, Justin Gilmer, Surya Ganguli, Jascha Sohl-Dickstein
Identity Matters in Deep Learning
Moritz Hardt, Tengyu Ma
A Learned Representation For Artistic Style
Vincent Dumoulin*, Jonathon Shlens, Manjunath Kudlur
Adversarial Training Methods for Semi-Supervised Text Classification
Takeru Miyato, Andrew M. Dai, Ian Goodfellow†
HyperNetworks
David Ha, Andrew Dai, Quoc V. Le
Learning to Remember Rare Events
Lukasz Kaiser, Ofir Nachum, Aurko Roy*, Samy Bengio
- 研讨会论文
Particle Value Functions
Chris J. Maddison, Dieterich Lawson, George Tucker, Nicolas Heess, Arnaud Doucet, Andriy Mnih, Yee Whye Teh
Neural Combinatorial Optimization with Reinforcement Learning
Irwan Bello, Hieu Pham, Quoc V. Le, Mohammad Norouzi, Samy Bengio
Short and Deep: Sketching and Neural Networks
Amit Daniely, Nevena Lazic, Yoram Singer, Kunal Talwar
Explaining the Learning Dynamics of Direct Feedback Alignment
Justin Gilmer, Colin Raffel, Samuel S. Schoenholz, Maithra Raghu, Jascha Sohl-Dickstein
Training a Subsampling Mechanism in Expectation
Colin Raffel, Dieterich Lawson
Tuning Recurrent Neural Networks with Reinforcement Learning
Natasha Jaques*, Shixiang (Shane) Gu*, Richard E. Turner, Douglas Eck
REBAR: Low-Variance, Unbiased Gradient Estimates for Discrete Latent Variable Models
George Tucker, Andriy Mnih, Chris J. Maddison, Jascha Sohl-Dickstein
Adversarial Examples in the Physical World
Alexey Kurakin, Ian Goodfellow†, Samy Bengio
Regularizing Neural Networks by Penalizing Confident Output Distributions
Gabriel Pereyra, George Tucker, Jan Chorowski, Lukasz Kaiser, Geoffrey Hinton
Unsupervised Perceptual Rewards for Imitation Learning
Pierre Sermanet, Kelvin Xu, Sergey Levine
Changing Model Behavior at Test-time Using Reinforcement Learning
Augustus Odena, Dieterich Lawson, Christopher Olah
Facebook 在本届 ICLR：18 篇论文，海报展示和研讨会
Facebook Research 官方博客也将他们在 ICLR 上的成果展现了出来。
Facebook 会参加本届 ICLR 的 18 场会议和研讨会，分享他们的最新研究。值得一提的是，知乎上非常活跃的大牛，我们熟悉的田渊栋老师有 2 篇论文被接收（一共提交了 3 篇论文）。
Facebook 人工智能实验室（FAIR）在本届 ICLR 提交的论文如下：
An Analytical Formula of Population Gradient for Two-Layered ReLU network and its Applications in Convergence and Critical Point Analysis
Yuandong Tian
Automatic Rule Extraction from Long Short Term Memory Networks
James Murdoch and Arthur Szlam
CommAI: Evaluating the Frst Steps Towards a Useful General AI
Marco Baroni, Armand Joulin, Allan Jabri, Germaan Kruszewski, Angeliki Lazaridou, Klemen Simonic, and Tomas Mikolov
Dialogue Learning With Human-in-the-Loop
Jiwei Li, Alexander H. Miller, Sumit Chopra, Marc’Aurelio Ranzato, and Jason Weston
DSD: Dense-Sparse-Dense Training for Deep Neural Networks
（原文没有给出作者和论文链接）
Efficient Softmax Approximation for GPUs
Édouard Grave, Armand Joulin, Moustapha Cissé, David Grangier, and Hervé Jégou
Episodic Exploration for Deep Deterministic Policies for StarCraft Micromanagement
Nicolas Usunier, Gabriel Synnaeve, Zeming Lin, and Soumith Chintala
Improving Neural Language Models with a Continuous Cache
Edouard Grave, Armand Joulin, and Nicolas Usunier
Learning End-to-End Goal-Oriented Dialog
Antoine Bordes, Y-Lan Boureau, and Jason Weston
Learning Through Dialogue Interactions by Asking Questions
Jiwei Li, Alexander H. Miller, Sumit Chopra, Marc’Aurelio Ranzato, and Jason Weston
LR-GAN: Layered Recursive Generative Adversarial Networks for Image Generation
Jianwei Yang, Anitha Kannan, Dhruv Batra, and Devi Parikh
Multi-Agent Cooperation and the Emergence of (Natural) Language
Angeliki Lazaridou, Alexander Peysakhovich, and Marco Baroni
Revisiting Classifier Two-Sample Tests
David Lopez-Paz and Maxime Oquab
Towards Principled Methods for Training Generative Adversarial Networks
Martin Arjovsky and Leon Bottou
Tracking the World State with Recurrent Entity Networks
Mikael Henaff, Jason Weston, Arthur Szlam, Antoine Bordes, and Yann LeCun
Training Agent for First-Person Shooter Game with Actor-Critic Curriculum Learning
Yuxin Wu and Yuandong Tian
Unsupervised Cross-Domain Image Generation
Yaniv Taigman, Adam Polyak, and Lior Wolf
Variable Computation in Recurrent Neural Networks
Yacine Jernite, Edouard Grave, Armand Joulin, and Tomas Mikolov
ICLR 2017 具体议程及最佳论文深入报道
关于 ICLR 2017 日程及最佳论文的详细介绍，包括三篇最佳论文的相关讨论及作者回复，请看新智元此前的报道：
