深度神经网络可解释性方法汇总,附Tensorflow代码实现


新智元推荐

编辑:元子

【新智元导读】理解神经网络:人们一直觉得深度学习可解释性较弱。然而,理解神经网络的研究一直也没有停止过,本文就来介绍几种神经网络的可解释性方法,并配有能够在Jupyter下运行的代码连接。来新智元 AI 朋友圈和AI大咖们一起讨论吧。

理解神经网络:人们一直觉得深度学习可解释性较弱。然而,理解神经网络的研究一直也没有停止过,本文就来介绍几种神经网络的可解释性方法,并配有能够在Jupyter下运行的代码连接。

Activation Maximization

通过激活最化来解释深度神经网络的方法一共有两种,具体如下:

1.1 Activation Maximization (AM)

相关代码如下:

http://nbviewer.jupyter.org/github/1202kbs/Understanding-NN/blob/master/1.1%20Activation%20Maximization.ipynb

1.2 Performing AM in Code Space

相关代码如下:

http://nbviewer.jupyter.org/github/1202kbs/Understanding-NN/blob/master/1.3%20Performing%20AM%20in%20Code%20Space.ipynb

Layer-wise Relevance Propagation

层方向的关联传播,一共有5种可解释方法。Sensitivity Analysis、Simple Taylor Decomposition、Layer-wise Relevance Propagation、Deep Taylor Decomposition、DeepLIFT。它们的处理方法是:先通过敏感性分析引入关联分数的概念,利用简单的Taylor Decomposition探索基本的关联分解,进而建立各种分层的关联传播方法。具体如下:

2.1 Sensitivity Analysis

相关代码如下:

http://nbviewer.jupyter.org/github/1202kbs/Understanding-NN/blob/master/2.1%20Sensitivity%20Analysis.ipynb

2.2 Simple Taylor Decomposition

相关代码如下:

http://nbviewer.jupyter.org/github/1202kbs/Understanding-NN/blob/master/2.2%20Simple%20Taylor%20Decomposition.ipynb

2.3 Layer-wise Relevance Propagation

相关代码如下:

http://nbviewer.jupyter.org/github/1202kbs/Understanding-NN/blob/master/2.3%20Layer-wise%20Relevance%20Propagation%20%281%29.ipynb

http://nbviewer.jupyter.org/github/1202kbs/Understanding-NN/blob/master/2.3%20Layer-wise%20Relevance%20Propagation%20%282%29.ipynb

2.4 Deep Taylor Decomposition

相关代码如下:

http://nbviewer.jupyter.org/github/1202kbs/Understanding-NN/blob/master/2.4%20Deep%20Taylor%20Decomposition%20%281%29.ipynb

http://nbviewer.jupyter.org/github/1202kbs/Understanding-NN/blob/master/2.4%20Deep%20Taylor%20Decomposition%20%282%29.ipynb

2.5 DeepLIFT

相关代码如下:

http://nbviewer.jupyter.org/github/1202kbs/Understanding-NN/blob/master/2.5%20DeepLIFT.ipynb

Gradient Based Methods

基于梯度的方法有:反卷积、反向传播, 引导反向传播,积分梯度和平滑梯度这几种。具体可以参考如下链接:

https://github.com/1202kbs/Understanding-NN/blob/master/models/grad.py

详细信息如下:

3.1 Deconvolution

相关代码如下:

http://nbviewer.jupyter.org/github/1202kbs/Understanding-NN/blob/master/3.1%20Deconvolution.ipynb

3.2 Backpropagation

相关代码如下:

http://nbviewer.jupyter.org/github/1202kbs/Understanding-NN/blob/master/3.2%20Backpropagation.ipynb

3.3 Guided Backpropagation

相关代码如下:

http://nbviewer.jupyter.org/github/1202kbs/Understanding-NN/blob/master/3.3%20Guided%20Backpropagation.ipynb

3.4 Integrated Gradients

相关代码如下:

http://nbviewer.jupyter.org/github/1202kbs/Understanding-NN/blob/master/3.4%20Integrated%20Gradients.ipynb

3.5 SmoothGrad

相关代码如下:

http://nbviewer.jupyter.org/github/1202kbs/Understanding-NN/blob/master/3.5%20SmoothGrad.ipynb

Class Activation Map

类激活映射的方法有3种,分别为:Class Activation Map、Grad-CAM、 Grad-CAM++。在MNIST上的代码可以参考:

https://github.com/deepmind/mnist-cluttered

每种方法的详细信息如下:

4.1 Class Activation Map

相关代码如下:

http://nbviewer.jupyter.org/github/1202kbs/Understanding-NN/blob/master/4.1%20CAM.ipynb

4.2 Grad-CAM

相关代码如下:

http://nbviewer.jupyter.org/github/1202kbs/Understanding-NN/blob/master/4.2%20Grad-CAM.ipynb

4.3 Grad-CAM++

相关代码如下:

http://nbviewer.jupyter.org/github/1202kbs/Understanding-NN/blob/master/4.3%20Grad-CAM-PP.ipynb

Quantifying Explanation Quality

虽然每一种解释技术都基于其自身的直觉或数学原理,但在更抽象的层次上确定好解释的特征并能够定量地测试这些特征也很重要。这里再推荐两种基于质量和评价的可解释性方法。具体如下:

5.1 Explanation Continuity

相关代码如下:

http://nbviewer.jupyter.org/github/1202kbs/Understanding-NN/blob/master/5.1%20Explanation%20Continuity.ipynb

5.2 Explanation Selectivity

相关代码如下:

http://nbviewer.jupyter.org/github/1202kbs/Understanding-NN/blob/master/5.2%20Explanation%20Selectivity.ipynb

参考文献

Sections 1.1 ~ 2.2 and 5.1 ~ 5.2

[1] Montavon, G., Samek, W., Müller, K., jun 2017. Methods for Interpreting and Understanding Deep Neural Networks. arXiv preprint arXiv:1706.07979, 2017.

Section 1.3

[2] Nguyen, A., Dosovitskiy, A., Yosinski, J., Brox, T., Clune, J., 2016. Synthesizing the preferred inputs for neurons in neural networks via deep generator networks. In: Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, December 5-10, 2016, Barcelona, Spain. pp. 3387-3395.

[3] A. Dosovitskiy and T. Brox. Generating images with perceptual similarity metrics based on deep networks. In NIPS, 2016.

Section 2.3

[4] Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K.R., Samek, W., 07 2015. On pixel-wise explanations for non-linear classi er decisions by layer-wise relevance propagation. PLOS ONE 10 (7), 1-46.

Section 2.4

[5] Montavon, G., Lapuschkin, S., Binder, A., Samek, W., Müller, K.R., 2017. Explaining nonlinear classi cation decisions with deep Taylor decomposition. Pattern Recognition 65, 211-222.

Section 2.5

[6] Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. Learning Important Features Through Propagating Activation Differences. arXiv preprint arXiv:1704.02685, 2017.

Section 3.1

[7] Zeiler, M. D., Fergus, R., 2014. Visualizing and understanding convolutional networks. In: Computer Vision - ECCV 2014 - 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part I. pp. 818-833.

Section 3.2

[8] K. Simonyan, A. Vedaldi, and A. Zisserman. Deep inside convolutional networks: Visualising image classification models and saliency maps. In Workshop at International Conference on Learning Representations, 2014.

Section 3.3

[9] Jost Tobias Springenberg, Alexey Dosovitskiy, Thomas Brox, and Martin Riedmiller. Striving for simplicity: The all convolutional net. arXiv preprint arXiv:1412.6806, 2014.

Section 3.4

[10] Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. arXiv preprint arXiv:1703.01365, 2017.

Section 3.5

[11] Daniel Smilkov, Nikhil Thorat, Been Kim, Fernanda Viégas, and Martin Wattenberg. SmoothGrad: removing noise by adding noise. arXiv preprint arXiv:1706.03825, 2017.

Section 4.1

[12] Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. Learning deep features for discriminative localization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2921–2929, 2016.

Section 4.2

[13] R. R.Selvaraju, A. Das, R. Vedantam, M. Cogswell, D. Parikh, and D. Batra. Grad-cam: Why did you say that? visual explanations from deep networks via gradient-based localization. arXiv:1611.01646, 2016.

Section 4.3

[14] A. Chattopadhyay, A. Sarkar, P. Howlader, and V. N. Balasubramanian. Grad-cam++: Generalized gradient-based visual explanations for deep convolutional networks. CoRR, abs/1710.11063, 2017.

本文分享自微信公众号 - 新智元(AI_era)

原文出处及转载信息见文内详细说明,如有侵权,请联系 yunjia_community@tencent.com 删除。

原始发表时间:2019-11-07

本文参与腾讯云自媒体分享计划,欢迎正在阅读的你也加入,一起分享。

发表于

我来说两句

0 条评论
登录 后参与评论

相关文章

来自专栏深度学习和计算机视觉

ROS kinetic 下单目摄像机的标定

https://github.com/bosch-ros-pkg/usb_cam

11230
来自专栏码洞

每个阿里程序员都必须搞懂的Maven基础知识

以前我们写代码时,jar包都默认放在一个叫 /lib 的目录下,然后把该目录设置为classpath可以读取到的目录,如下图所示:

9320
来自专栏磐创AI技术团队的专栏

我们能比卷积神经网络做得更好吗?

英国机器视觉会议(BMVC)大约两周前在英国卡迪夫结束,是计算机视觉和模式识别领域的顶级会议之一,具有28%的竞争接受率。与其他人相比,这是一个小活动,所以你有...

7440
来自专栏青笔原创

python 开源爬虫工具 kcrawler

使用 pip 安装成功后,会自动在系统搜索路径创建可执行程序:kcrawler, kcanjuke, kcjuejin。

8120
来自专栏大数据学习笔记

Neo4J:Apoc插件的下载与安装

版权声明:本文为博主原创文章,遵循 CC 4.0 BY-SA 版权协议,转载请附上原文出处链接和本声明。 ...

20320
来自专栏架构专题

还在收集资料?我这里有个github汇总

国内程序员都喜欢收集资料,但是又不看,github是重灾区。更有莫名其妙fork的,让人不得要领。这些资料其实也代表了收集者的一个思路,还是有不小参考价值的。

150100
来自专栏磐创AI技术团队的专栏

Github中文项目排行榜,你永远想不到开发者都用它干了什么

不久前,有 GitHub 用户吐槽说,GitHub 的每日趋势榜不按照国家和地区来区分,使得榜单上总会有很多点赞量很大的中文项目,有时候甚至会占据半壁江山。这位...

8710
来自专栏崔庆才的专栏

史上规模最大的中文知识图谱以及估值两个亿的 AI 核心代码

那么问题来了,世界上错综复杂的事物这么多,我们上哪里去学这么多知识啊?怎么快速把有意义的数据收集下来呢?

10220
来自专栏从零开始学自动化测试

httprunner学习20-extentreports无法加载问题(已解决)

最近有小伙伴反应使用httprunner的extentreports报告时,打开的页面样式全部丢失了,原本高大上的报告变成了丑八怪。 顿时心都凉了一大截,要是让...

12010
来自专栏Python数据科学

干货 | 深度学习必懂的13种概率分布

作为机器学习从业者,你需要知道概率分布相关的知识。这里有一份最常见的基本概率分布教程,大多数和使用 python 库进行深度学习有关。

9240

扫码关注云+社区

领取腾讯云代金券

年度创作总结 领取年终奖励