There are many deep learning resources freely available online,but it can be confusing knowing where to begin. Go from vague understanding of deep neural networks to knowledgeable practitioner in 7 steps!

By Matthew Mayo. Deep learning is a branch of machine learning, employing numerous similar, yet distinct, deep neural network architectures to solve various problems in natural language processing, computer vision, and bioinformatics, among other fields. Deep learning has experienced a tremendous recent research resurgence, and has been shown to deliver state of the art results in numerous applications. In essence, deep learning is the implementation of neural networks with more than a single hidden layer of neurons. This is, however, a very simplistic view of deep learning, and not one that is unanimously agreed upon. These "deep" architectures also vary quite considerably, with different implementations being optimized for different tasks or goals. The vast research being produced at such a constant rate is revealing new and innovative deep learning models at an ever-increasing pace.

Currently a white hot research topic, deep learning seems to be impacting all areas of machine learning and, by extension, data science. A look over recent papers in the relevant arXiv categories makes it easy to see that a large amount of what is being published is deep learning-related. Given the impressive results being produced, many researchers, practioners, and laypeople alike are wondering if deep learning is the edge of "true" artifical intelligence. This collection of reading materials and tutorials aims to provide a path for a deep neural networks newcomer to gain some understanding of this vast and complex topic. Though I do not assume any real understanding of neural networks or deep learning, I will assume your familiarity with general machine learning theory and practice to some degree. To overcome any deficiency you may have in the general areas of machine learning theory or practice you can consult the recent KDnuggets post 7 Steps to Mastering Machine Learning With Python. Since we will also see examples implemented in Python, some familiarity with the language will be useful. Introductory and review resources are also available in thepreviously mentioned post. This post will utilize freely-available materials from around the web in a cohesive order to first gain some understanding of deep neural networks at a theoretical level, and then move on to some practical implementations. As such, credit for the materials referenced lie solely with the creators, who will be noted alongside the resources. If you see that someone has not been properly credited for their work, please alert me to the oversight so that I may swiftly rectify it. A stark and honest disclaimer: deep learning is a complex and quickly-evolving field of both breadth and depth (pun unintended?), and as such this post does not claim to be an all-inclusive manual to becoming a deep learning expert; such a transformation would take greater time, many additional resources, and lots of practice building and testing models. I do, however, believe that utilizing the resources herein could help get you started on just such a path. Step 1: Introducing Deep Learning If you are reading this and interested in the topic, then you are probably already familiar with what deep neural networks are, if even at a basic level. Neural networks have a storied history, but we won't be getting into that. We do, however, want a common high level of understanding to begin with. First, have a look at the fantastic introductory videos fromDeepLearning.tv. At the time of this writing there are 14 videos; watch them all if you like, but definitely watch the first 5, covering the basics of neural nets and some of the more common architectures. Next, read over the NIPS 2015 Deep Learning Tutorial by Geoff Hinton, Yoshua Bengio, and Yann LeCun for an introduction at a slightly lower level. To round out our first step, read the first chapter of Neural Networks and Deep Learning, the fantastic, evolving online book by Michael Nielsen, which goes a step further but still keeps things fairly light. Step 2: Getting Technical Deep neural nets rely on a mathematical foundation of algebra and calculus. While this post will not produce any theoretical mathematicians, gaining some understanding of the basics before moving on would be helpful. First, watch Andrew Ng's linear algebra review videos. While not absolutely necessary, for those finding they want something deeper on this subject, consult the Linear Algebra Review and Reference from Ng's Stanford course, written by Zico Kolter and Chuong Do. Then look at this Introduction to the Derivative of a Function video by Professor Leonard. The video is succinct, the examples are clear, and it provides some understanding of what is actually going on during backpropagation from a mathematical standpoint. More on that soon. Next have a quick read over the Wikipedia entry for theSigmoid function, a bounded differentiable function often employed by individual neurons in a neural network. Finally, take a break from the maths and read this Deep Learning Tutorial by Google research scientist Quoc Le.

Step 3: Backpropagation and Gradient Descent An important part of neural networks, including modern deep architectures, is the backward propagation of errors through a network in order to update the weights used by neurons closer to the input. This is, quite bluntly, from where neural networks derive their "power," for lack of better term. Backpropagation for short (or even "backprop"), is paired with an optimization method which acts to minimize the weights that are subsequently distributed (via backpropagation), in order to minimize the loss function. A common optimization method in deep neural networks is gradient descent. First, read these introductory notes on gradient descent byMarc Toussaint of the University of Stuttgart. Next, have a look at this step by step example of backpropagation in action written by Matt Mazur. Moving on, read Jeremy Kun's informative blog post on coding backpropagation in Python. Having a look over the complete code is also suggested, as is attempting to replicate it yourself. Finally, read the second part of the Deep Learning Tutorialby Quoc Le, in order to get introduced to some specific common deep architectures and their uses.




原文发布于微信公众号 - 量化投资与机器学习(ZXL_LHTZ_JQXX)





0 条评论
登录 后参与评论



用 Mathematica 生成迷宫


《利用Python进行数据分析·第2版》第13章 Python建模库介绍13.1 pandas与模型代码的接口13.2 用Patsy创建模型描述13.3 statsmodels介绍13.4 sciki

本书中,我已经介绍了Python数据分析的编程基础。因为数据分析师和科学家总是在数据规整和准备上花费大量时间,这本书的重点在于掌握这些功能。 开发模型选用什么库...



【导读】专知内容组今天推出最新八篇情感分析(Sentiment Analysis)相关论文,欢迎查看!

来自专栏tkokof 的技术,小趣及杂念


在算法设计的学习中,每到“动态规划”一节,一般都会涉及到“矩阵连乘”问题(例如《Algorithms》,中文译名《算法概论》),可想而知该题的经典程度 :)


深度学习代码系列之——Deep Learning Toolbox For Matlab

A Matlab toolbox for Deep Learning Matlab/Octave toolbox for deep learning. I...



【导读】专知内容组整理了最近六篇机器翻译(Machine Translation)相关文章,为大家进行介绍,欢迎查看! 1. Machine Translat...



MP:matching pursuit匹配追踪 OMP:正交匹配追踪 主要介绍MP与OMP算法的思想与流程,解释为什么需要引入正交?   !!今天发现一个重大问...



更多资料: https://distill.pub/2016/augmented-rnns/#attentional-interfaces https://ww...



代码多来自《Introduction to Machine Learning with Python》. 该文集主要是自己的一个阅读笔记以及一些小思考,小总结...