# 告别选择困难症，我来带你剖析这些深度学习框架基本原理

1.张量

2.基于张量的操作

3.计算图和优化

4.自动微分工具

5. BLAS/cuBLAS和cuDNN的扩展

## 张量对象的操作

Sigmoid 层表示为 Matrix Operations，来自http://www.datasciencecentral.com/profiles/blogs/matrix-multiplication-in-neural-networks

1. NumPy: http://www.scipy-lectures.org/intro/numpy/operations.html
2. Theano: http://deeplearning.net/software/theano/library/tensor/basic.html
3. TensorFlow: https://www.tensorflow.org/api_docs/python/math_ops/

## 计算图和优化

1、只有在完成上一个操作后才开始下一步操作或者并行操作？

2、如何分配到不同的设备并在它们之间进行协调？

3、你如何避免冗余操作（乘以1，添加零），缓存有用的中间值，并将多个操作减少为一个（用mul替换mul（mul（mul（Tensor，2），2），2）（Tensor， 8））

1. Theano http://deeplearning.net/software/theano/extending/graphstructures.html
3. Caffe http://caffe.berkeleyvision.org/tutorial/net_layer_blob.html

## 参考文献

[1] It’s not straightforward how you would represent text as tensors. The first way is to use a one-hot encoding, which is a very sparse matrix and wastes a lot of space. A more dense representation is word vectors. These are pretty cool and I probably might write another post on them if enough people are interested!

[2] Also, as you will see in the Auto-differentiation part, it’s not clear how you would calculate the derivatives of words. They’re not even continuous!

[3] This is the (in)famous backpropagation algorithm and is central to learning in Multilayered neural networks.

[4] This also means moving the data to GPU or back. I have noticed that in Theano (possibly other frameworks as well), this is the most time-consuming part during execution.

[5] http://www.deeplearning.net/software/theano/optimizations.html

1607 篇文章140 人订阅

0 条评论

## 相关文章

### Unity3D游戏开发初探—2.初步了解3D模型基础

简而言之，3D模型就是三维的、立体的模型，D是英文Dimensions的缩写。

13630

37530

15530

26590

42780

34870

45680

36750

378110

47780