深度学习100+经典模型TensorFlow与Pytorch代码实现大合集

深度学习技术前沿公众号博主

发布于 2020-05-18 15:22:02

2.4K0

发布于 2020-05-18 15:22:02

文章被收录于专栏：深度学习技术前沿深度学习技术前沿

【导读】深度学习在过去十年获得了极大进展，出现很多新的模型，并且伴随TensorFlow和Pytorch框架的出现，有很多实现，但对于初学者和很多从业人员，如何选择合适的实现，是个选择。rasbt大神在Github上整理了关于深度学习模型TensorFlow和Pytorch代码实现集合，含有100个，各种各样的深度学习架构，模型，和技巧的集合Jupyter Notebooks，从基础的逻辑回归到神经网络到CNN到GNN等，可谓一网打尽，值得收藏！

地址：https://github.com/rasbt/deeplearning-models

传统机器学习

感知器 Perceptron [TensorFlow 1: GitHub | Nbviewer] https://github.com/rasbt/deeplearning-models/blob/master/tensorflow1_ipynb/basic-ml/perceptron.ipynb [PyTorch: GitHub | Nbviewer] https://nbviewer.jupyter.org/github/rasbt/deeplearning-models/blob/master/pytorch_ipynb/basic-ml/perceptron.ipynb

逻辑回归 Logistic Regression [TensorFlow 1: GitHub | Nbviewer] https://nbviewer.jupyter.org/github/rasbt/deeplearning-models/blob/master/tensorflow1_ipynb/basic-ml/logistic-regression.ipynb [PyTorch: GitHub | Nbviewer] https://nbviewer.jupyter.org/github/rasbt/deeplearning-models/blob/master/pytorch_ipynb/basic-ml/logistic-regression.ipynb

Softmax Regression (Multinomial Logistic Regression) [TensorFlow 1: GitHub | Nbviewer] https://github.com/rasbt/deeplearning-models/blob/master/tensorflow1_ipynb/basic-ml/softmax-regression.ipynb [PyTorch: GitHub | Nbviewer] https://github.com/rasbt/deeplearning-models/blob/master/pytorch_ipynb/basic-ml/softmax-regression.ipynb

Softmax Regression with MLxtend's plot_decision_regions on Iris [PyTorch: GitHub | Nbviewer] https://nbviewer.jupyter.org/github/rasbt/deeplearning-models/blob/master/pytorch_ipynb/basic-ml/softmax-regression-mlxtend-1.ipynb

多层感知器

多层感知器 Multilayer Perceptron [TensorFlow 1: GitHub | Nbviewer] https://nbviewer.jupyter.org/github/rasbt/deeplearning-models/blob/master/tensorflow1_ipynb/mlp/mlp-basic.ipynb [PyTorch: GitHub | Nbviewer] https://nbviewer.jupyter.org/github/rasbt/deeplearning-models/blob/master/pytorch_ipynb/mlp/mlp-basic.ipynb

带Dropout的多层感知器 Multilayer Perceptron with Dropout [TensorFlow 1: GitHub | Nbviewer] [PyTorch: GitHub | Nbviewer]
具有批处理规范化的多层感知器 Multilayer Perceptron with Batch Normalization [TensorFlow 1: GitHub | Nbviewer] [PyTorch: GitHub | Nbviewer]
Multilayer Perceptron with Backpropagation from Scratch [TensorFlow 1: GitHub | Nbviewer] [PyTorch: GitHub | Nbviewer]

卷积神经网络

基础

卷积神经网络 Convolutional Neural Network [TensorFlow 1: GitHub | Nbviewer] https://nbviewer.jupyter.org/github/rasbt/deeplearning-models/blob/master/tensorflow1_ipynb/cnn/cnn-basic.ipynb [PyTorch: GitHub | Nbviewer] https://nbviewer.jupyter.org/github/rasbt/deeplearning-models/blob/master/pytorch_ipynb/cnn/cnn-basic.ipynb
Convolutional Neural Network with He Initialization [PyTorch: GitHub | Nbviewer]

Concepts

Replacing Fully-Connnected by Equivalent Convolutional Layers [PyTorch: GitHub | Nbviewer]

Fully Convolutional

Fully Convolutional Neural Network [PyTorch: GitHub | Nbviewer]

LeNet

LeNet-5 on MNIST [PyTorch: GitHub | Nbviewer] https://nbviewer.jupyter.org/github/rasbt/deeplearning-models/blob/master/pytorch_ipynb/cnn/cnn-lenet5-mnist.ipynb
LeNet-5 on CIFAR-10 [PyTorch: GitHub | Nbviewer]
LeNet-5 on QuickDraw [PyTorch: GitHub | Nbviewer]

AlexNet

AlexNet on CIFAR-10 [PyTorch: GitHub | Nbviewer] https://nbviewer.jupyter.org/github/rasbt/deeplearning-models/blob/master/pytorch_ipynb/cnn/cnn-alexnet-cifar10.ipynb

VGG

Convolutional Neural Network VGG-16 [TensorFlow 1: GitHub | Nbviewer] https://nbviewer.jupyter.org/github/rasbt/deeplearning-models/blob/master/tensorflow1_ipynb/cnn/cnn-vgg16.ipynb [PyTorch: GitHub | Nbviewer]
VGG-16 Gender Classifier Trained on CelebA [PyTorch: GitHub | Nbviewer]
Convolutional Neural Network VGG-19 [PyTorch: GitHub | Nbviewer]

DenseNet

DenseNet-121 Digit Classifier Trained on MNIST [PyTorch: GitHub | Nbviewer] https://nbviewer.jupyter.org/github/rasbt/deeplearning-models/blob/master/pytorch_ipynb/cnn/cnn-densenet121-mnist.ipynb
DenseNet-121 Image Classifier Trained on CIFAR-10 [PyTorch: GitHub | Nbviewer]

ResNet

ResNet and Residual Blocks [PyTorch: GitHub | Nbviewer] https://nbviewer.jupyter.org/github/rasbt/deeplearning-models/blob/master/pytorch_ipynb/cnn/resnet-ex-1.ipynb
ResNet-18 Digit Classifier Trained on MNIST [PyTorch: GitHub | Nbviewer]
ResNet-18 Gender Classifier Trained on CelebA [PyTorch: GitHub | Nbviewer]
ResNet-34 Digit Classifier Trained on MNIST [PyTorch: GitHub | Nbviewer]
ResNet-34 Object Classifier Trained on QuickDraw [PyTorch: GitHub | Nbviewer]
ResNet-34 Gender Classifier Trained on CelebA [PyTorch: GitHub | Nbviewer]
ResNet-50 Digit Classifier Trained on MNIST [PyTorch: GitHub | Nbviewer]
ResNet-50 Gender Classifier Trained on CelebA [PyTorch: GitHub | Nbviewer]
ResNet-101 Gender Classifier Trained on CelebA [PyTorch: GitHub | Nbviewer]
ResNet-101 Trained on CIFAR-10 [PyTorch: GitHub | Nbviewer]
ResNet-152 Gender Classifier Trained on CelebA [PyTorch: GitHub | Nbviewer]

Network in Network

Network in Network CIFAR-10 Classifier [PyTorch: GitHub | Nbviewer] https://nbviewer.jupyter.org/github/rasbt/deeplearning-models/blob/master/pytorch_ipynb/cnn/nin-cifar10.ipynb

归一化层 Normalization Layers

BatchNorm before and after Activation for Network-in-Network CIFAR-10 Classifier [PyTorch: GitHub | Nbviewer] https://nbviewer.jupyter.org/github/rasbt/deeplearning-models/blob/master/pytorch_ipynb/cnn/nin-cifar10_batchnorm.ipynb
Filter Response Normalization for Network-in-Network CIFAR-10 Classifier [PyTorch: GitHub | Nbviewer]

度量学习 Metric Learning

Siamese Network with Multilayer Perceptrons [TensorFlow 1: GitHub | Nbviewer] https://nbviewer.jupyter.org/github/rasbt/deeplearning-models/blob/master/tensorflow1_ipynb/metric/siamese-1.ipynb

自编码器 Autoencoders

全连接自编码器 Fully-connected Autoencoders

Autoencoder (MNIST) [TensorFlow 1: GitHub | Nbviewer] https://nbviewer.jupyter.org/github/rasbt/deeplearning-models/blob/master/tensorflow1_ipynb/autoencoder/ae-basic.ipynb [PyTorch: GitHub | Nbviewer]
Autoencoder (MNIST) + Scikit-Learn Random Forest Classifier [TensorFlow 1: GitHub | Nbviewer] [PyTorch: GitHub | Nbviewer]

Convolutional Autoencoders

Convolutional Autoencoder with Deconvolutions / Transposed Convolutions [TensorFlow 1: GitHub | Nbviewer] [PyTorch: GitHub | Nbviewer]
Convolutional Autoencoder with Deconvolutions and Continuous Jaccard Distance [PyTorch: GitHub | Nbviewer]
Convolutional Autoencoder with Deconvolutions (without pooling operations) [PyTorch: GitHub | Nbviewer]
Convolutional Autoencoder with Nearest-neighbor Interpolation [TensorFlow 1: GitHub | Nbviewer] [PyTorch: GitHub | Nbviewer]
Convolutional Autoencoder with Nearest-neighbor Interpolation -- Trained on CelebA [PyTorch: GitHub | Nbviewer]
Convolutional Autoencoder with Nearest-neighbor Interpolation -- Trained on Quickdraw [PyTorch: GitHub | Nbviewer]

Variational Autoencoders

Variational Autoencoder [PyTorch: GitHub | Nbviewer]
Convolutional Variational Autoencoder [PyTorch: GitHub | Nbviewer]

Conditional Variational Autoencoders

Conditional Variational Autoencoder (with labels in reconstruction loss) [PyTorch: GitHub | Nbviewer]
Conditional Variational Autoencoder (without labels in reconstruction loss) [PyTorch: GitHub | Nbviewer]
Convolutional Conditional Variational Autoencoder (with labels in reconstruction loss) [PyTorch: GitHub | Nbviewer]
Convolutional Conditional Variational Autoencoder (without labels in reconstruction loss) [PyTorch: GitHub | Nbviewer]

生成式对抗网络 Generative Adversarial Networks (GANs)

Fully Connected GAN on MNIST [TensorFlow 1: GitHub | Nbviewer] https://nbviewer.jupyter.org/github/rasbt/deeplearning-models/blob/master/tensorflow1_ipynb/gan/gan.ipynb [PyTorch: GitHub | Nbviewer]
Fully Connected Wasserstein GAN on MNIST [PyTorch: GitHub | Nbviewer]
Convolutional GAN on MNIST [TensorFlow 1: GitHub | Nbviewer] [PyTorch: GitHub | Nbviewer]
Convolutional GAN on MNIST with Label Smoothing [TensorFlow 1: GitHub | Nbviewer] [PyTorch: GitHub | Nbviewer]
Convolutional Wasserstein GAN on MNIST [PyTorch: GitHub | Nbviewer]

图神经网络 Graph Neural Networks (GNNs)

Most Basic Graph Neural Network with Gaussian Filter on MNIST [PyTorch: GitHub | Nbviewer] https://nbviewer.jupyter.org/github/rasbt/deeplearning-models/blob/master/pytorch_ipynb/gnn/gnn-basic-1.ipynb
Basic Graph Neural Network with Edge Prediction on MNIST [PyTorch: GitHub | Nbviewer]
Basic Graph Neural Network with Spectral Graph Convolution on MNIST [PyTorch: GitHub | Nbviewer]

循环神经网络 Recurrent Neural Networks (RNNs)

Many-to-one: Sentiment Analysis / Classification

A simple single-layer RNN (IMDB) [PyTorch: GitHub | Nbviewer] https://nbviewer.jupyter.org/github/rasbt/deeplearning-models/blob/master/pytorch_ipynb/rnn/rnn_simple_imdb.ipynb
A simple single-layer RNN with packed sequences to ignore padding characters (IMDB) [PyTorch: GitHub | Nbviewer]
RNN with LSTM cells (IMDB) [PyTorch: GitHub | Nbviewer]
RNN with LSTM cells (IMDB) and pre-trained GloVe word vectors [PyTorch: GitHub | Nbviewer]
RNN with LSTM cells and Own Dataset in CSV Format (IMDB) [PyTorch: GitHub | Nbviewer]
RNN with GRU cells (IMDB) [PyTorch: GitHub | Nbviewer]
Multilayer bi-directional RNN (IMDB) [PyTorch: GitHub | Nbviewer]
Bidirectional Multi-layer RNN with LSTM with Own Dataset in CSV Format (AG News) [PyTorch: GitHub | Nbviewer]
Bidirectional Multi-layer RNN with LSTM with Own Dataset in CSV Format (Yelp Review Polarity) [PyTorch: GitHub | Nbviewer]
Bidirectional Multi-layer RNN with LSTM with Own Dataset in CSV Format (Amazon Review Polarity) [PyTorch: GitHub | Nbviewer]

Many-to-Many / Sequence-to-Sequence

A simple character RNN to generate new text (Charles Dickens) [PyTorch: GitHub | Nbviewer]

Ordinal Regression

Ordinal Regression CNN -- CORAL w. ResNet34 on AFAD-Lite [PyTorch: GitHub | Nbviewer]
Ordinal Regression CNN -- Niu et al. 2016 w. ResNet34 on AFAD-Lite [PyTorch: GitHub | Nbviewer]
Ordinal Regression CNN -- Beckham and Pal 2016 w. ResNet34 on AFAD-Lite [PyTorch: GitHub | Nbviewer]

Tips and Tricks

Cyclical Learning Rate [PyTorch: GitHub | Nbviewer]
Annealing with Increasing the Batch Size (w. CIFAR-10 & AlexNet) [PyTorch: GitHub | Nbviewer]
Gradient Clipping (w. MLP on MNIST) [PyTorch: GitHub | Nbviewer]