NIPS的最佳论文 强化学习Value iteration Network 及代码

TensorFlow实现：https://github.com/TheAbhiKumar/tensorflow-value-iteration-networks

下面文章作者 https://www.zhihu.com/people/ikerpeng/

代码实现介绍：

Value Iteration Networks in TensorFlow

Tamar, A., Wu, Y., Thomas, G., Levine, S., and Abbeel, P. Value Iteration Networks. Neural Information Processing Systems (NIPS) 2016

This repository contains an implementation of Value Iteration Networks in TensorFlow which won the Best Paper Award at NIPS 2016. This code is based on the original Theano implementation by the authors.

Training

• Download the 16x16 and 28x28 GridWorld datasets from the author's repository. This repository contains the 8x8 GridWorld dataset for convenience and its small size.

python3 train.py

If you want to monitor training progress change config.log to True and launch tensorboard --logdir /tmp/vintf/. The log directory is /tmp/vintf/ by default, but can be changed in config.logdir. The code currently runs the 8x8 GridWorld model by default.

The 8x8 GridWorld model converges in under 30 epochs with about ~98.5% accuracy. The paper lists that it should be around 99.6% and I was able to reproduce this with the Theano code. The TensorFlow model is not perfect as NaNs result when training with the same parameters as the Theano implementation on the 16x16 and 28x28 domain.

Dependencies

• Python >= 3.5
• TensorFlow >= 0.12
• SciPy >= 0.18.1 (to load the data)

Datasets

• The GridWorld dataset used is from the author's repository. It also contains Matlab scripts to generate the dataset. The code to process the dataset is from the original repository with minor modifications under this license
• The model was also originally tested on three other domains and the author's original code will be released eventually
  • Mars Rover Navigation
  • Continuous control
  • WebNav

Resources

• Value Iteration Networks on arXiv
• Aviv Tamar's (author) original implementation in Theano
• NIPS 2016 Supplemental
• ICML Slides
• Contact GitHub API Training Shop Blog About