A curated list of awesome Self-Supervised Learning resources. Inspired by awesome-deep-vision, awesome-adversarial-machine-learning, awesome-deep-learning-papers, and awesome-architecture-search
Self-Supervised Learning has become an exciting direction in Computer Vision, Machine Learning, and Robotics community. These are some of the awesome resources!
Contributing
Please help contribute this list by contacting me or add pull request
Markdown format:
- Paper Name.
[[pdf]](link)
[[code]](link) - Author 1, Author 2, and Author 3. *Conference'Year*
Change Log
July.29 ECCV'18 papers updated!
Table of Contents
Computer Vision
Image Representation Learning
Video Representation Learning
Geometry
Audio
Others
Machine Learning
Reinforcement Learning
Robotics
Talks
Computer Vision
Image Representation Learning
2015
Unsupervised Visual Representation Learning by Context Prediction.[pdf][code]
Doersch, Carl and Gupta, Abhinav and Efros, Alexei A. ICCV 2015
Unsupervised Learning of Visual Representations using Videos.[pdf][code]
Wang, Xiaolong and Gupta, Abhinav. ICCV 2015
Learning to See by Moving.[pdf][code]
Agrawal, Pulkit and Carreira, Joao and Malik, Jitendra. ICCV 2015
Learning image representations tied to ego-motion.[pdf][code]
Jayaraman, Dinesh and Grauman, Kristen. ICCV 2015
2016
Slow and steady feature analysis: higher order temporal coherence in video.[pdf]
Jayaraman, Dinesh and Grauman, Kristen. CVPR 2016
Context Encoders: Feature Learning by Inpainting.[pdf][code]
Pathak, Deepak and Krahenbuhl, Philipp and Donahue, Jeff and Darrell, Trevor and Efros, Alexei A. CVPR 2016
Colorful Image Colorization.[pdf][code]
Zhang, Richard and Isola, Phillip and Efros, Alexei A. ECCV 2016
Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles.[pdf][code]
Noroozi, Mehdi and Favaro, Paolo. ECCV 2016
Ambient Sound Provides Supervision for Visual Learning.[pdf][code]
Owens, Andrew and Wu, Jiajun and McDermott, Josh and Freeman, William and Torralba, Antonio. ECCV 2016
Learning Representations for Automatic Colorization.[pdf][code]
Larsson, Gustav and Maire, Michael and Shakhnarovich, Gregory. ECCV 2016
Unsupervised Visual Representation Learning by Graph-based Consistent Constraints.[pdf][code]
Li, Dong and Hung, Wei-Chih and Huang, Jia-Bin and Wang, Shengjin and Ahuja, Narendra and Yang, Ming-Hsuan. ECCV 2016
2017
Adversarial Feature Learning.[pdf][code]
Donahue, Jeff and Krahenbuhl, Philipp and Darrell, Trevor. ICLR 2017
Self-supervised learning of visual features through embedding images into text topic spaces.[pdf][code]
L. Gomez* and Y. Patel* and M. Rusiñol and D. Karatzas and C.V. Jawahar. CVPR 2017
Split-Brain Autoencoders: Unsupervised Learning by Cross-Channel Prediction.[pdf][code]
Zhang, Richard and Isola, Phillip and Efros, Alexei A. CVPR 2017
Learning Features by Watching Objects Move.[pdf][code]
Pathak, Deepak and Girshick, Ross and Dollar, Piotr and Darrell, Trevor and Hariharan, Bharath. CVPR 2017
Colorization as a Proxy Task for Visual Understanding.[pdf][code]
Larsson, Gustav and Maire, Michael and Shakhnarovich, Gregory. CVPR 2017
Learning Image Representations by Completing Damaged Jigsaw Puzzles.[pdf]
Kim, Dahun and Cho, Donghyeon and Yoo, Donggeun and Kweon, In So. WAVC 2018
Unsupervised Representation Learning by Predicting Image Rotations.[pdf][code]
Spyros Gidaris and Praveer Singh and Nikos Komodakis. ICLR 2018
Improvements to context based self-supervised learning.[pdf]
Terrell Mundhenk and Daniel Ho and Barry Chen. CVPR 2018
Self-Supervised Feature Learning by Learning to Spot Artifacts.
Simon Jenni and Universität Bern and Paolo Favaro. CVPR 2018
Boosting Self-Supervised Learning via Knowledge Transfer.[pdf]
Mehdi Noroozi and Ananth Vinjimoor and Paolo Favaro and Hamed Pirsiavash. CVPR 2018
Cross-domain Self-supervised Multi-task Feature Learning Using Synthetic Imagery.[pdf][code]
Zhongzheng Ren and Yong Jae Lee. CVPR 2018
ShapeCodes: Self-Supervised Feature Learning by Lifting Views to Viewgrids.[pdf]
Dinesh Jayaraman*, UC Berkeley; Ruohan Gao, University of Texas at Austin; Kristen Grauman. ECCV 2018
Video Representation Learning
Unsupervised Learning of Video Representations using LSTMs.[pdf][code]
Srivastava, Nitish and Mansimov, Elman and Salakhudinov, Ruslan. ICML 2015
Shuffle and Learn: Unsupervised Learning using Temporal Order Verification.[pdf][code]
Ishan Misra, C. Lawrence Zitnick and Martial Hebert. ECCV 2016
LSTM Self-Supervision for Detailed Behavior Analysis[pdf]
Biagio Brattoli*, Uta Büchler*, Anna-Sophia Wahl, Martin E. Schwab, and Björn Ommer. CVPR 2017
Self-Supervised Video Representation Learning With Odd-One-Out Networks.[pdf]
Basura Fernando and Hakan Bilen and Efstratios Gavves and Stephen Gould. CVPR 2017
Unsupervised Learning of Long-Term Motion Dynamics for Videos.[pdf]
Luo, Zelun and Peng, Boya and Huang, De-An and Alahi, Alexandre and Fei-Fei, Li. CVPR 2017
Geometry Guided Convolutional Neural Networks for Self-Supervised Video Representation Learning.[pdf]
Chuang Gan and Boqing Gong and Kun Liu and Hao Su and Leonidas J. Guibas. CVPR 2018
Improving Spatiotemporal Self-Supervision by Deep Reinforcement Learning.[pdf]
Biagio Brattoli*, Uta Büchler*, and Björn Ommer. ECCV 2018
Geometry
Self-supervised Learning of Motion Capture.[pdf][code][web]
Tung, Hsiao-Yu and Tung, Hsiao-Wei and Yumer, Ersin and Fragkiadaki, Katerina. NIPS 2017
Unsupervised Learning of Depth and Ego-Motion from Video.[pdf][code][web]
Zhou, Tinghui and Brown, Matthew and Snavely, Noah and Lowe, David G. CVPR 2017
Active Stereo Net: End-to-End Self-Supervised Learning for Active Stereo Systems.[project]
Yinda Zhang*, Sean Fanello, Sameh Khamis, Christoph Rhemann, Julien Valentin, Adarsh Kowdle, Vladimir Tankovich, Shahram Izadi, Thomas Funkhouser. ECCV 2018
Self-Supervised Relative Depth Learning for Urban Scene Understanding.[pdf][project]
Huaizu Jiang*, Erik Learned-Miller, Gustav Larsson, Michael Maire, Greg Shakhnarovich. ECCV 2018
Audio
Audio-Visual Scene Analysis with Self-Supervised Multisensory Features.[pdf][code]
Andrew Owens, Alexei A. Efros. ECCV 2018
Objects that Sound.[pdf]
R. Arandjelović, A. Zisserman. ECCV 2018
Learning to Separate Object Sounds by Watching Unlabeled Video.[pdf][project]
Ruohan Gao, Rogerio Feris, Kristen Grauman. ECCV 2018
Others
Self-learning Scene-specific Pedestrian Detectors using a Progressive Latent Model.[pdf]
Qixiang Ye, Tianliang Zhang, Qiang Qiu, Baochang Zhang, Jie Chen, Guillermo Sapiro. CVPR 2017
Fighting Fake News: Image Splice Detection via Learned Self-Consistency[pdf][code]
Minyoung Huh*, Andrew Liu*, Andrew Owens, Alexei A. Efros. ECCV 2018
Self-supervised Tracking by Colorization (Tracking Emerges by Colorizing Videos).[pdf]
Carl Vondrick*, Abhinav Shrivastava, Alireza Fathi, Sergio Guadarrama, Kevin Murphy. ECCV 2018
Machine Learning
Self-taught Learning: Transfer Learning from Unlabeled Data.[pdf]
Raina, Rajat and Battle, Alexis and Lee, Honglak and Packer, Benjamin and Ng, Andrew Y. ICML 2007
Representation Learning: A Review and New Perspectives.[pdf]
Bengio, Yoshua and Courville, Aaron and Vincent, Pascal. TPAMI 2013.
Reinforcement Learning
Curiosity-driven Exploration by Self-supervised Prediction.[pdf][code]
Deepak Pathak, Pulkit Agrawal, Alexei A. Efros, and Trevor Darrell. ICML 2017
coming soon...
Robotics
The Curious Robot: Learning Visual Representations via Physical Interactions.[pdf]
Lerrel Pinto and Dhiraj Gandhi and Yuanfeng Han and Yong-Lae Park and Abhinav Gupta. ECCV 2016
Time-Contrastive Networks: Self-Supervised Learning from Video.[pdf][Project]
Pierre Sermanet and Corey Lynch and Yevgen Chebotar and Jasmine Hsu and Eric Jang and Stefan Schaal and Sergey Levine. ICRA 2018
Learning to Poke by Poking: Experiential Learning of Intuitive Physics.[pdf]
Agrawal, Pulkit and Nair, Ashvin V and Abbeel, Pieter and Malik, Jitendra and Levine, Sergey. NIPS 2016
Supersizing Self-supervision: Learning to Grasp from 50K Tries and 700 Robot Hours. [pdf]
Pinto, Lerrel and Gupta, Abhinav. ICRA 2016
Supervision via Competition: Robot Adversaries for Learning Tasks.[pdf]
Pinto, Lerrel and Davidson, James and Gupta, Abhinav. ICRA 2017
Talks
Supersizing Self-Supervision: Learning Perception and Action without Human Supervision. Abhinav Gupta (CMU) [link]
Self-supervision, Meta-supervision, Curiosity: Making Computers Study Harder. Alyosha Efros (UCB) [link]
License
To the extent possible under law, Zhongzheng Ren has waived all copyright and related or neighboring rights to this work.