SelfSupervisedLearningResources.md

Offline Reinforcement Learning and Self-Supervised Learning Resources

Articles and papers

Offline Reinforcement Learning

• Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems, Sergey Levine et al, UC Berkeley, Google Research, 2020

• A Survey on Offline Reinforcement Learning: Taxonomy, Review and Open Problems, RF Prudencio et al, 2023

• Understanding The World Through Action, Sergey Levine, UC Berkeley, 2021

• When Should We Prefer Offline Reinforcement Learning Over Behavioral Cloning?, Aviral Kumar et al, UC Berkeley, Google Research, 2022

• Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reduction, Aviral Kumar et al, UC Berkeley, Google Brain, 2019

• Off-Policy Deep Reinforcement Learning without Exploration, S. Fujimoto et al, 2019

• Way Off-Policy Batch Deep Reinforcement Learning of Implicit Human Preferences in Dialog, N. Jaques et al, MIT, 2019

• Weakly-Supervised Reinforcement Learning for Controllable Behavior, L. Lee et al, CMU, NeurIPS, 2020

• Hierarchical Relative Enthropy Policy Search, Christian Daniel et al, TU Darmstadt, 2000

• Offline Meta-Reinforcement Learning with Online Self-Supervision, V. Pong et al, UC Berkeley, 2022

• Online Relative Entropy Policy Search using Reproducing Kernel Hilbert Space Embeddings, Z. Chen et al, Huawei, 2016

• Should I Run Offline Reinforcement Learning or Behavioral Cloning? Aviral Kumar et al, UC Berkeley, 2021

• Batch Reinforcement Learning, Sascha Lange, Thomas Gabel, Martin Riedmiller, Chapter of "Reinforcement Learning State of Art", 2012

• Neural Fitted Q Iteration - First Experiences with a Data Efficient Neural Reinforcement Learning Method (with annotations), M. Riedmiller, 2005

• Tree-Based Batch Mode Reinforcement Learning, Damien Ernst, Pierre Guerts, Louis Wehenkel, 2005

Self-Supervised Learning

• Cookbook of Self-Supervised Learning, Balestriero, R, Ibrahim, M, et al., 2023

Online Tutorials and Short Readings

Meta AI Research blog

• Self-supervised learning: the dark matter of intelligence

Youtube

• A General Purpose Navigation Model, Sergey Levine

Medium

• Decisions from Data: How Offline Reinforcement Learning Will Change How We Use Machine Learning, Sergey Levine, 2020

  https://medium.com/@sergey.levine/decisions-from-data-how-offline-reinforcement-learning-will-change-how-we-use-ml-24d98cb069b0

  related paper: Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems, Sergey Levine et al, UC Berkeley, Google Research, 2020

  related paper: D4RL: Datasets for Deep Data-Driven Reinforcement Learning, Justin Fu et al, UC Berkeley, 2021

• The Monster of Distributional Shift in Offline Reinforcement Learning and How to Pacify it

  https://medium.com/@athanasios.kapoutsis/the-monster-of-distribution-shift-in-offline-rl-and-how-to-pacify-it-4ea9a5db043

  related paper: Generalization in Humans and Deep Neural Networks, R. Geirhos et al, UoTubingen, 2020

  related paper: A Survey on Offline Reinforcement Learning: Taxonomy, Review and Open Problems, RF Prudencio et al, 2023

  related paper: Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems, Sergey Levine et al, UC Berkeley, Google Research, 2020

  related paper: When Should We Prefer Offline Reinforcement Learning Over Behavioral Cloning?, Aviral Kumar et al, UC Berkeley, Google Research, 2022

  related paper: Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reduction, Aviral Kumar et al, UC Berkeley, Google Brain, 2019

  related paper: Off-Policy Deep Reinforcement Learning without Exploration, S. Fujimoto et al, 2019

  related paper: Way Off-Policy Batch Deep Reinforcement Learning of Implicit Human Preferences in Dialog, N. Jaques et al, MIT, 2019

  related paper: Relative Entropy Policy Search, Jan Peters et al, Max Planck Institute, 2010

  related paper: Awac: accelerating online reinforcement learning with offline datasets, A. Nair, UC Berkeley, 2020

  related paper: Advantage-weighted regression: simple and scalable off-policy reinforcement learning, XB Peng et al, UC Berkeley, 2019

  related paper: Offline reinforcement learning with implicit q-learning, I. Kostrikov et al, UC Berkeley, 2021

  related paper: Conservative q-learning for offline reinforcement learning, Aviral Kumar et al, UC Berkeley, 2020

• Curious Agents: An Introduction with Dries Smith

  https://medium.com/@dries.epos/curious-agents-ebfee02ef024

  (code of this series: https://github.com/DriesSmit/CuriousAgents)

• Curious Agents II: Solving MountainCar without Rewards

  https://medium.com/@dries.epos/curious-agents-ii-solving-mountaincar-without-rewards-c49ae2177819

  (code of this series: https://github.com/DriesSmit/CuriousAgents)

• Curious Agents III: BYOL-Explore

  https://medium.com/@dries.epos/curious-agents-iii-byol-explore-93f34fa6146a

  (code of this series: https://github.com/DriesSmit/CuriousAgents)

• Curious Agents IV: BYOL-Hindsight

  https://medium.com/@dries.epos/curious-agents-iv-byol-hindsight-318c559175f0

  (code of this series: https://github.com/DriesSmit/CuriousAgents)

• Understanding the World Through Action: RL as a Foundation for Scalable Self-Supervised Learning with Sergey Levine

  https://medium.com/@sergey.levine/understanding-the-world-through-action-rl-as-a-foundation-for-scalable-self-supervised-learning-636e4e243001

• How Robots Can Learn End-to-End from Data with Sergey Levine

  https://medium.com/@sergey.levine/how-robots-can-learn-end-to-end-from-data-3d879b0a2ba1

• Self-Supervised Learning Using Projection Heads: Boost performance with unlabeled data with Daniel Warfield

  https://towardsdatascience.com/self-supervised-learning-using-projection-heads-b77af3911d33