Resource on Information Theory, Statistical Mechanics, Generative and Diffusion Models, Coding Theory
The Mathematical Theory of Communication, Claude Shannon and Warren Weaver, 1964
Information Theory, Robert B. Ash, 1965
Information Theory, Inference, and Learning Algorithms, David MacKay, 2005 (Ver 7)
Elements of Information Theory, Thomas M. Cover, Joy A. Thomas, 2006
Entropy and Information Theory, Robert M. Gray, Stanford U., First Edition, 2023
Physical Nature of Information, G. Falkovich, Lectures, June 5, 2023
Introduction to Information Theory and Applications, F. Bavaud, JC Chappelier, J Kohlas, 2005
Statistical Mechanics: Entropy, Order Parameters and Complexity, JP Sethna, 2006
Farwell to Entropy: Statistical Thermodynamics based on Information, Arieh Ben-Naim, 2008
Diffusion Processes and Stochastic Calculus, Fabrice Baudoin, 2019
Statistical Physics and Information Theory, Neri Merhav, Technion, Lecture notes
Numerical Solution of Stochastic Differential Equations, Peter E. Kloeden, Eckhart Platten, 1999
Brownian Motion and Stochastic Flow Systems, J. Michael Harrison, 1985
Modern Coding Theory, T. Richardson, R. Urbanke, 2007
Coding Theory: A First Course, Henk C.A. van Tilborg, 1993
Differential Space, Norbert Wiener, 1923
The Mathematical Theory of Communication, Claude Shannon, The Bell System Technical Journal, 1948
Communication Theory of Secrecy Systems, Claude Shannon, The Bell System Technical Journal, 1949
On Information and Sufficiency, S. Kullback, R.A. Leibler, 1951
Information Theory and Statistical Mechanics, Part I, E.T. Jaynes, Stanford U., 1957
Information Theory and Statistical Mechanics, Part II, E.T. Jaynes, Stanford U., 1957
Introduction to Diffusion Models for Deep Learning, Ryan O'Connor, 2022 (online blog)
What are Diffusion Models? Lilian Weng, OpenAI, 2021 (online blog)
Diffusion Models for Video Generation, Lilian Weng, OpenAI, 2024 (online blog)
Tutorial on Diffusion Models for Imaging and Vision, Stanley Chan, 2024
On Error Propagation of Diffusion Models, Y. Li, Michaela van der Schaar, U of Cambridge, 2024
Step By Step Diffusion : An Elementary Tutorial, P. Nakkiran et al, 2024
Understanding Diffusion Models: Unified Perspective, Calvin Luo, Google Brain, 2022
Lightweight Diffusion Models: A Survey, W. Song et al, 2024
Perspectives on Diffusion, Sander Dieleman, 2023
Introduction to Variational Methods for Graphical Models, M.I. Jordan et al, 1999
Diffusion Models Beat GANs on Image Synthesis, Prafulla Dharival, Alex Nichol, OpenAI, 2021
Generative Models of Images and Neural Networks, William Smith Peebles, PhD Thesis, 2023
Learning General Gaussian Mixtures with Efficient Score Matching, S. Chen et al, Harvard U, 2024
Improved Techniques for Training Score-Based Generative Models, Y. Song, S. Ermon, 2020
Denoising Diffusion Probabilistic Models, J. Ho et al, UC Berkeley, 2020
Denoising Diffusion Implicit Models, J. Song et al, ICLR 2021
Flow Matching for Generative Modeling, Yaron Lipman et al, Meta FAIR, 2023
Sampling, Diffusions, and Stochastic Localization, Andrea Montanari, 2023
Extracting Training Data From Diffusion Models, Nicholas Carlini et al, Deep Mind, 2023
High-Dimensional Dynamics of Generalization Error in Neural Networks, M.S. Advani et al, 2017
On The Mathematics of Diffusion Models, David McAllester, 2023
Reverse Time Diffusion Equation Models, Brian D.O. Anderson, 1982
Navier-Stokes, Fluid Dynamics, Image and Video Inpainting, M. Bertalmio et al, 2001
The Fokker-Planck Equation: Methods of Solution and Application, 2nd edition, H. Risken, 1996
Fokker-Planck Equation, Wikipedia
Klaus Schulten's lecture notes on non-equilibrium statistical mechanics, 1999
Generalized Fokker-Planck Equation: Derivation and Exact Solutions, SI Denisov et al, 2009
Probability Flow solution of the Fokker-Planck Equation, Nicholas Boffi, Eric Venden Eijnden, 2023
Scalable Diffusion Models with Transformers, William Peebles, UC Berkeley, 2022
Masked Diffusion Transformer is a Strong Image Synthesizer, S. Gao et al Sea AI Lab, Nankai U., 2023
FiT: Flexible Vision Transformer for Diffusion Model, Z. Lu et al, Feb 2024
DiffiT: Diffusion Vision Transformers for Image Generation, A. Hatamizadeh et al, 2024
Diffusion Transformer (DiT) Models: A Beginner’s Guide, Akruti Acharya, March 18, 2024
DiTFastAttn: Attention Compression for Diffusion Transformer Models, Zhihang Yuan et al, 2024
Tutorial on Variational Autoencoders, Carl Doersch, Carnegie Mellon, UC Berkeley, 2021
Introduction to Variational Autoencoders, Diedrik P. Kingma, Max Welling, 2019
Auto-Encoding Variational Bayes, Diedrik P. Kingma, Max Welling, 2022
The Sparse Autoencoder, Andrew Ng, Lecture Notes CS294A
Fisher information metric, Wikipedia
Methods of Information Geometry, Shun-ichi Amari, Hiroshi Nagaoka, 1993
An elementary introduction to information geometry, Frank Nielsen, Sony Corp., 2020
Far-from-Equilibrium Measurements of Thermodynamic Length, EH Feng, GE Crooks, UC Berkeley, 2008
The information geometry of mirror descent, G. Raskutti, S. Mukherjee, 2014
Thermodynamic Linear Algebra, Maxwell Aifer et al, Normal Computing Corporation, 2024
Conformal prediction under ambiguous ground truth, David Stutz et al, 2023, DeepMind
related repo: https://github.com/google-deepmind/uncertain_ground_truth
Maze Proof Establishes a ‘Backbone’ for Statistical Mechanics, L. Quanta Magazine, 2024
Backbone Exponent for Two Dimensional Percolation, Pierre Nollin et al, 2024
Near-Critical Percolation in Two Dimensions, Pierre Nollin, 2007
Continuum Limits for Critical Percolation and Other Stochastic Geometric Models, M. Aizenman, 1998
Critical Exponents for Two-Dimensional Percolation, S. Smirnov et al, 2001
Conductivity Exponent and Backbone Dimension in 2D Percolation, Peter Grassberger, 2018