DenseCL, SSL model improvements

• Adds the DenseCL: Dense Contrastive Learning for Self-Supervised Visual Pre-Training method. See the docs.
• Add TiCoTransform, thanks @radiradev!
• Improvements to the pre-commit hooks, thanks @SauravMaheshkar!
• Fix memory bank issue when using gather_distributed=True and training on a single GPU
• Fix student head update in DINO benchmark
• Various improvements to MaskedVisionTransformer
• Renaming of Lightly SSL to LightlySSL

Models

• AIM: Scalable Pre-training of Large Autoregressive Image Models
• Barlow Twins: Self-Supervised Learning via Redundancy Reduction, 2021
• Bootstrap your own latent: A new approach to self-supervised Learning, 2020
• DCL: Decoupled Contrastive Learning, 2021
• DenseCL: Dense Contrastive Learning for Self-Supervised Visual Pre-Training, 2021
• DINO: Emerging Properties in Self-Supervised Vision Transformers, 2021
• FastSiam: Resource-Efficient Self-supervised Learning on a Single GPU, 2022
• I-JEPA: Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture, 2023
• MAE: Masked Autoencoders Are Scalable Vision Learners, 2021
• MSN: Masked Siamese Networks for Label-Efficient Learning, 2022
• MoCo: Momentum Contrast for Unsupervised Visual Representation Learning, 2019
• NNCLR: Nearest-Neighbor Contrastive Learning of Visual Representations, 2021
• PMSN: Prior Matching for Siamese Networks, 2022
• SimCLR: A Simple Framework for Contrastive Learning of Visual Representations, 2020
• SimMIM: A Simple Framework for Masked Image Modeling, 2021
• SimSiam: Exploring Simple Siamese Representation Learning, 2020
• SMoG: Unsupervised Visual Representation Learning by Synchronous Momentum Grouping, 2022
• SwAV: Unsupervised Learning of Visual Features by Contrasting Cluster Assignments, M. Caron, 2020
• TiCo: Transformation Invariance and Covariance Contrast for Self-Supervised Visual Representation Learning, 2022
• VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning, Bardes, A. et. al, 2022
• VICRegL: VICRegL: Self-Supervised Learning of Local Visual Features, 2022

pydantic2 compatibility

• Lightly is now compatible with pydantic2
• migrated to pyproject

Models

• AIM: Scalable Pre-training of Large Autoregressive Image Models
• Barlow Twins: Self-Supervised Learning via Redundancy Reduction, 2021
• Bootstrap your own latent: A new approach to self-supervised Learning, 2020
• DCL: Decoupled Contrastive Learning, 2021
• DINO: Emerging Properties in Self-Supervised Vision Transformers, 2021
• FastSiam: Resource-Efficient Self-supervised Learning on a Single GPU, 2022
• I-JEPA: Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture, 2023
• MAE: Masked Autoencoders Are Scalable Vision Learners, 2021
• MSN: Masked Siamese Networks for Label-Efficient Learning, 2022
• MoCo: Momentum Contrast for Unsupervised Visual Representation Learning, 2019
• NNCLR: Nearest-Neighbor Contrastive Learning of Visual Representations, 2021
• PMSN: Prior Matching for Siamese Networks, 2022
• SimCLR: A Simple Framework for Contrastive Learning of Visual Representations, 2020
• SimMIM: A Simple Framework for Masked Image Modeling, 2021
• SimSiam: Exploring Simple Siamese Representation Learning, 2020
• SMoG: Unsupervised Visual Representation Learning by Synchronous Momentum Grouping, 2022
• SwAV: Unsupervised Learning of Visual Features by Contrasting Cluster Assignments, M. Caron, 2020
• TiCo: Transformation Invariance and Covariance Contrast for Self-Supervised Visual Representation Learning, 2022
• VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning, Bardes, A. et. al, 2022
• VICRegL: VICRegL: Self-Supervised Learning of Local Visual Features, 2022

Numpy version compatibility

Two changes w.r.t numpy version 2:

• Make lightly itself support numpy version 2: #1561
• Disallow numpy 2.0 in the requirements, as torchvision is not yet compatible with numpy 2: #1562

For more context, see #1558

Models

• AIM: Scalable Pre-training of Large Autoregressive Image Models
• Barlow Twins: Self-Supervised Learning via Redundancy Reduction, 2021
• Bootstrap your own latent: A new approach to self-supervised Learning, 2020
• DCL: Decoupled Contrastive Learning, 2021
• DINO: Emerging Properties in Self-Supervised Vision Transformers, 2021
• FastSiam: Resource-Efficient Self-supervised Learning on a Single GPU, 2022
• I-JEPA: Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture, 2023
• MAE: Masked Autoencoders Are Scalable Vision Learners, 2021
• MSN: Masked Siamese Networks for Label-Efficient Learning, 2022
• MoCo: Momentum Contrast for Unsupervised Visual Representation Learning, 2019
• NNCLR: Nearest-Neighbor Contrastive Learning of Visual Representations, 2021
• PMSN: Prior Matching for Siamese Networks, 2022
• SimCLR: A Simple Framework for Contrastive Learning of Visual Representations, 2020
• SimMIM: A Simple Framework for Masked Image Modeling, 2021
• SimSiam: Exploring Simple Siamese Representation Learning, 2020
• SMoG: Unsupervised Visual Representation Learning by Synchronous Momentum Grouping, 2022
• SwAV: Unsupervised Learning of Visual Features by Contrasting Cluster Assignments, M. Caron, 2020
• TiCo: Transformation Invariance and Covariance Contrast for Self-Supervised Visual Representation Learning, 2022
• VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning, Bardes, A. et. al, 2022
• VICRegL: VICRegL: Self-Supervised Learning of Local Visual Features, 2022

v1.5.7

Tiny improvements

• Increase download timeout for json files (#1556)
• Migrate coverage and mypy configuration to pyproject.toml #1549). Many thanks to @SauravMaheshkar for this improvement!

Models

• AIM: Scalable Pre-training of Large Autoregressive Image Models
• Barlow Twins: Self-Supervised Learning via Redundancy Reduction, 2021
• Bootstrap your own latent: A new approach to self-supervised Learning, 2020
• DCL: Decoupled Contrastive Learning, 2021
• DINO: Emerging Properties in Self-Supervised Vision Transformers, 2021
• FastSiam: Resource-Efficient Self-supervised Learning on a Single GPU, 2022
• I-JEPA: Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture, 2023
• MAE: Masked Autoencoders Are Scalable Vision Learners, 2021
• MSN: Masked Siamese Networks for Label-Efficient Learning, 2022
• MoCo: Momentum Contrast for Unsupervised Visual Representation Learning, 2019
• NNCLR: Nearest-Neighbor Contrastive Learning of Visual Representations, 2021
• PMSN: Prior Matching for Siamese Networks, 2022
• SimCLR: A Simple Framework for Contrastive Learning of Visual Representations, 2020
• SimMIM: A Simple Framework for Masked Image Modeling, 2021
• SimSiam: Exploring Simple Siamese Representation Learning, 2020
• SMoG: Unsupervised Visual Representation Learning by Synchronous Momentum Grouping, 2022
• SwAV: Unsupervised Learning of Visual Features by Contrasting Cluster Assignments, M. Caron, 2020
• TiCo: Transformation Invariance and Covariance Contrast for Self-Supervised Visual Representation Learning, 2022
• VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning, Bardes, A. et. al, 2022
• VICRegL: VICRegL: Self-Supervised Learning of Local Visual Features, 2022

v1.5.6

Changes

• Allow lightly-serve to run securely via https by passing ssl_cert and ssl_key

Models

• AIM: Scalable Pre-training of Large Autoregressive Image Models
• Barlow Twins: Self-Supervised Learning via Redundancy Reduction, 2021
• Bootstrap your own latent: A new approach to self-supervised Learning, 2020
• DCL: Decoupled Contrastive Learning, 2021
• DINO: Emerging Properties in Self-Supervised Vision Transformers, 2021
• FastSiam: Resource-Efficient Self-supervised Learning on a Single GPU, 2022
• I-JEPA: Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture, 2023
• MAE: Masked Autoencoders Are Scalable Vision Learners, 2021
• MSN: Masked Siamese Networks for Label-Efficient Learning, 2022
• MoCo: Momentum Contrast for Unsupervised Visual Representation Learning, 2019
• NNCLR: Nearest-Neighbor Contrastive Learning of Visual Representations, 2021
• PMSN: Prior Matching for Siamese Networks, 2022
• SimCLR: A Simple Framework for Contrastive Learning of Visual Representations, 2020
• SimMIM: A Simple Framework for Masked Image Modeling, 2021
• SimSiam: Exploring Simple Siamese Representation Learning, 2020
• SMoG: Unsupervised Visual Representation Learning by Synchronous Momentum Grouping, 2022
• SwAV: Unsupervised Learning of Visual Features by Contrasting Cluster Assignments, M. Caron, 2020
• TiCo: Transformation Invariance and Covariance Contrast for Self-Supervised Visual Representation Learning, 2022
• VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning, Bardes, A. et. al, 2022
• VICRegL: VICRegL: Self-Supervised Learning of Local Visual Features, 2022

Add unpatchify model utils operation

Changes

• Add unpatchify model utils operation to reconstruct an image from its patches. See the PR for more information. Thanks to @randombenj for implementing this!
• Fixes in CI regarding coverage.
• Fixes in lightly-serve that the server was sometimes not shut down correctly.

Models

• AIM: Scalable Pre-training of Large Autoregressive Image Models
• Barlow Twins: Self-Supervised Learning via Redundancy Reduction, 2021
• Bootstrap your own latent: A new approach to self-supervised Learning, 2020
• DCL: Decoupled Contrastive Learning, 2021
• DINO: Emerging Properties in Self-Supervised Vision Transformers, 2021
• FastSiam: Resource-Efficient Self-supervised Learning on a Single GPU, 2022
• I-JEPA: Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture, 2023
• MAE: Masked Autoencoders Are Scalable Vision Learners, 2021
• MSN: Masked Siamese Networks for Label-Efficient Learning, 2022
• MoCo: Momentum Contrast for Unsupervised Visual Representation Learning, 2019
• NNCLR: Nearest-Neighbor Contrastive Learning of Visual Representations, 2021
• PMSN: Prior Matching for Siamese Networks, 2022
• SimCLR: A Simple Framework for Contrastive Learning of Visual Representations, 2020
• SimMIM: A Simple Framework for Masked Image Modeling, 2021
• SimSiam: Exploring Simple Siamese Representation Learning, 2020
• SMoG: Unsupervised Visual Representation Learning by Synchronous Momentum Grouping, 2022
• SwAV: Unsupervised Learning of Visual Features by Contrasting Cluster Assignments, M. Caron, 2020
• TiCo: Transformation Invariance and Covariance Contrast for Self-Supervised Visual Representation Learning, 2022
• VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning, Bardes, A. et. al, 2022
• VICRegL: VICRegL: Self-Supervised Learning of Local Visual Features, 2022

Multi-GPU

Changes

• Fixes the GatherLayer for multiple GPUs. See PR for more information.
• Different typos in tutorials

Models

• AIM: Scalable Pre-training of Large Autoregressive Image Models
• Barlow Twins: Self-Supervised Learning via Redundancy Reduction, 2021
• Bootstrap your own latent: A new approach to self-supervised Learning, 2020
• DCL: Decoupled Contrastive Learning, 2021
• DINO: Emerging Properties in Self-Supervised Vision Transformers, 2021
• FastSiam: Resource-Efficient Self-supervised Learning on a Single GPU, 2022
• I-JEPA: Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture, 2023
• MAE: Masked Autoencoders Are Scalable Vision Learners, 2021
• MSN: Masked Siamese Networks for Label-Efficient Learning, 2022
• MoCo: Momentum Contrast for Unsupervised Visual Representation Learning, 2019
• NNCLR: Nearest-Neighbor Contrastive Learning of Visual Representations, 2021
• PMSN: Prior Matching for Siamese Networks, 2022
• SimCLR: A Simple Framework for Contrastive Learning of Visual Representations, 2020
• SimMIM: A Simple Framework for Masked Image Modeling, 2021
• SimSiam: Exploring Simple Siamese Representation Learning, 2020
• SMoG: Unsupervised Visual Representation Learning by Synchronous Momentum Grouping, 2022
• SwAV: Unsupervised Learning of Visual Features by Contrasting Cluster Assignments, M. Caron, 2020
• TiCo: Transformation Invariance and Covariance Contrast for Self-Supervised Visual Representation Learning, 2022
• VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning, Bardes, A. et. al, 2022
• VICRegL: VICRegL: Self-Supervised Learning of Local Visual Features, 2022

UX improvements

Changes

• Removed the hydra warning when using lightly-serve
• Improved the error messages and formatting of "well known" errors to improve the readability

Models

• AIM: Scalable Pre-training of Large Autoregressive Image Models
• Barlow Twins: Self-Supervised Learning via Redundancy Reduction, 2021
• Bootstrap your own latent: A new approach to self-supervised Learning, 2020
• DCL: Decoupled Contrastive Learning, 2021
• DINO: Emerging Properties in Self-Supervised Vision Transformers, 2021
• FastSiam: Resource-Efficient Self-supervised Learning on a Single GPU, 2022
• I-JEPA: Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture, 2023
• MAE: Masked Autoencoders Are Scalable Vision Learners, 2021
• MSN: Masked Siamese Networks for Label-Efficient Learning, 2022
• MoCo: Momentum Contrast for Unsupervised Visual Representation Learning, 2019
• NNCLR: Nearest-Neighbor Contrastive Learning of Visual Representations, 2021
• PMSN: Prior Matching for Siamese Networks, 2022
• SimCLR: A Simple Framework for Contrastive Learning of Visual Representations, 2020
• SimMIM: A Simple Framework for Masked Image Modeling, 2021
• SimSiam: Exploring Simple Siamese Representation Learning, 2020
• SMoG: Unsupervised Visual Representation Learning by Synchronous Momentum Grouping, 2022
• SwAV: Unsupervised Learning of Visual Features by Contrasting Cluster Assignments, M. Caron, 2020
• TiCo: Transformation Invariance and Covariance Contrast for Self-Supervised Visual Representation Learning, 2022
• VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning, Bardes, A. et. al, 2022
• VICRegL: VICRegL: Self-Supervised Learning of Local Visual Features, 2022

Better docs for MAE and TIMM

Changes

• add benchmark results for MAE
• add timm version info to docs

Models

• AIM: Scalable Pre-training of Large Autoregressive Image Models
• Barlow Twins: Self-Supervised Learning via Redundancy Reduction, 2021
• Bootstrap your own latent: A new approach to self-supervised Learning, 2020
• DCL: Decoupled Contrastive Learning, 2021
• DINO: Emerging Properties in Self-Supervised Vision Transformers, 2021
• FastSiam: Resource-Efficient Self-supervised Learning on a Single GPU, 2022
• I-JEPA: Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture, 2023
• MAE: Masked Autoencoders Are Scalable Vision Learners, 2021
• MSN: Masked Siamese Networks for Label-Efficient Learning, 2022
• MoCo: Momentum Contrast for Unsupervised Visual Representation Learning, 2019
• NNCLR: Nearest-Neighbor Contrastive Learning of Visual Representations, 2021
• PMSN: Prior Matching for Siamese Networks, 2022
• SimCLR: A Simple Framework for Contrastive Learning of Visual Representations, 2020
• SimMIM: A Simple Framework for Masked Image Modeling, 2021
• SimSiam: Exploring Simple Siamese Representation Learning, 2020
• SMoG: Unsupervised Visual Representation Learning by Synchronous Momentum Grouping, 2022
• SwAV: Unsupervised Learning of Visual Features by Contrasting Cluster Assignments, M. Caron, 2020
• TiCo: Transformation Invariance and Covariance Contrast for Self-Supervised Visual Representation Learning, 2022
• VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning, Bardes, A. et. al, 2022
• VICRegL: VICRegL: Self-Supervised Learning of Local Visual Features, 2022

MAE to use TIMM VIT

Changes

• Refactor MAE to use TIMM VIT
• Add AIM examples and docs
• Updated BYOL and MOCO benchmarks
• Use MMCRProjectionHead in examples

Models

• AIM: Scalable Pre-training of Large Autoregressive Image Models
• Barlow Twins: Self-Supervised Learning via Redundancy Reduction, 2021
• Bootstrap your own latent: A new approach to self-supervised Learning, 2020
• DCL: Decoupled Contrastive Learning, 2021
• DINO: Emerging Properties in Self-Supervised Vision Transformers, 2021
• FastSiam: Resource-Efficient Self-supervised Learning on a Single GPU, 2022
• I-JEPA: Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture, 2023
• MAE: Masked Autoencoders Are Scalable Vision Learners, 2021
• MSN: Masked Siamese Networks for Label-Efficient Learning, 2022
• MoCo: Momentum Contrast for Unsupervised Visual Representation Learning, 2019
• NNCLR: Nearest-Neighbor Contrastive Learning of Visual Representations, 2021
• PMSN: Prior Matching for Siamese Networks, 2022
• SimCLR: A Simple Framework for Contrastive Learning of Visual Representations, 2020
• SimMIM: A Simple Framework for Masked Image Modeling, 2021
• SimSiam: Exploring Simple Siamese Representation Learning, 2020
• SMoG: Unsupervised Visual Representation Learning by Synchronous Momentum Grouping, 2022
• SwAV: Unsupervised Learning of Visual Features by Contrasting Cluster Assignments, M. Caron, 2020
• TiCo: Transformation Invariance and Covariance Contrast for Self-Supervised Visual Representation Learning, 2022
• VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning, Bardes, A. et. al, 2022
• VICRegL: VICRegL: Self-Supervised Learning of Local Visual Features, 2022