Add Tian2022Deeper

Merge pull request #133 from Hollen0318/Hollen

notes for dataset and threat model

timm integration and new models

What's Changed

• Added new model ID's by @nmndeep in #52
• Modas2021_PRIME models added by @nmndeep in #57
• readme correction by @nmndeep in #60
• Remove all .cuda() calls by @dedeswim in #61
• Enable using ImageNet benchmarking for new models by @dedeswim in #64
• add Kang2021Stable model by @fra31 in #66
• sort models including external evaluations by @fra31 in #69
• add results from Erichson2022NoisyMix by @fra31 in #70
• Explicit encoding in setup.py to fix #72 by @dedeswim in #74
• Updated results for Sehwag2021Proxy by @VSehwag in #76
• Pang2022Robustness models added by @nmndeep in #77
• additional data flags corrected by @nmndeep in #78
• add models from Jia2022LAS-AT by @fra31 in #80
• remove unnecessary file when loading custom ImageNet by @CNOCycle in #75
• update info of Sridhar2021Robust models by @fra31 in #82
• ImageNet-3DCC and corruption updates by @ofkar in #85
• add models from Addepalli2022Efficient by @fra31 in #91
• throw error when too many images are requested by @fra31 in #96
• Add Debenedetti2022Light and support for timm by @dedeswim in #100

New Contributors

• @CNOCycle made their first contribution in #75
• @ofkar made their first contribution in #85

Full Changelog: v1.0...v1.1

v1.0 - Code corresponding to NeurIPS'21 Datasets and Benchmarks version of the whitepaper

Updates:

• New ImageNet leaderboards (Linf and common corruption) and relevant benchmarking functions (including ImageNet evaluation on a fixed subset of 10% of the test set).
• New models and leaderboard entries (now in total: 120+ evaluations, 80+ models).

v0.2.1 - Various fixes

This minor release improves some internals and fixes some bugs in the model zoo.

Internals improvements:

• Now, when no normalization is applied, the models in the model zoo are loaded with anonymous lambda functions instead of full-fledged classes to keep the code more concise and cleaner.

Bug fixes

• The CIFAR-100 version of Heyndrycks2019Using was missing rescaling in [-1, 1], hence leading to poor accuracy.
• The CIFAR-100 version of Rice2020Overfitting was missing a slightly different step from the other PreActResNets in the forward method of the PreActBlock.

v0.2 - CIFAR-100 and new models

This release adds the following new leaderboards:

• CIFAR-100 Linf
• CIFAR-100-C (common corruptions)

Moreover, it adds to the model zoo:

• 7 new models for CIFAR-10 Linf (28 in total)
• 2 new models for CIFAR-10 L2 (11 in total)
• 10 new models for CIFAR-100 Linf (10 in total)
• 2 new models for CIFAR-100-C (2 in total)

It also fixes some bugs and improves some internals:

• The the common corruptions datasets are now downloaded from the original Zenodo repositories, instead of Google Drive
• The benchmark function now raises a working if it is run on a model which is not in .eval() mode

v0.1

This is the first stable release of RobustBench. It includes the following features:

• A Model Zoo and a leaderboard containing up-to-date and State of the Art models trained on CIFAR-10 for robustness to L2 and Linf adversarial perturbations, and common corruptions.
• An updated API that makes it easier to add new datasets and threat models.
• A function to benchmark the robustness of new models on a given dataset and threat model.
• Functions to automatically generate the leaderboards of our website, and to generate LaTeX tables.