Skip to content

Implementation of ReXNet-tiny model using PyTorch

License

Notifications You must be signed in to change notification settings

ysbsb/ReXNet-PyTorch

Repository files navigation

ReXNet-tiny on CIFAR10 PyTorch

Implementation of ReXNet-tiny model on CIFAR10 using PyTorch

It's unofficial code! [origin paper]

Abstract

(1) simple and effective design principles to mitigate the representational bottleneck

(2) study the matrix rank on the features generated by ten thousand random networks

Method

rexnet-rank

rexnet

Run

ReXNet-tiny

CUDA_VISIBLE_DEVICES=0 python train.py --epochs 100 \
--savepath ./checkpoint/cifar10/sgd

ReXNet-tiny with SGDMix

CUDA_VISIBLE_DEVICES=0 python train_sgdmix.py --epochs 100 \
--savepath ./checkpoint/cifar10/sgdmix \
--name sgdmix

ReXNet-tiny with RandAugment12 and cutmix 0.5

CUDA_VISIBLE_DEVICES=0 python train_sgd_randaugcutout_cutmix.py \
--savepath ./checkpoint/cifar10/sgd_randaugsubin_cutmix12 \
--name sgd_randaugsubin_cutmix12 \
--rand_n 1 --rand_m 2

Experiement

1. ReXNet-tiny-cifar with optimizer

Model Input Res. Optimizer Top-1 acc. Params
ReXNet-tiny-cifar 32x32 Adam 93.82 1.9M
ReXNet-tiny-cifar 32x32 AdamW 93.82 1.9M
ReXNet-tiny-cifar 32x32 AdamP TODO 1.9M
ReXNet-tiny-cifar 32x32 SGD 94.90 1.9M
ReXNet-tiny-cifar 32x32 SGDP 94.43 1.9M
ReXNet-tiny-cifar 32x32 SGDMix 95.04 1.9M

2. Scheduler

Model Input Res. Optimizer Top-1 acc. Params
ReXNet-tiny-cifar, step 30 32x32 SGDMix 95.04 1.9M
ReXNet-tiny-cifar, multistep 30 32x32 SGDMix 95.13 1.9M
ReXNet-tiny-cifar, cosine 30 32x32 SGDMix 95.02 1.9M
Model Input Res. Optimizer Top-1 acc. Params
ReXNet-tiny-cifar, Multi step [30, 60, 90] 32x32 SGDMix TODO 1.9M

3. Augmentation

Model Input Res. Optimizer Top-1 acc. Params
ReXNet-tiny-cifar 32x32 Adam 93.82 1.9M
ReXNet-tiny-cifar + cutmix 32x32 Adam 94.97 1.9M
ReXNet-tiny-cifar + cutmix 32x32 SGDMix TODO 1.9M
ReXNet-tiny-cifar + randaug (1, 2) 32x32 Adam 94. 1.9M
ReXNet-tiny-cifar + randaug (1, 2) + cutmix 32x32 Adam 95.02 1.9M
ReXNet-tiny-cifar + randaug (1, 2) + cutmix 32x32 SGDMix TODO 1.9M

4. Comparision with efficientnet-cifar

Model Input Res. Optimizer Top-1 acc. Params
Efficientnet-tiny-cifar 32x32 Adam 93.18
ReXNet-tiny-cifar 32x32 Adam 93.82 1.9M

5. Model Compression (TODO)


Author

  • Subin Yang

About

Implementation of ReXNet-tiny model using PyTorch

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published