Graphs of accuracy and loss during final training process

[dhlee-jubilee]

Compared to the normal training process, train acc continues to increase and losses should continue to decline. However, the ENAS process shows that there are periodically periods in which the performance decreases drastically during the training process.
What parameters caused this results?

[AlanChou]

Hi, I suppose you're showing the results of running micro_final.sh or macro_final.sh.
This phenomenon is due to the cosine learning rate scheduling with restart.