fix incorrect learning rate warm-up after restarting from ckpt

[bozhang-hpc]

A simple fix to #181

[gahdritz]

Could you explain how this works? Where is resume_last_lr_step defined? When is self.last_lr_step ever assigned to anything except -1?

[bozhang-hpc]

Oh, I forgot the definition of resume_last_lr_step(). It's added in the 69a8a28 commit.

[gahdritz]

One more thing: this assumes that the resume_from_ckpt is a ZERO checkpoint. You should add a quick check that the resume_from_ckpt argument is a directory and not a file.

[bozhang-hpc]

The non deepspeed ckpt file is supported now.