Loss is not decrease

[Lyhour-Chhay]

I have been train the model my custom dataset. I have been trained up to 50 epoch. The training loss does not decrease around 1.6. The training is going well without NaN loss value. I already play around with hyper parameter such as learning rate vary from 0.01 to 0.0001 using Adam optimizer and SGD. I don't know what should I check more to make the loss decrease. Thank you very much.