Allow setting per-layer learning rates"

[shaydeci]

• Added support for passing initial_lr as a dictionary.
• Removed the usages of update_param_groups and initialize_param_groups. Affected recipes had equivalent initial_lr mapping added to them (tested).
• For the edge cases of having an instantiated optimiser, I assign names to the parameter_groups and extrac an initial_lr mapping from them so our schedulers can be used.

[BloodAxe]

I went pretty brutal on this PR, sorry :)
I mean, it's good. It written well, it introduce new features while keeping support of existing logic, and it's clean. Just a few tricky things here and there.

[BloodAxe]

And cherry on top - let's rename the PR title to be "Github release notes"-friendly. Something like: "Allow setting per-layer learning rates"

[BloodAxe]

One last remark from me - please add integration test for any model of your choice where we train a model for one short epoch with backbone lr=0, nech=0.1xLR, head=LR to ensure all works end-to-end
And then - LGTM :)

[shaydeci]

One last remark from me - please add integration test for any model of your choice where we train a model for one short epoch with backbone lr=0, nech=0.1xLR, head=LR to ensure all works end-to-end
And then - LGTM :)

Adde unit test training with frozen part of the net.