Different calculation results when running on CUP or MPS. (MAC M3 PRO OS14.5)

[OneManShow0815]

Hi all,
When running I get different calculation results when running (MAC M3 PRO OS14.5) on CUP or MPS.
My script runs a Quadratic equation instead of a linear discretion, but the rest is from Daniels 01. PyTorch Workflow Fundamentals.
In CPU is calculates good with results, on GPU it runs totally different with NaN on the values.


Only differences on the scripts are the transformations on tensors to CPU for the plot function due to numpy does not take MPS tensors. All running on Jupiter Notebook 7.0.8. all modules latest level, pytorch 2.2.2, python 3.11.9

Both notebooks are attached here:
Exercise_CPU_vs_MPS.zip
Any one a clue why that is or a fix?

Regards
Sven

[LuluW8071]

Try increasing learning rate to 1e-5 and see print results. To me it seems like loss just explodes.

On cpu: your test loss is increasing
On gpu: your test loss in extremely low from the start.

Also u might need to check ur quadratic eqn as well. I have found similar type of cases when i experimented during my start of learning phase as well.

[OneManShow0815]

Hi LuluW8071,

Thanks for your help. Great.
Found a solution. When I run the X not from 0 to 50 instead from 0 to 5 step 0.1. That works great.
Probably due the float32 which the MPS is using while CPU uses float64 as standard.

Finding is that a for loop with MPS takes longer as with COU, but with the same number of loops the result with MPS is much better as with CPU.
Any clue why?

Best Regards
Sven