Fix scatter, reduce

[awni]

• Fixes scatter for large sizes
• Probably will be part of a fix for some reductions for large sizes

TODO: measure impact on runtime to see if we need to specialize this.

[angeloskath]

Ok I think this is ready for a review at least the reduction part (or I am just tired of improving it). I will add some benchmarks and then go ahead and benchmark the scatter that is also in this PR.

It is a bit big and there are several changes on various parts so sorry about that. I will also write a long running test and a fuzzer to make sure that every size is properly checked in our nightly tests.

All reduce

When the array is big everything practically gets the memory bandwidth so it's all the same. But there is a difference for int8.

Row reduce

Col reduce

[awni]

🤤

[awni]

I don't know what kind of magic you did here..but those graphs are pretty amazing.