PERF: avoid creating many Series in apply_standard

[jbrockmendel]

• closes #xxxx
• tests added / passed
• passes black pandas
• passes git diff upstream/master -u -- "*.py" | flake8 --diff
• whatsnew entry

This avoids going through perennial-problem-causing libreduction code (xref #34014, #34080) and instead does the same trick in python-space by re-assigning block.values instead of creating new Series objects.

If we avoid libreduction but dont do this optimization, the most-affected asv is time_apply_ref_by_name that clocks in at 6.92x slower. This achieves parity on that asv.

ATM I'm still getting 4 test failures locally, need to troubleshoot. Update: passing

[jreback]

can you push this to an internals method instead?

[jbrockmendel]

im looking at that now. the other place where this pattern could be really useful is in groupby.ops, but its tougher there

[jreback]

sure also exposing an api for this would be ok as well (eg another internals method)

[jbrockmendel]

im still troubleshooting the groupby.ops usage, would like to punt on making this an internals method for the time being

[jreback]

does this fix #34506 ?

[jbrockmendel]

does this fix #34506 ?

The example from the OP there was fixed by #34913, but i still need to double-check the general case and add a test.

[jbrockmendel]

The groupby.ops analogue of this is continuing to be a PITA, so im exploring other avenues there (xred #34982). This should be good to go though, and as a follow-up we can rip out libreduction.Reducer.

[jreback]

thanks, yeah incremental refactors are good.

[jorisvandenbossche]

This commit caused some regressions, eg https://pandas.pydata.org/speed/pandas/#frame_methods.Apply.time_apply_np_mean,
https://pandas.pydata.org/speed/pandas/#frame_methods.Apply.time_apply_ref_by_name?x-axis-scale=date

Was that expected? (above you mention "achieves parity on that asv.")

[jorisvandenbossche]

Ah, I suppose this is just fixed by #35166