BUG: .apply with collections

[svjack]

consider following example:

pd.DataFrame([[0, 1, 2], [3, 4, 5]]).apply(lambda x: [x] * 2, axis = 1)
[[3, 4, 5], [3, 4, 5]]
[[3, 4, 5], [3, 4, 5]]

but if use numpy array to replace list, i.e.

pd.DataFrame([[0, 1, 2], [3, 4, 5]]).apply(lambda x: np.asarray([x]) * 2, axis = 1)
[[0, 2, 4]]
[[6, 4, 8]]

I can not understand the conclusion use list (first example)
if pandas take element in apply(axis = 1) as pd.Series
And [x] cast it to nest list for example ,
First Row [[0, 1, 2]] * 2 -> [[0, 1, 2], [0, 1, 2]]
Second Row [[3, 4, 5]] * 2 -> [[3, 4, 5], [3, 4, 5]]
should not have above conclusion.
It seems pandas only maintain the last row and apply it to all rows.
And when use replace [x] by {“x”: x},
Have similar problem.

I can understand use numpy collection and pandas collection
inside the elements of DataFrame (avoid use list tuple or dict) may be more safe.
But please explain this conclusion for me.

My pandas version is 1.1.3

[rhshadrach]

Thanks for the report! Agree this is unexpected, likely due to some optimizations apply uses under the hood. Further investigations and PRs to fix are welcome. As a temporary workaround, you can copy x to achieve the desired result:

df = pd.DataFrame([[1], [2]])
print(df.apply(lambda x: [x.copy()] * 2, axis = 1))

gives

0    [[1], [1]]
1    [[2], [2]]
dtype: object

[svjack]

Thanks for the report! Agree this is unexpected, likely due to some optimizations apply uses under the hood. Further investigations and PRs to fix are welcome. As a temporary workaround, you can copy x to achieve the desired result:

df = pd.DataFrame([[1], [2]])
print(df.apply(lambda x: [x.copy()] * 2, axis = 1))

gives

0    [[1], [1]]
1    [[2], [2]]
dtype: object

Does this related with:
Fixed regression in DataFrame.apply() where functions that altered the input in-place only operated on a single row (GH35462)
this may be a problem about BlockManager
In the series_generator in FrameColumnApply, the yield series if use its copy,
will not have this problem.
But this edit may sacrifice performance.
All about
“blk.values = arr”
in the series_generator.
use this single one row block to save values,
So the final v set will replace the previous value in the results dictionary in
apply_series_generator
It seems If you must fixed this problem,
you can not only use a Series object to swap data , in series_generator
This will increase the space complexiity.

[rhshadrach]

While somewhat related to #35462, I think this issue is different in at least one important aspect: the code the leads to this behavior does not mutate the input x.