Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: GH25495 incorrect dtype when using .loc to set Categorical value for column in 1-row DataFrame #29393

Merged
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
712e871
Fix indexing.setitem when type is pd.Categorical
keechongtan Nov 4, 2019
a23b3c9
Move categorical casting check further up
keechongtan Nov 6, 2019
550d68d
Move block creation to after we've check for exact match in shape
keechongtan Nov 6, 2019
c945ab1
Remove unused whitespace
keechongtan Nov 6, 2019
5d47754
Move test to TestDataFrameIndexingCategorical, add issue number
keechongtan Nov 7, 2019
2dfa594
Move checks to seperate elif
keechongtan Nov 8, 2019
3e847e9
Merge remote-tracking branch 'upstream/master' into bug/categorical-i…
keechongtan Nov 11, 2019
8c5a88e
Update whatsnew entry
keechongtan Nov 16, 2019
ca60804
Merge remote-tracking branch 'upstream/master' into bug/categorical-i…
keechongtan Nov 25, 2019
b1520b8
Merge branch 'bug/categorical-indexing-1row-df' of github.com:keechon…
keechongtan Nov 25, 2019
2e95853
Move tests to tests/frame/indexing/test_categorical.py
keechongtan Nov 25, 2019
39c95f4
Fix Pep8 Warning
keechongtan Nov 25, 2019
2b71592
Merge remote-tracking branch 'upstream/master' into bug/categorical-i…
keechongtan Nov 26, 2019
e5c1420
Merge remote-tracking branch 'upstream/master' into bug/categorical-i…
keechongtan Nov 29, 2019
4257fe8
use is_categorical_dtype(arr_value.dtype), as is_categorical_astype i…
keechongtan Dec 2, 2019
593dda1
Merge remote-tracking branch 'upstream/master' into bug/categorical-i…
keechongtan Dec 2, 2019
5512119
put exact_match as the first condition
keechongtan Dec 29, 2019
c5a7f6e
Merge remote-tracking branch 'upstream/master' into bug/categorical-i…
keechongtan Dec 29, 2019
4c339ac
Merge remote-tracking branch 'upstream/master' into bug/categorical-i…
keechongtan Jan 13, 2020
241bd7c
Remove merge conflict comment
keechongtan Jan 13, 2020
41e6ce4
Merge remote-tracking branch 'upstream/master' into bug/categorical-i…
keechongtan Jan 27, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions pandas/core/internals/blocks.py
Original file line number Diff line number Diff line change
Expand Up @@ -883,6 +883,14 @@ def setitem(self, indexer, value):
):
values[indexer] = value
try:
# GH25495 may need to convert to categorical block
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't put complicated things inside a try/except it tends to hide errors

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this could be much earlier in the comparisons

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've moved it out of the try/except.

However, the intended behaviour is that we only convert to categorical if it is a full assignment, and only change the values in a partial assignment. Hence the block creation needs to be done after we've checked the shape of the new values matches existing values.

if self.is_categorical_astype(
arr_value.dtype
) and not is_categorical_dtype(values):
return self.make_block(
Categorical(self.values, dtype=arr_value.dtype)
)

values = values.astype(arr_value.dtype)
except ValueError:
pass
Expand Down
9 changes: 9 additions & 0 deletions pandas/tests/frame/test_indexing.py
Original file line number Diff line number Diff line change
Expand Up @@ -204,6 +204,15 @@ def test_setitem_list_of_tuples(self, float_frame):
expected = Series(tuples, index=float_frame.index, name="tuples")
tm.assert_series_equal(result, expected)

def test_setitem_single_row_categorical(self):
df = DataFrame({"Alpha": ["a"], "Numeric": [0]})
keechongtan marked this conversation as resolved.
Show resolved Hide resolved
categories = pd.Categorical(df["Alpha"], categories=["a", "b", "c"])
df.loc[:, "Alpha"] = categories
keechongtan marked this conversation as resolved.
Show resolved Hide resolved

result = df["Alpha"]
expected = Series(categories, index=df.index, name="Alpha")
tm.assert_series_equal(result, expected)

def test_setitem_mulit_index(self):
# GH7655, test that assigning to a sub-frame of a frame
# with multi-index columns aligns both rows and columns
Expand Down