Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix scalar iloc rebase #17163

Closed
wants to merge 3 commits into from
Closed

Conversation

toobaz
Copy link
Member

@toobaz toobaz commented Aug 3, 2017

This is just a rebased version of #15778 . I want to make sense of the tests which failed, and which work fine instead on my machine.

Notice that I'm also working on a more complete approach to positional indexing, which will obsolete part of this PR, but that's going to take a while (and this PR, besides fixing the bug, is a step in that direction, since it rationalizes a bit positional indexing).

@gfyoung gfyoung added Indexing Related to indexing on series/frames, not to indexes themselves Internals Related to non-user accessible pandas implementation labels Aug 3, 2017
@toobaz
Copy link
Member Author

toobaz commented Aug 6, 2017

Wow. With #15778 , 2 circleci nodes out of 4 failed. Rebased, now only 1 fails. Trying a trivial rebase, just to be sure, sorry for the noise.

@codecov
Copy link

codecov bot commented Aug 6, 2017

Codecov Report

Merging #17163 into master will increase coverage by <.01%.
The diff coverage is 93.87%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #17163      +/-   ##
==========================================
+ Coverage   90.99%      91%   +<.01%     
==========================================
  Files         162      162              
  Lines       49508    49528      +20     
==========================================
+ Hits        45052    45072      +20     
  Misses       4456     4456
Flag Coverage Δ
#multiple 88.78% <93.87%> (+0.02%) ⬆️
#single 40.28% <79.59%> (-0.05%) ⬇️
Impacted Files Coverage Δ
pandas/core/indexing.py 94.05% <100%> (+0.1%) ⬆️
pandas/core/dtypes/cast.py 87.52% <72.72%> (-0.3%) ⬇️
pandas/io/gbq.py 25% <0%> (-58.34%) ⬇️
pandas/core/frame.py 97.66% <0%> (-0.1%) ⬇️
pandas/core/internals.py 94.03% <0%> (-0.02%) ⬇️
pandas/core/series.py 95.03% <0%> (-0.01%) ⬇️
pandas/io/formats/format.py 96.03% <0%> (-0.01%) ⬇️
pandas/io/parsers.py 95.45% <0%> (-0.01%) ⬇️
pandas/plotting/_converter.py 65.05% <0%> (+1.81%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 929c66f...aeb8bf7. Read the comment docs.

@codecov
Copy link

codecov bot commented Aug 6, 2017

Codecov Report

Merging #17163 into master will decrease coverage by 0.01%.
The diff coverage is 93.87%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #17163      +/-   ##
==========================================
- Coverage      91%   90.98%   -0.02%     
==========================================
  Files         162      162              
  Lines       49508    49538      +30     
==========================================
+ Hits        45055    45073      +18     
- Misses       4453     4465      +12
Flag Coverage Δ
#multiple 88.76% <93.87%> (ø) ⬆️
#single 40.29% <79.59%> (-0.04%) ⬇️
Impacted Files Coverage Δ
pandas/core/indexing.py 94.05% <100%> (+0.1%) ⬆️
pandas/core/dtypes/cast.py 87.52% <72.72%> (-0.3%) ⬇️
pandas/io/gbq.py 25% <0%> (-58.34%) ⬇️
pandas/core/frame.py 97.71% <0%> (-0.1%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7bef6d8...b8ee051. Read the comment docs.

@toobaz
Copy link
Member Author

toobaz commented Aug 7, 2017

(The failing test will be solved by #17002 - will rebase then)

@pep8speaks
Copy link

pep8speaks commented Aug 9, 2017

Hello @toobaz! Thanks for updating the PR.

Cheers ! There are no PEP8 issues in this Pull Request. 🍻

Comment last updated on August 09, 2017 at 09:49 Hours UTC

@@ -1085,3 +1085,24 @@ def cast_scalar_to_array(shape, value, dtype=None):
values.fill(fill_value)

return values


def _maybe_convert_indexer(indexer, until):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is likely duplicating some code in pandas.core.indexing. Also doesn't belong here as its purely an indexing converter (so put it in pandas.core.indexing)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also doesn't belong here as its purely an indexing converter (so put it in pandas.core.indexing)

I will use it also in BlockManager: can you confirm you suggestion is still valid?

Copy link
Contributor

@jreback jreback Aug 10, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that's fine. it just belongs in internal utlities related to indexing (and not internal casting type things). yes it is casting but is purely related to indexing (and not data types)


def _maybe_convert_indexer(indexer, until):
"""
Convert slice, tuple, list or scalar "indexer" to 1-d array of indices,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be ok to first do a PR to add a level to our indexing structure of code, IOW

pandas.core.indexing -> pandas.core.indexing.indexing, then you add other help modules as needed
e.g. pands.core.indexing.cast for example (where things like this could live).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just for completeness: recall that part of this (i.e. the InfoCleaner) is temporary, until I finish the work on BlockManager (while _maybe_convert_to_indexer will stay - it is used by the BlockManager too - as well as part of the new _setter). In the long run, I expect the amount and complexity of indexing code to actually decrease. So I'm a bit against more hierarchy. Actually, if you find this PR too messy, I'd rather close it and just wait for the work on BlockManager (which however will take more time, so the bug will have to wait).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah I'd rather not introduce a 'temporary' fix.

@toobaz
Copy link
Member Author

toobaz commented Aug 10, 2017

yeah I'd rather not introduce a 'temporary' fix.

OK, then never mind.

@toobaz toobaz closed this Aug 10, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Indexing Related to indexing on series/frames, not to indexes themselves Internals Related to non-user accessible pandas implementation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

BUG: .iloc indexing with duplicates
4 participants