REF: use BlockManager.apply for Rolling.count #35883
Conversation
| results = [] | ||
| for b in blocks: | ||
| result = b.notna().astype(int) | ||
| _, obj = self._create_blocks(self._selected_obj) |
There was a problem hiding this comment.
I was actually hoping that count could be defined in terms of self._apply so we can have less custom logic
def count(self):
window_func = self._get_cython_func_type("roll_count")
return self._apply(window_func, center=self.center, floor=0, name="count", **kwargs)
Might be not as straightforward (possibly) but just a heads up of where (hopefully) count should head.
There was a problem hiding this comment.
is this something you're planning on doing in the forseeable future? i agree thatd be a nicer solution
There was a problem hiding this comment.
I'll experiment with this tonight and get a PR up if tests pass
There was a problem hiding this comment.
Looks like this version allows (inconsistently with other rolling function) non-numeric data like datetimes and strings because of the ahead of time coercing with notna().astype(int). Taking longer than expected to tie out tests with using the roll_count cython version
There was a problem hiding this comment.
Not sure if anything can be shared, but it looks like DataFrameGroupBy.count is doing things kludgily blockwise and should also use an apply pattern
jreback
added
Refactor
…f-count-blockwise
|
Thanks @jbrockmendel. Couldn't find a straightforward simplification immediately, so this is a nice cleanup. |
* REF: remove unnecesary try/except * TST: add test for agg on ordered categorical cols (pandas-dev#35630) * TST: resample does not yield empty groups (pandas-dev#10603) (pandas-dev#35799) * revert accidental rebase * REF: use BlockManager.apply for Rolling.count Co-authored-by: Karthik Mathur <22126205+mathurk1@users.noreply.github.com> Co-authored-by: tkmz-n <60312218+tkmz-n@users.noreply.github.com>
* REF: remove unnecesary try/except * TST: add test for agg on ordered categorical cols (pandas-dev#35630) * TST: resample does not yield empty groups (pandas-dev#10603) (pandas-dev#35799) * revert accidental rebase * REF: use BlockManager.apply for Rolling.count Co-authored-by: Karthik Mathur <22126205+mathurk1@users.noreply.github.com> Co-authored-by: tkmz-n <60312218+tkmz-n@users.noreply.github.com>
black pandasgit diff upstream/master -u -- "*.py" | flake8 --diff