Updates SG PropertyGraph and cugraph-service to apply DataFrame.fillna() based on latest cuDF changes
#4408
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…perations on the entire vertex or edge DataFrame, to accommodate an update to cuDF which does not accept all fillna values as-is if applied to a categorical dtype.
…6-update_SG_PG_for_cudf
rlratzel
added
improvement
…ategoricals, updates tests to use latest datasets API.
rlratzel
changed the title
PropertyGraph to apply fillna() only to user-defined columns when non-dict/non-Series values passed inPropertyGraph and cugraph-service to apply DataFrame.fillna() based on latest cuDF changes
|
/merge |
|
/merge |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This handles a recent cuDF change by applying non-dict and non-Series values for a
fillna()call onPropertyGraphinstances only to the user-defined columns, with the assumption that savvy users that intend to update the "internal" columns, or users that are aware of their own categorical dtype columns, will use a dict or Series value to properly apply dtypes as needed.This also updates code in
cugraph-servicethat serializes dataframes to numpy bytes to properly convert NA values when categoricals are present.Notes:
PropertyGraphclass. The MG class needs further review as to how to best apply the same policy (and because there are other MG failing tests that need addressed). Since this is blocking CI for the SG case only, this PR is being submitted now and MG will be addressed later, which should be okay sincePropertyGraphis experimental.PropertyGraphwas not experimental.