Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

[MXNET-1253] [WIP] Fix large array issue in topk operator #13667

Closed
wants to merge 11 commits into from

Conversation

apeforest
Copy link
Contributor

@apeforest apeforest commented Dec 18, 2018

Description

This is a second batch of fixes for several sorting operators.

Checklist

Essentials

Please feel free to remove inapplicable items for your PR.

  • The PR title starts with [MXNET-$JIRA_ID], where $JIRA_ID refers to the relevant JIRA issue created (except PRs with tiny changes)
  • Changes are complete (i.e. I finished coding on this PR)
  • All changes have test coverage:
  • Unit tests are added for small changes to verify correctness (e.g. adding a new operator)
  • Nightly tests are added for complicated/long-running ones (e.g. changing distributed kvstore)
  • Build tests will be added for build configuration changes (e.g. adding a new build option with NCCL)
  • Code is well-documented:
  • For user-facing API changes, API doc string has been updated.
  • For new C++ functions in header files, their functionalities and arguments are documented.
  • For new examples, README.md is added to explain the what the example does, the source of the dataset, expected performance on test set and reference to the original paper if applicable
  • Check the API doc at http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html
  • To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change

Changes

  • topk
  • test_large_array.py

Comments

  • Removed dtype for topk operator for indices. It seems unnecessary (and confusing) to have float32 as a type for indices and not clear why user would ever want to define it. This may cause backward compatibility issue only if users set dtype for return indices of a topK operator.

@Roshrini
Copy link
Member

@apeforest Thanks for working on this.
@mxnet-label-bot Add [pr-work-in-progress, Operator]

@marcoabreu marcoabreu added Operator pr-work-in-progress PR is still work in progress labels Dec 18, 2018
@@ -58,7 +58,6 @@ struct TopKParam : public dmlc::Parameter<TopKParam> {
int k;
int ret_typ;
bool is_ascend;
int dtype;
Copy link
Member

@TaoLv TaoLv Dec 20, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like an incompatible change.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agree. could you please revert the change?

@Roshrini
Copy link
Member

Roshrini commented Jan 2, 2019

@apeforest Can you address comments and rebase this PR? Thanks

@stu1130
Copy link
Contributor

stu1130 commented Jan 16, 2019

@apeforest ping again for the update

@apeforest
Copy link
Contributor Author

Got fully occupied with another project. I will resume this PR when I get some slack next week.

@ankkhedia
Copy link
Contributor

@apeforest Did you get a chance to work on this PR?
#13627 depends on this fix.

@pinaraws
Copy link

@apeforest Any updates?

@piyushghai
Copy link
Contributor

@apeforest Can you rebase with the current master branch and look into the CI failures ?

@Roshrini
Copy link
Member

@apeforest Any update on this PR?

1 similar comment
@pinaraws
Copy link

@apeforest Any update on this PR?

@apeforest
Copy link
Contributor Author

apeforest commented Jun 4, 2019

@access2rohit is working on this in a new PR. Closing this one.

@apeforest apeforest closed this Jun 4, 2019
@apeforest apeforest deleted the bugfix/large-array-part2 branch February 25, 2020 18:34
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Operator pr-work-in-progress PR is still work in progress
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants