Skip to content
This repository has been archived by the owner on Jun 27, 2023. It is now read-only.

Size-based unsharding #94

Merged
merged 38 commits into from
Nov 12, 2021
Merged

Size-based unsharding #94

merged 38 commits into from
Nov 12, 2021

Conversation

schomatis
Copy link
Contributor

@schomatis schomatis commented Jun 1, 2021

Update 8/26: Working on tests in #99. Moving minor changes for last in #105.


Working on tests in separate PR (to be merged on top of this one): #99.


Review in progress.

Most important items to address:

@schomatis schomatis self-assigned this Jun 1, 2021
io/directory.go Outdated Show resolved Hide resolved
io/directory.go Outdated Show resolved Hide resolved
io/directory.go Outdated Show resolved Hide resolved
@schomatis schomatis force-pushed the schomatis/directory/unsharding branch from 8223156 to 068bcdd Compare June 20, 2021 15:42
This was referenced Jun 23, 2021
@schomatis

This comment has been minimized.

@schomatis schomatis force-pushed the schomatis/directory/unsharding branch from 068bcdd to 3301178 Compare June 29, 2021 19:42
@schomatis

This comment has been minimized.

Copy link
Contributor

@aschmahmann aschmahmann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks and sorry for the late review. It looks like there are still a number of FIXMEs in the PR which are needed pre-merge. I commented on the ones where it looks like you had questions, but if there are more please tag me.

It looks like the tests for this PR are split between here and #99. Which would you like me to take a look at? Should #99 be rebased on this PR (or just merged in?).

@Stebalien I left some comments/questions for you when convenient. Before we do the final merge here though I'll want another set of eyes.

io/directory.go Outdated Show resolved Hide resolved
io/directory.go Outdated Show resolved Hide resolved
io/directory.go Outdated Show resolved Hide resolved
io/directory.go Outdated Show resolved Hide resolved
io/directory.go Outdated Show resolved Hide resolved
io/directory.go Show resolved Hide resolved
io/directory.go Outdated Show resolved Hide resolved
io/directory.go Outdated Show resolved Hide resolved
hamt/hamt.go Outdated Show resolved Hide resolved
hamt/hamt.go Outdated Show resolved Hide resolved
Copy link
Member

@Stebalien Stebalien left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

naming nits

hamt/hamt.go Outdated Show resolved Hide resolved
hamt/hamt.go Outdated Show resolved Hide resolved
hamt/hamt.go Outdated Show resolved Hide resolved
hamt/hamt.go Outdated Show resolved Hide resolved
hamt/hamt.go Outdated Show resolved Hide resolved
hamt/hamt.go Outdated Show resolved Hide resolved
hamt/hamt.go Outdated Show resolved Hide resolved
hamt/hamt.go Outdated Show resolved Hide resolved
hamt/hamt.go Outdated Show resolved Hide resolved
io/directory.go Outdated Show resolved Hide resolved
io/directory.go Outdated Show resolved Hide resolved
io/directory.go Outdated Show resolved Hide resolved
io/directory_test.go Outdated Show resolved Hide resolved
io/directory_test.go Outdated Show resolved Hide resolved
@schomatis schomatis marked this pull request as ready for review October 21, 2021 14:24
schomatis and others added 15 commits October 22, 2021 18:24
Co-authored-by: Adin Schmahmann <adin.schmahmann@gmail.com>
Co-authored-by: Adin Schmahmann <adin.schmahmann@gmail.com>
* fix(dir): remove timeout and evaluate switch *before* removing entry

* BLOCKING: temporary fix

* remove Shard cid not used and not maintained

* update blocking readme

* resolve FIXME

* fix: add back concurrent option in dag.walk

* decouple switch from directory downgrade
- Add tests for automatic unsharding
- Modified some internals to be sufficiently extensible for testing
@schomatis schomatis force-pushed the schomatis/directory/unsharding branch from 51cb5fe to 9ea81f2 Compare October 22, 2021 21:25
Copy link
Contributor

@aschmahmann aschmahmann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

YESSSSSSSS 🎉

@aschmahmann aschmahmann merged commit bd53b6a into master Nov 12, 2021
@aschmahmann aschmahmann mentioned this pull request Dec 1, 2021
80 tasks
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants