Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug Fix: Handle multi-part lists properly in ReadPostingList #4574

Merged
merged 11 commits into from
Jan 22, 2020

Conversation

martinmr
Copy link
Contributor

@martinmr martinmr commented Jan 14, 2020

This PR changes ReadPostingList to skip over the parts of a multi-part list so that they are not accessed the next time ReadPostingList is called. A multi-part posting list should only be accessed via the main key. Accessing the posting list via one of the other keys was causing issues during rollup and adding spurious keys to the database, which eventually caused reads to return nil in the Jepsen uid-set test.


This change is Reviewable

If a key has a startUid > 0 it should not be read by ReadPostingList.
Instead, when the main key is accessed, the iterator should be advanced
so that these keys are skipped over.
@martinmr martinmr changed the title Skip over multi-part list keys. Handle multi-part lists properly in ReadPostingList Jan 14, 2020
@martinmr martinmr marked this pull request as ready for review January 15, 2020 00:19
@martinmr martinmr requested a review from a team January 15, 2020 00:24
Copy link
Contributor

@manishrjain manishrjain left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 2 files reviewed, 2 unresolved discussions (waiting on @manishrjain and @martinmr)


posting/list.go, line 491 at r1 (raw file):

func (l *List) Iterate(readTs uint64, afterUid uint64, f func(obj *pb.Posting) error) error {
	if l == nil {
		return nil

I'd create an error called errorNilList, and return it for all the funcs.


posting/mvcc.go, line 151 at r2 (raw file):

		// Trying to read a single part of a multi part list. This type of list
		// should be read once using the canonical list (with startUid equal to zero).
		return nil, nil

No nil error here. Should have a pre-defined error.

Copy link
Contributor Author

@martinmr martinmr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 2 files reviewed, 2 unresolved discussions (waiting on @manishrjain)


posting/list.go, line 491 at r1 (raw file):

Previously, manishrjain (Manish R Jain) wrote…

I'd create an error called errorNilList, and return it for all the funcs.

Removed the special nil handling in favor of not returning nil lists in ReadPostingList


posting/mvcc.go, line 151 at r2 (raw file):

Previously, manishrjain (Manish R Jain) wrote…

No nil error here. Should have a pre-defined error.

Done.

Copy link
Contributor

@parasssh parasssh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 2 files reviewed, 3 unresolved discussions (waiting on @manishrjain and @martinmr)

a discussion (no related file):
:lgtm:


Copy link
Contributor

@manishrjain manishrjain left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm: Hope you ran Jepsen!

Reviewed 2 of 2 files at r3.
Reviewable status: all files reviewed, 4 unresolved discussions (waiting on @manishrjain and @martinmr)


posting/list.go, line 1136 at r3 (raw file):

func (l *List) Facets(readTs uint64, param *pb.FacetParams, langs []string,
	listType bool) ([]*pb.Facets, error) {
	l.RLock()

revert all changes here.


posting/mvcc.go, line 150 at r3 (raw file):

	if err != nil {
		return nil, errors.Wrapf(err, "while reading posting list with key [%v]", key)
	}

Comment here that we used to have a bug here, as caught by . This PR fixes that.

@manishrjain manishrjain changed the title Handle multi-part lists properly in ReadPostingList Bug Fix: Handle multi-part lists properly in ReadPostingList Jan 22, 2020
Copy link
Contributor Author

@martinmr martinmr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 0 of 2 files reviewed, 4 unresolved discussions (waiting on @manishrjain and @martinmr)


posting/list.go, line 1136 at r3 (raw file):

Previously, manishrjain (Manish R Jain) wrote…

revert all changes here.

Done.


posting/mvcc.go, line 150 at r3 (raw file):

Previously, manishrjain (Manish R Jain) wrote…

Comment here that we used to have a bug here, as caught by . This PR fixes that.

Done.

@martinmr martinmr merged commit 9eb7c0a into master Jan 22, 2020
@martinmr martinmr deleted the martinmr/jepsen-fix branch January 22, 2020 20:21
danielmai pushed a commit that referenced this pull request Feb 6, 2020
This PR changes ReadPostingList to skip over the parts of a multi-part list so that
they are not accessed the next time ReadPostingList is called. A multi-part posting
list should only be accessed via the main key. Accessing the posting list via one of
the other keys was causing issues during rollup and adding spurious keys to the
database, which eventually caused reads to return nil in the Jepsen uid-set test.
danielmai added a commit that referenced this pull request Feb 6, 2020
* Revert "Disable splits in release 1.2 (#4672)"

This reverts commit 1846c68.

* Bug Fix: Handle multi-part lists properly in ReadPostingList (#4574)

This PR changes ReadPostingList to skip over the parts of a multi-part list so that
they are not accessed the next time ReadPostingList is called. A multi-part posting
list should only be accessed via the main key. Accessing the posting list via one of
the other keys was causing issues during rollup and adding spurious keys to the
database, which eventually caused reads to return nil in the Jepsen uid-set test.

* Add checks to verify expected number of list splits in list_test.go (#4609)

(cherry picked from commit 9ca59f8)

Co-authored-by: Martin Martinez Rivera <martinmr@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

3 participants