Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Validator client builder support #10749

Merged
merged 112 commits into from
Jun 6, 2022
Merged
Changes from 1 commit
Commits
Show all changes
112 commits
Select commit Hold shift + click to select a range
fabd6f2
Startinb builder service and interface
terencechain May 10, 2022
0973e08
Get header from builder
terencechain May 10, 2022
0a834ef
Add get builder block
terencechain May 12, 2022
ffb3ef2
Single validator registration
terencechain May 13, 2022
e4dcbc4
Add mev-builder http cli flag
terencechain May 14, 2022
81a6c3f
Add method to verify registration signature
terencechain May 14, 2022
d1fc816
Merge branch 'develop' of github.com:prysmaticlabs/prysm into builder-1
terencechain May 16, 2022
ca71dc0
Add builder registration
terencechain May 16, 2022
e2aa4b1
Add submit validator registration
terencechain May 16, 2022
253c773
suporting yaml
james-prysm May 17, 2022
fb8f064
fix yaml unmarshaling
kasey May 17, 2022
4935b1d
rolling back some changes from unmarshal from file
james-prysm May 17, 2022
8340c01
Merge branch 'develop' of github.com:prysmaticlabs/prysm into builder-1
terencechain May 17, 2022
288f338
adding yaml support
james-prysm May 18, 2022
e566041
Merge branch 'builder-1' into validator-client-builder-support
james-prysm May 18, 2022
5205f9b
adding register validator support
james-prysm May 18, 2022
9d29280
Merge branch 'develop' into validator-client-builder-support
james-prysm May 18, 2022
50b4244
added new validator requests into client/validator
james-prysm May 18, 2022
bd09469
Merge branch 'develop' into builder-1
james-prysm May 18, 2022
3cfa2de
Merge branch 'builder-1' into validator-client-builder-support
james-prysm May 18, 2022
449d12d
fixing gofmt
james-prysm May 18, 2022
1e7785e
updating flags and including gas limit, unit tests are still broken
james-prysm May 19, 2022
4961971
fixing bazel
james-prysm May 19, 2022
73cdbb4
more name changes and fixing unit tests
james-prysm May 20, 2022
fdd8848
fixing unit tests and renaming functions
james-prysm May 20, 2022
1a06daa
Merge branch 'develop' into builder-1
james-prysm May 20, 2022
8c9b1f6
Merge branch 'builder-1' into validator-client-builder-support
james-prysm May 20, 2022
bdc2d81
fixing unit tests and renaming to match changes
james-prysm May 20, 2022
9e16452
adding new test for yaml
james-prysm May 20, 2022
8e59afc
fixing bazel linter
james-prysm May 20, 2022
14c412e
reverting change on validator service proto
james-prysm May 20, 2022
e242eb0
Merge branch 'develop' into builder-1
james-prysm May 20, 2022
37193be
Merge branch 'develop' into builder-1
james-prysm May 20, 2022
a4981f0
adding clarifying logs
james-prysm May 20, 2022
399f216
renaming function name to be more descriptive
james-prysm May 23, 2022
b2ef039
Merge branch 'develop' into builder-1
james-prysm May 23, 2022
b47fd56
Merge branch 'builder-1' into validator-client-builder-support
james-prysm May 23, 2022
6da4f35
renaming variable
james-prysm May 24, 2022
e54dd78
Merge branch 'develop' into validator-client-builder-support
james-prysm May 24, 2022
5612b2f
Merge branch 'develop' into validator-client-builder-support
james-prysm May 24, 2022
e5792ea
rolling back some files that will be added from the builder-1 branch
james-prysm May 24, 2022
174cacb
reverting more
james-prysm May 24, 2022
34e6dc2
more reverting
james-prysm May 24, 2022
eb9ef3b
need placeholder
james-prysm May 24, 2022
dc49906
need placeholder
james-prysm May 24, 2022
eb37daa
Merge branch 'develop' into validator-client-builder-support
james-prysm May 24, 2022
8e4499d
fixing unit test
james-prysm May 24, 2022
3e1b328
Merge branch 'develop' into validator-client-builder-support
james-prysm May 24, 2022
108c9c8
fixing unit test
james-prysm May 24, 2022
3c9ce71
fixing unit test
james-prysm May 24, 2022
50abab8
fixing unit test
james-prysm May 24, 2022
0721273
fixing more unit tests
james-prysm May 24, 2022
f9b6815
fixing more unit tests
james-prysm May 24, 2022
bb31852
Merge branch 'develop' into validator-client-builder-support
james-prysm May 31, 2022
d6df7b4
Merge branch 'develop' into validator-client-builder-support
james-prysm May 31, 2022
2382712
Merge branch 'develop' into validator-client-builder-support
james-prysm May 31, 2022
57bb6e6
rolling back mockgen
james-prysm May 31, 2022
35c4311
Merge branch 'develop' into validator-client-builder-support
james-prysm May 31, 2022
0b932fe
Merge branch 'develop' into validator-client-builder-support
james-prysm Jun 1, 2022
91efa80
fixing bazel
james-prysm Jun 1, 2022
e005b4a
rolling back changes
james-prysm Jun 1, 2022
ca02322
removing duplicate function
james-prysm Jun 1, 2022
575408d
fixing client mock
james-prysm Jun 1, 2022
a89317f
removing unused type
james-prysm Jun 1, 2022
5f4836c
fixing missing brace
james-prysm Jun 1, 2022
fc5abb0
Merge branch 'develop' into validator-client-builder-support
james-prysm Jun 1, 2022
e6eed94
fixing bad field name
james-prysm Jun 1, 2022
e1b9e43
fixing bazel
james-prysm Jun 1, 2022
a8b9937
updating naming
james-prysm Jun 1, 2022
db80ef1
fixing bazel
james-prysm Jun 1, 2022
2aba632
Merge branch 'develop' into validator-client-builder-support
james-prysm Jun 1, 2022
5867cf8
fixing unit test
james-prysm Jun 1, 2022
a93a8d7
Merge branch 'develop' into validator-client-builder-support
james-prysm Jun 1, 2022
a37f4cb
fixing bazel linting
james-prysm Jun 1, 2022
90a5f73
unhandled err
james-prysm Jun 1, 2022
0c74cdd
fixing gofmt
james-prysm Jun 1, 2022
a3b167b
simplifying name based on feedback
james-prysm Jun 2, 2022
eb841aa
using corrected function
james-prysm Jun 2, 2022
4372811
moving default fee recipient and gaslimit to beaconconfig
james-prysm Jun 2, 2022
5cd4d6a
missing a few constant changes
james-prysm Jun 2, 2022
1aec13a
fixing bazel
james-prysm Jun 2, 2022
2cd333f
fixing more missed default renames
james-prysm Jun 2, 2022
6d634d5
fixing more constants in tests
james-prysm Jun 2, 2022
e24f6db
fixing bazel
james-prysm Jun 2, 2022
74f1ed3
Merge branch 'develop' into validator-client-builder-support
james-prysm Jun 2, 2022
992f89f
adding update proposer setting per epoch
james-prysm Jun 2, 2022
c41ca76
refactoring to reduce complexity
james-prysm Jun 2, 2022
4778b7c
adding unit test for proposer settings
james-prysm Jun 2, 2022
c3699d9
Update validator/client/validator.go
james-prysm Jun 2, 2022
263f5ac
trying out renaming based on feedback
james-prysm Jun 2, 2022
60e36d6
adjusting based on review comments
james-prysm Jun 2, 2022
baa79b9
Merge branch 'develop' into validator-client-builder-support
james-prysm Jun 2, 2022
701e09a
making tests more appropriate
james-prysm Jun 3, 2022
98962ee
fixing bazel
james-prysm Jun 3, 2022
178f441
Merge branch 'develop' into validator-client-builder-support
james-prysm Jun 3, 2022
3a2326d
updating flag description based on review feedback
james-prysm Jun 3, 2022
e971034
Merge branch 'develop' into validator-client-builder-support
james-prysm Jun 3, 2022
49c07b1
addressing review feedback
james-prysm Jun 3, 2022
3209082
Merge branch 'develop' into validator-client-builder-support
james-prysm Jun 3, 2022
2609a21
switching to pushing at start of epoch for more time
james-prysm Jun 3, 2022
fc1b0b6
Merge branch 'develop' into validator-client-builder-support
james-prysm Jun 3, 2022
6b2630a
Merge branch 'develop' into validator-client-builder-support
james-prysm Jun 4, 2022
64f4a57
Merge branch 'develop' into validator-client-builder-support
james-prysm Jun 6, 2022
a8d1082
Merge branch 'develop' into validator-client-builder-support
james-prysm Jun 6, 2022
7e943a5
Merge branch 'develop' into validator-client-builder-support
james-prysm Jun 6, 2022
5c570e5
adding new unit test and properly throwing error
james-prysm Jun 6, 2022
321d57e
switching keys in error to count
james-prysm Jun 6, 2022
b120c33
Merge branch 'develop' into validator-client-builder-support
james-prysm Jun 6, 2022
f33dfba
fixing log variable
james-prysm Jun 6, 2022
eb09781
resolving conflict
james-prysm Jun 6, 2022
82ba338
resolving more conflicts
james-prysm Jun 6, 2022
c2ef93d
adjusting error message
james-prysm Jun 6, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 5 additions & 6 deletions validator/client/validator.go
Original file line number Diff line number Diff line change
Expand Up @@ -980,18 +980,17 @@ func (v *validator) PushProposerSettings(ctx context.Context, km keymanager.IKey
return err
}
log.Infoln("Successfully prepared beacon proposer with fee recipient to validator index mapping.")
failedRegistrationPubkeys := make([]string, 0)
failedRegistrationCount := 0
for _, request := range registerValidatorRequests {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this running in a separate goroutine or blocking an unrelated thread? If the latter, should we consider running this in the background in its own goroutine on a timer?

Should we retry failures? What is the upper bound on the number of requests we could do here - maybe we should consider adding a context deadline?

Should the entire batch fail upon any individual request failing? The one validator that is due to be proposer in the next epoch could be at the end of the list and be not updated due to a previous unimportant validator registration failing. It would be cool to order the list so that proposers come first, but failing that I think we should put some thought into retries and how to deal with partial failures. For instance we could log the individual failures at debug log level, and at the end of the batch return an error that indicates x out of y validator registrations failed (maybe a different error message with the pubkey if just one failed).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when it runs per epoch I have it running in its own go routine you can see that in the runner.go file line 190. otherwise it runs when status updates and on startup which are not on their on go routine.

I think those are all possible future improvements, with it being set every6.4 minutes and on startup as well I wonder if that's 100% needed at this moment. I can make a note on the tracking of this feature but I don't think it should be part of this particular PR.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

with it being set every6.4 minutes and on startup as well I wonder if that's 100% needed at this moment.
I don't follow your meaning here, what is it that may not be needed based on the frequency of this running?

Are we building a prototype for integration testing, or is this something we want to be usable by end-users once this is merged? If it's a prototype then I agree that we can punt on resiliency, but I think accounting for retries and partial failures are basic requirements we need in place before this is used in production. I'll defer to @terencechain on clarifying the goal.

// calls beacon API but used for custom builders
if err := SubmitValidatorRegistration(ctx, v.validatorClient, v.node, km.Sign, request); err != nil {
// log the error and keep going
hexKey := hexutil.Encode(request.Pubkey)
failedRegistrationPubkeys = append(failedRegistrationPubkeys, hexKey)
log.Warnf("failed to register validator for custom builder for %s, error: %v ", hexKey, err)
failedRegistrationCount++
log.Warnf("failed to register validator for custom builder for %s, error: %v ", hexutil.Encode(request.Pubkey), err)
Copy link
Contributor

@kasey kasey Jun 6, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought the plan was to make all requests (so that a bad request doesn't ruin it for the rest), but at the end of the method return an error if any of the requests failed. @terencechain if some of the requests fail, should this method return an error?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated to address this, lt me know if this works , unit test added

}
}
if len(failedRegistrationPubkeys) != 0 {
return errors.Errorf("Register Validator requests failed for the following publickeys: %v", failedRegistrationPubkeys)
if failedRegistrationCount != 0 {
return errors.Errorf("Register validator requests failed %v times out of %v requests", failedRegistrationCount, len(registerValidatorRequests))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nitpick: it's nice to use the typed string formatters (eg %d for numbers, %s for strings) rather than %v where possible, because linters understand them and will complain when types are misaligned.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated

}
log.Infoln("Successfully submitted builder validator registration settings for custom builders.")
return nil
Expand Down