Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement new blocks subscriber and submit data to kafka #34

Merged
merged 34 commits into from
Nov 10, 2021
Merged

Implement new blocks subscriber and submit data to kafka #34

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
40cda62
Implement new blocks subscriber and submit data to kafka
DNK90 Oct 12, 2021
b5d4761
Fix job is added to job queue continuously
DNK90 Oct 12, 2021
37be217
Add SCRAM-SHA-256 authentication type and update description of authe…
DNK90 Oct 13, 2021
6394195
Continue the loop in worker when job reaches its maxTry instead of re…
DNK90 Oct 13, 2021
54eefb0
Change subscribe function from ChainHeadEvent (only emit canonical bl…
DNK90 Oct 14, 2021
bbc2f36
Add missing queueSize flag
DNK90 Oct 15, 2021
2221e34
Apply sending batch to optimize memory and fix deadlock when exiting
DNK90 Oct 17, 2021
91221ee
Add handle reorg event
DNK90 Oct 18, 2021
53d9e43
Send message to ChainEvent after finish syncing block using snap or fast
DNK90 Oct 19, 2021
32e9463
Add Remove and Rebirth logs event, Fix empty blockHash, Number, Trans…
DNK90 Oct 19, 2021
ef108c3
Implement get and publish past blocks in state
DNK90 Oct 19, 2021
becc2da
Add subscribe removedLogEvent and rebirthLogsEvent codes
DNK90 Oct 19, 2021
daa4114
Update Dockerfile and command usage
DNK90 Oct 21, 2021
7e338b0
Add contractAddress to transaction
DNK90 Oct 21, 2021
d26b915
Add checkConnection function to check connectivity to kafka, Fixed wr…
DNK90 Oct 21, 2021
82b838d
Correct docker-compose volumes path
DNK90 Oct 21, 2021
8fcb1eb
Correct mine condition
DNK90 Oct 21, 2021
86b40cd
Add number of transactions to block
DNK90 Oct 22, 2021
0c3a66b
Add CumulativeGasUsed to transaction
DNK90 Oct 26, 2021
4cb3252
Add Verbosity variable to docker-compose
DNK90 Oct 26, 2021
236eebd
Add comments for new ideas of resetting blockchain's state if bad blo…
DNK90 Oct 26, 2021
a0b0a4d
Add fromHeight validation for reOrg, rebirthLog and removeLog
DNK90 Oct 27, 2021
e8dcce5
Add RONIN_PARAMS to docker-compose
DNK90 Oct 27, 2021
68e24b3
Update sending confirmed block which behind 10 blocks with current block
DNK90 Oct 29, 2021
acac81c
Handle nil confirmed block
DNK90 Oct 29, 2021
8f36cf6
Add write timeout and log while publishing error
DNK90 Nov 4, 2021
5103124
Update the latest go-kafka
DNK90 Nov 4, 2021
81bed18
Fix typo, send wrong topic in confirmBlockTopic
DNK90 Nov 4, 2021
9c9c452
Update flags default value description
DNK90 Nov 5, 2021
48d074e
Enable miner by default in Dockerfile and entrypoint
DNK90 Nov 9, 2021
a1feced
Remove unused commented code
DNK90 Nov 9, 2021
51912f6
Add publishedTime to newBlock/newTx/newLogs to specify the exact time…
DNK90 Nov 10, 2021
01c9670
Update entrypoint that allows users custom their datadir
DNK90 Nov 10, 2021
435c760
Remove unnecessary comments in entrypoint.sh
DNK90 Nov 10, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 34 additions & 0 deletions cmd/ronin/main.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -197,6 +197,30 @@ var (
utils.MetricsInfluxDBPasswordFlag,
utils.MetricsInfluxDBTagsFlag,
}

subscriberFlags = []cli.Flag{
SubscriberFlag,
ChainEventFlag,
ChainSideEventFlag,
TransactionEventFlag,
ReorgTransactionEventFlag,
KafkaPartitionFlag,
KafkaUrlFlag,
KafkaAuthenticationFlag,
kafkaUsernameFlag,
kafkaPasswordFlag,
MaxRetryFlag,
NumberOfWorkerFlag,
BackOffFlag,
PublisherFlag,
FromHeightFlag,
LogsEventFlag,
QueueSizeFlag,
BlockConfirmedEventFlag,
TransactionConfirmedEventFlag,
LogsConfirmedEventFlag,
ConfirmBlockAtFlag,
}
)

func init() {
Expand Down Expand Up @@ -236,13 +260,15 @@ func init() {
// See snapshot.go
snapshotCommand,
}

sort.Sort(cli.CommandsByName(app.Commands))

app.Flags = append(app.Flags, nodeFlags...)
app.Flags = append(app.Flags, rpcFlags...)
app.Flags = append(app.Flags, consoleFlags...)
app.Flags = append(app.Flags, debug.Flags...)
app.Flags = append(app.Flags, metricsFlags...)
app.Flags = append(app.Flags, subscriberFlags...)

app.Before = func(ctx *cli.Context) error {
return debug.Setup(ctx)
Expand Down Expand Up @@ -318,6 +344,14 @@ func geth(ctx *cli.Context) error {
stack, backend := makeFullNode(ctx)
defer stack.Close()

// subscribe backend event if any
if ctx.GlobalBool(SubscriberFlag.Name) {
subs := NewSubscriber(backend, ctx)
defer subs.Close()
// wait until subscriber finishes its initiation
<-subs.Start()
}

startNode(ctx, stack, backend)
stack.Wait()
return nil
Expand Down
Loading