Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: abstract cluster operation #338

Merged
merged 4 commits into from
Dec 19, 2022
Merged

feat: abstract cluster operation #338

merged 4 commits into from
Dec 19, 2022

Conversation

wenfengwang
Copy link
Contributor

@wenfengwang wenfengwang commented Dec 13, 2022

What problem does this PR solve?

Issue Number: close #226

Problem Summary

abstract cluster operation

What is changed and how does it work?

type Topology struct {
	ControllerLeader string
	ControllerURLs   []string
	Uptime           time.Time
}

type Cluster interface {
	WaitForControllerReady() error
	Status() Topology
	IsReady() bool
	EventbusService() EventbusService
	SegmentService() SegmentService
	EventlogService() EventlogService
	TriggerService() TriggerService
	IDService() IDService
}

type EventbusService interface {
	IsExist(ctx context.Context, name string) bool
	CreateSystemEventbusIfNotExist(ctx context.Context, name string, logNum int, desc string) error
	Delete(ctx context.Context, name string) error
	RawClient() ctrlpb.EventBusControllerClient
}

type EventlogService interface {
	RawClient() ctrlpb.EventLogControllerClient
}

type TriggerService interface {
	RawClient() ctrlpb.TriggerControllerClient
	RegisterHeartbeat(ctx context.Context, interval time.Duration, reqFunc func() interface{}) error
}

type IDService interface {
	RawClient() ctrlpb.SnowflakeControllerClient
}

type SegmentService interface {
	RegisterHeartbeat(ctx context.Context, interval time.Duration, reqFunc func() interface{}) error
	RawClient() ctrlpb.SegmentControllerClient
}

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

@codecov
Copy link

codecov bot commented Dec 13, 2022

Codecov Report

Merging #338 (7a73f85) into main (09f3748) will increase coverage by 0.04%.
The diff coverage is 59.71%.

❗ Current head 7a73f85 differs from pull request most recent head b2c89d5. Consider uploading reports for the commit b2c89d5 to get more accurate results

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #338      +/-   ##
==========================================
+ Coverage   62.72%   62.77%   +0.04%     
==========================================
  Files         133      144      +11     
  Lines       12128    13234    +1106     
==========================================
+ Hits         7607     8307     +700     
- Misses       3977     4338     +361     
- Partials      544      589      +45     
Impacted Files Coverage Δ
internal/controller/eventbus/block/block.go 56.09% <0.00%> (ø)
internal/controller/eventbus/server/instance.go 72.04% <ø> (ø)
internal/controller/trigger/storage/secret.go 48.48% <ø> (ø)
...nternal/controller/trigger/storage/subscription.go 54.71% <ø> (ø)
...ernal/controller/trigger/storage/trigger_worker.go 63.41% <ø> (ø)
...rnal/primitive/interceptor/errinterceptor/error.go 0.00% <0.00%> (ø)
.../primitive/interceptor/memberinterceptor/member.go 0.00% <0.00%> (ø)
internal/primitive/vanus/id.go 19.68% <0.00%> (-0.65%) ⬇️
internal/raft/log/wal.go 100.00% <ø> (ø)
internal/store/block/raft/appender.go 0.00% <0.00%> (ø)
... and 81 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a74cef0...b2c89d5. Read the comment docs.

@wenfengwang wenfengwang marked this pull request as draft December 13, 2022 17:48
@wenfengwang wenfengwang marked this pull request as ready for review December 14, 2022 12:03
client/internal/vanus/eventlog/name_service.go Outdated Show resolved Hide resolved
internal/controller/eventbus/controller.go Show resolved Hide resolved
internal/controller/eventbus/controller.go Show resolved Hide resolved
internal/controller/trigger/controller.go Outdated Show resolved Hide resolved
pkg/cluster/controller.go Outdated Show resolved Hide resolved
pkg/cluster/raw_client/heartbeat.go Outdated Show resolved Hide resolved
@xdlbdy
Copy link
Contributor

xdlbdy commented Dec 15, 2022

the trigger worker start lost WaitForControllerReady

@wenfengwang
Copy link
Contributor Author

the trigger worker start lost WaitForControllerReady

Yes, because TriggerService is combined with EventbusService, it doesn't need to wait for the controller ready in the entrence. but in the line of https://github.com/linkall-labs/vanus/pull/338/files#diff-f90560b753f183a2ebcecf1e1563134bc75a35fc0fdf1eaab9f44f099ad2c3b0R516, here start a goroutine to wait controller is ready before creating eventbus

Signed-off-by: wenfeng <sxian.wang@gmail.com>
Signed-off-by: wenfeng <sxian.wang@gmail.com>
internal/controller/trigger/controller.go Outdated Show resolved Hide resolved
internal/controller/trigger/controller.go Outdated Show resolved Hide resolved
internal/controller/trigger/controller.go Outdated Show resolved Hide resolved
internal/controller/trigger/controller.go Show resolved Hide resolved
pkg/cluster/eventbus.go Outdated Show resolved Hide resolved
Signed-off-by: wenfeng <sxian.wang@gmail.com>
hwjiangkai
hwjiangkai previously approved these changes Dec 19, 2022
@xdlbdy
Copy link
Contributor

xdlbdy commented Dec 19, 2022

the trigger worker start lost WaitForControllerReady

Yes, because TriggerService is combined with EventbusService, it doesn't need to wait for the controller ready in the entrence. but in the line of https://github.com/linkall-labs/vanus/pull/338/files#diff-f90560b753f183a2ebcecf1e1563134bc75a35fc0fdf1eaab9f44f099ad2c3b0R516, here start a goroutine to wait controller is ready before creating eventbus

you are right, but I mean the trigger worker which is the Vanus component that runs the trigger, not the trigger controller. otherwise trigger worker status may be running which is not real because the controller is still not ready. Or maybe I mistake the feat of function WaitForControllerReady

@wenfengwang
Copy link
Contributor Author

the trigger worker start lost WaitForControllerReady

Yes, because TriggerService is combined with EventbusService, it doesn't need to wait for the controller ready in the entrence. but in the line of https://github.com/linkall-labs/vanus/pull/338/files#diff-f90560b753f183a2ebcecf1e1563134bc75a35fc0fdf1eaab9f44f099ad2c3b0R516, here start a goroutine to wait controller is ready before creating eventbus

you are right, but I mean the trigger worker which is the Vanus component that runs the trigger, not the trigger controller. otherwise trigger worker status may be running which is not real because the controller is still not ready. Or maybe I mistake the feat of function WaitForControllerReady

I got you. I wii add it.

Signed-off-by: wenfeng <sxian.wang@gmail.com>
Copy link
Contributor

@xdlbdy xdlbdy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@wenfengwang wenfengwang merged commit e696fa7 into main Dec 19, 2022
@wenfengwang wenfengwang deleted the feat-client-cluster branch January 5, 2023 08:09
wenfengwang added a commit that referenced this pull request Mar 23, 2023
* feat: abstract cluster operation

Signed-off-by: wenfeng <sxian.wang@gmail.com>

* fix ut&lint

Signed-off-by: wenfeng <sxian.wang@gmail.com>

* update

Signed-off-by: wenfeng <sxian.wang@gmail.com>

* add waiting into trigger

Signed-off-by: wenfeng <sxian.wang@gmail.com>

Signed-off-by: wenfeng <sxian.wang@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

create vanus system eventbus
3 participants