We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
我在数据库执行create table xxx_etl_init as select * from xxx 这样的dml语句,对xxx表存量数据做初始化etl, 对于数据量几千的表没什么问题, 但是当遇到数据量9万以上的表时,canal-server会出现以下日志——最初22:35的两条是正常的 客户端订阅轮训后产生的,但是接下来23:13:47以及之后不断出现的日志 则会导致重复订阅 :
2020-05-25 22:35:55.005 [New I/O server worker #1-1] INFO c.a.otter.canal.instance.core.AbstractCanalInstance - subscribe filter change to qcdbv2..* 2020-05-25 22:35:55.006 [New I/O server worker #1-1] WARN c.a.o.canal.parse.inbound.mysql.dbsync.LogEventConvert - --> init table filter : ^qcdbv2..$ 2020-05-25 23:13:47.771 [New I/O server worker #1-2] INFO c.a.otter.canal.instance.core.AbstractCanalInstance - subscribe filter change to qcdbv2.. 2020-05-25 23:13:47.772 [New I/O server worker #1-2] WARN c.a.o.canal.parse.inbound.mysql.dbsync.LogEventConvert - --> init table filter : ^qcdbv2..$ 2020-05-25 23:13:52.883 [New I/O server worker #1-3] INFO c.a.otter.canal.instance.core.AbstractCanalInstance - subscribe filter change to qcdbv2.. 2020-05-25 23:13:52.883 [New I/O server worker #1-3] WARN c.a.o.canal.parse.inbound.mysql.dbsync.LogEventConvert - --> init table filter : ^qcdbv2..$ 2020-05-25 23:41:35.519 [New I/O server worker #1-4] INFO c.a.otter.canal.instance.core.AbstractCanalInstance - subscribe filter change to qcdbv2.. 2020-05-25 23:41:35.519 [New I/O server worker #1-4] WARN c.a.o.canal.parse.inbound.mysql.dbsync.LogEventConvert - --> init table filter : ^qcdbv2..*$
canal.instance.memory.buffer.size =1024 canal.instance.memory.buffer.memunit = 1048576 canal.instance.memory.batch.mode = MEMSIZE
If there is an exception, please attach the exception trace:
Just put your stack trace here!
The text was updated successfully, but these errors were encountered:
这个日志是你客户端在发生重新订阅,检查一下客户端是否有异常?
Sorry, something went wrong.
No branches or pull requests
environment
Issue Description
我在数据库执行create table xxx_etl_init as select * from xxx 这样的dml语句,对xxx表存量数据做初始化etl,
对于数据量几千的表没什么问题,
但是当遇到数据量9万以上的表时,canal-server会出现以下日志——最初22:35的两条是正常的 客户端订阅轮训后产生的,但是接下来23:13:47以及之后不断出现的日志 则会导致重复订阅 :
Steps to reproduce
canal.instance.memory.buffer.size =1024
canal.instance.memory.buffer.memunit = 1048576
canal.instance.memory.batch.mode = MEMSIZE
Expected behaviour
Actual behaviour
If there is an exception, please attach the exception trace:
The text was updated successfully, but these errors were encountered: