Getting column names out of binlog "after" Row using Column::name_str()

[docteurklein]

Hi! I'm following closely what is done here https://github.com/blackbeam/rust_mysql_common/blob/f724a4efc0d14a3283ef2f90d651a6d099b98aa5/src/binlog/mod.rs#L753-L777

but all I get is strings like @1. my mysql server (8.2.0 ) is configured with --log-bin --binlog-format=row --binlog-row-image=minimal --binlog-row-metadata=full' .

Any idea what I am doing wrong? My code is visible here https://github.com/docteurklein/cdc-rs/blob/f95c7d8e152b090e93b9a93d96fa5358d46178ac/src/main.rs#L63

[blackbeam]

This is strange. Can you observe names using mysqlbinlog program with –print-table-metadata flag?

[docteurklein]

Sure, here is a sample of mysqlbinlog --print-table-metadata /var/lib/mysql/3cef2f35ad25-bin.000003, in which I can see the correct field definitions:

BINLOG '
U+e8ZRMBAAAAewAAACoSAAAAAFkAAAAAAAEABHBpbTEAIXBpbV9jYXRhbG9nX2F0dHJpYnV0ZV90
cmFuc2xhdGlvbgADA/wPAwL8AwIBAQACA/z/AAQZC2ZvcmVpZ25fa2V5BWxhYmVsBmxvY2FsZQgC
AAIMAeCiE7aS
U+e8ZR4BAAAAXAAAAIYSAAAAAFkAAAAAAAEAAgAD/wABAAAABQBhMSBVUwUAZW5fVVMAAQAAAAUA
YTEgRlIFAGZyX0ZSAAIAAAAFAGEyIEZSBQBmcl9GUn4LGco=
'/*!*/;
# at 4742
#240202 13:00:03 server id 1  end_log_pos 4773 CRC32 0x56c4a4f6         Xid = 17
COMMIT/*!*/;
# at 4773
#240202 13:14:09 server id 1  end_log_pos 4852 CRC32 0xa621760f         Anonymous_GTID  last_committed=13       sequence_number=14      rbr_only=yes    original_committed_timestamp=1706879649307538   immediate_commit_timestamp=1706879649307538        transaction_length=703
/*!50718 SET TRANSACTION ISOLATION LEVEL READ COMMITTED*//*!*/;
# original_commit_timestamp=1706879649307538 (2024-02-02 13:14:09.307538 UTC)
# immediate_commit_timestamp=1706879649307538 (2024-02-02 13:14:09.307538 UTC)
/*!80001 SET @@session.original_commit_timestamp=1706879649307538*//*!*/;
/*!80014 SET @@session.original_server_version=80036*//*!*/;
/*!80014 SET @@session.immediate_server_version=80036*//*!*/;
SET @@SESSION.GTID_NEXT= 'ANONYMOUS'/*!*/;
# at 4852
#240202 13:14:09 server id 1  end_log_pos 4936 CRC32 0xce5f02e4         Query   thread_id=21    exec_time=0     error_code=0
SET TIMESTAMP=1706879649/*!*/;
BEGIN
/*!*/;
# at 4936
#240202 13:14:09 server id 1  end_log_pos 5134 CRC32 0x614d9ea3         Table_map: `pim1`.`pim_catalog_product` mapped to number 85
# has_generated_invisible_primary_key=0
# Columns(`uuid` BINARY(16) NOT NULL,
#         `family_id` INT,
#         `product_model_id` INT,
#         `family_variant_id` INT,
#         `id` INT,
#         `is_enabled` TINYINT,
#         `identifier` TEXT CHARSET utf8mb4 COLLATE utf8mb4_0900_ai_ci,
#         `raw_values` JSON,
#         `created` TIMESTAMP,
#         `updated` TIMESTAMP)
# Primary Key(uuid)
# at 5134
#240202 13:14:09 server id 1  end_log_pos 5445 CRC32 0xd121006f         Update_rows: table id 85 flags: STMT_END_F

BINLOG '
oeq8ZRMBAAAAxgAAAA4UAAAAAFUAAAAAAAEABHBpbTEAE3BpbV9jYXRhbG9nX3Byb2R1Y3QACv4D
AwMDAfz1EREG/hACBAAA/gMBAQADBD/8/wAEZgR1dWlkCWZhbWlseV9pZBBwcm9kdWN0X21vZGVs
X2lkEWZhbWlseV92YXJpYW50X2lkAmlkCmlzX2VuYWJsZWQKaWRlbnRpZmllcgpyYXdfdmFsdWVz
B2NyZWF0ZWQHdXBkYXRlZAgBAAwC/8Cjnk1h
oeq8ZR8BAAAANwEAAEUVAAAAAFUAAAAAAAEAAgAK/////2gDEPsor9jByhHutbMCQqwSAAQBAAAA
BAAAAAEAAABHAAAAAAEARgALAAIAAA0AYTEBADkACwAOAAAZADxhbGxfY2hhbm5lbHM+AQAgAAsA
DQAMGAA8YWxsX2xvY2FsZXM+B3AxIGExIDFoAxD7KK/YwcoR7rWzAkKsEgAEAQAAAAQAAAABAAAA
hAAAAAACAIMAEgACABQABQAAGQAAUgBhMWRlc2MzAQA5AAsADgAAGQA8YWxsX2NoYW5uZWxzPgEA
IAALAA0ADBgAPGFsbF9sb2NhbGVzPgdwMSBhMSAxAQAxAAsADgAAGQA8YWxsX2NoYW5uZWxzPgEA
GAALAA0ABQQAPGFsbF9sb2NhbGVzPm8AIdE=
'/*!*/;
# at 5445
#240202 13:14:09 server id 1  end_log_pos 5476 CRC32 0x015573ed         Xid = 276
COMMIT/*!*/;
SET @@SESSION.GTID_NEXT= 'AUTOMATIC' /* added by mysqlbinlog */ /*!*/;
DELIMITER ;
# End of log file
/*!50003 SET COMPLETION_TYPE=@OLD_COMPLETION_TYPE*/;
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=0*/;

BTW, I put --binlog-row-image=full again, to include all the fields, but that shouldn't change much.

I also switched to 8.0 insteead of 8.2 (because only the 8.0-debian docker image contained the mysqlbinlog binary).
Still same results, the col names are @1, @2, ...

Thanks for the whole project BTW!

[docteurklein]

I noticed however when using mysqlbinlog -v (verbose) some interesting output:

### INSERT INTO `pim1`.`pim_catalog_product`                                                             
### SET                                                                                                                                                                                                            
###   @1='a\nb                                                                                                                                                                                                     
###   @2=1                                                                                                                                                                                                         
###   @3=NULL                                                                                                                                                                                                      
###   @4=NULL                                                                                            
###   @5=3                                                                                                                                                                                                         
###   @6=NULL                                                                                            
###   @7=NULL                                                                                            
###   @8='{"a1": {"<all_channels>": {"<all_locales>": "p3 a1 1"}}, "a2": {"<all_channels>": {"en_US": "p3 a2 us 1", "fr_FR": "p3 a2 fr 1"}}}'
###   @9=NULL                                                                                            
###   @10=NULL  

even tho this is not the original statement I executed.

[docteurklein]

Ok so I don't explain it but I switched to your mysql_async crate and I don't receive those column names anymore. Closing this for now!
Thanks :)

[docteurklein]

actually still getting those @ column names. It seems to depend on the mysql server I'm connecting to, but I can't isolate the problem.

[docteurklein]

I think I found the problem: I was not passing the --log-bin option in correct order, and --binlog-row-metadata=full was ignored (which is necessary to get column names).

Closing again. sorry for the noise!