List topics.
kafka-topics --bootstrap-server localhost:9092 --list
Create topic.
kafka-topics --bootstrap-server localhost:9092 --create --topic new_topic
Read topic config.
kafka-topics --bootstrap-server localhost:9092 --describe --topic new_topic
Add more partitions to topic
Partitions can be added but canot be deleted. Operations is irreversable.
kafka-topics --bootstrap-server localhost:9092 --alter --topic new_topic --partitions 4
List all configs for topic
kafka-configs --bootstrap-server localhost:9092 --describe --all --topic new_topic
Set retention time in msecs retention.ms
.
kafka-configs --bootstrap-server localhost:9092 --alter --topic new_topic --add-config retention.ms=10000
Remove retention time config option
kafka-configs --bootstrap-server localhost:9092 --alter --topic new_topic --delete-config retention.ms
Create config file connect.properties
with SSL options
security.protocol=SSL
ssl.keystore.location=keystore.jks
ssl.keystore.password=secret
ssl.key.password=secret
ssl.truststore.location=truststore.jks
ssl.truststore.password=secret
ssl.enabled.protocols=TLSv1.2,TLSv1.1,TLSv1
Create client, server keystores and truststore using KeyStore Explorer or Openssl and convert it to jks
- Root CA key pair (key and certificate) adding CA extention
- Server key pair signed with Root CA (Sing new key pair) and added extention TLS Web Server Authentication (1.3.6.1.5.5.7.3.1)
- Client key pair signed with Root CA and added extention TLS Web Client Authentication (1.3.6.1.5.5.7.3.2) Export Root CA to trustsore, save server and client key pairs to keystores
Use connect.properties as command config with kafka-topics
and kafka-configs
kafka-topics --command-config connect.properties --bootstrap-server "SSL://localhost:9093" --list
kafka-configs --command-config connect.properties --bootstrap-server "SSL://localhost:9093" --describe --all --topic new_topic
List
kafka-consumer-groups --bootstrap-server "localhost:9092" --list
Describe
kafka-consumer-groups --bootstrap-server "localhost:9092" --group worker-group --describe
Reset Offsets to latest offset. Can be used with dry run option --dry-run
.
Option --execute
confirms operation.
kafka-consumer-groups --bootstrap-server "localhost:9092" --topic worker-queue --group worker-group --reset-offsets --to-latest --execute