Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: One side TLS for Kafka #4916

Merged
merged 15 commits into from
Jun 28, 2023
Merged

feat: One side TLS for Kafka #4916

merged 15 commits into from
Jun 28, 2023

Conversation

RafalSkolasinski
Copy link
Contributor

@RafalSkolasinski RafalSkolasinski commented Jun 13, 2023

Building on top of #4658
Closes #4870

Tested with:

  • Strimzi Plain
  • Strimzi one sided TLS
  • Strimzi mTLS
  • Confluent Cloud

To make use of one -sided TLS simply leave security.kafka.ssl.client.secret value unset, e.g.

kafka:
  bootstrap: seldon-kafka-tls-bootstrap.kafka:9093

security:
  kafka:
    protocol: SSL
    ssl:
      client:
        secret:
        brokerValidationSecret: kafka-broker-auth
        endpointIdentificationAlgorithm: https

Note: this also adds propagation of

security:
  kafka:
    ssl:
      client:
        endpointIdentificationAlgorithm: https

Helm value to Golang client if SSL or SASL_SSL is being used.
Empty value will result in none value being set as otherwise golang client raises

Configuration property "ssl.endpoint.identification.algorithm" cannot be set to empty value.

@RafalSkolasinski RafalSkolasinski marked this pull request as draft June 13, 2023 17:32
@RafalSkolasinski RafalSkolasinski mentioned this pull request Jun 15, 2023
@ukclivecox ukclivecox added the v2 label Jun 17, 2023
@ukclivecox ukclivecox changed the title One side TLS for Kafka feat: One side TLS for Kafka Jun 17, 2023
RafalSkolasinski and others added 5 commits June 20, 2023 11:47
Previously this only happened if both were specified, indicating mTLS.
This now allows for normal, one-sided TLS and default trust (trust via the built-in CA bundle).

Add TLS provider method for retrieving just key store

Remove TLS provider method for retrieving all stores

Refactor Kafka SSL setup so client-provided auth is optional
@RafalSkolasinski RafalSkolasinski marked this pull request as ready for review June 20, 2023 12:28
@RafalSkolasinski
Copy link
Contributor Author

RafalSkolasinski commented Jun 20, 2023

For reference, strimzi listeners configuration for testing:

    listeners:
    - name: plain
      port: 9092
      tls: false
      type: loadbalancer
    - name: tls
      port: 9093
      tls: true
      type: loadbalancer
    - authentication:
        type: tls
      name: mtls
      port: 9094
      tls: true
      type: loadbalancer

values for plain test:

kafka:
  bootstrap: seldon-kafka-plain-bootstrap.kafka:9092

security:
  kafka:
    protocol: PLAINTEXT

values for one sided TLS test

kafka:
  bootstrap: seldon-kafka-tls-bootstrap.kafka:9093

security:
  kafka:
    protocol: SSL
    ssl:
      client:
        brokerValidationSecret: kafka-broker-auth

values for mTLS test

kafka:
  bootstrap: seldon-kafka-mtls-bootstrap.kafka:9094

security:
  kafka:
    protocol: SSL
    ssl:
      client:
        secret: kafka-client-auth
        brokerValidationSecret: kafka-broker-auth
        endpointIdentificationAlgorithm:

values for confluent cloud kafka test

kafka:
  bootstrap: <bootstrap servers URI>
  topics:
    replicationFactor: 3
  consumer:
    messageMaxBytes: 8388608
  producer:
    messageMaxBytes: 8388608

security:
  kafka:
    protocol: SASL_SSL
    sasl:
      mechanism: "PLAIN"
      client:
        username: <user name here>
        secret: confluent-kafka-secret
    ssl:
      client:
        secret: kafka-client-auth
        brokerValidationSecret: kafka-broker-auth
        endpointIdentificationAlgorithm: https

@SDJustus
Copy link

works as expected. Thank you very much! 💯

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants