title | weight | type |
---|---|---|
Apache Kafka Broker |
30 |
docs |
The Apache Kafka Broker is a native Broker implementation, that reduces network hops, supports any Kafka version, and has a better integration with Apache Kafka for the Knative Broker and Trigger model.
Notable features are:
Knative Eventing installation.
Install the Kafka broker by entering the following command:
kubectl apply --filename {{< artifact org="knative-sandbox" repo="eventing-kafka-broker" file="eventing-kafka-broker.yaml" >}}
Verify that kafka-broker-controller
, kafka-broker-receiver
and kafka-broker-dispatcher
are running,
by entering the following command:
kubectl get deployments.apps -n knative-eventing
Example output:
NAME READY UP-TO-DATE AVAILABLE AGE
eventing-controller 1/1 1 1 10s
eventing-webhook 1/1 1 1 9s
kafka-broker-controller 1/1 1 1 3s
kafka-broker-dispatcher 1/1 1 1 4s
kafka-broker-receiver 1/1 1 1 5s
A Kafka Broker object looks like this:
apiVersion: eventing.knative.dev/v1
kind: Broker
metadata:
annotations:
# case-sensitive
eventing.knative.dev/broker.class: Kafka
name: default
namespace: default
spec:
# Configuration specific to this broker.
config:
apiVersion: v1
kind: ConfigMap
name: kafka-broker-config
namespace: knative-eventing
# Optional dead letter sink, you can specify either:
# - deadLetterSink.ref, which is a reference to a Callable
# - deadLetterSink.uri, which is an absolute URI to a Callable (It can potentially be out of the Kubernetes cluster)
delivery:
deadLetterSink:
ref:
apiVersion: serving.knative.dev/v1
kind: Service
name: dlq-service
spec.config
should reference any ConfigMap
that looks like the following:
apiVersion: v1
kind: ConfigMap
metadata:
name: kafka-broker-config
namespace: knative-eventing
data:
# Number of topic partitions
default.topic.partitions: "10"
# Replication factor of topic messages.
default.topic.replication.factor: "1"
# A comma separated list of bootstrap servers. (It can be in or out the k8s cluster)
bootstrap.servers: "my-cluster-kafka-bootstrap.kafka:9092"
The above ConfigMap
is installed in the cluster. You can edit
the configuration or create a new one with the same values
depending on your needs.
To set the Kafka broker as the default implementation for all brokers in the Knative deployment,
you can apply global settings by modifying the config-br-defaults
ConfigMap in the knative-eventing
namespace.
This allows you to avoid configuring individual or per-namespace settings for each broker,
such as metadata.annotations.eventing.knative.dev/broker.class
or spec.config
.
The following YAML is an example of a config-br-defaults
ConfigMap using Kafka broker as the default implementation.
apiVersion: v1
kind: ConfigMap
metadata:
name: config-br-defaults
namespace: knative-eventing
data:
default-br-config: |
clusterDefault:
brokerClass: Kafka
apiVersion: v1
kind: ConfigMap
name: kafka-broker-config
namespace: knative-eventing
namespaceDefaults:
namespace1:
brokerClass: Kafka
apiVersion: v1
kind: ConfigMap
name: kafka-broker-config
namespace: knative-eventing
namespace2:
brokerClass: Kafka
apiVersion: v1
kind: ConfigMap
name: kafka-broker-config
namespace: knative-eventing
Knative exposes all available Kafka producer and consumer configurations that can be modified to suit your workloads.
You can change these configurations by modifying the config-kafka-broker-data-plane
ConfigMap
in
the knative-eventing
namespace.
Documentation for the settings available in this ConfigMap
is available on the
Apache Kafka website,
in particular, Producer configurations
and Consumer configurations.
The following YAML shows the default logging configuration for data plane components, that is created during the installation step:
apiVersion: v1
kind: ConfigMap
metadata:
name: kafka-broker-config-logging
namespace: knative-eventing
data:
config.xml: |
<configuration>
<appender name="jsonConsoleAppender" class="ch.qos.logback.core.ConsoleAppender">
<encoder class="net.logstash.logback.encoder.LogstashEncoder"/>
</appender>
<root level="INFO">
<appender-ref ref="jsonConsoleAppender"/>
</root>
</configuration>
To change the logging level to DEBUG
, you need to:
Apply the following kafka-broker-config-logging
ConfigMap
or replace level="INFO"
with level="DEBUG"
to the
ConfigMap
kafka-broker-config-logging
:
apiVersion: v1
kind: ConfigMap
metadata:
name: kafka-broker-config-logging
namespace: knative-eventing
data:
config.xml: |
<configuration>
<appender name="jsonConsoleAppender" class="ch.qos.logback.core.ConsoleAppender">
<encoder class="net.logstash.logback.encoder.LogstashEncoder"/>
</appender>
<root level="DEBUG">
<appender-ref ref="jsonConsoleAppender"/>
</root>
</configuration>
Restart the kafka-broker-receiver
and the kafka-broker-dispatcher
, by entering the following commands:
kubectl rollout restart deployment -n knative-eventing kafka-broker-receiver
kubectl rollout restart deployment -n knative-eventing kafka-broker-dispatcher
Вы можете оставить комментарий после Вход в систему
Неприемлемый контент может быть отображен здесь и не будет показан на странице. Вы можете проверить и изменить его с помощью соответствующей функции редактирования.
Если вы подтверждаете, что содержание не содержит непристойной лексики/перенаправления на рекламу/насилия/вульгарной порнографии/нарушений/пиратства/ложного/незначительного или незаконного контента, связанного с национальными законами и предписаниями, вы можете нажать «Отправить» для подачи апелляции, и мы обработаем ее как можно скорее.
Опубликовать ( 0 )