From 86410e14d7f2947dff7c6e3e91cdb80597ff3b4c Mon Sep 17 00:00:00 2001 From: Guozhang Wang Date: Tue, 5 Jun 2018 11:41:03 -0700 Subject: [PATCH] MINOR: Add missing configs for resilience settings Reviewers: Matthias J. Sax , Bill Bejeck --- .../developer-guide/config-streams.html | 35 +++++++++++++++---- 1 file changed, 29 insertions(+), 6 deletions(-) diff --git a/docs/streams/developer-guide/config-streams.html b/docs/streams/developer-guide/config-streams.html index 2b6ade5c513..6bba10d8210 100644 --- a/docs/streams/developer-guide/config-streams.html +++ b/docs/streams/developer-guide/config-streams.html @@ -259,6 +259,16 @@ The replication factor for changelog topics and repartition topics created by the application. 1 + retries + Medium + The number of retries for broker requests that return a retryable error. + 0 + + retry.backoff.ms + Medium + The amount of time in milliseconds, before a request is retried. This applies if the retries parameter is configured to be greater than 0. + 100 + state.cleanup.delay.ms Low The amount of time in milliseconds to wait before deleting state when a partition has migrated. @@ -514,8 +524,10 @@
-

Kafka consumers and producer configuration parameters

-

You can specify parameters for the Kafka consumers and producers that are used internally. +

Kafka consumers, producer and admin client configuration parameters

+

You can specify parameters for the Kafka consumers, producers, + and admin client that are used internally. + The consumer, producer and admin client settings are defined by specifying parameters in a StreamsConfig instance.

In this example, the Kafka consumer session timeout is configured to be 60000 milliseconds in the Streams settings:

Properties streamsSettings = new Properties();
 // Example of a "normal" setting for Kafka Streams
@@ -526,18 +538,21 @@

Naming

-

Some consumer and producer configuration parameters use the same parameter name. For example, send.buffer.bytes and - receive.buffer.bytes are used to configure TCP buffers; request.timeout.ms and retry.backoff.ms control retries - for client request. You can avoid duplicate names by prefix parameter names with consumer. or producer. (e.g., consumer.send.buffer.bytes and producer.send.buffer.bytes).

+

Some consumer, producer and admin client configuration parameters use the same parameter name, and Kafka Streams library itself also uses some parameters that share the same name with its embedded client. For example, send.buffer.bytes and + receive.buffer.bytes are used to configure TCP buffers; request.timeout.ms and retry.backoff.ms control retries for client request; + retries are used to configure how many retries are allowed when handling retriable errors from broker request responses. + You can avoid duplicate names by prefix parameter names with consumer., producer., or admin. (e.g., consumer.send.buffer.bytes and producer.send.buffer.bytes).

Properties streamsSettings = new Properties();
-// same value for consumer and producer
+// same value for consumer, producer, and admin client
 streamsSettings.put("PARAMETER_NAME", "value");
 // different values for consumer and producer
 streamsSettings.put("consumer.PARAMETER_NAME", "consumer-value");
 streamsSettings.put("producer.PARAMETER_NAME", "producer-value");
+streamsSettings.put("admin.PARAMETER_NAME", "admin-value");
 // alternatively, you can use
 streamsSettings.put(StreamsConfig.consumerPrefix("PARAMETER_NAME"), "consumer-value");
 streamsSettings.put(StreamsConfig.producerPrefix("PARAMETER_NAME"), "producer-value");
+streamsSettings.put(StreamsConfig.adminClientPrefix("PARAMETER_NAME"), "admin-value");

You could further separate consumer configuration by adding different prefixes:

    @@ -572,6 +587,14 @@ + auto.offset.reset + Global Consumer + none (cannot be changed) + + auto.offset.reset + Restore Consumer + none (cannot be changed) + auto.offset.reset Consumer earliest