Starting a KafkaStream was getting an error due to the fact that the TopologyBuilder.addSink method was not connecting the sink with it parent(s) processor/sources. Just needed to wire up the sink with it parent(s) in TopologyBuilder.addSink .
Author: bbejeck <bbejeck@gmail.com>
Reviewers: Guozhang Wang
Closes#572 from bbejeck/KAFKA-2872_kafka_stream_sink_not_connected_to_parent
Retry to find new directory and cleanup on exit.
Author: Rajini Sivaram <rajinisivaram@googlemail.com>
Reviewers: Ismael Juma, Guozhang Wang
Closes#399 from rajinisivaram/KAFKA-2718
Removed a config in tools-log4j.properties which prevented certain service classes from logging at TRACE level.
Author: Geoff Anderson <geoff@confluent.io>
Reviewers: Ewen Cheslack-Postava, Gwen Shapira
Closes#556 from granders/KAFKA-2820-systest-tool-loglevel
This is a hack which works. Is there a better way?
Build (v2) of the replication_test.py running here: http://jenkins.confluent.io/job/kafka_system_tests_branch_builder/185/
Author: Ben Stopford <benstopford@gmail.com>
Reviewers: Geoff Anderson, Gwen Shapira
Closes#520 from benstopford/fix-for-sasl-virtual-box
* Fix typo in api.html
* Mark security features as beta quality (similar to new consumer). Is there better wording?
* Improve wording and clarify things in a number of places
* Improve layout of `pre` blocks (tested locally, which doesn't seem to use the same stylesheets as the deployed version)
* Use producer.config in console-producer.sh command
* Improve SASL documentation structure
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Jun Rao, Magnus Edenhill, Gwen Shapira
Closes#550 from ijuma/documentation-improvements
Author: Geoff Anderson <geoff@confluent.io>
Reviewers: Ismael Juma, Guozhang Wang
Closes#537 from granders/KAFKA-2845-new-client-old-broker-compatibility
Restores control over log level in system test service class KafkaService.
Author: Geoff Anderson <geoff@confluent.io>
Reviewers: Ismael Juma, Ewen Cheslack-Postava
Closes#538 from granders/KAFKA-2820-systest-log-level
More performance improvements:
"In many cases, Gradle 2.9 is much faster than Gradle 2.8 when performing incremental builds.
Very large builds (many thousands of source files) could see incremental build speeds up to 80% faster than 2.7 and up to 40% faster than 2.8.
Gradle now uses a more efficient mechanism to scan the filesystem, making up-to-date checks significantly faster. This improvement is only available when running Gradle with Java 7 or newer.
Other improvements have been made to speed-up include and exclude pattern evaluation; these improvements apply to all supported Java versions.
Gradle now uses much less memory than previous releases when performing incremental builds. By de-duplicating Strings used as file paths in internal caches, and by reducing the overhead of listing classes under test for Java projects, some builds use 30-70% less memory that Gradle 2.8."
https://docs.gradle.org/current/release-notes
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Grant Henke, Guozhang Wang
Closes#549 from ijuma/gradle-2.9
Author: Jason Gustafson <jason@confluent.io>
Reviewers: Guozhang Wang <wangguoz@gmail.com>, Jun Rao <junrao@gmail.com>
Closes#530 from hachikuji/KAFKA-2841
This makes it easier to pass security properties in the same way
to `ConsoleConsumer` and `ConsoleProducer`.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Guozhang Wang <wangguoz@gmail.com>, Jun Rao <junrao@gmail.com>
Closes#544 from ijuma/producer-config-in-console-producer
…ecify a list of values for a config options.
Author: Parth Brahmbhatt <brahmbhatt.parth@gmail.com>
Reviewers: Jun Rao <junrao@gmail.com>
Closes#545 from Parth-Brahmbhatt/KAFKA-2852
guozhangwang
A restore consumer does not belong to a consumer group.
Author: Yasuhiro Matsuda <yasuhiro@confluent.io>
Reviewers: Guozhang Wang
Closes#543 from ymatsuda/no_group_for_restore_consumer
Author: Ashish Singh <asingh@cloudera.com>
Reviewers: Ismael Juma <ismael@juma.me.uk>, Jason Gustafson <jason@confluent.io>, Jun Rao <junrao@gmail.com>
Closes#534 from SinghAsDev/KAFKA-2746
Also mark `PrincipalBuilder` as `Unstable` and tweak docs.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Jun Rao <junrao@gmail.com>
Closes#542 from ijuma/kafka-2847-remove-principal-builder-class-from-client-configs
Often it is useful to link to a specific header within the documentation. Especially when referencing docs in the mailing lists.
This adds anchors and links for all headers in the docs.
Author: Grant Henke <granthenke@gmail.com>
Reviewers: Jun Rao <junrao@gmail.com>
Closes#498 from granthenke/doc-links
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Jason Gustafson <jason@confluent.io>, Ashish Singh <asingh@cloudera.com>, Jun Rao <junrao@gmail.com>
Closes#528 from ijuma/kafka-2831-consumer-group-command-zookeeper-new-consumer
guozhangwang
An optimization which may reduce unnecessary poll for standby tasks.
Author: Yasuhiro Matsuda <yasuhiro@confluent.io>
Reviewers: Guozhang Wang
Closes#535 from ymatsuda/remove_empty_standby_task
guozhangwang
* added a new config param "num.standby.replicas" (the default value is 0).
* added a new abstract class AbstractTask
* added StandbyTask as a subclass of AbstractTask
* modified StreamTask to a subclass of AbstractTask
* StreamThread
* standby tasks are created by calling StreamThread.addStandbyTask() from onPartitionsAssigned()
* standby tasks are destroyed by calling StreamThread.removeStandbyTasks() from onPartitionRevoked()
* In addStandbyTasks(), change log partitions are assigned to restoreConsumer.
* In removeStandByTasks(), change log partitions are removed from restoreConsumer.
* StreamThread polls change log records using restoreConsumer in the runLoop with timeout=0.
* If records are returned, StreamThread calls StandbyTask.update and pass records to each standby tasks.
Author: Yasuhiro Matsuda <yasuhiro@confluent.io>
Reviewers: Guozhang Wang
Closes#526 from ymatsuda/standby_task
Log warning message before truncating log in order to
display right offset value for the truncated log.
Author: Francois Visconte <f.visconte@criteo.com>
Reviewers: Dong Lin <lindong28@gmail.com>, Jun Rao <junrao@gmail.com>
Closes#287 from dopuskh3/KAFKA-2624
Author: Jason Gustafson <jason@confluent.io>
Reviewers: Ismael Juma <ismael@juma.me.uk>, Jun Rao <junrao@gmail.com>
Closes#519 from hachikuji/KAFKA-2821
Removed support for BLOCK_ON_BUFFER_FULL_CONFIG (block.on.buffer.full)
Removed support for METADATA_FETCH_TIMEOUT_CONFIG
Removed support for TIMEOUT_CONFIG (aka timeout.ms)
Added support for MAX_BLOCK_MS_CONFIG
Added support for REQUEST_TIMEOUT_MS_CONFIG
Author: Ben Stopford <benstopford@gmail.com>
Reviewers: Jun Rao <junrao@gmail.com>
Closes#525 from benstopford/KAFKA-2791
This avoids spurious log warning messages. Also tweak log message
if wrapResult.getStatus != CLOSED.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Rajini Sivaram <rajinisivaram@googlemail.com>, Jun Rao <junrao@gmail.com>
Closes#511 from ijuma/kafka-2817-unconnected-ssl-transport-layer-close
Bill Bejeck
Jun Rao
Ashish Singh
Jason Gustafson
Rajini Sivaram
Geoff Anderson
Ismael Juma
Ewen Cheslack-Postava
Ben Stopford
Grant Henke
Guozhang Wang
Parth Brahmbhatt
Yasuhiro Matsuda
Dong Lin
Francois Visconte
Sriharsha Chintalapani