kafka-console-consumer.sh is showing an incorrect number of
messages processed, counting one more message than the actual
number of processed messages.
Author: Luciano Afranllie <luafran@gmail.com>
Reviewers: Guozhang Wang
Closes#617 from luafran/console-consumer-number-of-processed-messages
Changes made for using getBaseConsumerConfigs from StreamingConfig.getConsumerConfigs.
Author: bbejeck <bbejeck@gmail.com>
Author: Bill Bejeck <bbejeck@gmail.com>
Reviewers: Guozhang Wang
Closes#596 from bbejeck/KAFKA-2902-StreamingConfig-use-getBaseConsumerConfigs
I originally tried to solve the problem by using tempfile, and creating and using scp() utility method that created a random local temp file every time it was called. However, it required passing miniKdc object to SecurityConfig setup_node which looked very invasive, since many tests use this method. Here is the PR for that, which I think we will close: https://github.com/apache/kafka/pull/609
This change is the least invasive change to solve conflicts between multiple tests jobs.
Author: Anna Povzner <anna@confluent.io>
Reviewers: Geoff Anderson
Closes#610 from apovzner/kafka_2851_01
For SSL and SASL replication tests, set security protocol for clients as well.
Author: Rajini Sivaram <rajinisivaram@googlemail.com>
Reviewers: Ismael Juma <ismael@juma.me.uk>, Ben Stopford <benstopford@gmail.com>, Geoff Anderson <geoff@confluent.io>, Jun Rao <junrao@gmail.com>
Closes#563 from rajinisivaram/KAFKA-2642
A few notes on the added test:
* I verified this test fails when changing between snappy 1.1.1.2 and 1.1.1.7 (per KAFKA-2189)
* The hard coded numbers are passing before and after lzo change
Author: Grant Henke <granthenke@gmail.com>
Reviewers: Ismael Juma, Guozhang Wang
Closes#552 from granthenke/lz4
This patch fixes some releative paths so bulding from a subproject directory like ($rootDir/core) will not fail
Author: Grant Henke <granthenke@gmail.com>
Reviewers: Ewen Chesklack-Postava
Closes#509 from granthenke/minor
This is a follow-up as the previous update was missing some changes.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Todd Palino, Guozhang Wang
Closes#607 from ijuma/java-tuning-settings
Tests rolling upgrade from PLAINTEXT to SSL
Author: Ben Stopford <benstopford@gmail.com>
Reviewers: Geoff Anderson, Ismael Juma
Closes#496 from benstopford/security-upgrade-test
Use the latest information from LinkedIn and mention that the latest
released version is recommended from a security perspective.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Grant Henke
Closes#603 from ijuma/java-version-recommendation
Author: Flavio Junqueira <fpj@apache.org>
Reviewers: Ismael Juma <ismael@juma.me.uk>, Ben Stopford <benstopford@gmail.com>, Jun Rao <junrao@gmail.com>
Closes#410 from fpj/KAFKA-2732
Currently we don't log exceptions raised when reading from the local log which makes tracking down the cause of problems a bit tricky.
Author: Ben Stopford <benstopford@gmail.com>
Reviewers: Guozhang Wang
Closes#593 from benstopford/small-patches
In `wrapForOutput` and `wrapForInput` methods of `org.apache.kafka.common.record.Compressor`, `Class.forName("[compression codec]")` and `getConstructor` methods are invoked for each `wrapForOutput` / `wrapForInput` call. Reflection calls are expensive and impact performance at high volumes. This patch adds a cache for `Constructor` to reduce the reflection overhead.
In our production deployments, this has reduced producer CPU usage by about 20%
Author: Maksim Logvinenko <mlogvinenko@gmail.com>
Reviewers: Ismael Juma
Closes#580 from logarithm/compressor-getclass-cache
Run tests with SSL, SASL_PLAINTEXT and SASL_SSL. Same security protocol is used for source and target Kafka.
Author: Rajini Sivaram <rajinisivaram@googlemail.com>
Reviewers: Geoff Andreson, Ben Stopford
Closes#559 from rajinisivaram/KAFKA-2643
Use Files.createTempDirectory to avoid reuse, for log directories create a new temp directory as parent
Author: Rajini Sivaram <rajinisivaram@googlemail.com>
Reviewers: Ismael Juma, Guozhang Wang
Closes#583 from rajinisivaram/KAFKA-2718-v2
Sanity check array size in requests before allocation
Author: Rajini Sivaram <rajinisivaram@googlemail.com>
Reviewers: Ismael Juma <ismael@juma.me.uk>, Ashish Singh <asingh@cloudera.com>, Jun Rao <junrao@gmail.com>
Closes#577 from rajinisivaram/KAFKA-2878
We can take advantage of the fact that major Scala versions are binary compatible (since 2.10) to make the build a little more user-friendly.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Ewen Cheslack=Postava, Grant Henke
Closes#574 from ijuma/install-all-and-major-instead-of-full-version
Since we created the 0.9.0 branch a while back.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Jun Rao <junrao@gmail.com>
Closes#575 from ijuma/bump-to-0.9.1.0-SNAPSHOT
Starting a KafkaStream was getting an error due to the fact that the TopologyBuilder.addSink method was not connecting the sink with it parent(s) processor/sources. Just needed to wire up the sink with it parent(s) in TopologyBuilder.addSink .
Author: bbejeck <bbejeck@gmail.com>
Reviewers: Guozhang Wang
Closes#572 from bbejeck/KAFKA-2872_kafka_stream_sink_not_connected_to_parent
Retry to find new directory and cleanup on exit.
Author: Rajini Sivaram <rajinisivaram@googlemail.com>
Reviewers: Ismael Juma, Guozhang Wang
Closes#399 from rajinisivaram/KAFKA-2718
Anna Povzner
Luciano Afranllie
bbejeck
Jason Gustafson
Rajini Sivaram
Grant Henke
Dong Lin
Ben Stopford
Ismael Juma
Alex Sherwin
Ewen Cheslack-Postava
Flavio Junqueira
Tomasz Nurkiewicz
Maksim Logvinenko
Jesse Anderson
Yasuhiro Matsuda
Guozhang Wang
Jun Rao
jholoman
Gwen Shapira
Ashish Singh
Geoff Anderson