onurkaraman becketqin Do you have time to review this patch? It addresses the ticket that jjkoshy filed in KAFKA-2668.
Author: Dong Lin <lindong28@gmail.com>
Reviewers: Onur Karaman <okaraman@linkedin.com>, Joel Koshy <jjkoshy@gmail.com>, Guozhang Wang <wangguoz@gmail.com>, Jun Rao <junrao@gmail.com>
Closes#328 from lindong28/KAFKA-2668
`Iterator.continually` is more efficient (it doesn't allocate a `Cons` instance per element) and we don't need the extra functionality provided by `Stream.continually`.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Ewen Cheslack-Postava
Closes#106 from ijuma/kafka-2399-replace-stream-continually
guozhangwang
* fix ProcessorStateManager to use correct ktable partitions
* more ktable tests
Author: Yasuhiro Matsuda <yasuhiro@confluent.io>
Reviewers: Guozhang Wang
Closes#635 from ymatsuda/more_ktable_test
It used to be used by MirrorMaker but its usage was removed in KAFKA-1997.
Author: Grant Henke <granthenke@gmail.com>
Reviewers: Jiangjie Qin
Closes#638 from granthenke/remove-dll
Removes all the System.currentTimeMillis calls to help with performance on small messages.
Author: Jay Kreps <jay.kreps@gmail.com>
Reviewers: Guozhang Wang
Closes#632 from jkreps/producer-perf-regression
guozhangwang
* added KTable API and impl
* added standby support for KTable
Author: Yasuhiro Matsuda <yasuhiro@confluent.io>
Reviewers: Guozhang Wang
Closes#604 from ymatsuda/add_ktable
kafka-console-consumer.sh is showing an incorrect number of
messages processed, counting one more message than the actual
number of processed messages.
Author: Luciano Afranllie <luafran@gmail.com>
Reviewers: Guozhang Wang
Closes#617 from luafran/console-consumer-number-of-processed-messages
Changes made for using getBaseConsumerConfigs from StreamingConfig.getConsumerConfigs.
Author: bbejeck <bbejeck@gmail.com>
Author: Bill Bejeck <bbejeck@gmail.com>
Reviewers: Guozhang Wang
Closes#596 from bbejeck/KAFKA-2902-StreamingConfig-use-getBaseConsumerConfigs
I originally tried to solve the problem by using tempfile, and creating and using scp() utility method that created a random local temp file every time it was called. However, it required passing miniKdc object to SecurityConfig setup_node which looked very invasive, since many tests use this method. Here is the PR for that, which I think we will close: https://github.com/apache/kafka/pull/609
This change is the least invasive change to solve conflicts between multiple tests jobs.
Author: Anna Povzner <anna@confluent.io>
Reviewers: Geoff Anderson
Closes#610 from apovzner/kafka_2851_01
For SSL and SASL replication tests, set security protocol for clients as well.
Author: Rajini Sivaram <rajinisivaram@googlemail.com>
Reviewers: Ismael Juma <ismael@juma.me.uk>, Ben Stopford <benstopford@gmail.com>, Geoff Anderson <geoff@confluent.io>, Jun Rao <junrao@gmail.com>
Closes#563 from rajinisivaram/KAFKA-2642
A few notes on the added test:
* I verified this test fails when changing between snappy 1.1.1.2 and 1.1.1.7 (per KAFKA-2189)
* The hard coded numbers are passing before and after lzo change
Author: Grant Henke <granthenke@gmail.com>
Reviewers: Ismael Juma, Guozhang Wang
Closes#552 from granthenke/lz4
This patch fixes some releative paths so bulding from a subproject directory like ($rootDir/core) will not fail
Author: Grant Henke <granthenke@gmail.com>
Reviewers: Ewen Chesklack-Postava
Closes#509 from granthenke/minor
This is a follow-up as the previous update was missing some changes.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Todd Palino, Guozhang Wang
Closes#607 from ijuma/java-tuning-settings
Tests rolling upgrade from PLAINTEXT to SSL
Author: Ben Stopford <benstopford@gmail.com>
Reviewers: Geoff Anderson, Ismael Juma
Closes#496 from benstopford/security-upgrade-test
Use the latest information from LinkedIn and mention that the latest
released version is recommended from a security perspective.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Grant Henke
Closes#603 from ijuma/java-version-recommendation
Author: Flavio Junqueira <fpj@apache.org>
Reviewers: Ismael Juma <ismael@juma.me.uk>, Ben Stopford <benstopford@gmail.com>, Jun Rao <junrao@gmail.com>
Closes#410 from fpj/KAFKA-2732
Currently we don't log exceptions raised when reading from the local log which makes tracking down the cause of problems a bit tricky.
Author: Ben Stopford <benstopford@gmail.com>
Reviewers: Guozhang Wang
Closes#593 from benstopford/small-patches
In `wrapForOutput` and `wrapForInput` methods of `org.apache.kafka.common.record.Compressor`, `Class.forName("[compression codec]")` and `getConstructor` methods are invoked for each `wrapForOutput` / `wrapForInput` call. Reflection calls are expensive and impact performance at high volumes. This patch adds a cache for `Constructor` to reduce the reflection overhead.
In our production deployments, this has reduced producer CPU usage by about 20%
Author: Maksim Logvinenko <mlogvinenko@gmail.com>
Reviewers: Ismael Juma
Closes#580 from logarithm/compressor-getclass-cache
Run tests with SSL, SASL_PLAINTEXT and SASL_SSL. Same security protocol is used for source and target Kafka.
Author: Rajini Sivaram <rajinisivaram@googlemail.com>
Reviewers: Geoff Andreson, Ben Stopford
Closes#559 from rajinisivaram/KAFKA-2643
Use Files.createTempDirectory to avoid reuse, for log directories create a new temp directory as parent
Author: Rajini Sivaram <rajinisivaram@googlemail.com>
Reviewers: Ismael Juma, Guozhang Wang
Closes#583 from rajinisivaram/KAFKA-2718-v2
Sanity check array size in requests before allocation
Author: Rajini Sivaram <rajinisivaram@googlemail.com>
Reviewers: Ismael Juma <ismael@juma.me.uk>, Ashish Singh <asingh@cloudera.com>, Jun Rao <junrao@gmail.com>
Closes#577 from rajinisivaram/KAFKA-2878
We can take advantage of the fact that major Scala versions are binary compatible (since 2.10) to make the build a little more user-friendly.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Ewen Cheslack=Postava, Grant Henke
Closes#574 from ijuma/install-all-and-major-instead-of-full-version
Yasuhiro Matsuda
Jason Gustafson
Dong Lin
Ismael Juma
Vahid Hashemian
Grant Henke
Guozhang Wang
Jay Kreps
Flavio Junqueira
Anna Povzner
Luciano Afranllie
bbejeck
Rajini Sivaram
Ben Stopford
Alex Sherwin
Ewen Cheslack-Postava
Tomasz Nurkiewicz
Maksim Logvinenko
Jesse Anderson