It behaves better on Windows and provides more useful error messages.
Also:
* Minor inconsistency fix in `kafka.server.OffsetCheckpoint`.
* Remove delete from `streams.state.OffsetCheckpoint` constructor (similar to the change in `kafka.server.OffsetCheckpoint` in 836cb19633 (diff-2503b32f29cbbd61ed8316f127829455L29)).
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Ewen Cheslack-Postava <ewen@confluent.io>
Closes#771 from ijuma/kafka-3105-use-atomic-move-with-fallback-instead-of-rename
Follow up PR as per comments in the ticket.
junrao It should be correct now as `curBrokers` included only live brokers and live/dead brokers are computed based on it. Could you take a look when you have time?
Author: David Jacot <david.jacot@gmail.com>
Reviewers: Jun Rao <junrao@gmail.com>
Closes#756 from dajac/KAFKA-3085
Buffer is rewound before the compression rate metric is updated which results in 0 compress rate. The fix is to include a new compressRate field to record the latest compression rate in ```close()``` and return it to sensor
Author: Xiao, Tao <taxiao@ebay.com>
Reviewers: Guozhang Wang
Closes#784 from xiaotao183/kafka-2993
guozhangwang
When ```WindowedSerializer``` is specified in ```to(...)``` or ```through(...)``` for a key, we use ```WindowedStreamPartitioner```.
Author: Yasuhiro Matsuda <yasuhiro@confluent.io>
Reviewers: Guozhang Wang
Closes#779 from ymatsuda/partitioner
topic level config delete config options
use --delete-config instead of --deleteConfig
Author: eamon <zhang.eamon@hotmail.com>
Reviewers: Ismael Juma, Grant Henke, Guozhang Wang
Closes#612 from EamonZhang/trunk
Remove deletion of tmp file in `OffsetCheckpoint`'s constructor. This delete causes unintuitive behaviour like `LogRecoveryTest` causing a `System.exit` because the test creates an instance of `OffsetCheckpoint` in order to call `read()` on it (while unexpectedly deleting a file being written by another instance of `OffsetCheckpoint`).
Also:
* Improve error-handling in `OffsetCheckpoint`
* Also include minor performance improvements in `read()`
* Minor clean-ups to `ReplicaManager` and `LogRecoveryTest`
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Ewen Cheslack-Postava <ewen@confluent.io>
Closes#759 from ijuma/kafka-3063-log-recovery-test-exits-jvm
I'm also fixing a bug in the testChroot test case.
Author: Flavio Junqueira <fpj@apache.org>
Reviewers: Ewen Cheslack-Postava <ewen@confluent.io>
Closes#736 from fpj/KAFKA-3069
Note that KAFKA-3077 will be required to run these tests.
Author: Ashish Singh <asingh@cloudera.com>
Reviewers: Ewen Cheslack-Postava <ewen@confluent.io>
Closes#747 from SinghAsDev/KAFKA-3078
Simple fixes that have tripped users.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Ewen Cheslack-Postava <ewen@confluent.io>
Closes#745 from ijuma/security-doc-improvements
This is handy when debugging certain kinds of Jenkins failures.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Guozhang Wang <wangguoz@gmail.com>, Ewen Cheslack-Postava <ewen@confluent.io>
Closes#739 from ijuma/gradle-show-standard-streams
Fix two minor typos in Kafka official website page.
Author: Gabriel Zhang <smalldirector@yahoo.com>
Reviewers: Ismael Juma, Guozhang Wang
Closes#742 from smalldirector/kafka-document-typos-fix
Patch by fpj and benstopford.
Author: flavio junqueira <fpj@apache.org>
Author: Flavio Junqueira <fpj@apache.org>
Author: Ben Stopford <benstopford@gmail.com>
Reviewers: Ben Stopford <benstopford@gmail.com>, Geoff Anderson <geoff@confluent.io>, Ewen Cheslack-Postava <ewen@confluent.io>
Closes#683 from fpj/KAFKA-2979
Added option to use custom partitioning logic within each topology sink.
Author: Randall Hauch <rhauch@gmail.com>
Reviewers: Guozhang Wang
Closes#309 from rhauch/kafka-2649
Author: Grant Henke <granthenke@gmail.com>
Reviewers: Ismael Juma <ismael@juma.me.uk>, Jun Rao <junrao@gmail.com>
Closes#686 from granthenke/compaction
Deprecates ErrorMapping.scala in core in favor or Errors.java in common.
Duplicated exceptions in core are deprecated as well, to ensure the mapping is correct.
Author: Grant Henke <granthenke@gmail.com>
Reviewers: Jason Gustafson <jason@confluent.io>, Ismael Juma <ismael@juma.me.uk>, Ewen Cheslack-Postava <ewen@confluent.io>
Closes#616 from granthenke/error-mapping
Author: David Jacot <david.jacot@gmail.com>
Reviewers: Ismael Juma <ismael@juma.me.uk>, Grant Henke <granthenke@gmail.com>, Jun Rao <junrao@gmail.com>
Closes#196 from dajac/KAFKA-2072-part-2
Author: Michael G. Noll <michael@confluent.io>
Reviewers: Flavio Junqueira <fpj@apache.org>, Jun Rao <junrao@gmail.com>
Closes#573 from miguno/KAFKA-2874
Use invalid hostname to ensure that test works in all environments
Author: Rajini Sivaram <rajinisivaram@googlemail.com>
Reviewers: Ismael Juma <ismael@juma.me.uk>, Jun Rao <junrao@gmail.com>
Closes#546 from rajinisivaram/KAFKA-2850
The check was implemented in KAFKA-340 : If we are shutting down a broker when the ISR of a partition includes only that broker, we could lose some messages that have been previously committed. For clean shutdown, we need to guarantee that there is at least 1 other broker in ISR after the broker is shut down.
When we are deleting the topic, this check can be avoided.
Author: Mayuresh Gharat <mgharat@mgharat-ld1.linkedin.biz>
Reviewers: Dong Lin <lindong28@gmail.com>, Rajini Sivaram <rajinisivaram@googlemail.com>, Jun Rao <junrao@gmail.com>
Closes#729 from MayureshGharat/kafka-2937
guozhangwang added .git/refs/heads/ file existence check.
Author: Manikumar reddy O <manikumar.reddy@gmail.com>
Reviewers: Ismael Juma, Guozhang Wang
Closes#209 from omkreddy/KAFKA-1901
Author: Konrad <konkalita@gmail.com>
Author: konradkalita <konkalita@gmail.com>
Reviewers: Ismael Juma <ismael@juma.me.uk>, Jun Rao <junrao@gmail.com>
Closes#732 from konradkalita/Kafka-3058
I verified this by trying to produce to __consumer_offsets and the logged message looks like:
[2015-12-22 10:34:40,897] INFO [KafkaApi-0] Closing connection due to error during produce request with correlation id 1 from client id console-producer with ack=0
Topic and partition to exceptions: [__consumer_offsets,43] -> kafka.common.InvalidTopicException (kafka.server.KafkaApis)
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Jun Rao <junrao@gmail.com>
Closes#709 from ijuma/kafka-3010-include-error-in-log-when-ack-0
Also include some trivial clean-ups in `ProducerConfig`and `BaseProducer`.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Gwen Shapira
Closes#710 from ijuma/use-new-producer-properties-in-config
guozhangwang
An implementation of local store for join window. This implementation uses "rolling" of RocksDB instances for timestamp based truncation.
Author: Yasuhiro Matsuda <yasuhiro@confluent.io>
Reviewers: Guozhang Wang
Closes#726 from ymatsuda/windowed_join
Fix it by making it possible to pass the `doLog` parameter to `AbstractConfig`. As explained in the code comments, this means that we can continue to benefit from ZK default settings as specified in `KafkaConfig` without having to duplicate code.
Also:
* Removed unused private methods from `KafkaConfig`
* Removed `case` modifier from `KafkaConfig` so that `hashCode`, `equals`
and `toString` from `AbstractConfig` are used.
* Made `props` a `val` and added `apply` method to `KafkaConfig` to
remain binary compatible.
* Call authorizer.close even if an exception is thrown during `configure`.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Guozhang Wang
Closes#725 from ijuma/kafka-3052-broker-properties-get-logged-twice-if-acl-enabled
This commit contains minor grammatical fixes. Some of the changes are just removing rogue commas, which can be hard to see in the diff.
This contribution is my original work and I license the work to the project under the project's open source license.
Author: Samuel Julius Hecht <samjhecht@gmail.com>
Reviewers: Gwen Shapira
Closes#721 from samjhecht/minor-docs-edits
sasl.kerberos.service.name surround by double quote didn't work, have to remove.
Author: BINLEI XUE <kongpo0412@gmail.com>
Reviewers: Gwen Shapira
Closes#720 from secjex/patch-1
In Kafka 0.9, request.required.acks=-1 which configration of producer is replaced by acks=all,
but this old config is remained in docs.
Author: Sasaki Toru <sasakitoa@nttdata.co.jp>
Reviewers: Gwen Shapira
Closes#716 from sasakitoa/acks_doc
David Jacot
Jason Gustafson
Ismael Juma
Tao Xiao
Yasuhiro Matsuda
Guozhang Wang
Eamon Zhang
Ján Koščo
Flavio Junqueira
Ashish Singh
Dong Lin
Grant Henke
Praveen Devarao
Gabriel Zhang
Randall Hauch
Michael G. Noll
Rajini Sivaram
Mayuresh Gharat
Manikumar reddy O
Konrad
Gwen Shapira
Kishore Senji
Samuel Julius Hecht
BINLEI XUE
Sasaki Toru