Author: Jiangjie Qin <becket.qin@gmail.com>
Reviewers: Ismael Juma <ismael@juma.me.uk>, Jun Rao <junrao@gmail.com>
Closes#1112 from becketqin/KAFKA-3442
Also included a minor efficiency improvement in kafka.cluster.EndPoint.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Gwen Shapira
Closes#1105 from ijuma/kafka-3431-replace-broker-end-point-with-node
This fixes an issue with long topic names by considering, during topic
validation, the '-' and the partition id that is appended to the log
folder created for each topic partition.
Author: Vahid Hashemian <vahidhashemian@us.ibm.com>
Reviewers: Gwen Shapira, Grant Henke
Closes#898 from vahidhashemian/KAFKA-3219
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Grant Henke, Gwen Shapira
Closes#1100 from ijuma/kafka-3426-invalid-protocol-type-errors-invalid-sizes
It is not true in practice. Maybe the implied feature is not yet implemented or removed.
These lines can be super misleading.
Please merge.
Thank you.
Author: gaob13 <gaob13@mails.tsinghua.edu.cn>
Reviewers: Ismael Juma, Ewen Cheslack-Postava
Closes#793 from gaob13/trunk
Author: Jason Gustafson <jason@confluent.io>
Reviewers: Ismael Juma <ismael@juma.me.uk>, Ewen Cheslack-Postava <ewen@confluent.io>
Closes#1108 from hachikuji/KAFKA-3412
Note: This goes only to trunk. 0.10.0 branch will need a separate PR with different versions.
Author: Gwen Shapira <cshapi@gmail.com>
Reviewers: Ismael Juma <ismael@juma.me.uk>, Ewen Cheslack-Postava <ewen@confluent.io>
Closes#1109 from gwenshap/minor-fix-version-trunk
This will need to be double-committed.
Author: Gwen Shapira <cshapi@gmail.com>
Reviewers: Ismael Juma <ismael@juma.me.uk>, Ewen Cheslack-Postava <ewen@confluent.io>
Closes#1107 from gwenshap/fix-doc-version
Also improves Java docs for the Streams high-level DSL.
Author: Guozhang Wang <wangguoz@gmail.com>
Reviewers: Ismael Juma, Michael G. Noll
Closes#1097 from guozhangwang/KNewJavaDoc
Author: Grant Henke <granthenke@gmail.com>
Reviewers: Ismael Juma <ismael@juma.me.uk>, Jun Rao <junrao@gmail.com>
Closes#1091 from granthenke/fetch-error
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Larkin Lowrey <llowrey@gmail.com>, Jun Rao <junrao@gmail.com>
Closes#1103 from ijuma/kafka-3378-follow-up
The main impediment to bringing up aws machines in parallel using vagrant was the interaction between `vagrant-hostmanager` and `vagrant-aws`. If you disable hostmanager during the `up` phase, and run it after the cluster is up, parallel bringup is possible. The only caveat is that machines must be brought up in small-ish batches to prevent rate limit errors from AWS since `vagrant-aws` doesn't seem to have mechanisms to
This PR:
- disables `vagrant-hostmanager` during bringup
- adds a wrapper script to make it convenient to bring machines up in batches on aws
Author: Geoff Anderson <geoff@confluent.io>
Reviewers: Ewen Cheslack-Postava <ewen@confluent.io>
Closes#982 from granders/vagrant-disable-hostmanager
…e calls
Changes the SimpleAclAuthorizer to:
- Track and utilize the zookeeper version when updating zookeeper to prevent data loss in the case of stale reads and race conditions
- Update local cache when modifying ACLs
- Add debug logging
Author: Grant Henke <granthenke@gmail.com>
Author: Grant Henke <granthenke@users.noreply.github.com>
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Flavio Junqueira, Jun Rao, Ismael Juma, Gwen Shapira
Closes#1006 from granthenke/simple-authorizer-fix
This is a different implementation to the one in #1085 by Larkin Lowrey (llowrey). The hard part here was actually finding the problem and all credit goes to llowrey.
This PR also fixes our handling of `finishConnect` (we now check the return value).
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Jun Rao
Closes#1094 from ijuma/KAFKA-3378-instantly-connecting-socket-channels
minor fix for prompt the user, or we will get a error message:
>Missing required argument "[time]"
Author: Xin Wang <best.wangxin@163.com>
Reviewers: Ashish Singh, Guozhang Wang
Closes#1068 from vesense/patch-4
Author: Jason Gustafson <jason@confluent.io>
Reviewers: Ismael Juma <ismael@juma.me.uk>, Grant Henke <granthenke@gmail.com>, Ewen Cheslack-Postava <me@ewencp.org>, Jun Rao <junrao@gmail.com>
Closes#1064 from hachikuji/KAFKA-3394
Also include:
1) remove streams specific configs before passing to producer and consumer to avoid warning message;
2) add `ConsumerRecord` timestamp extractor and set as the default extractor.
Author: Guozhang Wang <wangguoz@gmail.com>
Reviewers: Michael G. Noll, Ewen Cheslack-Postava
Closes#1093 from guozhangwang/KConfigWarn
ewencp gwenshap granders could you have a look please? Thanks.
Author: Eno Thereska <eno.thereska@gmail.com>
Reviewers: Ewen Cheslack-Postava <ewen@confuent.io>
Closes#1096 from enothereska/systest-hotfix-name
Also update server tests to always use new producer.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Gwen Shapira
Closes#1092 from ijuma/kafka-2982-deprecate-old-producers
becketqin apovzner please have a look. becketqin the test fails when the producer and consumer are 0.9.x and the message format changes on the fly.
Author: Eno Thereska <eno.thereska@gmail.com>
Reviewers: Ewen Cheslack-Postava, Ismael Juma, Gwen Shapira
Closes#1070 from enothereska/kafka-3202-format-change-fly
becketqin Can you take a look?
Author: Dong Lin <lindong28@gmail.com>
Reviewers: Ismael Juma <ismael@juma.me.uk>, Jun Rao <junrao@gmail.com>
Closes#1009 from lindong28/KAFKA-3330
…ibraries
This ensures duplicates are not copied in the distribution without rewriting all of the tar'ing logic. A larger improvement could be made to the packaging code, but that should be tracked by another jira.
Author: Grant Henke <granthenke@gmail.com>
Reviewers: Gwen Shapira, Ismael Juma
Closes#1075 from granthenke/libs-duplicates
I also slightly tweaked the wording on a couple of warnings.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Gwen Shapira
Closes#1072 from ijuma/kafka-3373-follow-up
`Map.values` returns `DefaultValuesIterable` where the default implementation of `toSeq` is (sadly) `toStream`. `Stream` is a lazy collection and it can reflect changes to the underlying map before it's `forced`.
I verified that the test failed before my change.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Gwen Shapira
Closes#1088 from ijuma/kafka-3414-get-alive-brokers-no-mutation
apovzner becketqin please have a look if you can. Thanks.
Author: Eno Thereska <eno.thereska@gmail.com>
Reviewers: Anna Povzner, Gwen Shapira
Closes#1059 from enothereska/kafka-3188-compatibility
A new consumer config option 'exclude.internal.topics' was added to
allow excluding internal topics when wildcards are used to specify
consumers.
The new option takes a boolean value, with a default 'false' value (i.e.
no exclusion).
This patch is co-authored with rajinisivaram edoardocomar mimaison
Author: edoardo <ecomar@uk.ibm.com>
Author: Vahid Hashemian <vahidhashemian@us.ibm.com>
Reviewers: Ismael Juma, Jun Rao, Gwen Shapira
Closes#1082 from edoardocomar/KAFKA-2832
This contribution is my original work, and I license it under the project's open source license.
CC jkreps
Author: Drausin Wulsin <daedalus2718@gmail.com>
Author: John Doe <daedalus2718@gmail.com>
Reviewers: Jason Gustafson
Closes#1055 from drausin/bugfix/consumer-records-iterator
guozhangwang ymatsuda : please review.
Author: Michael G. Noll <michael@confluent.io>
Reviewers: Guozhang Wang <wangguoz@gmail.com>
Closes#1081 from miguno/KAFKA-3411
This is a KIP-42 followup.
Currently, If sending the record fails before it gets to the server, ProducerInterceptor.onAcknowledgement() is called with metadata == null, and non-null exception. However, it is useful to pass topic and partition, if known, to ProducerInterceptor.onAcknowledgement() as well. This patch ensures that ProducerInterceptor.onAcknowledgement() gets record metadata with topic and maybe partition. If partition is not set in 'record' and KafkaProducer.send() fails before partition gets assigned, then ProducerInterceptor.onAcknowledgement() gets RecordMetadata with partition == -1. Only time when ProducerInterceptor.onAcknowledgement() gets null record metadata is when the client passes null record to KafkaProducer.send().
Author: Anna Povzner <anna@confluent.io>
Reviewers: Ismael Juma <ismael@juma.me.uk>, Ashish Singh <asingh@cloudera.com>, Jun Rao <junrao@gmail.com>
Closes#1015 from apovzner/kip42-3
This ZkClient version adds authentication validation and a conditional delete method needed for other patches
Author: Grant Henke <granthenke@gmail.com>
Reviewers: Ismael Juma, Gwen Shapira
Closes#1084 from granthenke/zkclient-08
becketqin have a look if this looks reasonable to you. Thanks.
Author: Eno Thereska <eno.thereska@gmail.com>
Reviewers: Geoff Anderson <geoff@confluent.io>, Ewen Cheslack-Postava <ewen@confluent.io>
Closes#1051 from enothereska/kafka-3371
Added commitRecord(SourceRecord record) to SourceTask. This method is called during the callback from producer.send() when the message has been sent successfully. Added commitTaskRecord(SourceRecord record) to WorkerSourceTask to handle calling commitRecord on the SourceTask. Updated tests for calls to commitRecord.
Author: Jeremy Custenborder <jcustenborder@gmail.com>
Reviewers: Jason Gustafson <jason@confluent.io>, Ewen Cheslack-Postava <ewen@confluent.io>
Closes#950 from jcustenborder/KAFKA-3260
Also clean-up `LogTest` a little.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Guozhang Wang
Closes#1071 from ijuma/kafka-3047-explicit-offset-assignment-corrupt-log-test
Please see https://cwiki.apache.org/confluence/display/KAFKA/KIP-36+Rack+aware+replica+assignment for the overall design.
The update to TopicMetadataRequest/TopicMetadataResponse will be done in a different PR.
Author: Allen Wang <awang@netflix.com>
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Ismael Juma <ismael@juma.me.uk>, Jason Gustafson <jason@confluent.io>, Grant Henke <granthenke@gmail.com>, Jun Rao <junrao@gmail.com>
Closes#132 from allenxwang/KAFKA-1215
Jason Gustafson
Jiangjie Qin
Ismael Juma
Grant Henke
Ewen Cheslack-Postava
Vahid Hashemian
gaob13
Gwen Shapira
Guozhang Wang
Geoff Anderson
Jun Rao
Pierre-Yves Ritschard
Xin Wang
Eno Thereska
Dong Lin
Liquan Pei
edoardo
Drausin Wulsin
Michael G. Noll
Anna Povzner
Jeremy Custenborder
Yasuhiro Matsuda
Allen Wang