Author: Tom Lee <github@tomlee.co>
Reviewers: Onur Karaman <okaraman@linkedin.com>, Jiangjie Qin <jiangjie@linkedin.com>, Grant Henke <ghenke@cloudera.com>, Jason Gustafson <jason@confluent.io>, Guozhang Wang <wangguoz@gmail.com>
Closes#962 from hachikuji/KAFKA-2698
guozhangwang
* add table aggregate to the system test
* actually create change log partition replica
Author: Yasuhiro Matsuda <yasuhiro@confluent.io>
Reviewers: Guozhang Wang <wangguoz@gmail.com>
Closes#966 from ymatsuda/enh_systest
There is a possibility that the state directory locking fails when another stream thread is taking long to close all tasks. Simple retries should alleviate the problem.
Author: Yasuhiro Matsuda <yasuhiro@confluent.io>
Reviewers: Guozhang Wang <wangguoz@gmail.com>
Closes#899 from ymatsuda/minor2
This PR includes a number of clean-ups:
* Code style
* Documentation wording improvements
* Efficiency improvements
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Jun Rao <junrao@gmail.com>
Closes#943 from ijuma/kafka-3259-kip-31-32-clean-ups
Also update `kafka-merge-pr.py` and `tests/kafkatest/__init__.py`.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Ewen Cheslack-Postava <ewen@confluent.io>
Closes#963 from ijuma/update-trunk-0.10.0.0-SNAPSHOT
…able remote debugging to Kafka tools scripts
Author: Christian Posta <christian.posta@gmail.com>
Reviewers: Grant Henke, Gwen Shapira
Closes#955 from christian-posta/ceposta-enable-jvm-debugging-opts
Author: Jason Gustafson <jason@confluent.io>
Reviewers: Grant Henke <granthenke@gmail.com>, Ewen Cheslack-Postava <ewen@confluent.io>
Closes#931 from hachikuji/KAFKA-3007
guozhangwang
My bad. I removed ZOOKEEPER_CONNECT_CONFIG from consumer's config by mistake. It is needed by our own partition assigner running in consumers.
Author: Yasuhiro Matsuda <yasuhiro@confluent.io>
Reviewers: Guozhang Wang <wangguoz@gmail.com>
Closes#959 from ymatsuda/hotfix3
KAFKA-3242: minor rename / logging change to references to 'adding partitions' to indicate 'modifying partitions'
Author: Ben Stopford <benstopford@gmail.com>
Reviewers: Grant Henke
Closes#924 from benstopford/small_changes
StreamThread should keep going after a commit was failed due to a group rebalance.
Currently the thread just dies.
guozhangwang
Author: Yasuhiro Matsuda <yasuhiro@confluent.io>
Reviewers: Guozhang Wang <wangguoz@gmail.com>
Closes#933 from ymatsuda/catch_commit_failure
Author: zhuchen1018 <amandazhu19620701@gmail.com>
Reviewers: Grant Henke <granthenke@gmail.com>, Joel Koshy <jjkoshy.w@gmail.com>, Dong Lin <lindong28@gmail.com>
Closes#935 from zhuchen1018/minor-remove-unused-imports
This is the latest version in Maven even though HISTORY.md includes releases all the way to 4.5.0.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Grant Henke <ghenke@cloudera.com>, Guozhang Wang <wangguoz@gmail.com>
Closes#937 from ijuma/update-rocks-db-for-streams
These are minor, but no reason to make our example code look worse than it has to.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Grant Henke
Closes#940 from ijuma/example-style-improvements
Author: Frank Scholten <frank@frankscholten.nl>
Reviewers: Eno Thereska <eno.thereska@gmail.com>, Ewen Cheslack-Postava <ewen@confluent.io>
Closes#941 from frankscholten/tests/cluster-connection-states
… with the Vagranfile.local file
Author: Christian Posta <christian.posta@gmail.com>
Reviewers: Gwen Shapira
Closes#942 from christian-posta/ceposta-doco
See KIP-31 and KIP-32 for details.
A few notes on the patch:
1. This patch implements KIP-31 and KIP-32. The patch includes features in both KAFKA-3025, KAFKA-3026 and KAFKA-3036
2. All unit tests passed.
3. The unit tests were run with new and old message format.
4. When message format conversion occurs during consumption, the consumer will not be able to detect the message size too large situation. I did not try to fix this because the situation seems rare and only happen during migration phase.
Author: Jiangjie Qin <becket.qin@gmail.com>
Author: Ismael Juma <ismael@juma.me.uk>
Author: Jiangjie (Becket) Qin <becket.qin@gmail.com>
Reviewers: Jason Gustafson <jason@confluent.io>, Anna Povzner <anna@confluent.io>, Ismael Juma <ismael@juma.me.uk>, Guozhang Wang <wangguoz@gmail.com>, Jun Rao <junrao@gmail.com>
Closes#764 from becketqin/KAFKA-3025
Removing streams' specific config params from producer/consumer configs to reduce warning messages.
Author: Yasuhiro Matsuda <yasuhiro@confluent.io>
Reviewers: Guozhang Wang <wangguoz@gmail.com>
Closes#906 from ymatsuda/clean_config
1. Added a test case to prove commit() on SourceTask was not being called.
2. Added commitSourceTask() which logs potential exceptions.
3. Added after call to finishSuccessfulFlush().
Author: Jeremy Custenborder <jeremy@scarcemedia.com>
Reviewers: Ewen Cheslack-Postava <ewen@confluent.io>
Closes#909 from jcustenborder/KAFKA-3225
Author: zhuchen1018 <amandazhu19620701@gmail.com>
Reviewers: Dong Lin <lindong28@gmail.com>, Guozhang Wang <wangguoz@gmail.com>
Closes#911 from zhuchen1018/KAFKA-2757
…tionListener introduced as part of KAFKA-2211
Author: Parth Brahmbhatt <brahmbhatt.parth@gmail.com>
Reviewers: Flavio Junqueira <fpj@apache.org>, Ismael Juma <ismael@juma.me.uk>, Sriharsha Chintalapani <mail@harsha.io>
Closes#679 from Parth-Brahmbhatt/KAFKA-2547 and squashes the following commits:
1722c76 [Parth Brahmbhatt] Addressing review comments.
376f77d [Parth Brahmbhatt] Merge branch 'trunk' of http://git-wip-us.apache.org/repos/asf/kafka into KAFKA-2547
a13b963 [Parth Brahmbhatt] Addressing comments from Reviewers.
1007137 [Parth Brahmbhatt] KAFKA-2547: Make DynamicConfigManager to use the ZkNodeChangeNotificationListener introduced as part of KAFKA-2211
…uests equivalent
Author: Grant Henke <granthenke@gmail.com>
Reviewers: Ismael Juma <ismael@juma.me.uk>, Sriharsha Chintalapani <mail@harsha.io>
Closes#896 from granthenke/update-metadata and squashes the following commits:
2eb5d59 [Grant Henke] Address reviews
497258d [Grant Henke] KAFKA-2508: Replace UpdateMetadata{Request,Response} with o.a.k.c.requests equivalent
A few minor fixes to reconcile differences between the windows and unix versions of the kafka/zookeeper start scripts that were causing cross-platform inconsistencies during deployment.
- Resolve differences in CLASSPATH setup between .bat and .sh start scripts
- .bat start scripts honor externally provided KAFKA_HEAP_OPTS and KAFKA_LOG4J_OPTS consistent with .sh
- .bat start scripts configure log4j similar to .sh
Author: Matt Fluet <matt.fluet@appian.com>
Reviewers: Eno Thereska <eno.thereska@gmail.com>, Grant Henke <granthenke@gmail.com>, Ewen Cheslack-Postava <ewen@confluent.io>
Closes#908 from fluetm/scripts-patch
Patch version bumps for bouncy castle, minikdc, snappy, slf4j, scalatest and powermock. Notable fixes:
* Snappy: fixes a resource leak
* Bouncy castle: security fixes
Also update Gradle to 2.11 (where the notable change is improved IDE integration) and the grgit build dependency.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Grant Henke <granthenke@gmail.com>, Ewen Cheslack-Postava <ewen@confluent.io>
Closes#903 from ijuma/kafka-3227-conservative-update-of-kafka-deps
…ent ID
- Adds NULLABLE_STRING Type to the protocol
- Changes client_id in the REQUEST_HEADER to NULLABLE_STRING with a default of ""
- Fixes server handling of invalid ApiKey request and other invalid requests
Author: Grant Henke <granthenke@gmail.com>
Reviewers: Ismael Juma <ismael@juma.me.uk>, Joel Koshy <jjkoshy.w@gmail.com>
Closes#866 from granthenke/null-clientid
Remove the batch from the RecordAccumulator once its closed while aborting batches. Make sure we don't accept new batch appends to RecordAccumulator while the producer is being closed.
Author: Mayuresh Gharat <mgharat@mgharat-ld1.linkedin.biz>
Reviewers: Jiangjie Qin, Ismael Juma, Guozhang Wang
Closes#825 from MayureshGharat/KAFKA-3147
Pass through the root StateStore in the init method so the inner StateStore can register that object.
Author: tomdearman <tom.dearman@gmail.com>
Reviewers: Yasuhiro Matsuda
Closes#904 from tomdearman/KAFKA-3229
* We need to poll periodically even when all partitions are paused in order to respond to a possible rebalance promptly.
* There is a race condition when two (or more) threads try to clean up the same state directory. One of the thread fails with FileNotFoundException. Thus the new code simply catches it and ignore.
Author: Yasuhiro Matsuda <yasuhiro@confluent.io>
Reviewers: Gwen Shapira
Closes#893 from ymatsuda/hotfix
There are multi-byte characters In quickstart.html and security.html.
This PR will fix it.
Author: Sasaki Toru <sasakitoa@nttdata.co.jp>
Reviewers: Grant Henke
Closes#897 from sasakitoa/remove_multi_byte_character
This is the most of the KIP-42: Producer and consumer interceptor. (Except exposing CRC and record sizes to the interceptor, which is coming as a separate PR; tracked by KAFKA-3196).
This PR includes:
1. Add ProducerInterceptor interface and call its callbacks from appropriate places in Kafka Producer.
2. Add ConsumerInterceptor interface and call its callbacks from appropriate places in Kafka Consumer.
3. Add unit tests for interceptor changes
4. Add integration test for both mutable consumer and producer interceptors.
Author: Anna Povzner <anna@confluent.io>
Reviewers: Jason Gustavson, Ismael Juma, Gwen Shapira
Closes#854 from apovzner/kip42
Ismael Juma
Jason Gustafson
Tom Lee
Yasuhiro Matsuda
Gwen Shapira
Ewen Cheslack-Postava
Christian Posta
Guozhang Wang
Xin Wang
Ben Stopford
zhuchen1018
Jiangjie Qin
Frank Scholten
Kim Christensen
Jun Rao
Jeremy Custenborder
Parth Brahmbhatt
Grant Henke
Matt Fluet
Mayuresh Gharat
tomdearman
Sasaki Toru
Anna Povzner
Ashish Singh