Observation: when doing "gradlew releaseTarGz" the streams jar was not included in the tarball. Adding a line to include it. ijuma guozhangwang could you please review. Thanks.
Author: Eno Thereska <eno.thereska@gmail.com>
Reviewers: Guozhang Wang <wangguoz@gmail.com>
Closes#984 from enothereska/trunk
guozhangwang
Author: Kim Christensen <kich@mvno.dk>
Reviewers: Ismael Juma <ismael@juma.me.uk>, Guozhang Wang <wangguoz@gmail.com>
Closes#912 from kichristensen/KAFKA-3133
guozhangwang made the changes as requested, I reverted my original commit and that seems to have closed the other pull request - sorry if that mucks up the process a bit
Author: tomdearman <tom.dearman@gmail.com>
Reviewers: Guozhang Wang <wangguoz@gmail.com>
Closes#978 from tomdearman/KAFKA-3278
Added CompressionTest that tests 4 producers, each using a different compression type and one not using compression.
Enabled VerifiableProducer to run producers with different compression types (passed in the constructor). This includes enabling each producer to output unique values, so that the verification process in ProduceConsumeValidateTest is correct (counts acks from all producers).
Also a fix for console consumer to raise an exception if it sees the incorrect consumer output (before we swallowed an exception, so was hard to debug the issue).
Author: Anna Povzner <anna@confluent.io>
Reviewers: Geoff Anderson, Jason Gustafson
Closes#958 from apovzner/kafka-3214
…list is not supported for new consumers
Author: Ashish Singh <asingh@cloudera.com>
Reviewers: Grant Henke <granthenke@gmail.com>, Ewen Cheslack-Postava <ewen@confluent.io>
Closes#923 from SinghAsDev/KAFKA-3243
This is my original work and I license the work to the project under the project's open source license.
Author: Richard Whaling <rwhaling@spantree.net>
Reviewers: Jason Gustafson, Gwen Shapira
Closes#968 from rwhaling/docs/kafkaconsumer-heartbeat-doc-improvement
Author: Tom Lee <github@tomlee.co>
Reviewers: Onur Karaman <okaraman@linkedin.com>, Jiangjie Qin <jiangjie@linkedin.com>, Grant Henke <ghenke@cloudera.com>, Jason Gustafson <jason@confluent.io>, Guozhang Wang <wangguoz@gmail.com>
Closes#962 from hachikuji/KAFKA-2698
guozhangwang
* add table aggregate to the system test
* actually create change log partition replica
Author: Yasuhiro Matsuda <yasuhiro@confluent.io>
Reviewers: Guozhang Wang <wangguoz@gmail.com>
Closes#966 from ymatsuda/enh_systest
There is a possibility that the state directory locking fails when another stream thread is taking long to close all tasks. Simple retries should alleviate the problem.
Author: Yasuhiro Matsuda <yasuhiro@confluent.io>
Reviewers: Guozhang Wang <wangguoz@gmail.com>
Closes#899 from ymatsuda/minor2
This PR includes a number of clean-ups:
* Code style
* Documentation wording improvements
* Efficiency improvements
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Jun Rao <junrao@gmail.com>
Closes#943 from ijuma/kafka-3259-kip-31-32-clean-ups
Also update `kafka-merge-pr.py` and `tests/kafkatest/__init__.py`.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Ewen Cheslack-Postava <ewen@confluent.io>
Closes#963 from ijuma/update-trunk-0.10.0.0-SNAPSHOT
…able remote debugging to Kafka tools scripts
Author: Christian Posta <christian.posta@gmail.com>
Reviewers: Grant Henke, Gwen Shapira
Closes#955 from christian-posta/ceposta-enable-jvm-debugging-opts
Author: Jason Gustafson <jason@confluent.io>
Reviewers: Grant Henke <granthenke@gmail.com>, Ewen Cheslack-Postava <ewen@confluent.io>
Closes#931 from hachikuji/KAFKA-3007
guozhangwang
My bad. I removed ZOOKEEPER_CONNECT_CONFIG from consumer's config by mistake. It is needed by our own partition assigner running in consumers.
Author: Yasuhiro Matsuda <yasuhiro@confluent.io>
Reviewers: Guozhang Wang <wangguoz@gmail.com>
Closes#959 from ymatsuda/hotfix3
KAFKA-3242: minor rename / logging change to references to 'adding partitions' to indicate 'modifying partitions'
Author: Ben Stopford <benstopford@gmail.com>
Reviewers: Grant Henke
Closes#924 from benstopford/small_changes
StreamThread should keep going after a commit was failed due to a group rebalance.
Currently the thread just dies.
guozhangwang
Author: Yasuhiro Matsuda <yasuhiro@confluent.io>
Reviewers: Guozhang Wang <wangguoz@gmail.com>
Closes#933 from ymatsuda/catch_commit_failure
Author: zhuchen1018 <amandazhu19620701@gmail.com>
Reviewers: Grant Henke <granthenke@gmail.com>, Joel Koshy <jjkoshy.w@gmail.com>, Dong Lin <lindong28@gmail.com>
Closes#935 from zhuchen1018/minor-remove-unused-imports
This is the latest version in Maven even though HISTORY.md includes releases all the way to 4.5.0.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Grant Henke <ghenke@cloudera.com>, Guozhang Wang <wangguoz@gmail.com>
Closes#937 from ijuma/update-rocks-db-for-streams
These are minor, but no reason to make our example code look worse than it has to.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Grant Henke
Closes#940 from ijuma/example-style-improvements
Author: Frank Scholten <frank@frankscholten.nl>
Reviewers: Eno Thereska <eno.thereska@gmail.com>, Ewen Cheslack-Postava <ewen@confluent.io>
Closes#941 from frankscholten/tests/cluster-connection-states
… with the Vagranfile.local file
Author: Christian Posta <christian.posta@gmail.com>
Reviewers: Gwen Shapira
Closes#942 from christian-posta/ceposta-doco
See KIP-31 and KIP-32 for details.
A few notes on the patch:
1. This patch implements KIP-31 and KIP-32. The patch includes features in both KAFKA-3025, KAFKA-3026 and KAFKA-3036
2. All unit tests passed.
3. The unit tests were run with new and old message format.
4. When message format conversion occurs during consumption, the consumer will not be able to detect the message size too large situation. I did not try to fix this because the situation seems rare and only happen during migration phase.
Author: Jiangjie Qin <becket.qin@gmail.com>
Author: Ismael Juma <ismael@juma.me.uk>
Author: Jiangjie (Becket) Qin <becket.qin@gmail.com>
Reviewers: Jason Gustafson <jason@confluent.io>, Anna Povzner <anna@confluent.io>, Ismael Juma <ismael@juma.me.uk>, Guozhang Wang <wangguoz@gmail.com>, Jun Rao <junrao@gmail.com>
Closes#764 from becketqin/KAFKA-3025
Removing streams' specific config params from producer/consumer configs to reduce warning messages.
Author: Yasuhiro Matsuda <yasuhiro@confluent.io>
Reviewers: Guozhang Wang <wangguoz@gmail.com>
Closes#906 from ymatsuda/clean_config
1. Added a test case to prove commit() on SourceTask was not being called.
2. Added commitSourceTask() which logs potential exceptions.
3. Added after call to finishSuccessfulFlush().
Author: Jeremy Custenborder <jeremy@scarcemedia.com>
Reviewers: Ewen Cheslack-Postava <ewen@confluent.io>
Closes#909 from jcustenborder/KAFKA-3225
Author: zhuchen1018 <amandazhu19620701@gmail.com>
Reviewers: Dong Lin <lindong28@gmail.com>, Guozhang Wang <wangguoz@gmail.com>
Closes#911 from zhuchen1018/KAFKA-2757
…tionListener introduced as part of KAFKA-2211
Author: Parth Brahmbhatt <brahmbhatt.parth@gmail.com>
Reviewers: Flavio Junqueira <fpj@apache.org>, Ismael Juma <ismael@juma.me.uk>, Sriharsha Chintalapani <mail@harsha.io>
Closes#679 from Parth-Brahmbhatt/KAFKA-2547 and squashes the following commits:
1722c76 [Parth Brahmbhatt] Addressing review comments.
376f77d [Parth Brahmbhatt] Merge branch 'trunk' of http://git-wip-us.apache.org/repos/asf/kafka into KAFKA-2547
a13b963 [Parth Brahmbhatt] Addressing comments from Reviewers.
1007137 [Parth Brahmbhatt] KAFKA-2547: Make DynamicConfigManager to use the ZkNodeChangeNotificationListener introduced as part of KAFKA-2211
…uests equivalent
Author: Grant Henke <granthenke@gmail.com>
Reviewers: Ismael Juma <ismael@juma.me.uk>, Sriharsha Chintalapani <mail@harsha.io>
Closes#896 from granthenke/update-metadata and squashes the following commits:
2eb5d59 [Grant Henke] Address reviews
497258d [Grant Henke] KAFKA-2508: Replace UpdateMetadata{Request,Response} with o.a.k.c.requests equivalent
A few minor fixes to reconcile differences between the windows and unix versions of the kafka/zookeeper start scripts that were causing cross-platform inconsistencies during deployment.
- Resolve differences in CLASSPATH setup between .bat and .sh start scripts
- .bat start scripts honor externally provided KAFKA_HEAP_OPTS and KAFKA_LOG4J_OPTS consistent with .sh
- .bat start scripts configure log4j similar to .sh
Author: Matt Fluet <matt.fluet@appian.com>
Reviewers: Eno Thereska <eno.thereska@gmail.com>, Grant Henke <granthenke@gmail.com>, Ewen Cheslack-Postava <ewen@confluent.io>
Closes#908 from fluetm/scripts-patch
Patch version bumps for bouncy castle, minikdc, snappy, slf4j, scalatest and powermock. Notable fixes:
* Snappy: fixes a resource leak
* Bouncy castle: security fixes
Also update Gradle to 2.11 (where the notable change is improved IDE integration) and the grgit build dependency.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Grant Henke <granthenke@gmail.com>, Ewen Cheslack-Postava <ewen@confluent.io>
Closes#903 from ijuma/kafka-3227-conservative-update-of-kafka-deps
…ent ID
- Adds NULLABLE_STRING Type to the protocol
- Changes client_id in the REQUEST_HEADER to NULLABLE_STRING with a default of ""
- Fixes server handling of invalid ApiKey request and other invalid requests
Author: Grant Henke <granthenke@gmail.com>
Reviewers: Ismael Juma <ismael@juma.me.uk>, Joel Koshy <jjkoshy.w@gmail.com>
Closes#866 from granthenke/null-clientid
Remove the batch from the RecordAccumulator once its closed while aborting batches. Make sure we don't accept new batch appends to RecordAccumulator while the producer is being closed.
Author: Mayuresh Gharat <mgharat@mgharat-ld1.linkedin.biz>
Reviewers: Jiangjie Qin, Ismael Juma, Guozhang Wang
Closes#825 from MayureshGharat/KAFKA-3147
Guozhang Wang
Eno Thereska
Kim Christensen
Parth Brahmbhatt
Tom Dearman
Anna Povzner
Ashish Singh
Richard Whaling
Ismael Juma
Jason Gustafson
Tom Lee
Yasuhiro Matsuda
Gwen Shapira
Ewen Cheslack-Postava
Christian Posta
Xin Wang
Ben Stopford
zhuchen1018
Jiangjie Qin
Frank Scholten
Jun Rao
Jeremy Custenborder
Grant Henke
Matt Fluet
Mayuresh Gharat