Author: Jason Gustafson <jason@confluent.io>
Reviewers: Ismael Juma <ismael@juma.me.uk>, Jun Rao <junrao@gmail.com>
Closes#2019 from hachikuji/KAFKA-4298
Author: Konstantine Karantasis <konstantine@confluent.io>
Reviewers: Ismael Juma <ismael@juma.me.uk>, Jason Gustafson <jason@confluent.io>
Closes#1995 from kkonstantine/KAFKA-4254-Update-producers-metadata-before-failing-on-non-existent-partition
If evict is called on a NamedCache and the cache is empty an NPE is thrown. This was reported on the user list from a developer running 0.10.1.
Author: Damian Guy <damian.guy@gmail.com>
Reviewers: Eno Thereska, Matthias J. Sax, Guozhang Wang
Closes#2024 from dguy/cache-bug
https://issues.apache.org/jira/browse/KAFKA-4025
this patch sets the file.encoding system property to UTF-8 before invoking rat during the build process and resets it to the original value afterwards
Author: radai-rosenblatt <radai.rosenblatt@gmail.com>
Reviewers: Joel Koshy <jjkoshy.w@gmail.com>
Closes#1710 from radai-rosenblatt/fix-build-on-windows
Records that are deleted/removed from the CachingKeyValueStore shouldn't appear in range and all queries.
Modified the iterator such that it skips over the deleted records.
Author: Damian Guy <damian.guy@gmail.com>
Reviewers: Eno Thereska, Matthias J. Sax, Guozhang Wang
Closes#2001 from dguy/kafka-4283
Removed a non-existing reference in table of contents and fixed some section numbers.
Author: Vahid Hashemian <vahidhashemian@us.ibm.com>
Reviewers: Jason Gustafson <jason@confluent.io>
Closes#2004 from vahidhashemian/doc/fix_toc
And improve readability by adding proper punctuations.
Author: Vahid Hashemian <vahidhashemian@us.ibm.com>
Reviewers: Jason Gustafson <jason@confluent.io>
Closes#2002 from vahidhashemian/doc/fix_typos
Make the latest version of our docs follow the latest site template structure.
Author: Derrick Or <derrickor@gmail.com>
Reviewers: Jason Gustafson <jason@confluent.io>
Closes#2003 from derrickdoo/docs-updates
Remove topics under /admin/delete_topics path in zk if deleteTopic is disabled. The topic should never be enqueued for deletion.
Author: MayureshGharat <gharatmayuresh15@gmail.com>
Reviewers: Joel Koshy <jjkoshy.w@gmail.com>, Jiangjie Qin <becket.qin@gmail.com>, Jun Rao <junrao@gmail.com>
Closes#846 from MayureshGharat/kafka-3175
Author: Jakub Dziworski <jakub.dziworski@gmail.com>
Reviewers: Guozhang Wang <wangguoz@gmail.com>, Jason Gustafson <jason@confluent.io>
Closes#1996 from JakubDziworski/doc_fix_streams_link
Author: Jakub Dziworski <jakub.dziworski@gmail.com>
Reviewers: Guozhang Wang <wangguoz@gmail.com>, Jason Gustafson <jason@confluent.io>
Closes#1994 from JakubDziworski/doc_fix
Author: Vahid Hashemian <vahidhashemian@us.ibm.com>
Reviewers: Jason Gustafson <jason@confluent.io>
Closes#1990 from vahidhashemian/doc/quickstart_update_windows
Test expects all records to be published successfully, which cannot be guaranteed with acks=0 since failures are not retried.
Author: Rajini Sivaram <rajinisivaram@googlemail.com>
Reviewers: Ben Stopford <benstopford@gmail.com>, Ismael Juma <ismael@juma.me.uk>
Closes#1991 from rajinisivaram/KAFKA-4265
Author: Jiangjie Qin <becket.qin@gmail.com>
Reviewers: Ismael Juma <ismael@juma.me.uk>, Jason Gustafson <jason@confluent.io>
Closes#1993 from becketqin/KAFKA-4274
Author: Matthias J. Sax <matthias@confluent.io>
Reviewers: Damian Guy <damian.guy@gmail.com>, Guozhang Wang <wangguoz@gmail.com>
Closes#1940 from mjsax/hotfix-resetTool
restoreConsumer.assign(..) in removeStandbyTasks was logging an (ignorable) exception due to the restoreConsumer being closed. Moved the restoreConsumer.assign(..) to shutdownTasksAndState as this is done prior to the closing of consumers.
Author: Damian Guy <damian.guy@gmail.com>
Reviewers: Guozhang Wang <wangguoz@gmail.com>
Closes#1986 from dguy/hotfix-assign
We recently switched `SystemTimer` to use `hiResClockMs` (based on `nanoTime`), but we
were still using `System.currentTimeMillis` in the test. That would sometimes mean
that we would measure elapsed time as lower than expected.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Rajini Sivaram <rajinisivaram@googlemail.com>, Jason Gustafson <jason@confluent.io>
Closes#1890 from ijuma/fix-test-request-satisfaction-transient-failure
To prevent test from completing without throttling before config change takes effect, produce more messages in the test.
Author: Rajini Sivaram <rajinisivaram@googlemail.com>
Reviewers: Ben Stopford <benstopford@gmail.com>, Ismael Juma <ismael@juma.me.uk>
Closes#1982 from rajinisivaram/KAFKA-4262
Changed the ordering in `StreamThread.shutdown`
1. commitAll (we need to commit so that any cached data is flushed through the topology)
2. close all tasks
3. producer.flush() - so any records produced during close are flushed and we have offsets for them
4. close all state managers
5. close producers/consumers
6. remove the tasks
Also in `onPartitionsRevoked`
1. commitAll
2. close all tasks
3. producer.flush
4. close all state managers
Author: Damian Guy <damian.guy@gmail.com>
Reviewers: Eno Thereska, Guozhang Wang
Closes#1970 from dguy/kafka-4253
This is a minor change to fix the most glaring issues. We have another JIRA to revamp the upgrade docs.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Jeff Klukas <jeff@klukas.net>, Jiangjie Qin <becket.qin@gmail.com>, Jason Gustafson <jason@confluent.io>
Closes#1971 from ijuma/kafka-4257-upgrade-docs-inconsitencies
custom ip resolver in test driver makes incorrect assumption when calling vm.communicate.execute, causing driver to fail launching with Vagrant 1.8.6, due to https://github.com/mitchellh/vagrant/pull/7676
Author: Xavier Léauté <xavier@confluent.io>
Reviewers: Ewen Cheslack-Postava <ewen@confluent.io>
Closes#1962 from xvrl/fix-vagrant-resolver
Grow read and write buffers of cleaner up to the maximum message size of the log being cleaned if the topic has larger max message size than the default config of the broker.
Author: Rajini Sivaram <rajinisivaram@googlemail.com>
Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>, Jiangjie Qin <becket.qin@gmail.com>, Jason Gustafson <jason@confluent.io>
Closes#1758 from rajinisivaram/KAFKA-4019
In doc it stays:
_"Our topic is divided into a set of totally ordered partitions, each of which is consumed by one consumer at any given time."_
And consumer is described as:
_"We'll call **processes** that subscribe to topics and process the feed of published messages **consumers**."_
Which might lead to a wrong conclusion - that each partition can be read by one process at any given time.
I think this statements misses information about **consumer groups**, so i propose:
_"Our topic is divided into a set of totally ordered partitions, each of which is consumed by exactly one consumer (from each subscribed consumer groups) at any given time"_
This contribution is my original work and I license the work to the project under the project's open source license.
Author: pilo <jakub.pilimon@4finance.com>
Reviewers: Jiangjie Qin <becket.qin@gmail.com>, Jason Gustafson <jason@confluent.io>
Closes#1900 from pilloPl/minor/doc-fix
the wrong word in this sentence " allows us to bring machies up in parallel on AWS." .
“machies” change to "machines".
Author: Yang Wei <yangvlive@gmail.com>
Reviewers: Jason Gustafson <jason@confluent.io>
Closes#1949 from yangwei71/trunk
pass through user-defined consumer and producer properties from StreamsConfig to ConsumerConfig and ProducerConfig.
For example, consumer interceptor support, i.e, consumer.interceptor.classes=SomeClass
Author: Damian Guy <damian.guy@gmail.com>
Reviewers: Guozhang Wang <wangguoz@gmail.com>
Closes#1943 from dguy/interceptor
Reopening of https://github.com/apache/kafka/pull/1428
Author: Edoardo Comar <ecomar@uk.ibm.com>
Author: Mickael Maison <mickael.maison@gmail.com>
Reviewers: Grant Henke <granthenke@gmail.com>, Ismael Juma <ismael@juma.me.uk>, Jason Gustafson <jason@confluent.io>
Closes#1908 from edoardocomar/KAFKA-3396
Author: Jason Gustafson <jason@confluent.io>
Reviewers: Vahid Hashemian <vahidhashemian@us.ibm.com>, Jiangjie Qin <becket.qin@gmail.com>, Ismael Juma <ismael@juma.me.uk>
Closes#1936 from hachikuji/KAFKA-3824
Temporarily disable the offset commit (when auto commit is enabled) in the new consumer's `unsubscribe()` method towards a workaround for the issue reported in [KAFKA-3491](https://issues.apache.org/jira/browse/KAFKA-3491).
For now, a call to `unsubscribe()` can be made to reset the offsets in case processing the batch received from the most recent `poll()` is interrupted (due to some exception).
Author: Vahid Hashemian <vahidhashemian@us.ibm.com>
Reviewers: Jason Gustafson <jason@confluent.io>
Closes#1944 from vahidhashemian/KAFKA-4234
Jason Gustafson
Konstantine Karantasis
Damian Guy
radai-rosenblatt
Eno Thereska
Vahid Hashemian
Andrea Cosentino
Joshi
Derrick Or
MayureshGharat
Gwen Shapira
Jakub Dziworski
Rajini Sivaram
Jiangjie Qin
Matthias J. Sax
Ismael Juma
Flavio Junqueira
Guozhang Wang
Xavier Léauté
pilo
Ben Stopford
Yang Wei
Edoardo Comar