Both the headers and requests have regressed to just show object ids instead of their contents from their underlying structs. I'm guessing this regression came from commit [fc1cfe475e8ae8458d8ddf119ce18d0c64653a70](fc1cfe475e)
Author: Onur Karaman <okaraman@linkedin.com>
Reviewers: Jiangjie Qin <becket.qin@gmail.com>
Closes#2678 from onurkaraman/KAFKA-4891
This commmit brings improved test coverage for window store fetch method
and WindowStoreIterator
Author: Andrey Dyachkov <andrey.dyachkov@zalando.de>
Reviewers: Damian Guy, Guozhang Wang
Closes#2672 from adyach/trunk
Author: Michael G. Noll <michael@confluent.io>
Reviewers: Dongjin Lee, Eno Thereska, Damian Guy, Colin P. McCabe, Matthas J. Sax, Guozhang Wang
Closes#2554 from miguno/KAFKA-4769
Author: Eno Thereska <eno@confluent.io>
Author: Eno Thereska <eno.thereska@gmail.com>
Reviewers: Damian Guy, Guozhang Wang
Closes#2663 from enothereska/minor-rocksdb-parallel
Remove generic type of class ClientState and generic T inTaskAssignor.
Author: sharad.develop <sharad.develop@gmail.com>
Reviewers: Matthias J. Sax, Guozhang Wang
Closes#2616 from sharad-develop/KAFKA-4738
The record should be created with CreateTime (like in the producer). The conversion to
LogAppendTime happens automatically (if necessary).
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Jason Gustafson <jason@confluent.io>
Closes#2657 from ijuma/kafka-4861-log-append-time-breaks-group-data-manager
ijuma ewencp cmccabe harshach Please review.
Here is a sample run:
https://travis-ci.org/raghavgautam/kafka/builds/191714520
In this run 214 tests were run and 144 tests passed.
I will open separate jiras for fixing failures.
Author: Raghav Kumar Gautam <raghav@apache.org>
Reviewers: Sriharsha Chintalapani <harsha@hortonworks.com>, Ewen Cheslack-Postava <ewen@confluent.io>
Closes#2376 from raghavgautam/trunk
It’s a minor optimisation, but simple enough.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Apurva Mehta <apurva.1618@gmail.com>, Jason Gustafson <jason@confluent.io>
Closes#2658 from ijuma/has-in-flight-requests
Remove unnecessary 'flush', the underlying stream should handle it on close.
Author: Will Droste <william.droste@arris.com>
Reviewers: Ismael Juma <ismael@juma.me.uk>
Closes#2528 from wdroste/trunk
Add application.id to StreamThread name
Author: sharad.develop <sharad.develop@gmail.com>
Reviewers: Matthias J. Sax, Guozhang Wang
Closes#2617 from sharad-develop/KAFKA-4722
Also fix a potential reordering bug and include a few clean-ups.
Author: Dong Lin <lindong28@gmail.com>
Reviewers: Jiangjie (Becket) Qin <becket.qin@gmail.com>, Jason Gustafson <jason@confluent.io>, Ismael Juma <ismael@juma.me.uk>
Closes#2641 from lindong28/KAFKA-4820-followup
restrict the locating of segments in `Segments#segments(..)` to only the segments that are currently available, i.e., rather than searching the hashmap for many segments that don't exist.
Author: Damian Guy <damian.guy@gmail.com>
Reviewers: Guozhang Wang <wangguoz@gmail.com>
Closes#2645 from dguy/session-windows-testing
Author: Jason Gustafson <jason@confluent.io>
Reviewers: Apurva Mehta <apurva.1618@gmail.com>, Ismael Juma <ismael@juma.me.uk>
Closes#2646 from hachikuji/rename-record-batch
`Long.MaxValue` for the linger overflows in `RecordBatch#maybeExpire` when added to
the current timestamp.
Then causes an error to be set for the batch by `Sender` (not happening every time since
it depends on the timing of `Sender`):
That error then causes a call to `ProduceRequestResult#done` on the batch, which then
makes the check for "not done" fail.
Author: Armin Braun <me@obrown.io>
Reviewers: Ismael Juma <ismael@juma.me.uk>
Closes#2639 from original-brownbear/KAFKA-3155
Increase the reliability of the one temporal comparison in ReassignPartitionsClusterTest by imposing a delay after ZK is updated. This should be more reliable than just increasing the amount of data.
This relates to a previous PR: https://github.com/apache/kafka/pull/1982
Author: Ben Stopford <benstopford@gmail.com>
Reviewers: Ismael Juma <ismael@juma.me.uk>
Closes#1997 from benstopford/KAFKA-4266
A couple of updates were missed in the [PR](https://github.com/apache/kafka/pull/2475) that replaced the use of error codes with Errors objects.
Author: Vahid Hashemian <vahidhashemian@us.ibm.com>
Reviewers: Ismael Juma <ismael@juma.me.uk>
Closes#2635 from vahidhashemian/minor/Errors_refactoring_leftover
This resolves the following issues in the ProcessorTopologyTestDriver:
- It should not create an internal changelog topic when using `through()` and `table()`
- It should forward the produced record back into the topology if it is to a source topic
Jira ticket: https://issues.apache.org/jira/browse/KAFKA-4828
The contribution is my original work and I license the work to the project under the project's open source license.
Author: Hamidreza Afzali <hrafzali@gmail.com>
Reviewers: Matthias J. Sax, Guozhang Wang
Closes#2629 from hrafzali/KAFKA-4828_ProcessorTopologyTestDriver_through
KAFKA-4567 - Connect Producer and Consumer ignore ssl parameters configured for worker
Added brief explanation to the docs about parameter inheritance of Kafka consumers and producers from the worker config.
Author: Sönke Liebau <soenke.liebau@opencore.com>
Reviewers: Gwen Shapira
Closes#2511 from soenkeliebau/KAFKA-4567
There were some minor differences in the basic consumer config and streams config that are now rectified. In addition, in AWS environments the socket size makes a big difference to performance and I've tuned it up accordingly. I've also increased the number of records now that perf is higher.
Author: Eno Thereska <eno@confluent.io>
Reviewers: Guozhang Wang <wangguoz@gmail.com>
Closes#2634 from enothereska/minor-standardize-params
Author: Dong Lin <lindong28@gmail.com>
Reviewers: Jason Gustafson <jason@confluent.io>, Ismael Juma <ismael@juma.me.uk>, Jiangjie Qin <becket.qin@gmail.com>
Closes#2619 from lindong28/KAFKA-4820
* Turned off Nagle on the sending sockets to force the socket to physically acknowledge after the first write in `sendRequest`
* Added a `200ms` delay between write attempts (necessary on Linux, but not Mac)
Author: Armin Braun <armin.braun@1und1.de>
Reviewers: Ismael Juma <ismael@juma.me.uk>
Closes#2632 from original-brownbear/KAFKA-3182
MINOR: Fix ResetIntegrationTest test failures
KAFKA-2857 follow-up.
Author: Vahid Hashemian <vahidhashemian@us.ibm.com>
Reviewers: Jason Gustafson <jason@confluent.io>, Ismael Juma <ismael@juma.me.uk>
Closes#2636 from vahidhashemian/minor/kafka-2857-followup
Fix tests/docker/Dockerfile to put the old Kafka distributions in the
correct spot for tests. Also, run_tests.sh should exit with an error
code if image rebuilding fails, rather than silently falling back to an
older image.
Author: Colin P. Mccabe <cmccabe@confluent.io>
Reviewers: Ewen Cheslack-Postava <ewen@confluent.io>
Closes#2613 from cmccabe/dockerfix
There won't be a 0.10.3.0.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Jason Gustafson <jason@confluent.io>
Closes#2628 from ijuma/bump-version-to-0.11.0.0-SNAPSHOT
This applies to new-consumer based groups and would avoid scenarios in which user issues a `--describe` query while the group is initializing.
Example: The following could occur for a newly created group.
```
kafkakafka:~/workspace/kafka$ bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 --describe --group g
Note: This will only show information about consumers that use the Java consumer API (non-ZooKeeper-based consumers).
Error: Executing consumer group command failed due to The group coordinator is not available.
```
With this PR the group is queried repeatedly at specific intervals within a preset (and configurable) timeout `group-init-timeout` to circumvent unfortunate situations like above.
Author: Vahid Hashemian <vahidhashemian@us.ibm.com>
Reviewers: Jason Gustafson <jason@confluent.io>
Closes#2538 from vahidhashemian/KAFKA-2857
If leader node of one more more partitions in a consumer subscription are temporarily unavailable, request metadata refresh so that partitions skipped for assignment dont have to wait for metadata expiry before reassignment. Metadata refresh is also requested if a subscribe topic or assigned partition doesn't exist.
Author: Rajini Sivaram <rajinisivaram@googlemail.com>
Reviewers: Vahid Hashemian <vahidhashemian@us.ibm.com>, Ismael Juma <ismael@juma.me.uk>, Jason Gustafson <jason@confluent.io>
Closes#2622 from rajinisivaram/KAFKA-4631
added \<pre> tags to not break javadoc display of the ASCII diagrams.
see broken ascii here:
https://kafka.apache.org/0102/javadoc/org/apache/kafka/streams/KafkaStreams.State.html
fix can be checked with gradle :streams:javadoc and then checking streams/build/docs/javadoc/org/apache/kafka/streams/KafkaStreams.State.html
I also fixed the diagram in StreamThread.java however currently no javadoc is generated for that one (since it's internal)
enothereska please have a look
Author: Clemens Valiente <clemens.valiente@trivago.com>
Reviewers: Eno Thereska, Matthias J. Sax, Guozhang Wang
Closes#2621 from cvaliente/KAFKA-4800-ASCII-diagrams
Have fetcherThreadMap keyed off brokerId + fetcherId instead of broker + fetcherId, but did not consider the case where port is changed.
Author: huxi <huxi@zhenrongbao.com>
Reviewers: Jun Rao <junrao@gmail.com>
Closes#2606 from amethystic/kafka4811_ReplicaFetchThread_fail_create
Avoid calling time.milliseconds more often than necessary. Cleaning and committing logic can use the timestamp at the start of the loop with minimal consequences. 5-10% improvements noticed with request rates of 450K records/second.
Also tidy up benchmark code a bit more.
Author: Eno Thereska <eno.thereska@gmail.com>
Author: Eno Thereska <eno@confluent.io>
Reviewers: Matthias J. Sax, Damian Guy, Guozhang Wang
Closes#2603 from enothereska/minor-reduce-milliseconds2
Detect when a rebalance has happened due to one or more existing nodes bouncing. Keep assignment of previous active tasks the same and only assign the tasks that were not active to the new clients.
Author: Damian Guy <damian.guy@gmail.com>
Reviewers: Guozhang Wang <wangguoz@gmail.com>
Closes#2609 from dguy/kstreams-575
Author: Damian Guy <damian.guy@gmail.com>
Reviewers: Guozhang Wang <wangguoz@gmail.com>, Ewen Cheslack-Postava <me@ewencp.org>, Jason Gustafson <jason@confluent.io>, Ismael Juma <ismael@juma.me.uk>
Closes#2594 from dguy/checkstyle
This resolves the issue in the ProcessorTopologyTestDriver that the extracted timestamp is not forwarded with the produced record to the internal topics.
JIRA ticket: https://issues.apache.org/jira/browse/KAFKA-4789
The contribution is my original work and I license the work to the project under the project's open source license.
guozhangwang dguy
Author: Hamidreza Afzali <hrafzali@gmail.com>
Reviewers: Damian Guy, Guozhang Wang
Closes#2590 from hrafzali/KAFKA-4789_ProcessorTopologyTestDriver_timestamp
Author: Michael G. Noll <michael@confluent.io>
Reviewers: Damian Guy, Matthias J. Sax, Guozhang Wang
Closes#2607 from miguno/trunk-flatMapValues-docstring
Author: Matthias J. Sax <matthias@confluent.io>
Reviewers: Jason Gustafson <jason@confluent.io>, Ewen Cheslack-Postava <ewen@confluent.io>
Closes#2303 from mjsax/licenseHeader
…0.x and not 0.8
Author: Colin P. Mccabe <cmccabe@confluent.io>
Reviewers: Ewen Cheslack-Postava <ewen@confluent.io>
Closes#2602 from cmccabe/KAFKA-4809
Onur Karaman
Andrey Dyachkov
Michael G. Noll
Aaron Coburn
Eno Thereska
sharad.develop
Ismael Juma
Raghav Kumar Gautam
simplesteph
Will Droste
Jason Gustafson
Dong Lin
Damian Guy
Armin Braun
Matthias J. Sax
Ben Stopford
bbejeck
Vahid Hashemian
Hamidreza Afzali
Sönke Liebau
Akhilesh Naidu
Colin P. Mccabe
Armin Braun
Rajini Sivaram
Clemens Valiente
Stevo Slavic
huxi
Eno Thereska