Removed the check for expiring only those batches whose metadata is unavailable. Now the batches will be expired irrespective of whether the leader is available or not, as soon as it reaches the requestimeout threshold.
Author: Mayuresh Gharat <mgharat@mgharat-ld1.linkedin.biz>
Reviewers: Jun Rao <junrao@gmail.com>
Closes#503 from MayureshGharat/kafka-2805
guozhangwang
When the rebalance happens each consumer reports the following information to the coordinator.
* Client UUID (a unique id assigned to an instance of KafkaStreaming)
* Task ids of previously running tasks
* Task ids of valid local states on the client's state directory
TaskAssignor does the following
* Assign a task to a client which was running it previously. If there is no such client, assign a task to a client which has its valid local state.
* Try to balance the load among stream threads.
* A client may have more than one stream threads. The assignor tries to assign tasks to a client proportionally to the number of threads.
Author: Yasuhiro Matsuda <yasuhiro@confluent.io>
Reviewers: Guozhang Wang
Closes#497 from ymatsuda/task_assignment
Process any remaining data in the network read buffer in `SslTransportLayer` when `read()` is invoked. On handshake completion, there could be application data ready to be processed that was read into `netReadBuffer` during handshake processing. `read()` is already invoked from `Selector` after handshake completion, but data already read into the `netReadBuffer` was not being processed. This PR adds a check for remaining data and continues with processing data if data is available.
Author: Rajini Sivaram <rajinisivaram@googlemail.com>
Reviewers: Jun Rao <junrao@gmail.com>
Closes#493 from rajinisivaram/KAFKA-2801
Author: Ewen Cheslack-Postava <me@ewencp.org>
Reviewers: Ben Stopford, Geoff Anderson, Guozhang Wang
Closes#432 from ewencp/kafka-2752-copycat-clean-bounce-test
-- Refactored multi-consumer integration group assignment validation tests for round-robin assignment
-- Added multi-consumer integration tests for session timeout expiration:
1. When a consumer stops polling
2. When a consumer calls close()
-- Fixes to issues found with session timeout expiration tests woth help from Jason Gustafson: Try to avoid SendFailedException exception by cancelling the scheduled tasks and ensuring metadata update before sending group leave requests + send leave group request with retries.
Author: Anna Povzner <anna@confluent.io>
Reviewers: Jason Gustafson, Guozhang Wang
Closes#472 from apovzner/cpkafka-81
This PR adds failover to simple end to end mirror maker test
Marked as WIP for 2 reasons:
- We may want to add a couple more test cases where kafka is being used to store offsets
- There appears to be a test failure in the hard failover case
Author: Geoff Anderson <geoff@confluent.io>
Reviewers: Ewen Cheslack-Postava
Closes#427 from granders/KAFKA-2258-mirrormaker-test
The bug causes only the first instance of group metadata in the topic to be written to the cache (because of the putIfNotExists in addGroup). Coordinator fail-over won't work properly unless the cache is loaded with the right metadata.
Author: Jason Gustafson <jason@confluent.io>
Reviewers: Guozhang Wang
Closes#462 from hachikuji/hotfix-group-loading
Close socket channel in finally block to avoid file descriptor leak when remote end closes the connection
Author: Rajini Sivaram <rajinisivaram@googlemail.com>
Reviewers: Ismael Juma <ismael@juma.me.uk>, Jun Rao <junrao@gmail.com>
Closes#460 from rajinisivaram/KAFKA-2779
Looks like this println might have been left in there by mistake.
Author: Rajini Sivaram <rajinisivaram@googlemail.com>
Reviewers: Guozhang Wang
Closes#429 from rajinisivaram/latency-args-print
Noticed that there was a small typo in section 4.1 of the Design documentation on the [website](https://kafka.apache.org/documentation.html#majordesignelements) ('new' vs. 'knew'). This patch corrects that.
Author: Chris Pinola <chris@pinola.co>
Reviewers: Guozhang Wang
Closes#391 from chrnola/minor/design-doc-typo
Mayuresh Gharat
Yasuhiro Matsuda
Ewen Cheslack-Postava
Gwen Shapira
Guozhang Wang
Rajini Sivaram
Grant Henke
Parth Brahmbhatt
Jason Gustafson
Geoff Anderson
Anna Povzner
Ashish Singh
Flavio Junqueira
Chris Pinola