src-kafka

Commit Graph

Author	SHA1	Message	Date
Ismael Juma	39eb31feae	MINOR: Optimise performance of `Topic.validate()` I included a JMH benchmark and the results follow. The implementation in this PR takes no more than 1/10th of the time when compared to trunk. I also included results for an alternative implementation that is a little slower than the one in the PR. Trunk: ```text TopicBenchmark.testValidate topic avgt 15 134.107 ± 3.956 ns/op TopicBenchmark.testValidate longer-topic-name avgt 15 316.241 ± 13.379 ns/op TopicBenchmark.testValidate very-long-topic-name_with_more_text avgt 15 636.026 ± 30.272 ns/op ``` Implementation in the PR: ```text TopicBenchmark.testValidate topic avgt 15 13.153 ± 0.383 ns/op TopicBenchmark.testValidate longer-topic-name avgt 15 26.139 ± 0.896 ns/op TopicBenchmark.testValidate very-long-topic-name.with_more_text avgt 15 44.829 ± 1.390 ns/op ``` Alternative implementation where boolean validChar = Character.isLetterOrDigit(c) \|\| c == '.' \|\| c == '_' \|\| c == '-'; ```text TopicBenchmark.testValidate topic avgt 15 18.883 ± 1.044 ns/op TopicBenchmark.testValidate longer-topic-name avgt 15 36.696 ± 1.220 ns/op TopicBenchmark.testValidate very-long-topic-name_with_more_text avgt 15 65.956 ± 0.669 ns/op ``` Author: Ismael Juma <ismael@juma.me.uk> Reviewers: Guozhang Wang <wangguoz@gmail.com> Closes #3234 from ijuma/optimise-topic-is-valid	8 years ago
Mario Molina	dc5bf4bd45	KAFKA-5218; New Short serializer, deserializer, serde Author: Mario Molina <mmolimar@gmail.com> Reviewers: Matthias J. Sax <matthias@confluent.io>, Damian Guy <damian.guy@gmail.com>, Michael G. Noll <michael@confluent.io>, Guozhang Wang <wangguoz@gmail.com> Closes #3017 from mmolimar/KAFKA-5218	8 years ago
Rajini Sivaram	73ca0d215e	KAFKA-5320: Include all request throttling in client throttle metrics Author: Rajini Sivaram <rajinisivaram@googlemail.com> Reviewers: Jun Rao <junrao@gmail.com>, Ismael Juma <ismael@juma.me.uk> Closes #3137 from rajinisivaram/KAFKA-5320	8 years ago
Konstantine Karantasis	45f2261763	KAFKA-3487: Support classloading isolation in Connect (KIP-146) Author: Konstantine Karantasis <konstantine@confluent.io> Reviewers: Randall Hauch <rhauch@gmail.com>, Ewen Cheslack-Postava <ewen@confluent.io> Closes #3028 from kkonstantine/KAFKA-3487-Support-classloading-isolation-in-Connect	8 years ago
Ismael Juma	972b754536	KAFKA-3267; Describe and Alter Configs Admin APIs (KIP-133) Author: Ismael Juma <ismael@juma.me.uk> Reviewers: Jun Rao <junrao@gmail.com> Closes #3076 from ijuma/kafka-3267-describe-alter-configs-protocol	8 years ago
Colin P. Mccabe	9815e18fef	KAFKA-3266; Describe, Create and Delete ACLs Admin APIs (KIP-140) Includes server-side code, protocol and AdminClient. Author: Colin P. Mccabe <cmccabe@confluent.io> Reviewers: Ismael Juma <ismael@juma.me.uk> Closes #2941 from cmccabe/KAFKA-3266	8 years ago
Rajini Sivaram	4c75f31a5f	KAFKA-5179; Log connection termination during authentication Author: Rajini Sivaram <rajinisivaram@googlemail.com> Reviewers: Ismael Juma, Jun Rao Closes #2980 from rajinisivaram/KAFKA-5179	8 years ago
Colin P. Mccabe	4aed28d189	KAFKA-3265; Add a public AdminClient API in Java (KIP-117) Author: Colin P. Mccabe <cmccabe@confluent.io> Reviewers: Dan Norwood <norwood@confluent.io>, Ismael Juma <ismael@juma.me.uk> Closes #2472 from cmccabe/KAFKA-3265	8 years ago
Apurva Mehta	a82f194b21	KAFKA-4818; Exactly once transactional clients Author: Apurva Mehta <apurva@confluent.io> Reviewers: Guozhang Wang <wangguoz@gmail.com>, Ismael Juma <ismael@juma.me.uk>, Jason Gustafson <jason@confluent.io> Closes #2840 from apurvam/exactly-once-transactional-clients	8 years ago
Matthias J. Sax	148f8c2545	KAFKA-4986; Producer per StreamTask support (KIP-129) Enable producer per task if exactly-once config is enabled. Author: Matthias J. Sax <matthias@confluent.io> Reviewers: Eno Thereska <eno@confluent.io>, Damian Guy <damian.guy@gmail.com>, Ismael Juma <ismael@juma.me.uk> Closes #2773 from mjsax/exactly-once-streams-producer-per-task	8 years ago
Matthias J. Sax	865d82af2c	KAFKA-4990; Request/response classes for transactions (KIP-98) Author: Matthias J. Sax <matthias@confluent.io> Author: Guozhang Wang <wangguoz@gmail.com> Author: Jason Gustafson <jason@confluent.io> Reviewers: Apurva Mehta <apurva@confluent.io>, Jun Rao <junrao@gmail.com>, Ismael Juma <ismael@juma.me.uk> Closes #2799 from mjsax/kafka-4990-add-api-stub-config-parameters-request-types	8 years ago
Ben Stopford	0baea2ac13	KIP-101: Alter Replication Protocol to use Leader Epoch rather than High Watermark for Truncation This PR replaces https://github.com/apache/kafka/pull/2743 (just raising from Confluent repo) This PR describes the addition of Partition Level Leader Epochs to messages in Kafka as a mechanism for fixing some known issues in the replication protocol. Full details can be found here: [KIP-101 Reference](https://cwiki.apache.org/confluence/display/KAFKA/KIP-101+-+Alter+Replication+Protocol+to+use+Leader+Epoch+rather+than+High+Watermark+for+Truncation) The key elements are: - Epochs are stamped on messages as they enter the leader. - Epochs are tracked in both leader and follower in a new checkpoint file. - A new API allows followers to retrieve the leader's latest offset for a particular epoch. - The logic for truncating the log, when a replica becomes a follower, has been moved from Partition into the ReplicaFetcherThread - When partitions are added to the ReplicaFetcherThread they are added in an initialising state. Initialising partitions request leader epochs and then truncate their logs appropriately. This test provides a good overview of the workflow `EpochDrivenReplicationProtocolAcceptanceTest.shouldFollowLeaderEpochBasicWorkflow()` The corrupted log use case is covered by the test `EpochDrivenReplicationProtocolAcceptanceTest.offsetsShouldNotGoBackwards()` Remaining work: There is a do list here: https://docs.google.com/document/d/1edmMo70MfHEZH9x38OQfTWsHr7UGTvg-NOxeFhOeRew/edit?usp=sharing Author: Ben Stopford <benstopford@gmail.com> Author: Jun Rao <junrao@gmail.com> Reviewers: Ismael Juma <ismael@juma.me.uk>, Jun Rao <junrao@gmail.com> Closes #2808 from benstopford/kip-101-v2	8 years ago
Apurva Mehta	bdf4cba047	KAFKA-4817; Add idempotent producer semantics This is from the KIP-98 proposal. The main points of discussion surround the correctness logic, particularly the Log class where incoming entries are validated and duplicates are dropped, and also the producer error handling to ensure that the semantics are sound from the users point of view. There is some subtlety in the idempotent producer semantics. This patch only guarantees idempotent production upto the point where an error has to be returned to the user. Once we hit a such a non-recoverable error, we can no longer guarantee message ordering nor idempotence without additional logic at the application level. In particular, if an application wants guaranteed message order without duplicates, then it needs to do the following in the error callback: 1. Close the producer so that no queued batches are sent. This is important for guaranteeing ordering. 2. Read the tail of the log to inspect the last message committed. This is important for avoiding duplicates. Author: Apurva Mehta <apurva@confluent.io> Author: hachikuji <jason@confluent.io> Author: Apurva Mehta <apurva.1618@gmail.com> Author: Guozhang Wang <wangguoz@gmail.com> Author: fpj <fpj@apache.org> Author: Jason Gustafson <jason@confluent.io> Reviewers: Jason Gustafson <jason@confluent.io>, Ismael Juma <ismael@juma.me.uk>, Jun Rao <junrao@gmail.com> Closes #2735 from apurvam/exactly-once-idempotent-producer	8 years ago
Colin P. Mccabe	d345d53e4e	KAFKA-4902; Utils#delete should correctly handle I/O errors and symlinks Author: Colin P. Mccabe <cmccabe@confluent.io> Reviewers: Jun Rao <junrao@gmail.com>, Apurva Mehta <apurva@confluent.io>, Ismael Juma <ismael@juma.me.uk> Closes #2691 from cmccabe/KAFKA-4902	8 years ago
Jason Gustafson	5bd06f1d54	KAFKA-4816; Message format changes for idempotent/transactional producer (KIP-98) Author: Jason Gustafson <jason@confluent.io> Reviewers: Jun Rao <junrao@gmail.com>, Apurva Mehta <apurva@confluent.io>, Guozhang Wang <wangguoz@gmail.com>, Ismael Juma <ismael@juma.me.uk> Closes #2614 from hachikuji/exactly-once-message-format	8 years ago
Colin P. Mccabe	d9b784e147	KAFKA-4796; Fix some findbugs warnings in Kafka Java client Author: Colin P. Mccabe <cmccabe@confluent.io> Reviewers: Ismael Juma <ismael@juma.me.uk> Closes #2593 from cmccabe/KAFKA-4796	8 years ago
Damian Guy	de05c9d3a0	MINOR: Add code quality checks (and suppressions) to checkstyle.xml Author: Damian Guy <damian.guy@gmail.com> Reviewers: Guozhang Wang <wangguoz@gmail.com>, Ewen Cheslack-Postava <me@ewencp.org>, Jason Gustafson <jason@confluent.io>, Ismael Juma <ismael@juma.me.uk> Closes #2594 from dguy/checkstyle	8 years ago

17 Commits (4b4102884c8d317fe362523a9f5af20a27a4ee52)