src-kafka

226 Tags

413 MiB

Author	SHA1	Message	Date
Apurva Mehta	bdf4cba047	KAFKA-4817; Add idempotent producer semantics This is from the KIP-98 proposal. The main points of discussion surround the correctness logic, particularly the Log class where incoming entries are validated and duplicates are dropped, and also the producer error handling to ensure that the semantics are sound from the users point of view. There is some subtlety in the idempotent producer semantics. This patch only guarantees idempotent production upto the point where an error has to be returned to the user. Once we hit a such a non-recoverable error, we can no longer guarantee message ordering nor idempotence without additional logic at the application level. In particular, if an application wants guaranteed message order without duplicates, then it needs to do the following in the error callback: 1. Close the producer so that no queued batches are sent. This is important for guaranteeing ordering. 2. Read the tail of the log to inspect the last message committed. This is important for avoiding duplicates. Author: Apurva Mehta <apurva@confluent.io> Author: hachikuji <jason@confluent.io> Author: Apurva Mehta <apurva.1618@gmail.com> Author: Guozhang Wang <wangguoz@gmail.com> Author: fpj <fpj@apache.org> Author: Jason Gustafson <jason@confluent.io> Reviewers: Jason Gustafson <jason@confluent.io>, Ismael Juma <ismael@juma.me.uk>, Jun Rao <junrao@gmail.com> Closes #2735 from apurvam/exactly-once-idempotent-producer	8 years ago
Colin P. Mccabe	d345d53e4e	KAFKA-4902; Utils#delete should correctly handle I/O errors and symlinks Author: Colin P. Mccabe <cmccabe@confluent.io> Reviewers: Jun Rao <junrao@gmail.com>, Apurva Mehta <apurva@confluent.io>, Ismael Juma <ismael@juma.me.uk> Closes #2691 from cmccabe/KAFKA-4902	8 years ago
Jason Gustafson	5bd06f1d54	KAFKA-4816; Message format changes for idempotent/transactional producer (KIP-98) Author: Jason Gustafson <jason@confluent.io> Reviewers: Jun Rao <junrao@gmail.com>, Apurva Mehta <apurva@confluent.io>, Guozhang Wang <wangguoz@gmail.com>, Ismael Juma <ismael@juma.me.uk> Closes #2614 from hachikuji/exactly-once-message-format	8 years ago
Colin P. Mccabe	d9b784e147	KAFKA-4796; Fix some findbugs warnings in Kafka Java client Author: Colin P. Mccabe <cmccabe@confluent.io> Reviewers: Ismael Juma <ismael@juma.me.uk> Closes #2593 from cmccabe/KAFKA-4796	8 years ago
Damian Guy	de05c9d3a0	MINOR: Add code quality checks (and suppressions) to checkstyle.xml Author: Damian Guy <damian.guy@gmail.com> Reviewers: Guozhang Wang <wangguoz@gmail.com>, Ewen Cheslack-Postava <me@ewencp.org>, Jason Gustafson <jason@confluent.io>, Ismael Juma <ismael@juma.me.uk> Closes #2594 from dguy/checkstyle	8 years ago

5 Commits (b611cfa5c0f4941a491781424afd9b699bdb894e)