Added commitRecord(SourceRecord record) to SourceTask. This method is called during the callback from producer.send() when the message has been sent successfully. Added commitTaskRecord(SourceRecord record) to WorkerSourceTask to handle calling commitRecord on the SourceTask. Updated tests for calls to commitRecord.
Author: Jeremy Custenborder <jcustenborder@gmail.com>
Reviewers: Jason Gustafson <jason@confluent.io>, Ewen Cheslack-Postava <ewen@confluent.io>
Closes#950 from jcustenborder/KAFKA-3260
* Fix and suppress number of unchecked warnings (except for Kafka Streams)
* Add `SafeVarargs` annotation to fix warnings
* Suppress unfixable deprecation warnings
* Replace deprecated by non-deprecated usage where possible
* Avoid reflective calls via structural types in Scala
* Tweak compiler settings for scalac and javac
Once we drop Java 7 and Scala 2.10, we can tweak the compiler settings further so that they warn us about more things.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Grant Henke, Gwen Shapira, Guozhang Wang
Closes#1042 from ijuma/kafka-3375-suppress-depreccated-tweak-compiler
Added offsetBackingStore config to StandaloneConfig and DistributedConfig;
Added config for offset.storage.topic and config.storage.topic into DistributedConfig;
Author: jinxing <jinxing@fenbi.com>
Reviewers: Jason Gustafson <jason@confluent.io>, Ewen Cheslack-Postava <ewen@confluent.io>
Closes#734 from ZoneMayor/trunk-KAFKA-2934
…ssage to assist with figuring this out.
Author: Gwen Shapira <cshapi@gmail.com>
Reviewers: Ewen Cheslack-Postava <ewen@confluent.io>
Closes#993 from gwenshap/KAFKA-2944
See KIP-31 and KIP-32 for details.
A few notes on the patch:
1. This patch implements KIP-31 and KIP-32. The patch includes features in both KAFKA-3025, KAFKA-3026 and KAFKA-3036
2. All unit tests passed.
3. The unit tests were run with new and old message format.
4. When message format conversion occurs during consumption, the consumer will not be able to detect the message size too large situation. I did not try to fix this because the situation seems rare and only happen during migration phase.
Author: Jiangjie Qin <becket.qin@gmail.com>
Author: Ismael Juma <ismael@juma.me.uk>
Author: Jiangjie (Becket) Qin <becket.qin@gmail.com>
Reviewers: Jason Gustafson <jason@confluent.io>, Anna Povzner <anna@confluent.io>, Ismael Juma <ismael@juma.me.uk>, Guozhang Wang <wangguoz@gmail.com>, Jun Rao <junrao@gmail.com>
Closes#764 from becketqin/KAFKA-3025
1. Added a test case to prove commit() on SourceTask was not being called.
2. Added commitSourceTask() which logs potential exceptions.
3. Added after call to finishSuccessfulFlush().
Author: Jeremy Custenborder <jeremy@scarcemedia.com>
Reviewers: Ewen Cheslack-Postava <ewen@confluent.io>
Closes#909 from jcustenborder/KAFKA-3225
This fixes failure of the FileStreamSinkTaskTest unit test on Windows
Signed-off-by: glikson <gliksonil.ibm.com>
Author: glikson <glikson@il.ibm.com>
Reviewers: Guozhang Wang <wangguoz@gmail.com>, Ewen Cheslack-Postava <ewen@confluent.io>
Closes#708 from glikson/file-ut-newline
Summary of code changes
------------------------------------
1) Added a new Checkstyle rule to flag any code using star imports
2) Fixed ALL existing code violations using star imports
Testing
-----------
Local build was successful
ALL JUnits ran successfully on local.
ewencp - Request you to please review changes. Thank you !
I state that the contribution is my original work and I license the work to the project under the project's open source license.
Author: manasvigupta <manasvigupta@yahoo.co.in>
Reviewers: Ewen Cheslack-Postava <ewen@confluent.io>
Closes#700 from manasvigupta/KAFKA-3009
We always return a `Struct` from `Schema.read` and this means that
we can remove a large number of casts.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Ewen Cheslack-Postava <ewen@confluent.io>
Closes#684 from ijuma/schema-read-should-return-struct
A few issues found via static analysis.
Author: Edward Ribeiro <edward.ribeiro@gmail.com>
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Gwen Shapira, Sriharsha Chintalapani, Guozhang Wang
Closes#652 from ijuma/use-equals-instead-of-==
onurkaraman becketqin Do you have time to review this patch? It addresses the ticket that jjkoshy filed in KAFKA-2668.
Author: Dong Lin <lindong28@gmail.com>
Reviewers: Onur Karaman <okaraman@linkedin.com>, Joel Koshy <jjkoshy@gmail.com>, Guozhang Wang <wangguoz@gmail.com>, Jun Rao <junrao@gmail.com>
Closes#328 from lindong28/KAFKA-2668
Gradle does not handle subprojects with the same name (top-level tools vs
connect/tools) properly, making the dependency impossible to express correctly
since we need to move the ThroughputThrottler class into the top level tools
project. Moving the current set of tools into the runtime jar works fine since
they are only used for system tests at the moment.
Author: Ewen Cheslack-Postava <me@ewencp.org>
Reviewers: Gwen Shapira
Closes#512 from ewencp/kafka-2807-redux
Author: Ewen Cheslack-Postava <me@ewencp.org>
Reviewers: Ben Stopford, Geoff Anderson, Guozhang Wang
Closes#432 from ewencp/kafka-2752-copycat-clean-bounce-test
Ewen Cheslack-Postava
Pierre-Yves Ritschard
Liquan Pei
Jeremy Custenborder
Ismael Juma
Gwen Shapira
Jason Gustafson
jinxing
Anna Povzner
Jiangjie Qin
Jeremy Custenborder
Kishore Senji
glikson
manasvigupta
Vahid Hashemian
Edward Ribeiro
Dong Lin
Grant Henke