Ewen Cheslack-Postava
75f2b8c8b4
KAFKA-2782: Fix KafkaBasedLogTest assertion and move it to the main test thread.
...
Author: Ewen Cheslack-Postava <me@ewencp.org>
Reviewers: Gwen Shapira
Closes #463 from ewencp/kafka-2782-fix-kafka-based-log-test-assertion
9 years ago
Ewen Cheslack-Postava
13ba57dcfc
KAFKA-2778: Use zero loss settings by default for Connect source producers.
...
Author: Ewen Cheslack-Postava <me@ewencp.org>
Reviewers: Gwen Shapira
Closes #459 from ewencp/kafka-2778-connect-source-zero-loss-settings
9 years ago
Ewen Cheslack-Postava
bc76e6704e
KAFKA-2775: Move exceptions into API package for Kafka Connect.
...
Author: Ewen Cheslack-Postava <me@ewencp.org>
Reviewers: Gwen Shapira
Closes #457 from ewencp/kafka-2775-exceptions-in-api-package
9 years ago
Jason Gustafson
2b04004de8
HOTFIX: bug updating cache when loading group metadata
...
The bug causes only the first instance of group metadata in the topic to be written to the cache (because of the putIfNotExists in addGroup). Coordinator fail-over won't work properly unless the cache is loaded with the right metadata.
Author: Jason Gustafson <jason@confluent.io>
Reviewers: Guozhang Wang
Closes #462 from hachikuji/hotfix-group-loading
9 years ago
Ewen Cheslack-Postava
e9fc7b8c84
KAFKA-2776: Fix lookup of schema conversion cache size in JsonConverter.
...
Author: Ewen Cheslack-Postava <me@ewencp.org>
Reviewers: Gwen Shapira
Closes #458 from ewencp/kafka-2776-json-converter-cache-config-fix
9 years ago
Ewen Cheslack-Postava
a24f9a23a6
KAFKA-2781; Only require signing artifacts when uploading archives.
...
Author: Ewen Cheslack-Postava <me@ewencp.org>
Reviewers: Jun Rao <junrao@gmail.com>
Closes #461 from ewencp/kafka-2781-no-signing-for-install
9 years ago
Rajini Sivaram
efbebc6e84
KAFKA-2779; Close SSL socket channel on remote connection close
...
Close socket channel in finally block to avoid file descriptor leak when remote end closes the connection
Author: Rajini Sivaram <rajinisivaram@googlemail.com>
Reviewers: Ismael Juma <ismael@juma.me.uk>, Jun Rao <junrao@gmail.com>
Closes #460 from rajinisivaram/KAFKA-2779
9 years ago
Ewen Cheslack-Postava
f2031d4063
KAFKA-2774: Rename Copycat to Kafka Connect
...
Author: Ewen Cheslack-Postava <me@ewencp.org>
Reviewers: Gwen Shapira
Closes #456 from ewencp/kafka-2774-rename-copycat
9 years ago
Ewen Cheslack-Postava
f4b87deefe
KAFKA-2480: Handle retriable and non-retriable exceptions thrown by sink tasks.
...
Author: Ewen Cheslack-Postava <me@ewencp.org>
Reviewers: Gwen Shapira
Closes #450 from ewencp/kafka-2480-unrecoverable-task-errors
9 years ago
Jason Gustafson
83fb734603
HOTFIX: fix group coordinator edge cases around metadata storage callback
...
Author: Jason Gustafson <jason@confluent.io>
Reviewers: Guozhang Wang
Closes #451 from hachikuji/hotfix-group-coordinator
9 years ago
Ashish Singh
34d9976656
KAFKA-2761: enable whitelist regex subscription for new consumer in ConsoleConsumer
...
#412 is a pre-req.
Author: Ashish Singh <asingh@cloudera.com>
Reviewers: Guozhang Wang
Closes #445 from SinghAsDev/KAFKA-2761
9 years ago
Jason Gustafson
23cc9c77b2
KAFKA-2723: new consumer exception cleanup
...
Author: Jason Gustafson <jason@confluent.io>
Reviewers: Guozhang Wang
Closes #441 from hachikuji/K2723
9 years ago
Gwen Shapira
b4e1bdf48b
KAFKA-2666: Docs: Automatically generate documentation from config classes
...
…the way we always planned to
Author: Gwen Shapira <cshapi@gmail.com>
Reviewers: Jun Rao, Guozhang Wang
Closes #382 from gwenshap/KAFKA-2666
9 years ago
Flavio Junqueira
30bf985348
KAFKA-2767; Upgrade ZkClient version to 0.7
...
Author: Flavio Junqueira <fpj@apache.org>
Reviewers: Jun Rao <junrao@gmail.com>
Closes #449 from fpj/KAFKA-2767
9 years ago
Guozhang Wang
15524d0970
KAFKA-2766: use new producer by default in tooling
...
Also update the API docs for new consumer.
Author: Guozhang Wang <wangguoz@gmail.com>
Reviewers: Gwen Shapira
Closes #448 from guozhangwang/K2766
9 years ago
Ewen Cheslack-Postava
d297b3af26
KAFKA-2765: Add versions to Copycat Connector and Task interfaces and log versions when instantiating connectors and tasks.
...
Author: Ewen Cheslack-Postava <me@ewencp.org>
Reviewers: Gwen Shapira
Closes #446 from ewencp/connector-versions
9 years ago
Rajini Sivaram
130a561ad5
MINOR: Remove unnecessary println
...
Looks like this println might have been left in there by mistake.
Author: Rajini Sivaram <rajinisivaram@googlemail.com>
Reviewers: Guozhang Wang
Closes #429 from rajinisivaram/latency-args-print
9 years ago
Chris Pinola
09ff686738
MINOR: Fix homophone typo in Design documentation
...
Noticed that there was a small typo in section 4.1 of the Design documentation on the [website](https://kafka.apache.org/documentation.html#majordesignelements ) ('new' vs. 'knew'). This patch corrects that.
Author: Chris Pinola <chris@pinola.co>
Reviewers: Guozhang Wang
Closes #391 from chrnola/minor/design-doc-typo
9 years ago
Ashish Singh
a9e4ce79fe
KAFKA-2734: kafka-console-consumer should allow empty topic for old consumer.
...
…ot specifying topic
Author: Ashish Singh <asingh@cloudera.com>
Reviewers: Sriharsha Chintalapani, Guozhang Wang
Closes #412 from SinghAsDev/KAFKA-2734
9 years ago
Ashish Singh
a423316d38
KAFKA-2760: Clean up interface of AdminClient.describeConsumerGroup.
...
…roupId)
Author: Ashish Singh <asingh@cloudera.com>
Reviewers: Jason Gustafson, Guozhang Wang
Closes #442 from SinghAsDev/KAFKA-2760
9 years ago
Ewen Cheslack-Postava
c006c5916e
KAFKA-2764: Change use of Properties in Copycat to Maps.
...
Author: Ewen Cheslack-Postava <me@ewencp.org>
Reviewers: Gwen Shapira, Guozhang Wang
Closes #444 from ewencp/kafka-2764-maps-not-properties
9 years ago
Ewen Cheslack-Postava
a76660ac81
KAFKA-2713: Run task start and stop methods in worker threads so they execute in parallel and cannot block the herder thread.
...
Author: Ewen Cheslack-Postava <me@ewencp.org>
Reviewers: Gwen Shapira
Closes #443 from ewencp/kafka-2713-task-start-stop-threaded
9 years ago
Jason Gustafson
b1eaa46a51
HOTFIX: unsubscribe does not clear user assignment properly
...
Author: Jason Gustafson <jason@confluent.io>
Reviewers: Gwen Shapira
Closes #439 from hachikuji/unsubscribe-hotfix
9 years ago
Ashish Singh
c3c0c04e62
KAFKA-2490: support new consumer in ConsumerGroupCommand
...
Author: Ashish Singh <asingh@cloudera.com>
Reviewers: Guozhang Wang, Jason Gustafson
Closes #299 from SinghAsDev/KAFKA-2490
9 years ago
Grant Henke
33e879a389
KAFKA-2702: ConfigDef toHtmlTable() sorts in a way that is a bit conf…
...
…using
Author: Grant Henke <granthenke@gmail.com>
Reviewers: Gwen Shapira
Closes #379 from granthenke/config-html
9 years ago
Guozhang Wang
76bcccd61f
KAFKA-2745: Update JavaDoc for new / updated consumer APIs
...
Author: Guozhang Wang <wangguoz@gmail.com>
Reviewers: Jeff Holoman, Jason Gustafson, Gwen Shapira
Closes #425 from guozhangwang/K2745
9 years ago
Matthew Bruce
cbdd8218c1
KAFKA-2756: Use request version Id instead of latest version Id to parse the corresponding response.
...
Author: Guozhang Wang <wangguoz@gmail.com>
Reviewers: Guozhang Wang
Closes #438 from guozhangwang/K2756
9 years ago
Guozhang Wang
0273c4379f
MINOR: follow-up KAFKA-2730 to use two tags for broker id and fetcher id combination
...
Author: Guozhang Wang <wangguoz@gmail.com>
Reviewers: Ismael Juma, Guozhang Wang
Closes #434 from guozhangwang/K2730-hotfix
9 years ago
Jeff Holoman
bf2563e2f7
KAFKA-2735: BrokerEndPoint should support uppercase hostnames.
...
Added support for uppercase hostnames in BrokerEndPoint. Added unit test
to cover this scenario.
Author: jholoman <jeff.holoman@gmail.com>
Reviewers: Grant Henke, Guozhang Wang
Closes #415 from jholoman/KAFKA-2735
9 years ago
Ashish Singh
490bbc9118
KAFKA-2755: Suspicious reference forwarding cause NPE on handleDescribeGroup.
...
…xistent consumer group
Author: Ashish Singh <asingh@cloudera.com>
Reviewers: Guozhang Wang
Closes #435 from SinghAsDev/KAFKA-2755
9 years ago
Jason Gustafson
7eee11451e
KAFKA-2753: improve SyncGroup error handling in client
...
Author: Jason Gustafson <jason@confluent.io>
Reviewers: Guozhang Wang
Closes #433 from hachikuji/KAFKA-2753
9 years ago
Ewen Cheslack-Postava
a4551773ca
KAFKA-2748: Ensure sink tasks commit offsets upon rebalance and rewind if the SinkTask flush fails.
...
Also fix the incorrect consumer group ID setting which was giving each task its
own group instead of one for the entire sink connector.
Author: Ewen Cheslack-Postava <me@ewencp.org>
Reviewers: Guozhang
Closes #431 from ewencp/kafka-2748-sink-task-rebalance-commit
9 years ago
Gwen Shapira
d23785ff2d
KAFKA-2738: Replica FetcherThread should connect to leader endpoint matching its inter-broker security protocol.
...
…atching its inter-broker security protocol
Author: Gwen Shapira <cshapi@gmail.com>
Reviewers: Jun Rao, Guozhang Wang
Closes #428 from gwenshap/KAFKA-2738
9 years ago
Ewen Cheslack-Postava
b30d68a4e3
KAFKA-2743: Make forwarded task reconfiguration requests asynchronous on backoff on retrying.
...
Author: Ewen Cheslack-Postava <me@ewencp.org>
Reviewers: Guozhang Wang
Closes #422 from ewencp/task-reconfiguration-async-with-backoff
9 years ago
Ewen Cheslack-Postava
7d6515fb8f
KAFKA-2741: Make SourceTaskContext and SinkTaskContext interfaces and keep implementations in runtime jar.
...
Author: Ewen Cheslack-Postava <me@ewencp.org>
Reviewers: Guozhang Wang
Closes #420 from ewencp/task-context-interfaces
9 years ago
Ewen Cheslack-Postava
4a9e7607b9
KAFKA-2742: Fix SourceTaskOffsetCommitter to handle removal of commit tasks when they are already in progress.
...
Author: Ewen Cheslack-Postava <me@ewencp.org>
Reviewers: Guozhang Wang
Closes #421 from ewencp/wait-on-in-progress-source-offset-commits
9 years ago
Jason Gustafson
68f42210a1
MINOR: add test case for fetching from a compacted topic
...
Author: Jason Gustafson <jason@confluent.io>
Reviewers: Guozhang Wang
Closes #426 from hachikuji/compacted-topics
9 years ago
Jason Gustafson
ef5d168cc8
KAFKA-2697: client-side support for leave group
...
Author: Jason Gustafson <jason@confluent.io>
Reviewers: Ewen Cheslack-Postava, Guozhang Wang
Closes #414 from hachikuji/KAFKA-2697
9 years ago
Jason Gustafson
8de62253ad
HOTFIX: Fix incorrect version used for group metadata version
...
Author: Jason Gustafson <jason@confluent.io>
Reviewers: Guozhang Wang
Closes #424 from hachikuji/hotfix-metadata-storage
9 years ago
Ewen Cheslack-Postava
70a784b64a
KAFKA-2744: Commit source task offsets after task is completely stopped
...
Author: Ewen Cheslack-Postava <me@ewencp.org>
Reviewers: Guozhang Wang
Closes #423 from ewencp/commit-source-offsets-after-work-thread-exits
9 years ago
Jason Gustafson
c39e79bb5a
KAFKA-2691: Improve handling of authorization failure during metadata refresh
...
Author: Jason Gustafson <jason@confluent.io>
Reviewers: Jun Rao
Closes #394 from hachikuji/KAFKA-2691
9 years ago
Guozhang Wang
c30ee50d82
KAFKA-2730: use thread-id as metrics tags
...
Author: Guozhang Wang <wangguoz@gmail.com>
Reviewers: Gwen Shapira
Closes #416 from guozhangwang/K2730
9 years ago
Anna Povzner
aa73554c19
KAFKA-2737: Added single- and multi-consumer integration tests for round-robin assignment
...
Two tests:
1. One consumer subscribes to 2 topics, each with 2 partitions; includes adding and removing a topic.
2. Several consumers subscribe to 2 topics, several partition each; includes adding one more consumer after initial assignment is done and verified.
Author: Anna Povzner <anna@confluent.io>
Reviewers: Guozhang Wang
Closes #413 from apovzner/cpkafka-76
9 years ago
Yasuhiro Matsuda
421de0a3f9
KAFKA-2727: Topology partial construction
...
guozhangwang
Author: Yasuhiro Matsuda <yasuhiro@confluent.io>
Reviewers: Guozhang Wang
Closes #411 from ymatsuda/topology_partial_construction
9 years ago
Michael G. Noll
7ded64bc2e
KAFKA-2740: Convert Windows bin scripts from CRLF to LF line encodings
...
There are no functional changes to the modified scripts.
Author: Michael G. Noll <michael@confluent.io>
Reviewers: Guozhang Wang
Closes #419 from miguno/KAFKA-2740
9 years ago
Jiangjie Qin
70a7d5786c
KAFKA-2722; Improve ISR change propagation.
...
The patch has two changes:
1. fixed a bug in controller that it sends UpdateMetadataRequest of all the partitions in the cluster.
2. Uses the following rules to propagate ISR change: 1) if there are ISR changes pending propagation and the last ISR change is more than five seconds ago, propagate the changes. 2) if there is ISR change at T in the recent five seconds, delay the propagation until T + 5s. 3) if the last propagation is more than 1 min ago, ignore rule No.2 and propagate ISR change if there are changes pending propagation.
This algorithm avoids a fixed configuration of ISR propagation interval as we discussed about in KIP-29.
Author: Jiangjie Qin <becket.qin@gmail.com>
Reviewers: Jun Rao <junrao@gmail.com>
Closes #402 from becketqin/KAFKA-2722
9 years ago
Rajini Sivaram
98db5ea94f
KAFKA-2644; Run relevant ducktape tests with SASL_PLAINTEXT and SASL_SSL
...
Run sanity check, replication tests and benchmarks with SASL/Kerberos using MiniKdc.
Author: Rajini Sivaram <rajinisivaram@googlemail.com>
Reviewers: Geoff Anderson <geoff@confluent.io>, Jun Rao <junrao@gmail.com>
Closes #358 from rajinisivaram/KAFKA-2644
9 years ago
Jason Gustafson
596c203af1
KAFKA-2687: Add support for ListGroups and DescribeGroup APIs
...
Author: Jason Gustafson <jason@confluent.io>
Reviewers: Guozhang Wang, Jun Rao
Closes #388 from hachikuji/K2687
9 years ago
Gwen Shapira
f413143edd
KAFKA-2441: SSL/TLS in official docs
...
Author: Gwen Shapira <cshapi@gmail.com>
Reviewers: Guozhang Wang
Closes #406 from gwenshap/KAFKA-2441
9 years ago
Liquan Pei
5aa5f19d38
KAFKA-2480: Add backoff timeout and support rewinds
...
Author: Liquan Pei <liquanpei@gmail.com>
Reviewers: Ewen Cheslack-Postava, Gwen Shapira
Closes #340 from Ishiihara/backoff
9 years ago