src-kafka

Commit Graph

Author	SHA1	Message	Date
Guozhang Wang	b6c29e3810	MINOR: Add InterfaceStability.Unstable annotations to some Kafka Streams public APIs Also improves Java docs for the Streams high-level DSL. Author: Guozhang Wang <wangguoz@gmail.com> Reviewers: Ismael Juma, Michael G. Noll Closes #1097 from guozhangwang/KNewJavaDoc	9 years ago
Gwen Shapira	702d560c55	Changing version to 0.10.1.0-SNAPSHOT	9 years ago
Grant Henke	4f048c4f19	MINOR: Fix FetchRequest.getErrorResponse for version 1 Author: Grant Henke <granthenke@gmail.com> Reviewers: Ismael Juma <ismael@juma.me.uk>, Jun Rao <junrao@gmail.com> Closes #1091 from granthenke/fetch-error	9 years ago
Ismael Juma	95eabc8c8b	KAFKA-3378; Follow-up to ensure we `finishConnect` for immediately connected keys Author: Ismael Juma <ismael@juma.me.uk> Reviewers: Larkin Lowrey <llowrey@gmail.com>, Jun Rao <junrao@gmail.com> Closes #1103 from ijuma/kafka-3378-follow-up	9 years ago
Geoff Anderson	c188a68e2b	MINOR: Add vagrant up wrapper for simple parallel bringup on aws The main impediment to bringing up aws machines in parallel using vagrant was the interaction between `vagrant-hostmanager` and `vagrant-aws`. If you disable hostmanager during the `up` phase, and run it after the cluster is up, parallel bringup is possible. The only caveat is that machines must be brought up in small-ish batches to prevent rate limit errors from AWS since `vagrant-aws` doesn't seem to have mechanisms to This PR: - disables `vagrant-hostmanager` during bringup - adds a wrapper script to make it convenient to bring machines up in batches on aws Author: Geoff Anderson <geoff@confluent.io> Reviewers: Ewen Cheslack-Postava <ewen@confluent.io> Closes #982 from granders/vagrant-disable-hostmanager	9 years ago
Grant Henke	bfac36ad0e	KAFKA-3328: SimpleAclAuthorizer can lose ACLs with frequent add/remov… …e calls Changes the SimpleAclAuthorizer to: - Track and utilize the zookeeper version when updating zookeeper to prevent data loss in the case of stale reads and race conditions - Update local cache when modifying ACLs - Add debug logging Author: Grant Henke <granthenke@gmail.com> Author: Grant Henke <granthenke@users.noreply.github.com> Author: Ismael Juma <ismael@juma.me.uk> Reviewers: Flavio Junqueira, Jun Rao, Ismael Juma, Gwen Shapira Closes #1006 from granthenke/simple-authorizer-fix	9 years ago
Ewen Cheslack-Postava	eb823281a5	KAFKA-3424: Add CORS support to Connect REST API Author: Ewen Cheslack-Postava <me@ewencp.org> Reviewers: Gwen Shapira Closes #1099 from ewencp/cors-rest-support	9 years ago
Jun Rao	6553679718	KAFKA-3427: broker can return incorrect version of fetch response when the broker hits an unknown exception Author: Jun Rao <junrao@gmail.com> Reviewers: Ismael Juma, Becket Qin Closes #1101 from junrao/kafka-3427	9 years ago
Ismael Juma	b6c6291e1f	KAFKA-3378; Client blocks forever if SocketChannel connects instantly This is a different implementation to the one in #1085 by Larkin Lowrey (llowrey). The hard part here was actually finding the problem and all credit goes to llowrey. This PR also fixes our handling of `finishConnect` (we now check the return value). Author: Ismael Juma <ismael@juma.me.uk> Reviewers: Jun Rao Closes #1094 from ijuma/KAFKA-3378-instantly-connecting-socket-channels	9 years ago
Pierre-Yves Ritschard	4332175c11	KAFKA-3006: standardize KafkaConsumer API to use Collection Author: Pierre-Yves Ritschard <pyr@spootnik.org> Reviewers: Jason Gustafson, Gwen Shapira Closes #1098 from hachikuji/KAFKA-3006	9 years ago
Xin Wang	07360cb0d2	KAFKA-3397: use -1(latest) as time default value for tools.GetOffsetShell minor fix for prompt the user, or we will get a error message: >Missing required argument "[time]" Author: Xin Wang <best.wangxin@163.com> Reviewers: Ashish Singh, Guozhang Wang Closes #1068 from vesense/patch-4	9 years ago
Jason Gustafson	d9bf55171b	KAFKA-3394; allow null offset metadata in commit API Author: Jason Gustafson <jason@confluent.io> Reviewers: Ismael Juma <ismael@juma.me.uk>, Grant Henke <granthenke@gmail.com>, Ewen Cheslack-Postava <me@ewencp.org>, Jun Rao <junrao@gmail.com> Closes #1064 from hachikuji/KAFKA-3394	9 years ago
Guozhang Wang	5d0cd7667f	KAFKA-3422: Add overloading functions without serdes in Streams DSL Also include: 1) remove streams specific configs before passing to producer and consumer to avoid warning message; 2) add `ConsumerRecord` timestamp extractor and set as the default extractor. Author: Guozhang Wang <wangguoz@gmail.com> Reviewers: Michael G. Noll, Ewen Cheslack-Postava Closes #1093 from guozhangwang/KConfigWarn	9 years ago
Eno Thereska	0d8cbbcb20	HOTFIX: Renamed tests to match expected suffix ewencp gwenshap granders could you have a look please? Thanks. Author: Eno Thereska <eno.thereska@gmail.com> Reviewers: Ewen Cheslack-Postava <ewen@confuent.io> Closes #1096 from enothereska/systest-hotfix-name	9 years ago
Ismael Juma	e89a9ce1a4	KAFKA-2982; Mark the old Scala producer and related classes as deprecated Also update server tests to always use new producer. Author: Ismael Juma <ismael@juma.me.uk> Reviewers: Gwen Shapira Closes #1092 from ijuma/kafka-2982-deprecate-old-producers	9 years ago
Guozhang Wang	dea0719e99	KAFKA-3336: Unify Serializer and Deserializer into Serialization Author: Guozhang Wang <wangguoz@gmail.com> Reviewers: Michael G. Noll, Ismael Juma Closes #1066 from guozhangwang/K3336	9 years ago
Eno Thereska	f57dabbe56	KAFKA-3202: System test that changes message version on the fly becketqin apovzner please have a look. becketqin the test fails when the producer and consumer are 0.9.x and the message format changes on the fly. Author: Eno Thereska <eno.thereska@gmail.com> Reviewers: Ewen Cheslack-Postava, Ismael Juma, Gwen Shapira Closes #1070 from enothereska/kafka-3202-format-change-fly	9 years ago
Dong Lin	579d473ce9	KAFKA-3330; Truncate log cleaner offset checkpoint if the log is truncated becketqin Can you take a look? Author: Dong Lin <lindong28@gmail.com> Reviewers: Ismael Juma <ismael@juma.me.uk>, Jun Rao <junrao@gmail.com> Closes #1009 from lindong28/KAFKA-3330	9 years ago
Grant Henke	61281f5c53	KAFKA-3250: release tarball is unnecessarily large due to duplicate l… …ibraries This ensures duplicates are not copied in the distribution without rewriting all of the tar'ing logic. A larger improvement could be made to the packaging code, but that should be tracked by another jira. Author: Grant Henke <granthenke@gmail.com> Reviewers: Gwen Shapira, Ismael Juma Closes #1075 from granthenke/libs-duplicates	9 years ago
Ismael Juma	3f28d77449	KAFKA-3373; MINOR: follow-up, a few val renames remaining I also slightly tweaked the wording on a couple of warnings. Author: Ismael Juma <ismael@juma.me.uk> Reviewers: Gwen Shapira Closes #1072 from ijuma/kafka-3373-follow-up	9 years ago
Ismael Juma	8ef804dc19	KAFKA-3414; Return of MetadataCache.getAliveBrokers should not be mutated by cache updates `Map.values` returns `DefaultValuesIterable` where the default implementation of `toSeq` is (sadly) `toStream`. `Stream` is a lazy collection and it can reflect changes to the underlying map before it's `forced`. I verified that the test failed before my change. Author: Ismael Juma <ismael@juma.me.uk> Reviewers: Gwen Shapira Closes #1088 from ijuma/kafka-3414-get-alive-brokers-no-mutation	9 years ago
Liquan Pei	c07d017227	KAFKA-3315: Add REST and Connector API to expose connector configuration Author: Liquan Pei <liquanpei@gmail.com> Reviewers: Ewen Cheslack-Postava <ewen@confluent.io> Closes #964 from Ishiihara/expose-connector-config	9 years ago
Eno Thereska	a1eb12d7c6	KAFKA-3188: Compatibility test for old and new clients with 0.10 broker apovzner becketqin please have a look if you can. Thanks. Author: Eno Thereska <eno.thereska@gmail.com> Reviewers: Anna Povzner, Gwen Shapira Closes #1059 from enothereska/kafka-3188-compatibility	9 years ago
edoardo	30e78fa006	KAFKA-2832: Add a consumer config option to exclude internal topics A new consumer config option 'exclude.internal.topics' was added to allow excluding internal topics when wildcards are used to specify consumers. The new option takes a boolean value, with a default 'false' value (i.e. no exclusion). This patch is co-authored with rajinisivaram edoardocomar mimaison Author: edoardo <ecomar@uk.ibm.com> Author: Vahid Hashemian <vahidhashemian@us.ibm.com> Reviewers: Ismael Juma, Jun Rao, Gwen Shapira Closes #1082 from edoardocomar/KAFKA-2832	9 years ago
Drausin Wulsin	dce06766da	KAFKA-3392: ConsumerRecords iterator throws NoSuchElementException when a TopicPartition is empty This contribution is my original work, and I license it under the project's open source license. CC jkreps Author: Drausin Wulsin <daedalus2718@gmail.com> Author: John Doe <daedalus2718@gmail.com> Reviewers: Jason Gustafson Closes #1055 from drausin/bugfix/consumer-records-iterator	9 years ago
Michael G. Noll	958e10c87c	KAFKA-3411: Streams: stop using "job" terminology, rename job.id to application.id guozhangwang ymatsuda : please review. Author: Michael G. Noll <michael@confluent.io> Reviewers: Guozhang Wang <wangguoz@gmail.com> Closes #1081 from miguno/KAFKA-3411	9 years ago
Anna Povzner	9a836d0154	KAFKA-3303; Pass partial record metadata to ProducerInterceptor.onAcknowledgement on error This is a KIP-42 followup. Currently, If sending the record fails before it gets to the server, ProducerInterceptor.onAcknowledgement() is called with metadata == null, and non-null exception. However, it is useful to pass topic and partition, if known, to ProducerInterceptor.onAcknowledgement() as well. This patch ensures that ProducerInterceptor.onAcknowledgement() gets record metadata with topic and maybe partition. If partition is not set in 'record' and KafkaProducer.send() fails before partition gets assigned, then ProducerInterceptor.onAcknowledgement() gets RecordMetadata with partition == -1. Only time when ProducerInterceptor.onAcknowledgement() gets null record metadata is when the client passes null record to KafkaProducer.send(). Author: Anna Povzner <anna@confluent.io> Reviewers: Ismael Juma <ismael@juma.me.uk>, Ashish Singh <asingh@cloudera.com>, Jun Rao <junrao@gmail.com> Closes #1015 from apovzner/kip42-3	9 years ago
Ismael Juma	858047a12b	KAFKA-3402; Restore behaviour of MetadataCache.getTopicMetadata when unsupported security protocol is received Author: Ismael Juma <ismael@juma.me.uk> Reviewers: Jason Gustafson, Grant Henke Closes #1073 from ijuma/kafka-3402-restore-get-topic-metadata-behaviour	9 years ago
Grant Henke	cd8bd606cb	KAFKA-3403: Upgrade ZkClient to 0.8 This ZkClient version adds authentication validation and a conditional delete method needed for other patches Author: Grant Henke <granthenke@gmail.com> Reviewers: Ismael Juma, Gwen Shapira Closes #1084 from granthenke/zkclient-08	9 years ago
Eno Thereska	b0f811804f	KAFKA-3371: ClientCompatibilityTest system test failing becketqin have a look if this looks reasonable to you. Thanks. Author: Eno Thereska <eno.thereska@gmail.com> Reviewers: Geoff Anderson <geoff@confluent.io>, Ewen Cheslack-Postava <ewen@confluent.io> Closes #1051 from enothereska/kafka-3371	9 years ago
Ewen Cheslack-Postava	6834b91e34	MINOR: KAFKA-3260 follow up, fix commitRecord calls in tests Author: Ewen Cheslack-Postava <me@ewencp.org> Reviewers: Guozhang Wang <wangguoz@gmail.com> Closes #1080 from ewencp/minor-kafka-3260-followup-tests	9 years ago
Jeremy Custenborder	6eacc0de30	KAFKA-3260 - Added SourceTask.commitRecord Added commitRecord(SourceRecord record) to SourceTask. This method is called during the callback from producer.send() when the message has been sent successfully. Added commitTaskRecord(SourceRecord record) to WorkerSourceTask to handle calling commitRecord on the SourceTask. Updated tests for calls to commitRecord. Author: Jeremy Custenborder <jcustenborder@gmail.com> Reviewers: Jason Gustafson <jason@confluent.io>, Ewen Cheslack-Postava <ewen@confluent.io> Closes #950 from jcustenborder/KAFKA-3260	9 years ago
Ismael Juma	fd6efbe0b7	MINOR: Remove unused import in `WordCountJob` to fix checkstyle failure Author: Ismael Juma <ismael@juma.me.uk> Reviewers: Ewen Cheslack-Postava <ewen@confluent.io> Closes #1077 from ijuma/fix-streams-checkstyle-failure	9 years ago
Ismael Juma	90c623d10f	MINOR: Add test that verifies fix for KAFKA-3047 Also clean-up `LogTest` a little. Author: Ismael Juma <ismael@juma.me.uk> Reviewers: Guozhang Wang Closes #1071 from ijuma/kafka-3047-explicit-offset-assignment-corrupt-log-test	9 years ago
Yasuhiro Matsuda	355076cd26	MINOR: kstream/ktable counting method with default long serdes guozhangwang miguno Author: Yasuhiro Matsuda <yasuhiro@confluent.io> Reviewers: Michael G. Noll, Guozhang Wang Closes #1065 from ymatsuda/count_serdes	9 years ago
Allen Wang	951e30adc6	KAFKA-1215; Rack-Aware replica assignment option Please see https://cwiki.apache.org/confluence/display/KAFKA/KIP-36+Rack+aware+replica+assignment for the overall design. The update to TopicMetadataRequest/TopicMetadataResponse will be done in a different PR. Author: Allen Wang <awang@netflix.com> Author: Ismael Juma <ismael@juma.me.uk> Reviewers: Ismael Juma <ismael@juma.me.uk>, Jason Gustafson <jason@confluent.io>, Grant Henke <granthenke@gmail.com>, Jun Rao <junrao@gmail.com> Closes #132 from allenxwang/KAFKA-1215	9 years ago
MayureshGharat	deb2b004cb	KAFKA-3013: Display the topic-partition in the exception message for expired batches in recordAccumulator Added topic-partition information to the exception message on batch expiry in RecordAccumulator Author: MayureshGharat <gharatmayuresh15@gmail.com> Reviewers: Gwen Shapira, Lin Dong, Ismael Juma Closes #695 from MayureshGharat/kafka-3013	9 years ago
Ismael Juma	241c3ebb28	KAFKA-3375; Suppress deprecated warnings where reasonable and tweak compiler settings * Fix and suppress number of unchecked warnings (except for Kafka Streams) * Add `SafeVarargs` annotation to fix warnings * Suppress unfixable deprecation warnings * Replace deprecated by non-deprecated usage where possible * Avoid reflective calls via structural types in Scala * Tweak compiler settings for scalac and javac Once we drop Java 7 and Scala 2.10, we can tweak the compiler settings further so that they warn us about more things. Author: Ismael Juma <ismael@juma.me.uk> Reviewers: Grant Henke, Gwen Shapira, Guozhang Wang Closes #1042 from ijuma/kafka-3375-suppress-depreccated-tweak-compiler	9 years ago
Jiangjie Qin	ffbe624e6f	KAFKA-3373; add 'log' prefix to configurations in KIP-31/32 Author: Jiangjie Qin <becket.qin@gmail.com> Reviewers: Gwen Shapira Closes #1049 from becketqin/KAFKA-3373	9 years ago
Liquan Pei	cf40acc2b1	MINOR: Remove unused method, redundant in interface definition and add final for object used in sychronization guozhangwang Very minor cleanup. Author: Liquan Pei <liquanpei@gmail.com> Reviewers: Guozhang Wang <wangguoz@gmail.com> Closes #1063 from Ishiihara/minor-cleanup	9 years ago
Yasuhiro Matsuda	c1a56c6839	KAFKA-3395: prefix job id to internal topic names guozhangwang Author: Yasuhiro Matsuda <yasuhiro@confluent.io> Reviewers: Guozhang Wang <wangguoz@gmail.com> Closes #1062 from ymatsuda/k3395	9 years ago
Manikumar reddy O	ac7b2e95d3	KAFKA-2551; Update Unclean leader election docs Author: Manikumar reddy O <manikumar.reddy@gmail.com> Reviewers: Ismael Juma <ismael@juma.me.uk>, Sriharsha Chintalapani <mail@harsha.io> Closes #1054 from omkreddy/KAFKA-2551	9 years ago
Ismael Juma	c9311d5f4e	KAFKA-3047: Explicit offset assignment in Log.append can corrupt the log This fix was suggested by Maciek Makowski, who also reported the problem. Author: Ismael Juma <ismael@juma.me.uk> Reviewers: Guozhang Wang <wangguoz@gmail.com> Closes #1029 from ijuma/KAFKA-3047-log-append-can-corrupt-the-log	9 years ago
Ashish Singh	a162f6bf66	KAFKA-3380; Add system test for GetOffsetShell tool Author: Ashish Singh <asingh@cloudera.com> Reviewers: Gwen Shapira Closes #1048 from SinghAsDev/KAFKA-3380	9 years ago
Jiangjie Qin	02d4da5f64	KAFKA-2960 KAFKA-1148; Clear purgatory for partitions before becoming follower Author: Jiangjie Qin <becket.qin@gmail.com> Reviewers: Aditya Auradkar <aauradkar@linkedin.com>, Ismael Juma <ismael@juma.me.uk>, Joel Koshy <jjkoshy.w@gmail.com>, Jun Rao <junrao@gmail.com>, Guozhang Wang <wangguoz@gmail.com> Closes #1018 from becketqin/KAFKA-2960	9 years ago
Jason Gustafson	764d8ca9eb	KAFKA-2073: migrate to client-side topic metadata request/response Author: Jason Gustafson <jason@confluent.io> Author: Ismael Juma <ismael@juma.me.uk> Author: hachikuji <jason@confluent.io> Reviewers: Grant Henke, Ismael Juma, Gwen Shapira, Flavio Junquiera Closes #988 from hachikuji/KAFKA-2073	9 years ago
Manikumar reddy O	287cce251f	MINOR: update compression design doc to include lz4 protocol Author: Manikumar reddy O <manikumar.reddy@gmail.com> Reviewers: Ismael Juma, Guozhang Wang Closes #1040 from omkreddy/MINOR-DOC	9 years ago
Guozhang Wang	9c4c5ae1cd	MINOR: Add unit test for internal topics Author: Guozhang Wang <wangguoz@gmail.com> Reviewers: Yasuhiro Matsuda <yasuhiro@confluent.io>, Ewen Cheslack-Postava <ewen@confluent.io> Closes #1047 from guozhangwang/KInternal	9 years ago
Eno Thereska	04ef9c3547	MINOR: Increased default EC2 instance size AWS instance size increased to m3.xlarge to allow all system tests to pass. ijuma ewencp have a look please. Author: Eno Thereska <eno.thereska@gmail.com> Reviewers: Gwen Shapira Closes #1046 from enothereska/minor-aws	9 years ago
Jason Gustafson	e403b3c4bf	KAFKA-3318: clean up consumer logging and error messages Author: Jason Gustafson <jason@confluent.io> Reviewers: Ismael Juma Closes #1036 from hachikuji/KAFKA-3318	9 years ago

1 2 3 4 5 ...

2206 Commits (31e263e8294e94de3d2c44d2ab3a827ab904e247) All Branches Search

2206 Commits (31e263e8294e94de3d2c44d2ab3a827ab904e247)

All Branches