src-kafka

Commit Graph

Author	SHA1	Message	Date
Ewen Cheslack-Postava	fc0ea25025	MINOR: Add another common error case for CorruptRecordException's error message Author: Ewen Cheslack-Postava <me@ewencp.org> Reviewers: Ismael Juma <ismael@juma.me.uk> Closes #3541 from ewencp/corrupt-record-null-key-compacted-topic	7 years ago
Gwen Shapira	ea8845bbd6	KAFKA-5384: Enable topic deletion by default https://cwiki.apache.org/confluence/display/KAFKA/KIP-162+-+Enable+topic+deletion+by+default Author: Gwen Shapira <cshapi@gmail.com> Reviewers: Ismael Juma Closes #3241 from gwenshap/KAFKA-5384	7 years ago
Kamal C	bbc3900185	MINOR: Use static imports in KafkaLog4jAppender Instead of redefining the constants. Author: Kamal C <kamal.chandraprakash@gmail.com> Reviewers: Ismael Juma <ismael@juma.me.uk> Closes #3349 from Kamal15/log4j2	7 years ago
Ismael Juma	454f0f1ca1	MINOR: Remove flaky assertion in verifyCloseOldestConnectionWithStagedReceives This seems to fail a lot in Jenkins although it always passes locally for me. Removing the assertion to restore Jenkins stability while we investigate in more detail. Author: Ismael Juma <ismael@juma.me.uk> Reviewers: Damian Guy <damian.guy@gmail.com>, Rajini Sivaram <rajinisivaram@googlemail.com> Closes #3543 from ijuma/remove-selector-test-assertion	7 years ago
Ewen Cheslack-Postava	d663005fdd	MINOR: Do not wait for first line of console consumer output since we now have a more reliable test using JMX Waiting for the first line of output was added in KAFKA-2527 when JmxMixin was originally added as a heuristic to determine when the process was ready. We've since determined this is not good enough given JmxTool's limitations and now include a separate, more reliable check before starting JmxTool. This check is also dangerous since a consumer that is started before data is available in the topic, it won't output anything to stdout and only logs errors to a separate log file. This means we may have a long delay between starting the process and starting JMX monitoring. Since we have a more reliable check for liveness via JMX now (and in cases that need it, partition assignment metrics via JMX), we should no longer need to wait for the first line of output. Author: Ewen Cheslack-Postava <me@ewencp.org> Reviewers: Ismael Juma <ismael@juma.me.uk>, Apurva Mehta <apurva@confluent.io> Closes #3447 from ewencp/dont-wait-first-line-console-consumer	7 years ago
Rajini Sivaram	28c83d9667	KAFKA-5587; Remove channel only after staged receives are delivered When idle connections are closed, ensure that channels with staged receives are retained in `closingChannels` until all staged receives are completed. Also ensure that only one staged receive is completed in each poll, even when channels are closed. Author: Rajini Sivaram <rajinisivaram@googlemail.com> Reviewers: Jun Rao <junrao@gmail.com>, Ismael Juma <ismael@juma.me.uk> Closes #3526 from rajinisivaram/KAFKA-5587	7 years ago
Jan Burkhardt	e2fe19d22a	KAFKA-5600; Fix group loading regression causing stale metadata/offset cache the while loop was too big and need to be closed earlier to see the fix, ignore whitespace since most of it is indentation this bug was introduced by commit `5bd06f1d54` Author: Jan Burkhardt <jan.burkhardt@just.social> Reviewers: Ismael Juma <ismael@juma.me.uk>, Jason Gustafson <jason@confluent.io> Closes #3538 from bjrke/trunk	7 years ago
Jeremy Custenborder	6d7a81b478	KAFKA-5579: check for null Author: Jeremy Custenborder <jcustenborder@gmail.com> Reviewers: Randall Hauch <rhauch@gmail.com>, Ewen Cheslack-Postava <ewen@confluent.io> Closes #3517 from jcustenborder/KAFKA-5579	7 years ago
Rajini Sivaram	cf94b188f1	MINOR: Use correct connectionId in SocketServer log message Also add connection id to KafkaChannel exception message Author: Rajini Sivaram <rajinisivaram@googlemail.com> Reviewers: Jason Gustafson <jason@confluent.io>, Ismael Juma <ismael@juma.me.uk> Closes #3529 from rajinisivaram/MINOR-log-connection-id	7 years ago
Xianyang Liu	d0ce0a95de	MINOR: Correct the ConsumerPerformance print format Currently, the output of `ConsumerPerformance` looks strange. The `header` format as follow: ``` "time, threadId, data.consumed.in.MB, MB.sec, data.consumed.in.nMsg, nMsg.sec" ``` while the `body` as follow: ``` println("%s, %d, %.4f, %.4f, %d, %.4f".format(dateFormat.format(endMs), id, totalMBRead, 1000.0 * (mbRead / elapsedMs), messagesRead, ((messagesRead - lastMessagesRead) / elapsedMs) * 1000.0)) ``` So we get the follow result: ``` time, data.consumeed.in.MB, MB.sec, data.consumeed.in.nMsg, nMsg.Sec 09:52:00, 0, 1100.3086, 220.0177, 563358, 112649.0702 ``` So the `header` and `body` mismatching. And also, this pr makes the functions more readable. Author: Xianyang Liu <xianyang.liu@intel.com> Reviewers: Ismael Juma <ismael@juma.me.uk> Closes #3417 from ConeyLiu/consumertest	7 years ago
Guozhang Wang	873eeae9f7	MINOR: Change log level for group member failure from debug to info Author: Guozhang Wang <wangguoz@gmail.com> Reviewers: Jason Gustafson <jason@confluent.io> Closes #3523 from guozhangwang/KMinor-group-coordinator-member-failure-info-log4j	7 years ago
Ismael Juma	1685e7112c	MINOR: Enable a number of xlint scalac warnings Update the code where possible to fix the warnings. The unused warning introduced in Scala 2.12 is quite handy and provides a reason to compile with Scala 2.12. Author: Ismael Juma <ismael@juma.me.uk> Reviewers: Jason Gustafson <jason@confluent.io> Closes #3464 from ijuma/scala-xlint	7 years ago
Balint Molnar	e391045473	KAFKA-5127; Replace pattern matching with foreach where the case None is ignored Author: Balint Molnar <balintmolnar91@gmail.com> Reviewers: Vahid Hashemian <vahidhashemian@us.ibm.com>, Ismael Juma <ismael@juma.me.uk>, Jason Gustafson <jason@confluent.io> Closes #2919 from baluchicken/KAFKA-5127	7 years ago
Gregor Uhlenheuer	2d2e9adb5d	KAFKA-5584; Fix integer overflow in Log.size It may lead to wrong metrics and it may break size-based retention. Author: Gregor Uhlenheuer <kongo2002@googlemail.com> Reviewers: Ismael Juma <ismael@juma.me.uk> Closes #3521 from kongo2002/KAFKA-5584	7 years ago
Evgeny Veretennikov	0547a08254	MINOR: typo in variable name "unkownInfo" Author: Evgeny Veretennikov <evg.veretennikov@gmail.com> Reviewers: Apurva Mehta <apurva@confluent.io>, Ismael Juma <ismael@juma.me.uk> Closes #3433 from evis/unkown_info_typo	7 years ago
Kamal C	ef27515943	MINOR: Typo error corrected in the KStream Javadoc. Author: Kamal C <kamal.chandraprakash@gmail.com> Reviewers: Matthias J. Sax <matthias@confluent.io>, Damian Guy <damian.guy@gmail.com> Closes #3520 from Kamal15/stream_doc	7 years ago
Joel Dice	8068a3560b	MINOR: consume from outputTopic in EosIntegrationTest.runSimpleCopyTest Previously, the code mistakenly consumed from inputTopic, which worked, but didn't actually verify that the messages were correctly copied from inputTopic to outputTopic. Author: Joel Dice <jdice@mersive.com> Reviewers: Matthias J. Sax <matthias@confluent.io>, Damian Guy <damian.guy@gmail.com> Closes #3522 from dicej/trunk	7 years ago
Eno Thereska	4416739915	MINOR: Upgraded RocksDB to 5.3.6 Solves a problem as described here: https://groups.google.com/forum/#!topic/confluent-platform/cR02K2o6BIw Author: Eno Thereska <eno.thereska@gmail.com> Reviewers: Damian Guy <damian.guy@gmail.com> Closes #3472 from enothereska/minor-upgrade-rocksdb-5.1.2	7 years ago
Manikumar Reddy O	9c6fc95f91	MINOR: Update docstring for "offsets.retention.minutes" config Author: Manikumar Reddy O <manikumar.reddy@gmail.com> Reviewers: Jason Gustafson <jason@confluent.io> Closes #2562 from omkreddy/MINOR-DOC	7 years ago
Bill Bejeck	2628695356	MINOR: need to be backwards compatible with deprecated default configs until removed …s until removed Author: Bill Bejeck <bill@confluent.io> Reviewers: Damian Guy <damian.guy@gmail.com>, Matthias J. Sax <matthias@confluent.io>, Guozhang Wang <wangguoz@gmail.com> Closes #3461 from bbejeck/MINOR_make_sure_deprecated_streams_configs_still_usable	7 years ago
Matthias J. Sax	990382b94c	KAFKA-5566: fixed race condition between flush and commit Author: Matthias J. Sax <matthias@confluent.io> Reviewers: Guozhang Wang <wangguoz@gmail.com> Closes #3504 from mjsax/kafka-5566-queryable-state-test	7 years ago
Eno Thereska	a1f97c8dc4	KAFKA-5157; Options for handling corrupt data during deserialization This is the implementation of KIP-161: https://cwiki.apache.org/confluence/display/KAFKA/KIP-161%3A+streams+deserialization+exception+handlers Author: Eno Thereska <eno.thereska@gmail.com> Reviewers: Damian Guy <damian.guy@gmail.com>, Matthias J. Sax <matthias@confluent.io> Closes #3423 from enothereska/KAFKA-5157-deserialization-exceptions	7 years ago
Guozhang Wang	6a5cd67b9b	HOTFIX: fix a few typos on streams quickstart Author: Guozhang Wang <wangguoz@gmail.com> Reviewers: Eno Thereska <eno.thereska@gmail.com>, Damian Guy <damian.guy@gmail.com> Closes #3510 from guozhangwang/KHotfix-streams-quickstart	7 years ago
Eno Thereska	7bfe008ae1	MINOR: Move quickstart under streams Author: Eno Thereska <eno.thereska@gmail.com> Reviewers: Michael G. Noll <michael@confluent.io>, Damian Guy <damian.guy@gmail.com>, Guozhang Wang <wangguoz@gmail.com> Closes #3494 from enothereska/minor-quickstart-docs rename section name and bold font for section names	7 years ago
Eno Thereska	bb8eff1e70	MINOR: Reduce logging level Author: Eno Thereska <eno.thereska@gmail.com> Reviewers: Matthias J. Sax <matthias@confluent.io>, Guozhang Wang <wangguoz@gmail.com> Closes #3505 from enothereska/minor-less-logs	7 years ago
Matthias J. Sax	35f9fa804b	KAFKA-5464: Follow up. Increase poll timeout Author: Matthias J. Sax <matthias@confluent.io> Reviewers: Ismael Juma <ismael@juma.me.uk>, Damian Guy <damian.guy@gmail.com>, Guozhang Wang <wangguoz@gmail.com> Closes #3496 from mjsax/KAFKA-5464-follow-up	7 years ago
Damian Guy	75f1b757e1	HOTFIX: fix paths in streams index Author: Damian Guy <damian.guy@gmail.com> Reviewers: Ismael Juma <ismael@juma.me.uk> Closes #3502 from dguy/doc-fixes	7 years ago
Ismael Juma	c574c58a16	MINOR: Improve versioning in docs when a full version is required Also fix quickstart. Author: Ismael Juma <ismael@juma.me.uk> Reviewers: Damian Guy <damian.guy@gmail.com> Closes #3501 from ijuma/introduce-full-version-variable-in-docs	7 years ago
Eno Thereska	071b8f975f	KAFKA-5566; fix unit test shouldAllowToQueryAfterThreadDied Ensure only one thread dies, not both. Author: Eno Thereska <eno.thereska@gmail.com> Reviewers: Damian Guy <damian.guy@gmail.com> Closes #3500 from enothereska/hotfix-queryable-state	7 years ago
Matthias J. Sax	6ea36bc6fb	HOTFIX: disable flaky system tests Author: Matthias J. Sax <matthias@confluent.io> Reviewers: Eno Thereska <eno.thereska@gmail.com>, Damian Guy <damian.guy@gmail.com> Closes #3497 from mjsax/disable-flaky-system-tests	7 years ago
Damian Guy	0046a881d3	HOTFIX: fix broken streams test Author: Damian Guy <damian.guy@gmail.com> Reviewers: Eno Thereska <eno.thereska@gmail.com>, Ismael Juma <ismael@juma.me.uk> Closes #3493 from dguy/hotfix-test-failure	7 years ago
huxihx	6d203361a0	KAFKA-5508; Documentation for altering topics Operations documentation should guide user to employ `kafka-configs.sh` to add/remove configs for a topic. Author: huxihx <huxi_2b@hotmail.com> Reviewers: Damian Guy <damian.guy@gmail.com> Closes #3429 from huxihx/KAFKA-5508_Doc_for_altering_topics	7 years ago
Eno Thereska	9078554232	KAFKA-5372; fixes to state transitions Several fixes to state transition logic: - Kafka streams will now be in ERROR when all threads are DEAD or when global thread stops unexpectedly - Fixed transition logic in corner cases when thread is already dead or Kafka Streams is already closed - Fixed incorrect transition diagram in StreamThread - Unit tests to verify transitions Also: - re-enabled throwing an exception when an unexpected state change happens - fixed a whole bunch of EoS tests that did not start a thread - added more comments. Author: Eno Thereska <eno.thereska@gmail.com> Reviewers: Matthias J. Sax <matthias@confluent.io>, Guozhang Wang <wangguoz@gmail.com>, Damian Guy <damian.guy@gmail.com> Closes #3432 from enothereska/KAFKA-5372-state-transitions	7 years ago
Matthias J. Sax	f53f5eaa1b	KAFKA-5464; StreamsKafkaClient should not use StreamsConfig.POLL_MS_CONFIG Author: Matthias J. Sax <matthias@confluent.io> Reviewers: Eno Thereska <eno.thereska@gmail.com>, Damian Guy <damian.guy@gmail.com> Closes #3439 from mjsax/kafka-5464-streamskafkaclient-poll	7 years ago
Paolo Patierno	e68a94056f	KAFKA-5557: Using a logPrefix inside the StreamPartitionAssignor Added logPrefix for avoiding stream thread name formatting replicated more times Author: ppatierno <ppatierno@live.com> Reviewers: Matthias J. Sax <matthias@confluent.io>, Guozhang Wang <wangguoz@gmail.com> Closes #3488 from ppatierno/kafka-5557	7 years ago
Matthias J. Sax	70e949d522	KAFKA-5167: Release state locks in case of failure Author: Matthias J. Sax <matthias@confluent.io> Reviewers: Damian Guy <damian.guy@gmail.com>, Guozhang Wang <wangguoz@gmail.com> Closes #3449 from mjsax/kafka-5167-streams-task-gets-stuck-after-re-balance-due-to-LockException	7 years ago
Paolo Patierno	3ab0456db8	KAFKA-5525: Streams reset tool should have same console output with or without dry-run Fixed console output to be consistent with/without dry-run option Author: ppatierno <ppatierno@live.com> Reviewers: Matthias J. Sax <matthias@confluent.io>, Guozhang Wang <wangguoz@gmail.com> Closes #3443 from ppatierno/kafka-5525	7 years ago
Damian Guy	6cae5ec668	MINOR: add IQ docs to streams documentation Author: Damian Guy <damian.guy@gmail.com> Reviewers: Matthias J. Sax <matthias@confluent.io>, Guozhang Wang <wangguoz@gmail.com> Closes #3484 from dguy/iq-doc	7 years ago
Rajini Sivaram	48c2160c75	MINOR: Wait for tasks to terminate to avoid exception in test teardown Author: Rajini Sivaram <rajinisivaram@googlemail.com> Reviewers: Ismael Juma <ismael@juma.me.uk> Closes #3483 from rajinisivaram/MINOR-consumerbounce-test	7 years ago
ppatierno	ef5867dce3	KAFKA-5469; Created state changelog topics not logged correctly Fixed debug logging for the created state changelog topics Added toString() for InternalTopicMetadata and InternalTopicConfig for above debug logging Author: ppatierno <ppatierno@live.com> Reviewers: Damian Guy <damian.guy@gmail.com> Closes #3368 from ppatierno/kafka-5469	7 years ago
Rajini Sivaram	afb91a02ae	MINOR: Add test brokers to list as they are created to ensure cleanup Author: Rajini Sivaram <rajinisivaram@googlemail.com> Reviewers: Apurva Mehta <apurva@confluent.io>, Ismael Juma <ismael@juma.me.uk> Closes #3471 from rajinisivaram/MINOR-shutdown-brokers	7 years ago
Eno Thereska	cbafc08a5f	MINOR: compatibility tests for streams Update system tests to make use of the newly released 0.11 version. Add on to https://github.com/apache/kafka/pull/3454 Author: Eno Thereska <eno.thereska@gmail.com> Reviewers: Damian Guy <damian.guy@gmail.com> Closes #3479 from enothereska/minor-compatibility-tests	7 years ago
Guozhang Wang	2c6e9a9653	MINOR: Follow-up Streams doc changes to break into sub-pages Also fixed a bunch of broken links (details can be found in `34f8ecea0d`) Author: Guozhang Wang <wangguoz@gmail.com> Reviewers: Eno Thereska <eno.thereska@gmail.com>, Bill Bejeck <bbejeck@gmail.com>, Damian Guy <damian.guy@gmail.com> Closes #3473 from guozhangwang/KMinor-streams-doc-breakdown	7 years ago
Guozhang Wang	c9f777cc44	HOTFIX: Add back the copy-constructor of abstract stream Author: Guozhang Wang <wangguoz@gmail.com> Reviewers: Damian Guy <damian.guy@gmail.com> Closes #3428 from guozhangwang/KHotfix-add-copy-constructor-abstract-stream	7 years ago
Jeremy Custenborder	ab8e9d1755	KAFKA-5548: Extended validation for SchemaBuilder methods. More input validation for SchemaBuilder methods. Author: Jeremy Custenborder <jcustenborder@gmail.com> Reviewers: Ewen Cheslack-Postava <ewen@confluent.io> Closes #3474 from jcustenborder/KAFKA-5548	7 years ago
Jason Gustafson	9238aeaa21	KAFKA-5522; ListOffsets should bound timestamp search by LSO in read_committed Author: Jason Gustafson <jason@confluent.io> Reviewers: Ismael Juma <ismael@juma.me.uk>, Apurva Mehta <apurva@confluent.io>, Guozhang Wang <wangguoz@gmail.com> Closes #3456 from hachikuji/KAFKA-5522	7 years ago
Ben Stopford	9c2de4920e	KAFKA-5542; Improve Java doc for LeaderEpochFileCache.endOffsetFor() Author: Ben Stopford <benstopford@gmail.com> Reviewers: Jun Rao <junrao@gmail.com> Closes #3468 from benstopford/KAFKA-5542	7 years ago
Ismael Juma	49ed16daf4	MINOR: Compatibility and upgrade tests for 0.11.0.x Author: Ismael Juma <ismael@juma.me.uk> Reviewers: Eno Thereska <eno.thereska@gmail.com>, Ewen Cheslack-Postava <me@ewencp.org> Closes #3454 from ijuma/test-upgrades-from-0.11.0.x	7 years ago
Kamal C	342f34a199	MINOR: InFlightRequests#isEmpty(node) method corrected. - In clearAll method, get operation is removed. - variable name `requestTimeout` changed to `requestTimeoutMs` for clarity Author: Kamal C <kamal.chandraprakash@gmail.com> Reviewers: Rajini Sivaram <rajinisivaram@googlemail.com> Closes #3467 from Kamal15/frequest	7 years ago
Apurva Mehta	6471822079	KAFKA-5020; Update message format in implementation docs Author: Apurva Mehta <apurva@confluent.io> Reviewers: Jason Gustafson <jason@confluent.io> Closes #3360 from apurvam/KAFKA-5020-message-format-docs-update	7 years ago

... 3 4 5 6 7 ...

4122 Commits (beeed86600f3571635e83e6a3b6ee0c38679c7fe) All Branches Search

4122 Commits (beeed86600f3571635e83e6a3b6ee0c38679c7fe)

All Branches