This PR includes the same code as https://github.com/apache/kafka/pull/1261 but is rebased on latest trunk.
Author: Michael G. Noll <michael@confluent.io>
Reviewers: Matthias J. Sax, Guozhang Wang
Closes#1277 from miguno/KAFKA-3613-v2
Author: Eno Thereska <eno.thereska@gmail.com>
Reviewers: Ismael Juma, Damian Guy, Michael G. Noll, Guozhang Wang
Closes#1260 from enothereska/KAFKA-3612-integration-tests
guozhangwang
Author: Damian Guy <damian.guy@gmail.com>
Reviewers: Matthias J. Sax, Guozhang Wang
Closes#1272 from dguy/kstreamimpl-to-npe and squashes the following commits:
49d48fb [Damian Guy] actually commit the fix
07ce589 [Damian Guy] fix npe in KStreamImpl.to(..)
74d396d [Damian Guy] fix npe in KStreamImpl.to(..)
Author: Matthias J. Sax <matthias@confluent.io>
Reviewers: Ismael Juma, Michael G. Noll, Guozhang Wang
Closes#1266 from mjsax/kafka-3599-minorCodeCleanup
Feel free to review guozhangwang enothereska mjsax .
Author: Michael G. Noll <michael@confluent.io>
Reviewers: Matthias J. Sax, Michael G. Noll, Eno Thereska
Closes#1262 from miguno/KAFKA-3614
This contribution is my original work and I license the work to the Kafka project under the project's open source license.
cc guozhangwang miguno ymatsuda
Author: Jeff Klukas <jeff@klukas.net>
Reviewers: Michael G. Noll <michael@confluent.io>, Guozhang Wang <wangguoz@gmail.com>
Closes#1270 from jklukas/streams-doc-fix
Author: Guozhang Wang <wangguoz@gmail.com>
Reviewers: Ismael Juma, Josh Gruenberg, Michael G. Noll, Ewen Cheslack-Postava
Closes#1229 from guozhangwang/K3499
Kafka Streams seems to hold file handles on the `.lock` files for the state dirs, resulting in an explosion of filehandles over time. Running `lsof` shows the number of open filehandles on the `.lock` file increasing rapidly over time. In a separate test project, I reproduced the issue and determined that in order for the filehandle to be relinquished the `FileChannel` instance must be properly closed. Applying this patch seems to resolve the issue in my job.
Author: Greg Fodor <gfodor@gmail.com>
Reviewers: Guozhang Wang <wangguoz@gmail.com>
Closes#1267 from gfodor/bug/state-lock-filehandle-leak
Author: Damian Guy <damian.guy@gmail.com>
Reviewers: Guozhang Wang <wangguoz@gmail.com>, Michael G. Noll <michael@confluent.io>
Closes#1255 from dguy/kgroupedtable-count-test
Author: Guozhang Wang <wangguoz@gmail.com>
Reviewers: Eno Thereska <eno.thereska@gmail.com>, Ismael Juma <ismael@juma.me.uk>
Closes#1246 from guozhangwang/K3589
Author: Matthias J. Sax <matthias@confluent.io>
Reviewers: Guozhang Wang <wangguoz@gmail.com>
Closes#1231 from mjsax/kafka-3337-extact-key-selector-from-agg
For enums and other constant strings, use locale independent case conversions to enable comparisons to work regardless of the default locale.
Author: Rajini Sivaram <rajinisivaram@googlemail.com>
Reviewers: Manikumar Reddy, Ismael Juma, Guozhang Wang, Gwen Shapira
Closes#1220 from rajinisivaram/KAFKA-3548
… With KStream the method selectKey was added to enable getting a key from values before perfoming aggregation-by-key operations on original streams that have null keys.
Author: bbejeck <bbejeck@gmail.com>
Reviewers: Guozhang Wang <wangguoz@gmail.com>
Closes#1222 from bbejeck/KAFKA-3430_allow_users_to_set_key_KTable_toStream
Author: Grant Henke <granthenke@gmail.com>
Reviewers: Ismael Juma <ismael@juma.me.uk>, Ewen Cheslack-Postava <ewen@confluent.io>
Closes#1217 from granthenke/close-consumers
Author: Eno Thereska <eno.thereska@gmail.com>
Reviewers: Ismael Juma <ismael@juma.me.uk>, Guozhang Wang <wangguoz@gmail.com>
Closes#1203 from enothereska/KAFKA-3504-logcompaction
Addresses comments from previous PR [#1187]
Changed print and writeAsText method return signature to void
Flush System.out on close
Changed IllegalStateException to TopologyBuilderException
Updated MockProcessorContext.topic method to return a String
Renamed KStreamPrinter to KeyValuePrinter
Updated the printing of null keys to 'null' to match ConsoleConsumer
Updated JavaDoc stating need to override toString
Author: bbejeck <bbejeck@gmail.com>
Reviewers: Dan Norwood, Guozhang Wang
Closes#1209 from bbejeck/KAFKA-3338_Adding_print/writeAsText_to_Streams_DSL
miguno guozhangwang please have a look if you can.
Author: Eno Thereska <eno.thereska@gmail.com>
Reviewers: Michael G. Noll <michael@confluent.io>, Guozhang Wang <wangguoz@gmail.com>
Closes#1193 from enothereska/kafka-3512-ForEach
Author: Guozhang Wang <wangguoz@gmail.com>
Reviewers: Anna Povzner <anna@confluent.io>, Ewen Cheslack-Postava <ewen@confluent.io>
Closes#1190 from guozhangwang/K3505
guozhangwang
Setting the timestamp in produced records in SinkNode. This forces the producer record's timestamp same as the context's timestamp.
Author: Yasuhiro Matsuda <yasuhiro@confluent.io>
Reviewers: Guozhang Wang <wangguoz@gmail.com>
Closes#1137 from ymatsuda/set_timestamp_in_sinknode
guozhangwang
Without clearing the dirty flags, RocksDBStore will perform flush for every new record. This bug made the store performance painfully slower.
Author: Yasuhiro Matsuda <yasuhiro@confluent.io>
Reviewers: Guozhang Wang <wangguoz@gmail.com>
Closes#1163 from ymatsuda/clear_dirty_flag
…nology
Hi all,
This is my first contribution and I hope it will be good.
The PR is related to this issue:
https://issues.apache.org/jira/browse/KAFKA-3449
Thanks a lot,
Andrea
Author: Andrea Cosentino <ancosen@gmail.com>
Reviewers: Yasuhiro Matsuda, Guozhang Wang
Closes#1134 from oscerd/KAFKA-3449
Fix NPE in StoreChangeLogger caused by a record out of window retention period.
guozhangwang
Author: Yasuhiro Matsuda <yasuhiro@confluent.io>
Reviewers: Guozhang Wang <wangguoz@gmail.com>
Closes#1124 from ymatsuda/logger_npe
Replace `update` with `withPartitions`, which returns a copy instead of mutating the instance.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Guozhang Wang <wangguoz@gmail.com>
Closes#1118 from ijuma/kafka-3432-cluster-update-thread-safety
Also improves Java docs for the Streams high-level DSL.
Author: Guozhang Wang <wangguoz@gmail.com>
Reviewers: Ismael Juma, Michael G. Noll
Closes#1097 from guozhangwang/KNewJavaDoc
Also include:
1) remove streams specific configs before passing to producer and consumer to avoid warning message;
2) add `ConsumerRecord` timestamp extractor and set as the default extractor.
Author: Guozhang Wang <wangguoz@gmail.com>
Reviewers: Michael G. Noll, Ewen Cheslack-Postava
Closes#1093 from guozhangwang/KConfigWarn
guozhangwang ymatsuda : please review.
Author: Michael G. Noll <michael@confluent.io>
Reviewers: Guozhang Wang <wangguoz@gmail.com>
Closes#1081 from miguno/KAFKA-3411
guozhangwang Very minor cleanup.
Author: Liquan Pei <liquanpei@gmail.com>
Reviewers: Guozhang Wang <wangguoz@gmail.com>
Closes#1063 from Ishiihara/minor-cleanup
Matthias J. Sax
Michael G. Noll
Eno Thereska
Damian Guy
Guozhang Wang
Jeff Klukas
Greg Fodor
Rajini Sivaram
Bill Bejeck
Grant Henke
Ismael Juma
Yasuhiro Matsuda
Andrea Cosentino
Pierre-Yves Ritschard
Liquan Pei
Manikumar reddy O
Ewen Cheslack-Postava