While playing with client got the next exception:
```java
java.lang.IllegalArgumentException: Invalid partition given with record: 1 is not in the range [0...1].
```
It's obviously incorrect, so I've fixed it.
Author: Igor Stepanov <igor.stepanov@keystonett.com>
Reviewers: Guozhang Wang <wangguoz@gmail.com>
Closes#1210 from stepio/trunk
Addresses comments from previous PR [#1187]
Changed print and writeAsText method return signature to void
Flush System.out on close
Changed IllegalStateException to TopologyBuilderException
Updated MockProcessorContext.topic method to return a String
Renamed KStreamPrinter to KeyValuePrinter
Updated the printing of null keys to 'null' to match ConsoleConsumer
Updated JavaDoc stating need to override toString
Author: bbejeck <bbejeck@gmail.com>
Reviewers: Dan Norwood, Guozhang Wang
Closes#1209 from bbejeck/KAFKA-3338_Adding_print/writeAsText_to_Streams_DSL
This should make Log.read act the same when startOffset is larger than maxOffset as it would if startOffset was larger than logEndOffset. The current behavior can result in an IllegalArgumentException from LogSegment if a consumer attempts to fetch an offset above the high watermark which is present in the leader's log. It seems more correct if Log.read presents the view of the log to consumers as if it simply ended at maxOffset (high watermark).
I've tried to describe an example scenario of this happening here https://issues.apache.org/jira/browse/KAFKA-725?focusedCommentId=15221673&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15221673
I'm not sure I understand why ReplicaManager sets maxOffset to the high watermark, and not high watermark + 1. Isn't the high watermark the last committed message, and readable by consumers?
Tests passed for me locally on second try, seems like it just hit a flaky test.
Author: Stig Rohde Døssing <sdo@it-minds.dk>
Reviewers: Jiangjie Qin <becket.qin@gmail.com>, Guozhang Wang <wangguoz@gmail.com>
Closes#1178 from srdo/KAFKA-725
miguno guozhangwang please have a look if you can.
Author: Eno Thereska <eno.thereska@gmail.com>
Reviewers: Michael G. Noll <michael@confluent.io>, Guozhang Wang <wangguoz@gmail.com>
Closes#1193 from enothereska/kafka-3512-ForEach
Author: Guozhang Wang <wangguoz@gmail.com>
Reviewers: Anna Povzner <anna@confluent.io>, Ewen Cheslack-Postava <ewen@confluent.io>
Closes#1190 from guozhangwang/K3505
This PR: https://github.com/apache/kafka/pull/958 fixed the use of prop_file in the situation when we have multiple producers (before, every producer will add to the config). However, it assumes that self.prop_file is initially "". This is correct for all existing tests, but it precludes us from extending verifiable producer and adding more properties to the producer config (same as console consumer).
This is a small PR to change the behavior to the original, but also make verifiable producer use prop_file method to be consistent with console consumer.
Also few more fixes to verifiable producer came up during the review:
-- fixed each_produced_at_least() method
-- more straightforward use of compression types
granders please review.
Author: Anna Povzner <anna@confluent.io>
Reviewers: Geoff Anderson <geoff@confluent.io>, Ewen Cheslack-Postava <ewen@confluent.io>
Closes#1192 from apovzner/fix_verifiable_producer
Fail unsent requests only when returning from KafkaConsumer.poll().
Author: Rajini Sivaram <rajinisivaram@googlemail.com>
Reviewers: Jason Gustafson <jason@confluent.io>, Ismael Juma <ismael@juma.me.uk>, Ewen Cheslack-Postava <ewen@confluent.io>
Closes#1183 from rajinisivaram/KAFKA-3488
Allows the the maximum retires when writing to zookeeper to be overridden in tests and sets the value to Int.MaxValue to avoid transient failure.
Author: Grant Henke <granthenke@gmail.com>
Reviewers: Ismael Juma <ismael@juma.me.uk>, Ewen Cheslack-Postava <ewen@confluent.io>
Closes#1156 from granthenke/transient-acl-test
I also fixed a few issues in `SocketServerTest` and included a few clean-ups.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Jun Rao <junrao@gmail.com>
Closes#1172 from ijuma/kafka-3489-update-request-metrics-if-client-closes
* Make all fields accessed outside of a lock `volatile`
* Only allow mutation within the class
* Remove unnecessary `AtomicInteger` since mutation always happens inside a lock
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Guozhang Wang <wangguoz@gmail.com>
Closes#1188 from ijuma/kafka-3510-offset-index-thread-safety
I believe this addresses KAFKA-3384.
The POSIX kill manpage is at http://pubs.opengroup.org/onlinepubs/9699919799/utilities/kill.html
Author: Matt McClure <mlm@aya.yale.edu>
Reviewers: Geoff Anderson <geoff@confluent.io>, Ewen Cheslack-Postava <ewen@confluent.io>
Closes#1148 from matthewlmcclure/KAFKA-3384
guozhangwang
Setting the timestamp in produced records in SinkNode. This forces the producer record's timestamp same as the context's timestamp.
Author: Yasuhiro Matsuda <yasuhiro@confluent.io>
Reviewers: Guozhang Wang <wangguoz@gmail.com>
Closes#1137 from ymatsuda/set_timestamp_in_sinknode
Sentence was missing "as", minor grammar clean up.
Author: Paul Cavallaro <paulcavallaro@gmail.com>
Reviewers: Ewen Cheslack-Postava <ewen@confluent.io>
Closes#1151 from paulcavallaro/docs-fix
Also removed the code for handling negative timeouts in `blockingReady` as `Selector.poll` has not supported that for a while.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Jun Rao <junrao@gmail.com>
Closes#1177 from ijuma/kafka-3495-blocking-send-and-receive-request-timeout
Use a different keytab for server and client in SASL tests
Also:
* Improve approach used to build the JAAS files programmatically
* Delete stale `kafka_jaas.conf` file
* Move `FourLetterWords` to its own file, add `Zk` prefix and clean-up its usage
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Harsha Chintalapani, Gwen Shapira
Closes#533 from ijuma/separate-keytabs-for-sasl-tests
Before this patch arrays containing primitive types were not output:
```
Metadata Request (Version: 0) => [topics]
```
After this patch the type is listed:
```
Metadata Request (Version: 0) => [topics]
topics => STRING
```
Author: Grant Henke <granthenke@gmail.com>
Reviewers: Ashish Singh, Gwen Shapira
Closes#1174 from granthenke/protocol-arrays
Zookeeper clients that are not closed after the server is shutdown keep trying to reconnect, reloading JAAS configuration. This impacts subsequent tests which rely on JAAS config to be reset.
Author: Rajini Sivaram <rajinisivaram@googlemail.com>
Reviewers: Flavio Junqueira <fpj@apache.org>, Ewen Cheslack-Postava <ewen@confluent.io>
Closes#1171 from rajinisivaram/KAFKA-2910
This also fixes KAFKA-3453 and KAFKA-2866.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Gwen Shapira
Closes#1155 from ijuma/kafka-3475-introduce-our-minikdc
Previous version of ducktape was found to have a memory leak which caused occasional failures in nightly runs.
Author: Geoff Anderson <geoff@confluent.io>
Reviewers: Ewen Cheslack-Postava <ewen@confluent.io>
Closes#1165 from granders/minor-advance-ducktape-to-0.4.0
guozhangwang
Without clearing the dirty flags, RocksDBStore will perform flush for every new record. This bug made the store performance painfully slower.
Author: Yasuhiro Matsuda <yasuhiro@confluent.io>
Reviewers: Guozhang Wang <wangguoz@gmail.com>
Closes#1163 from ymatsuda/clear_dirty_flag
Also:
* Rename remaining `CoreUtils.rm` to `delete` for consistency
* Use `try with resources` in `Utils` to simplify code
* Silence compiler warning due to exception catch clause in `TestUtils`
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Guozhang Wang <wangguoz@gmail.com>
Closes#1153 from ijuma/remove-redundant-core-utils-rm
This was recently introduced in:
1fbe445dde
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Jun Rao <junrao@gmail.com>
Closes#1152 from ijuma/fix-typos-in-record-accumulator
Author: Jiangjie Qin <becket.qin@gmail.com>
Reviewers: Ismael Juma <ismael@juma.me.uk>, Jun Rao <junrao@gmail.com>
Closes#1056 from becketqin/KAFKA-3388
…nology
Hi all,
This is my first contribution and I hope it will be good.
The PR is related to this issue:
https://issues.apache.org/jira/browse/KAFKA-3449
Thanks a lot,
Andrea
Author: Andrea Cosentino <ancosen@gmail.com>
Reviewers: Yasuhiro Matsuda, Guozhang Wang
Closes#1134 from oscerd/KAFKA-3449
Ismael Juma
Guozhang Wang
Igor Stepanov
Ewen Cheslack-Postava
bbejeck
Jason Gustafson
Stig Rohde Døssing
Eno Thereska
Anna Povzner
Rajini Sivaram
Matthias J. Sax
Grant Henke
Matt McClure
Yasuhiro Matsuda
Liquan Pei
Paul Cavallaro
Flavio Junqueira
Ashish Singh
Geoff Anderson
Jiangjie Qin
Andrea Cosentino