Also removed the code for handling negative timeouts in `blockingReady` as `Selector.poll` has not supported that for a while.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Jun Rao <junrao@gmail.com>
Closes#1177 from ijuma/kafka-3495-blocking-send-and-receive-request-timeout
Use a different keytab for server and client in SASL tests
Also:
* Improve approach used to build the JAAS files programmatically
* Delete stale `kafka_jaas.conf` file
* Move `FourLetterWords` to its own file, add `Zk` prefix and clean-up its usage
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Harsha Chintalapani, Gwen Shapira
Closes#533 from ijuma/separate-keytabs-for-sasl-tests
Zookeeper clients that are not closed after the server is shutdown keep trying to reconnect, reloading JAAS configuration. This impacts subsequent tests which rely on JAAS config to be reset.
Author: Rajini Sivaram <rajinisivaram@googlemail.com>
Reviewers: Flavio Junqueira <fpj@apache.org>, Ewen Cheslack-Postava <ewen@confluent.io>
Closes#1171 from rajinisivaram/KAFKA-2910
This also fixes KAFKA-3453 and KAFKA-2866.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Gwen Shapira
Closes#1155 from ijuma/kafka-3475-introduce-our-minikdc
Also:
* Rename remaining `CoreUtils.rm` to `delete` for consistency
* Use `try with resources` in `Utils` to simplify code
* Silence compiler warning due to exception catch clause in `TestUtils`
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Guozhang Wang <wangguoz@gmail.com>
Closes#1153 from ijuma/remove-redundant-core-utils-rm
Author: Jiangjie Qin <becket.qin@gmail.com>
Reviewers: Ismael Juma <ismael@juma.me.uk>, Jun Rao <junrao@gmail.com>
Closes#1112 from becketqin/KAFKA-3442
Also included a minor efficiency improvement in kafka.cluster.EndPoint.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Gwen Shapira
Closes#1105 from ijuma/kafka-3431-replace-broker-end-point-with-node
This fixes an issue with long topic names by considering, during topic
validation, the '-' and the partition id that is appended to the log
folder created for each topic partition.
Author: Vahid Hashemian <vahidhashemian@us.ibm.com>
Reviewers: Gwen Shapira, Grant Henke
Closes#898 from vahidhashemian/KAFKA-3219
It is not true in practice. Maybe the implied feature is not yet implemented or removed.
These lines can be super misleading.
Please merge.
Thank you.
Author: gaob13 <gaob13@mails.tsinghua.edu.cn>
Reviewers: Ismael Juma, Ewen Cheslack-Postava
Closes#793 from gaob13/trunk
Author: Jason Gustafson <jason@confluent.io>
Reviewers: Ismael Juma <ismael@juma.me.uk>, Ewen Cheslack-Postava <ewen@confluent.io>
Closes#1108 from hachikuji/KAFKA-3412
…e calls
Changes the SimpleAclAuthorizer to:
- Track and utilize the zookeeper version when updating zookeeper to prevent data loss in the case of stale reads and race conditions
- Update local cache when modifying ACLs
- Add debug logging
Author: Grant Henke <granthenke@gmail.com>
Author: Grant Henke <granthenke@users.noreply.github.com>
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Flavio Junqueira, Jun Rao, Ismael Juma, Gwen Shapira
Closes#1006 from granthenke/simple-authorizer-fix
minor fix for prompt the user, or we will get a error message:
>Missing required argument "[time]"
Author: Xin Wang <best.wangxin@163.com>
Reviewers: Ashish Singh, Guozhang Wang
Closes#1068 from vesense/patch-4
Author: Jason Gustafson <jason@confluent.io>
Reviewers: Ismael Juma <ismael@juma.me.uk>, Grant Henke <granthenke@gmail.com>, Ewen Cheslack-Postava <me@ewencp.org>, Jun Rao <junrao@gmail.com>
Closes#1064 from hachikuji/KAFKA-3394
Also update server tests to always use new producer.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Gwen Shapira
Closes#1092 from ijuma/kafka-2982-deprecate-old-producers
becketqin Can you take a look?
Author: Dong Lin <lindong28@gmail.com>
Reviewers: Ismael Juma <ismael@juma.me.uk>, Jun Rao <junrao@gmail.com>
Closes#1009 from lindong28/KAFKA-3330
I also slightly tweaked the wording on a couple of warnings.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Gwen Shapira
Closes#1072 from ijuma/kafka-3373-follow-up
`Map.values` returns `DefaultValuesIterable` where the default implementation of `toSeq` is (sadly) `toStream`. `Stream` is a lazy collection and it can reflect changes to the underlying map before it's `forced`.
I verified that the test failed before my change.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Gwen Shapira
Closes#1088 from ijuma/kafka-3414-get-alive-brokers-no-mutation
A new consumer config option 'exclude.internal.topics' was added to
allow excluding internal topics when wildcards are used to specify
consumers.
The new option takes a boolean value, with a default 'false' value (i.e.
no exclusion).
This patch is co-authored with rajinisivaram edoardocomar mimaison
Author: edoardo <ecomar@uk.ibm.com>
Author: Vahid Hashemian <vahidhashemian@us.ibm.com>
Reviewers: Ismael Juma, Jun Rao, Gwen Shapira
Closes#1082 from edoardocomar/KAFKA-2832
This is a KIP-42 followup.
Currently, If sending the record fails before it gets to the server, ProducerInterceptor.onAcknowledgement() is called with metadata == null, and non-null exception. However, it is useful to pass topic and partition, if known, to ProducerInterceptor.onAcknowledgement() as well. This patch ensures that ProducerInterceptor.onAcknowledgement() gets record metadata with topic and maybe partition. If partition is not set in 'record' and KafkaProducer.send() fails before partition gets assigned, then ProducerInterceptor.onAcknowledgement() gets RecordMetadata with partition == -1. Only time when ProducerInterceptor.onAcknowledgement() gets null record metadata is when the client passes null record to KafkaProducer.send().
Author: Anna Povzner <anna@confluent.io>
Reviewers: Ismael Juma <ismael@juma.me.uk>, Ashish Singh <asingh@cloudera.com>, Jun Rao <junrao@gmail.com>
Closes#1015 from apovzner/kip42-3
Also clean-up `LogTest` a little.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Guozhang Wang
Closes#1071 from ijuma/kafka-3047-explicit-offset-assignment-corrupt-log-test
Please see https://cwiki.apache.org/confluence/display/KAFKA/KIP-36+Rack+aware+replica+assignment for the overall design.
The update to TopicMetadataRequest/TopicMetadataResponse will be done in a different PR.
Author: Allen Wang <awang@netflix.com>
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Ismael Juma <ismael@juma.me.uk>, Jason Gustafson <jason@confluent.io>, Grant Henke <granthenke@gmail.com>, Jun Rao <junrao@gmail.com>
Closes#132 from allenxwang/KAFKA-1215
* Fix and suppress number of unchecked warnings (except for Kafka Streams)
* Add `SafeVarargs` annotation to fix warnings
* Suppress unfixable deprecation warnings
* Replace deprecated by non-deprecated usage where possible
* Avoid reflective calls via structural types in Scala
* Tweak compiler settings for scalac and javac
Once we drop Java 7 and Scala 2.10, we can tweak the compiler settings further so that they warn us about more things.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Grant Henke, Gwen Shapira, Guozhang Wang
Closes#1042 from ijuma/kafka-3375-suppress-depreccated-tweak-compiler
This fix was suggested by Maciek Makowski, who also reported the problem.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Guozhang Wang <wangguoz@gmail.com>
Closes#1029 from ijuma/KAFKA-3047-log-append-can-corrupt-the-log
Author: Jiangjie Qin <becket.qin@gmail.com>
Reviewers: Aditya Auradkar <aauradkar@linkedin.com>, Ismael Juma <ismael@juma.me.uk>, Joel Koshy <jjkoshy.w@gmail.com>, Jun Rao <junrao@gmail.com>, Guozhang Wang <wangguoz@gmail.com>
Closes#1018 from becketqin/KAFKA-2960
By using `getHostString` (introduced in Java 7) instead of `getHostName`.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Jason Gustafson, Grant Henke
Closes#1030 from ijuma/kafka-3352-avoid-dns-reverse-look-ups
JAAS configuration may be set using other methods and hence the check for System property doesn't always match where the actual configuration used by Kafka is loaded from.
Author: Rajini Sivaram <rajinisivaram@googlemail.com>
Reviewers: Ismael Juma <ismael@juma.me.uk>, Sriharsha Chintalapani <harsha@hortonworks.com>, Flavio Junqueira <fpj@apache.org>, Ewen Cheslack-Postava <ewen@confluent.io>
Closes#967 from rajinisivaram/KAFKA-3279
…Response with o.a.k.c.requests equivalent
Author: Grant Henke <granthenke@gmail.com>
Reviewers: Ismael Juma
Closes#927 from granthenke/offset-refactor
https://issues.apache.org/jira/browse/KAFKA-1476
Let me know if these kind of contributions should have their own requisite JIRA opened in advance.
Cheers..
Author: Christian Posta <christian.posta@gmail.com>
Reviewers: Gwen Shapira
Closes#945 from christian-posta/ceposta-tidy-up-consumer-groups-describe
* Include request id when parsing of request header fails
* Don't mute selector on a connection that was closed due to an error (otherwise a second exception is thrown)
* Throw appropriate exception from `ApiKeys.fromId` if invalid id is passed
* Fail fast in `AbstractRequest.getRequest` if we fail to handle an instance of `ApiKeys` (if this happens, it's a programmer error and the code in `getRequest` needs to be updated)
I ran into the top two issues while trying to figure out why a connection from a producer to a broker was failing (and it made things harder than necessary). While fixing them, I noticed the third and fourth issues.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Gwen Shapira
Closes#1017 from ijuma/kafka-3341-improve-error-handling-invalid-requests
The fix basically ensures that the throttleTimeSensor is non-null before handing off to record the metric value. We also record the throttle time to 0 so that we don't recreate the sensor always.
Author: Aditya Auradkar <aauradkar@linkedin.com>
Reviewers: Jiangjie Qin <becket.qin@gmail.com>, Jun Rao <junrao@gmail.com>
Closes#989 from auradkar/KAFKA-3310
* Change `MessageFormat.writeTo` to take a `ConsumerRecord`
* Change `MessageReader.readMessage()` to use `ProducerRecord`
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Jun Rao <junrao@gmail.com>
Closes#972 from ijuma/kafka-3273-message-formatter-and-reader-resilient
The ability to specify a deserializer for keys and values was added in a recent commit (845c6eae1f), but it contained a few issues.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Guozhang Wang <wangguoz@gmail.com>
Closes#987 from ijuma/console-consumer-cleanups
Ismael Juma
Ashish Singh
Rajini Sivaram
Grant Henke
Jason Gustafson
Jiangjie Qin
Vahid Hashemian
gaob13
Jun Rao
Pierre-Yves Ritschard
Xin Wang
Dong Lin
edoardo
Anna Povzner
Allen Wang
Christian Posta
Aditya Auradkar
Guozhang Wang
Parth Brahmbhatt