Author: Liquan Pei <liquanpei@gmail.com>
Reviewers: Geoff Anderson <geoff@confluent.io>, Jun Rao <junrao@gmail.com>, Ismael Juma <ismael@juma.me.uk>
Closes#1364 from Ishiihara/add-quote-classpath
Initially proposed by ijuma in https://github.com/apache/kafka/pull/1362#issuecomment-218293662
mjsax commented:
> StreamThread.close() should be extended to call metrics.close() (the class need a private member to reference the Metrics object, too)
The `Metrics` instance is created in the `KafkaStreams` constructor and shared between all threads, so closing it within the threads doesn't seem like the right approach. This PR calls `Metrics.close()` in `KafkaStreams.close()` instead.
cc guozhangwang
Author: Jeff Klukas <jeff@klukas.net>
Reviewers: Ismael Juma, Guozhang Wang
Closes#1379 from jklukas/close-streams-metrics
This is a follow up of KAKFA-3421 to update the connect developer guide to include the configuration validation. Also includes a couple of minor fixes.
Author: Liquan Pei <liquanpei@gmail.com>
Reviewers: Jason Gustafson <jason@confluent.io>, Ewen Cheslack-Postava <ewen@confluent.io>
Closes#1366 from Ishiihara/connect-dev-doc
The information from this file is displayed in:
https://projects.apache.org/project.html?kafka
I removed the tag instead of fixing it because it's one more thing to maintain and neither Hadoop nor ZooKeeper have it (httpd does though).
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Guozhang Wang <wangguoz@gmail.com>
Closes#1378 from ijuma/remove-release-from-doap
I removed the hamcrest matcher to unbreak the build, but we probably want to tweak the `import-control.xml` as it currently only allows it for `<subpackage name="integration">`, which is weird.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Guozhang Wang <wangguoz@gmail.com>
Closes#1380 from ijuma/fix-streams-config-test-checkstyle
This is an improved version of https://github.com/apache/kafka/pull/1374, where we include a unit test.
/cc ijuma and guozhangwang
Author: Guozhang Wang <wangguoz@gmail.com>
Author: Michael G. Noll <michael@confluent.io>
Reviewers: Michael G. Noll <michael@confluent.io>, Ismael Juma <ismael@juma.me.uk>
Closes#1377 from miguno/streamsconfig-multiple-bootstrap-servers
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Guozhang Wang <wangguoz@gmail.com>, Jun Rao <junrao@gmail.com>
Closes#1373 from ijuma/fix-producer-buffer-size-upgrade-note
Author: Jason Gustafson <jason@confluent.io>
Reviewers: Roger Hoover <roger.hoover@gmail.com>, Ismael Juma <ismael@juma.me.uk>, Ewen Cheslack-Postava <ewen@confluent.io>
Closes#1358 from hachikuji/KAFKA-3583
Author: Jason Gustafson <jason@confluent.io>
Reviewers: Geoff Anderson <geoff@confluent.io>, Ismael Juma <ismael@juma.me.uk>
Closes#1365 from hachikuji/KAFKA-3694
Propagate IOException in SslTransportLayer channel.close to be consistent with PlaintextTransportLayer, close authenticator on channel close even if transport layer close fails.
Author: Rajini Sivaram <rajinisivaram@googlemail.com>
Reviewers: Ismael Juma <ismael@juma.me.uk>
Closes#1370 from rajinisivaram/minor-channelclose2
ZkUtils.getSequenceId() method is used to generate broker id sequence numbers. During startup, each broker updates the data at /brokers/seqid zk path and returns stat.getVersion as next sequence id.
stat.getVersion returns "1" for first data update. So ZkUtils.getSequenceId() should return "1" on first data update.
Author: Manikumar reddy O <manikumar.reddy@gmail.com>
Reviewers: Flavio Junqueira <fpj@apache.org>, Ismael Juma <ismael@juma.me.uk>
Closes#1224 from omkreddy/KAFKA-3525
We have had transient failures in this method when Jenkins is
overloaded.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Ewen Cheslack-Postava <ewen@confluent.io>
Closes#1359 from ijuma/increase-producer-close-timeout-in-test
Author: Guozhang Wang <wangguoz@gmail.com>
Reviewers: Michael G. Noll <michael@confluent.io>, Ismael Juma <ismael@juma.me.uk>
Closes#1321 from guozhangwang/KStreamsJavaDoc
Add a test for changing SASL mechanism using rolling upgrade and a test for rolling upgrade from 0.9.0.x to 0.10.0 with SASL/GSSAPI.
Author: Rajini Sivaram <rajinisivaram@googlemail.com>
Reviewers: Ben Stopford <benstopford@gmail.com>, Geoff Anderson <geoff@confluent.io>, Ismael Juma <ismael@juma.me.uk>
Closes#1290 from rajinisivaram/KAFKA-3634
_copyDependantTestLibs_ was added temporarily as a dependency of _jar_ task to enable SASL system tests to be run with MiniKdc without changing the automated system test runs which run _gradlew clean jar_. Since the build target _systemTestLibs_ is already in Kafka build.gradle, the Confluent automated test runs can now run _gradlew clean systemTestLibs_ instead. This PR provides the final change to remove _copyDependantTestLibs_ from the _jar_ task. This should be committed only after the Confluent automated sytem test build script is updated, to avoid breaking any builds.
Author: Rajini Sivaram <rajinisivaram@googlemail.com>
Reviewers: Ismael Juma <ismael@juma.me.uk>
Closes#430 from rajinisivaram/minor-systemtestlibs
Author: Jason Gustafson <jason@confluent.io>
Reviewers: Ismael Juma <ismael@juma.me.uk>
Closes#1354 from hachikuji/minor-add-missing-assignor-tostrings
After testing KAFKA-3160 a bit more, I found that the error code was not being set properly in ProduceResponse. This happened because the validation error is raised in the CompressionFactory constructor, which was not wrapped in a try / catch.
ijuma junrao
(This contribution is my original work and I license the work under Apache 2.0.)
Author: Dana Powers <dana.powers@gmail.com>
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Jun Rao <junrao@gmail.com>, Gwen Shapira <cshapi@gmail.com>, Ismael Juma <ismael@juma.me.uk>
Closes#1344 from dpkp/decompress_error_code
Removed the over pessimistic require and instead attempt to fill the dedup buffer. Use the (only) map until full;
this may allow to process all dirty segment (optimism) or may happen in the middle of a dirt segment.
In either case, do compaction using the map loaded that way.
This patch was developed with edoardocomar
Author: Mickael Maison <mickael.maison@gmail.com>
Reviewers: Jun Rao <junrao@gmail.com>, Guozhang Wang <wangguoz@gmail.com>
Closes#1332 from mimaison/KAFKA-3587
Test sends large request using multiple writes of length followed by request body. The first write should succeed, but since the server closes the connection on processing the length that is too big, subsequent writes may fail. Modified test to handle this exception.
Author: Rajini Sivaram <rajinisivaram@googlemail.com>
Reviewers: Ismael Juma <ismael@juma.me.uk>
Closes#1349 from rajinisivaram/KAFKA-3662
Add references to the new consumer property 'max.partition.fetch.bytes' along with the old consumer property 'fetch.message.max.bytes' in the corresponding warning messages of TopicCommand.
Also, create and leverage a static variable for the default value of the new consumer property.
Also, use 'DEFAULT_...' for default propoerty constant names in the code instead of '..._DEFAULT'.
Author: Vahid Hashemian <vahidhashemian@us.ibm.com>
Reviewers: Manikumar reddy O <manikumar.reddy@gmail.com>, Ashish Singh <asingh@cloudera.com>, Grant Henke <granthenke@gmail.com>, Ismael Juma <ismael@juma.me.uk>
Closes#1239 from vahidhashemian/KAFKA-3579
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Jun Rao <junrao@gmail.com>
Closes#1338 from ijuma/kafka-3670-controlled-shutdown-leader-selector-preferred-replica
This contribution is my original work and I license the work under Apache 2.0.
Author: Dana Powers <dana.powers@gmail.com>
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Jun Rao <junrao@gmail.com>, Ismael Juma <ismael@juma.me.uk>
Closes#1212 from dpkp/KAFKA-3160
so that unresolvable DNS names are ignored and only throw an error if no other bootstrap servers are resolvable.
Author: Jonathan Bond <jbond@netflix.com>
Reviewers: Ewen Cheslack-Postava <me@ewencp.org>, Grant Henke <granthenke@gmail.com>, Ismael Juma <ismael@juma.me.uk>
Closes#792 from bondj/KAFKA-3112
…value for doLog
Author: Mark Grover <mark@apache.org>
Reviewers: Grant Henke <granthenke@gmail.com>, Ismael Juma <ismael@juma.me.uk>
Closes#1334 from markgrover/kafka-3669-trunk
This patch adds logic for the following:
- remove hard-coded paths to various scripts and jars in kafkatest service classes
- provide a mechanism for overriding path resolution logic with a "pluggable" path resolver class
Author: Geoff Anderson <geoff@confluent.io>
Reviewers: Ewen Cheslack-Postava <ewen@confluent.io>
Closes#1245 from granders/configurable-install-path
Pull request to update the consumer API links in the docs.
Author: Dustin Cote <dustin@dustins-mbp.attlocal.net>
Reviewers: Ismael Juma <ismael@juma.me.uk>
Closes#1331 from cotedm/KAFKA-3666
Whenever the BufferPool throws a "Failed to allocate memory within the configured max blocking time" exception, it should also remove the condition object from the waiters deque
Author: MayureshGharat <gharatmayuresh15@gmail.com>
Reviewers: Chen Zhu <amandazhu19620701@gmail.com>, Ismael Juma <ismael@juma.me.uk>
Closes#1314 from MayureshGharat/kafka-3651
AbstractPartitionAssignor.assign has an ambiguous line in its documentation:
> param partitionsPerTopic The number of partitions for each subscribed topic (may be empty for some topics)
Does empty mean the topic has an entry with value zero, or that the entry is excluded from the map altogether? The current implementation in AbstractPartitionAssignor excludes the entry from partitionsPerTopic if the topic isn't in the metadata.
RoundRobinAssignorTest.testOneConsumerNonexistentTopic interprets emptiness as providing the topic with a zero value.
RangeAssignor interprets emptiness as excluding the entry from the map.
RangeAssignorTest.testOneConsumerNonexistentTopic interprets emptiness as providing the topic with a zero value.
This implementation chooses to solve the NPE by deciding to exclude topics from partitionsPerTopic when the topic is not in the metadata.
Author: Onur Karaman <okaraman@linkedin.com>
Reviewers: Jason Gustafson <jason@confluent.io>, Ismael Juma <ismael@juma.me.uk>
Closes#1326 from onurkaraman/KAFKA-3661
Liquan Pei
Jeff Klukas
Ismael Juma
Guozhang Wang
Jason Gustafson
Rajini Sivaram
Ben Stopford
Manikumar reddy O
Vahid Hashemian
Ashish Singh
Ewen Cheslack-Postava
Dana Powers
Mickael Maison
Kaufman Ng
Jonathan Bond
Chen Zhu
Mark Grover
Geoff Anderson
Yuto Kawamura
Dustin Cote
Mayuresh Gharat
Onur Karaman