This makes it consistent with MirrorMaker with the old consumer.
Author: huxi <huxi@zhenrongbao.com>
Author: amethystic <huxi_2b@hotmail.com>
Reviewers: Vahid Hashemian <vahidhashemian@us.ibm.com>, Ismael Juma <ismael@juma.me.uk>
Closes#2072 from amethystic/kafka-4351_Regex_behavior_change_for_new_consumer
Mx4jLoader.scala should explicitly `return true` if the class is successfully loaded and started, otherwise it will return false even if the class is loaded.
Author: Edward Ribeiro <edward.ribeiro@gmail.com>
Reviewers: Ewen Cheslack-Postava <ewen@confluent.io>
Closes#2295 from eribeiro/mx4jloader-bug
During Acceptor initialization, if "Address already in use" error is encountered,
the shutdown latch in each Processor is never counted down. Consequently,
the Kafka server hangs when `Processor.shutdown` is called.
Author: huxi <huxi@zhenrongbao.com>
Author: amethystic <huxi_2b@hotmail.com>
Reviewers: Jun Rao <junrao@gmail.com>, Ismael Juma <ismael@juma.me.uk>
Closes#2156 from amethystic/kafka-4428_Kafka_noexit_for_port_already_use
adding a LogConfig value validator. gwenshap or junrao would you mind taking a look?
Author: Dustin Cote <dustin@confluent.io>
Reviewers: Ismael Juma <ismael@juma.me.uk>, Ewen Cheslack-Postava <ewen@confluent.io>
Closes#1796 from cotedm/retentionbytesvalidation
The current output for the `--generate` option looks like this
```
Current partition replica assignment
{"version":1,"partitions":[{"topic":"t1","partition":0,"replicas":[0]}]}
Proposed partition reassignment configuration
{"version":1,"partitions":[{"topic":"t1","partition":0,"replicas":[1]}]}
```
This PR simply changes it to
```
Current partition replica assignment
{"version":1,"partitions":[{"topic":"t1","partition":0,"replicas":[0]}]}
Proposed partition reassignment configuration
{"version":1,"partitions":[{"topic":"t1","partition":0,"replicas":[1]}]}
```
to make it more readable.
Author: Vahid Hashemian <vahidhashemian@us.ibm.com>
Reviewers: Guozhang Wang <wangguoz@gmail.com>
Closes#1868 from vahidhashemian/minor/improve_output_format_of_reassign_partitions
This was changed in b58b6a1bef and caused the `ReplicaVerificationToolTest.test_replica_lags`
system test to start failing.
I also added a unit test and a couple of other minor clean-ups.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Jason Gustafson <jason@confluent.io>
Closes#2280 from ijuma/kafka-4554-fix-replica-buffer-verify-checksum
Author: Ashish Singh <asingh@cloudera.com>
Reviewers: Jason Gustafson <jason@confluent.io>, Colin P. Mccabe <cmccabe@confluent.io>, Dana Powers <dana.powers@gmail.com>, Gwen Shapira <cshapi@gmail.com>, Grant Henke <granthenke@gmail.com>, Ismael Juma <ismael@juma.me.uk>
Closes#1251 from SinghAsDev/KAFKA-3600
The latter return `Iterable` instead of `Iterator` so that enhanced foreach can be used
in Java.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Jason Gustafson <jason@confluent.io>
Closes#2261 from ijuma/deepEntries-shallowEntries
Fix OffsetIndex overflow when replicating a highly compacted topic.
https://issues.apache.org/jira/browse/KAFKA-4451
Author: Michael Schiff <schiff.michael@gmail.com>
Author: Michael Schiff <michael.schiff@tubemogul.com>
Reviewers: Jun Rao <junrao@gmail.com>
Closes#2210 from michaelschiff/bug/4451
Author: Dong Lin <lindong28@gmail.com>
Author: Dong Lin <dolin@linkedin.com>
Reviewers: Jiangjie Qin <becket.qin@gmail.com>
Closes#2241 from lindong28/KAFKA-4521
Author: Jason Gustafson <jason@confluent.io>
Reviewers: Ismael Juma <ismael@juma.me.uk>, Guozhang Wang <wangguoz@gmail.com>, Jun Rao <junrao@gmail.com>
Closes#2140 from hachikuji/KAFKA4390
Collecting socket server metrics during shutdown may throw NullPointerException
Author: Xavier Léauté <xavier@confluent.io>
Reviewers: Ismael Juma <ismael@juma.me.uk>
Closes#2221 from xvrl/fix-metrics-npe-on-shutdown
Author: Dong Lin <lindong28@gmail.com>
Reviewers: Ismael Juma <ismael@juma.me.uk>, Jiangjie Qin <becket.qin@gmail.com>
Closes#2170 from lindong28/KAFAK-4445
Fix possible integer overflow.
Author: Kim Christensen <kich@mvno.dk>
Reviewers: Ismael Juma <ismael@juma.me.uk>
Closes#2200 from kichristensen/MiscalculatedOffsetRetention
Author: Jason Gustafson <jason@confluent.io>
Reviewers: Ismael Juma, Jun Rao, Jiangjie Qin, Guozhang Wang
Closes#2195 from hachikuji/KAFKA-3994-linked-queue
NPE was caused by `log.logSegments.toArray` resulting in array containing `null` values. The exact reason still remains somewhat a mystery to me, but it seems that the culprit is `JavaConverters` in combination with concurrent data structure access.
Here's a simple code example to prove that:
```scala
import java.util.concurrent.ConcurrentSkipListMap
// Same as `JavaConversions`, but allows explicit conversions via `asScala`/`asJava` methods.
import scala.collection.JavaConverters._
case object Value
val m = new ConcurrentSkipListMap[Int, Value.type]
new Thread { override def run() = { while (true) m.put(9000, Value) } }.start()
new Thread { override def run() = { while (true) m.remove(9000) } }.start()
new Thread { override def run() = { while (true) { println(m.values.asScala.toArray.headOption) } } }.start()
```
Running the example will occasionally print `Some(null)` indicating that there's something shady going on during `toArray` conversion.
`null`s magically disappear by making the following change:
```diff
- println(m.values.asScala.toArray.headOption)
+ println(m.values.asScala.toSeq.headOption)
```
Author: Anton Karamanov <ataraxer@yandex-team.ru>
Reviewers: Ismael Juma <ismael@juma.me.uk>, Guozhang Wang <wangguoz@gmail.com>
Closes#2204 from ataraxer/KAFKA-4205
Also:
* Make all implementations of `Time` thread-safe as they are accessed from multiple threads in some cases.
* Change default implementation of `MockTime` to use two separate variables for `nanoTime` and `currentTimeMillis` as they have different `origins`.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Ewen Cheslack-Postava <ewen@confluent.io>, Shikhar Bhushan <shikhar@confluent.io>, Jason Gustafson <jason@confluent.io>, Eno Thereska <eno.thereska@gmail.com>, Damian Guy <damian.guy@gmail.com>
Closes#2095 from ijuma/kafka-2247-consolidate-time-interfaces
Author: Alexey Ozeritsky <aozeritsky@yandex-team.ru>
Reviewers: Ismael Juma <ismael@juma.me.uk>, Jason Gustafson <jason@confluent.io>
Closes#2125 from resetius/KAFKA-4399
Removed stale comment left behind, minor fixes (UpdateMetadataRequest instead of MetadataUpdateRequest) and remove redundant comments.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Jiangjie (Becket) Qin <becket.qin@gmail.com>, Dong Lin <lindong28@gmail.com>
Closes#2194 from ijuma/kafka-4443-minor-follow-up
The last patch submitted by MayureshGharat (back in Dec 15) has been rebased to the latest trunk. I took care of a couple of test failures (MetricsTest) along the way. jjkoshy , granders , avianey , you may be interested in this PR.
Author: Sumant Tambe <sutambe@yahoo.com>
Author: Mayuresh Gharat <mgharat@mgharat-ld1.linkedin.biz>
Author: MayureshGharat <gharatmayuresh15@gmail.com>
Reviewers: Joel Koshy <jjkoshy.w@gmail.com>
Closes#1664 from sutambe/async-delete-topic
Author: Dong Lin <lindong28@gmail.com>
Reviewers: Jiangjie Qin <becket.qin@gmail.com>, Jun Rao <junrao@gmail.com>
Closes#2168 from lindong28/KAFKA-4443
Author: Jun He <jun.he@airbnb.com>
Reviewers: Jiangjie (Becket) Qin <becket.qin@gmail.com>, Jun Rao <junrao@gmail.com>, Ismael Juma <ismael@juma.me.uk>
Closes#2127 from jun-he/KAFKA-4384
- bug-fix follow up
- Resetter fails if no intermediate topic is used because seekToEnd() commit ALL partitions to EOL
Author: Matthias J. Sax <matthias@confluent.io>
Reviewers: Michael G. Noll, Roger Hoover, Guozhang Wang
Closes#2138 from mjsax/kafka-4331-streams-resetter-bugfix
Fixes static initialization order dependency between KafkaConfig and LogConfig. jjkoshy please take a look.
Author: Sumant Tambe <sutambe@yahoo.com>
Reviewers: Ismael Juma <ismael@juma.me.uk>
Closes#2120 from sutambe/logconfig-static-init
Author: MayureshGharat <gharatmayuresh15@gmail.com>
Reviewers: Jiangjie Qin <becket.qin@gmail.com>, Jason Gustafson <jason@confluent.io>, Ismael Juma <ismael@juma.me.uk>, Jun Rao <junrao@gmail.com>
Closes#2116 from MayureshGharat/KAFKA-4362
Process requests received from channels before they were closed. For consumers, wait for coordinator requests to complete before returning from close.
Author: Rajini Sivaram <rajinisivaram@googlemail.com>
Reviewers: Jason Gustafson <jason@confluent.io>, Jun Rao <junrao@gmail.com>, Ismael Juma <ismael@juma.me.uk>
Closes#1836 from rajinisivaram/KAFKA-3703
Author: Dong Lin <lindong28@gmail.com>
Reviewers: Onur Karaman <okaraman@linkedin.com>, Jun Rao <junrao@gmail.com>, Ismael Juma <ismael@juma.me.uk>
Closes#2148 from lindong28/KAFKA-4420
JavaConversions are deprecated in 2.12 in favour of JavaConverters.
Author: Bernard Leach <leachbj@bouncycastle.org>
Reviewers: Ismael Juma <ismael@juma.me.uk>
Closes#2101 from leachbj/4377-java-converters
I think we should add a space here, otherwise the two words will join together.
And replace the host string with a constant, otherwise when I need to modify the host, I need to modify several files.
Author: 郑谦00117553 <00117553@zte.intra>
Reviewers: Guozhang Wang <wangguoz@gmail.com>
Closes#2132 from ZhengQian1/trunk
The consumer can deadlock on shutdown if a topic event fires during shutdown. The shutdown acquires the rebalance lock and then the topic-event-watcher lock. The topic event watcher acquires these in the reverse order. Shutdown should not need to acquire the topic-event-watcher’s lock - all it does is unsubscribes from topic events.
Author: Joel Koshy <jjkoshy@gmail.com>
Reviewers: Jiangjie Qin <becket.qin@gmail.com>
Closes#2129 from jjkoshy/KAFKA-4409
This bug was recently introduced via 2959bc2ad.
Author: Himani Arora <1himani.arora@gmail.com>
Reviewers: Ismael Juma <ismael@juma.me.uk>
Closes#2091 from himani1/minor_fix
huxi
Edward Ribeiro
Jeff Widman
Dustin Cote
Rajini Sivaram
Vahid Hashemian
Manikumar Reddy O
Dong Lin
Ismael Juma
Jason Gustafson
Rekha Joshi
Json Tu
Ashish Singh
Michael Schiff
Jiangjie Qin
Xavier Léauté
Kim Christensen
Anton Karamanov
Alexey Ozeritsky
Mayuresh Gharat
Jun He
Matthias J. Sax
Sumant Tambe
Bernard Leach
Qian Zheng
Joel Koshy
Tu Yang
Himani Arora