Author: Bill Bejeck <bill@confluent.io>
Reviewers: Matthias J. Sax <matthias@confluent.io>, Michael G. Noll <michael@confluent.io>, Damian Guy <damian.guy@gmail.com>
Closes#3994 from bbejeck/KAFKA-5985_document_need_to_close_iterators
This is less likely to break custom metric reporters and since the method
is deprecated, people will be warned about this potential issue.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Ewen Cheslack-Postava <ewen@confluent.io>, Xavier Léauté <xavier@confluent.io>
Closes#3996 from ijuma/avoid-exception-in-measurable-value
Adds new metrics to support health checks:
1. Error rates for each request type, per-error code
2. Request size and temporary memory size
3. Message conversion rate and time
4. Successful and failed authentication rates
5. ZooKeeper latency and status
6. Client version
Author: Rajini Sivaram <rajinisivaram@googlemail.com>
Reviewers: Ismael Juma <ismael@juma.me.uk>
Closes#3705 from rajinisivaram/KAFKA-5746-new-metrics
This is the polished version.
1. The old merge() method in StreamsBuilder has been removed,
2. The merge() method in KStreamBuilder was changed so that it would use the single variable argument
rather than several variable arguments in the KStreamImpl implementation
3. The merge() method in KStream has been declared as final and tests have been added to test correctness.
Author: Richard Yu <richardyu@Richards-Air.attlocal.net>
Reviewers: Matthias J. Sax <matthias@confluent.io>, Bill Bejeck <bill@confluent.io>, Guozhang Wang <wangguoz@gmail.com>, Damian Guy <damian.guy@gmail.com>
Closes#3916 from ConcurrencyPractitioner/trunk
Make documentation consistent across methods and throw
IllegalStateException instead of IllegalArgumentException in
some cases.
Also include a couple of minor fixes in upgrade.html.
Author: Dong Lin <lindong28@gmail.com>
Reviewers: Ismael Juma <ismael@juma.me.uk>
Closes#3781 from lindong28/minor-admin-client-comment
Author: Matthias J. Sax <matthias@confluent.io>
Reviewers: Michael G. Noll <michael@confluent.io>, Guozhang Wang <wangguoz@gmail.com>
Closes#3949 from mjsax/minor-update-streams-quickstart
Author: Mickael Maison <mickael.maison@gmail.com>
Reviewers: Ismael Juma <ismael@juma.me.uk>, Jason Gustafson <jason@confluent.io>
Closes#3036 from mimaison/KAFKA-3356
Author: Manjula K <manjula@kafka-summit.org>
Author: manjuapu <manjula@confluent.io>
Reviewers: Guozhang Wang <wangguoz@gmail.com>
Closes#3950 from manjuapu/customer-logo-stream
Author: Manjula K <manjula@kafka-summit.org>
Author: manjuapu <manjula@confluent.io>
Reviewers: Guozhang Wang <wangguoz@gmail.com>
Closes#3940 from manjuapu/customer-logo-stream
Developed with edoardocomar
Author: Mickael Maison <mickael.maison@gmail.com>
Reviewers: Edoardo Comar <ecomar@uk.ibm.com>, Rajini Sivaram <rajinisivaram@googlemail.com>
Closes#3906 from mimaison/KAFKA-5735
Author: Damian Guy <damian.guy@gmail.com>
Reviewers: Matthias J. Sax <matthias@confluent.io>, Guozhang Wang <wangguoz@gmail.com>
Closes#3921 from dguy/minor-kip-173-docs
Author: Florian Hussonnois <florian.hussonnois@gmail.com>
Reviewers: Matthias J. Sax <matthias@confluent.io>, Guozhang Wang <wangguoz@gmail.com>
Closes#3811 from fhussonnois/KAFKA-5839
minor fixes
I found several minor issues with the Kafka Streams tutorial:
* Some typos
* "As shown above, it illustrate that the constructed ..." instead of "As shown above, it illustrate_s_ that the constructed ..."
* "same as Pipe.java below" instead of "same as Pipe.java _above_"
* Wrong class name in the `LineSplit` example
* Incorrect imports for the code examples
* Missing `import org.apache.kafka.streams.kstream.KStream;` in `LineSplit` and `WordCount` example
* Unnecessary (and potentially confusing) split by whitespaces in the `WorkCount` class (the split into words happened already in `LineSplit`)
Author: Jakub Scholz <www@scholzj.com>
Reviewers: Matthias J. Sax <matthias@confluent.io>, Guozhang Wang <wangguoz@gmail.com>
Closes#3883 from scholzj/stream-tutorial-typos
Add overloads of `filter`, `filterNot`, `mapValues` that take `Materialized` as a param to `KTable`. Deprecate overloads using `storeName` and `storeSupplier`
Author: Damian Guy <damian.guy@gmail.com>
Reviewers: Matthias J. Sax <matthias@confluent.io>, Guozhang Wang <wangguoz@gmail.com>
Closes#3807 from dguy/ktable-filter-map
Part of KIP-182
- Add `Printed` class and `KStream#print(Printed)`
- deprecate all other `print` and `writeAsText` methods
Author: Damian Guy <damian.guy@gmail.com>
Reviewers: Bill Bejeck <bill@confluent.io>, Matthias J. Sax <matthias@confluent.io>, Guozhang Wang <wangguoz@gmail.com>
Closes#3768 from dguy/kafka-5652-printed
Add the `WindowedKStream` interface and implementation of methods that don't require `Materialized`
Author: Damian Guy <damian.guy@gmail.com>
Reviewers: Bill Bejeck <bill@confluent.io>, Matthias J. Sax <matthias@confluent.io>, Guozhang Wang <wangguoz@gmail.com>
Closes#3809 from dguy/kgrouped-stream-windowed-by
Part of KIP-182
- Add `StateStoreBuilder` interface and `WindowStateStoreBuilder`, `KeyValueStateStoreBuilder`, and `SessionStateStoreBuilder` implementations
- Add `StoreSupplier`, `WindowBytesStoreSupplier`, `KeyValueBytesStoreSupplier`, `SessionBytesStoreSupplier` interfaces and implementations
- Add new methods to `Stores` to create the newly added `StoreSupplier` and `StateStoreBuilder` implementations
- Update `Topology` and `InternalTopology` to use the interfaces
Author: Damian Guy <damian.guy@gmail.com>
Reviewers: Bill Bejeck <bill@confluent.io>, Matthias J. Sax <matthias@confluent.io>, Guozhang Wang <wangguoz@gmail.com>
Closes#3767 from dguy/kafka-5650
Add the `Produced` class and `KStream` overloads that use it:
`KStream#to(String, Produced)`
`KStream#through(String, Produced)`
Deprecate all other to and through methods accept the single param methods that take a topic param
Author: Damian Guy <damian.guy@gmail.com>
Reviewers: Bill Bejeck <bill@confluent.io>, Matthias J. Sax <matthias@confluent.io>, Guozhang Wang <wangguoz@gmail.com>
Closes#3770 from dguy/kafka-5652-produced
The iterator interface usage has some examples missing explicit close operation after usage. We should remind the user to do so because un-closed iterator will leave the underlying file descriptor open, thus eating up memory.
guozhangwang Ishiihara
Author: Boyang Chen <bychen@pinterest.com>
Author: Boyang Chen <bchen11@outlook.com>
Reviewers: Guozhang Wang <wangguoz@gmail.com>
Closes#3714 from abbccdda/add_iterator_close_on_doc
minor fixes
Author: Guozhang Wang <wangguoz@gmail.com>
Reviewers: Bill Bejeck <bill@confluent.io>, Matthias J. Sax <matthias@confluent.io>, Damian Guy <damian.guy@gmail.com>
Closes#3796 from guozhangwang/kip-138-minor-renames
Part of KIP-182
- Add the `Serialized` class
- implement overloads of `KStream#groupByKey` and KStream#groupBy`
- deprecate existing methods that have more than default arguments
Author: Damian Guy <damian.guy@gmail.com>
Reviewers: Bill Bejeck <bill@confluent.io>, Matthias J. Sax <matthias@confluent.io>, Guozhang Wang <wangguoz@gmail.com>
Closes#3772 from dguy/kafka-5817
Author: Matthias J. Sax <matthias@confluent.io>
Reviewers: Sharon Liu <sharonliu.cup@gmail.com>, Damian Guy <damian.guy@gmail.com>, Guozhang Wang <wangguoz@gmail.com>
Closes#3787 from mjsax/kafka-5823-kip120-docs
Added a simple table of contents for the developer section.
Author: Eno Thereska <eno.thereska@gmail.com>
Reviewers: Damian Guy <damian.guy@gmail.com>
Closes#3760 from enothereska/minor-docs-toc
1. Core concepts (added the stream time definition), upgrade guide and developer guide.
2. Related Java docs changes.
Author: Guozhang Wang <wangguoz@gmail.com>
Reviewers: Bill Bejeck <bill@confluent.io>, Damian Guy <damian.guy@gmail.com>
Closes#3732 from guozhangwang/KMinor-kip138-docs
1. Add upgrade section for 1.0.0, including Streams API changes section.
2. Add metrics name changes section.
Author: Guozhang Wang <wangguoz@gmail.com>
Reviewers: Eno Thereska <eno.thereska@gmail.com>, Damian Guy <damian.guy@gmail.com>
Closes#3687 from guozhangwang/KMinor-metrics-upgrade-guide
Needs to come after https://github.com/apache/kafka/pull/3701
Originally reviewed as part of #3490.
Author: Eno Thereska <eno.thereska@gmail.com>
Reviewers: Damian Guy <damian.guy@gmail.com>
Closes#3711 from enothereska/minor-docs-stateless
Originally reviewed as part of https://github.com/apache/kafka/pull/3490.
Author: Eno Thereska <eno.thereska@gmail.com>
Reviewers: Damian Guy <damian.guy@gmail.com>
Closes#3701 from enothereska/minor-docs-create-source-streams
0. Minor fixes on the existing examples to merge all on a single input topic; also do not use `common.utils.Exit` as it is for internal usage only.
1. Add the archetype project for the quickstart. Steps to try it out:
a. `mvn install` on the quickstart directory.
b. `mvn archetype:generate \
-DarchetypeGroupId=org.apache.kafka \
-DarchetypeArtifactId=streams-quickstart-java \
-DarchetypeVersion=1.0.0-SNAPSHOT \
-DgroupId=streams-quickstart \
-DartifactId=streams-quickstart \
-Dversion=0.1 \
-Dpackage=StreamsQuickstart \
-DinteractiveMode=false` at any directory to create the project.
c. build the streams jar with version `1.0.0-SNAPSHOT` to local maven repository with `./gradlew installAll`; `cd streams-quickstart; mvn clean package`
d. create the input / output topics, start the console producer and consumer.
e. start the program: `mvn exec:java -Dexec.mainClass=StreamsQuickstart.Pipe/LineSplit/WordCount`.
f. type data on console producer and observe data on console consumer.
Author: Guozhang Wang <wangguoz@gmail.com>
Reviewers: Damian Guy <damian.guy@gmail.com>, Bill Bejeck <bbejeck@gmail.com>, Ewen Cheslack-Postava <me@ewencp.org>, Eno Thereska <eno.thereska@gmail.com>
Closes#3630 from guozhangwang/KMinor-streams-quickstart-tutorial
Author: Bill Bejeck <bill@confluent.io>
Reviewers: Eno Thereska <eno.thereska@gmail.com>, Damian Guy <damian.guy@gmail.com>
Closes#3629 from bbejeck/docs-updates-for-kip-167
These handlers were previously used on the broker to handle uncaught exceptions, but now the broker users the new Java request objects exclusively.
Author: Jason Gustafson <jason@confluent.io>
Reviewers: Ismael Juma <ismael@juma.me.uk>
Closes#3646 from hachikuji/remove-old-request-error-handlers
This is important because Bits.reserveMemory calls System.gc() hoping to free native
memory in order to avoid throwing an OutOfMemoryException. This call is currently
a no-op due to -XX:+DisableExplicitGC.
It's worth mentioning that -XX:MaxDirectMemorySize can be used to increase the
amount of native memory available for allocation of direct byte buffers.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Apurva Mehta <apurva@confluent.io>, Manikumar Reddy <manikumar.reddy@gmail.com>, Jason Gustafson <jason@confluent.io>
Closes#3371 from ijuma/kafka-5470-explicit-gc-invokes-concurrent
Author: Damian Guy <damian.guy@gmail.com>
Reviewers: Guozhang Wang <wangguoz@gmail.com>, Eno Thereska <eno.thereska@gmail.com>
Closes#3604 from dguy/memory-management-docs
A couple of fixes to metric names to match the KIP
- Removed extra strings in the metric names that are already in the tags
- add a separate metric for "all"
Author: Eno Thereska <eno.thereska@gmail.com>
Reviewers: Guozhang Wang <wangguoz@gmail.com>
Closes#3491 from enothereska/hotfix-metric-names
1. Remove rest deprecation warnings in streams:jar.
2. Consolidate all unit test classes' reflections to access internal topology builder from packages other than `o.a.k.streams`. We need to refactor the hierarchies of StreamTask, StreamThread and KafkaStreams to remove these hacky reflections.
3. Minor fixes such as reference path, etc.
4. Minor edits on web docs for the describe function under developer-guide.
Author: Guozhang Wang <wangguoz@gmail.com>
Reviewers: Bill Bejeck <bill@confluent.io>, Ismael Juma <ismael@juma.me.uk>, Damian Guy <damian.guy@gmail.com>
Closes#3603 from guozhangwang/K5671-followup-comments
Author: Matthias J. Sax <matthias@confluent.io>
Reviewers: Damian Guy <damian.guy@gmail.com>, Bill Bejeck <bill@confluent.io>, Guozhang Wang <wangguoz@gmail.com>
Closes#3602 from mjsax/kafka-5671-add-streamsbuilder
Bill Bejeck
Ismael Juma
Rajini Sivaram
Manikumar Reddy
Manjula K
Richard Yu
Dong Lin
Matthias J. Sax
Mickael Maison
Damian Guy
manjuapu
Florian Hussonnois
Jakub Scholz
Guozhang Wang
Boyang Chen
James Cheng
Eno Thereska
Michal Borowiecki
Jason Gustafson