With KIP-175 there are a number of consumer group command options that can be used to describe a consumer group.
This PR updates the documentation on consumer group command to mention those options.
A new mechanism was added recently to the Metrics framework to make it easier to generate the documentation. It uses a registry with a MetricsNameTemplate for each metric, and then those templates are used when creating the actual metrics. The metrics framework provides utilities that can generate the HTML documentation from the registry of templates.
This change moves the recently-added Connect metrics over to use these templates and to then generate the metric documentation for Connect.
This PR is based upon #3975 and can be rebased once that has been merged.
Author: Randall Hauch <rhauch@gmail.com>
Reviewers: Ewen Cheslack-Postava <ewen@confluent.io>
Closes#3987 from rhauch/kafka-5990
And remove the deprecated `ConsumerOffsetChecker` example.
Author: Vahid Hashemian <vahidhashemian@us.ibm.com>
Reviewers: Jason Gustafson <jason@confluent.io>
Closes#3405 from vahidhashemian/KAFKA-5495
A couple of fixes to metric names to match the KIP
- Removed extra strings in the metric names that are already in the tags
- add a separate metric for "all"
Author: Eno Thereska <eno.thereska@gmail.com>
Reviewers: Guozhang Wang <wangguoz@gmail.com>
Closes#3491 from enothereska/hotfix-metric-names
This contribution is my original work and I license the work to the project under the project's open source license.
Author: Jeff Klukas <jeff@klukas.net>
Reviewers: Ismael Juma <ismael@juma.me.uk>
Closes#3249 from jklukas/docs-no-more-yammer-metrics-in-client
Author: Vahid Hashemian <vahidhashemian@us.ibm.com>
Reviewers: Ismael Juma <ismael@juma.me.uk>
Closes#3415 from vahidhashemian/doc/make_topic_config_a_section
Operations documentation should guide user to employ `kafka-configs.sh` to add/remove configs for a topic.
Author: huxihx <huxi_2b@hotmail.com>
Reviewers: Damian Guy <damian.guy@gmail.com>
Closes#3429 from huxihx/KAFKA-5508_Doc_for_altering_topics
Author: Guozhang Wang <wangguoz@gmail.com>
Reviewers: Derrick Or <derrickor@gmail.com>, Ismael Juma <ismael@juma.me.uk>
Closes#3214 from guozhangwang/KMinor-doc-java-brush
Author: Rajini Sivaram <rajinisivaram@googlemail.com>
Reviewers: Ismael Juma <ismael@juma.me.uk>, Jun Rao <junrao@gmail.com>
Closes#3169 from rajinisivaram/MINOR-producer-metrics
Autogenerate docs for the Consumer Fetcher's metrics. This is a smaller subset of the original PR https://github.com/apache/kafka/pull/1202.
CC ijuma benstopford hachikuji
Author: James Cheng <jylcheng@yahoo.com>
Reviewers: Ismael Juma <ismael@juma.me.uk>, Guozhang Wang <wangguoz@gmail.com>
Closes#2993 from wushujames/fetcher_metrics_docs
1. Update value for queued.max.requests to 500
2. Removed invalid config 'controller.message.queue.size'
3. Removed flush configs including 'log.flush.interval.messages', 'log.flush.interval.ms' and 'log.flush.scheduler.interval.ms'
Author: huxi <huxi@zhenrongbao.com>
Reviewers: Grant Henke <granthenke@gmail.com>, Jun Rao <junrao@gmail.com>
Closes#2490 from amethystic/kafka4727_server_config_doc_update
- Seperate Streams documentation out to a standalone page.
- Setup templates to use handlebars.js
- Create template variables to swap in frequently updated values like version number from a single file templateData.js
Author: Derrick Or <derrickor@gmail.com>
Reviewers: Guozhang Wang <wangguoz@gmail.com>
Closes#2245 from derrickdoo/docTemplates
Removed a non-existing reference in table of contents and fixed some section numbers.
Author: Vahid Hashemian <vahidhashemian@us.ibm.com>
Reviewers: Jason Gustafson <jason@confluent.io>
Closes#2004 from vahidhashemian/doc/fix_toc
And improve readability by adding proper punctuations.
Author: Vahid Hashemian <vahidhashemian@us.ibm.com>
Reviewers: Jason Gustafson <jason@confluent.io>
Closes#2002 from vahidhashemian/doc/fix_typos
This is to imply that the Java consumer/producer are the recommended consumer/producer now.
Author: Vahid Hashemian <vahidhashemian@us.ibm.com>
Reviewers: Jason Gustafson <jason@confluent.io>
Closes#1921 from vahidhashemian/KAFKA-3697
Details in KIP-55.
Author: Rajini Sivaram <rajinisivaram@googlemail.com>
Reviewers: Jun Rao <junrao@gmail.com>
Closes#1847 from rajinisivaram/KAFKA-4079
added new consumer metrics section
refactored common metrics into new section
updated TOC
Author: Kaufman Ng <kaufman@confluent.io>
Reviewers: Jason Gustafson <jason@confluent.io>, Ewen Cheslack-Postava <ewen@confluent.io>
Closes#1361 from coughman/KAFKA-3479-consumer-metrics-doc
This contribution is my original work, and I license the work to the project under the project's open source license.
Author: Samuel Taylor <staylor@square-root.com>
Reviewers: Ismael Juma <ismael@juma.me.uk>
Closes#1630 from ssaamm/trunk
I've updated the ops documentation with information on using the XFS filesystem, based on LinkedIn's testing (and subsequent switch from EXT4).
I've also added some information to clarify the potential risk to the suggested EXT4 options (again, based on my experience with a multiple broker failure situation).
Author: Todd Palino <tpalino@linkedin.com>
Reviewers: Sriharsha Chintalapani <harsha@hortonworks.com>, Dana Powers <dana.powers@gmail.com>, Ismael Juma <ismael@juma.me.uk>
Closes#1605 from toddpalino/trunk
Adding sizing recommendations for file descriptors to the ops guide.
Author: Dustin Cote <dustin@confluent.io>
Author: Dustin Cote <dustin@dustins-mbp.attlocal.net>
Reviewers: Gwen Shapira
Closes#1353 from cotedm/KAFKA-3683 and squashes the following commits:
8120318 [Dustin Cote] Adding file descriptor sizing recommendations
0908aa9 [Dustin Cote] Merge https://github.com/apache/kafka into trunk
32315e4 [Dustin Cote] Merge branch 'trunk' of https://github.com/cotedm/kafka into trunk
13309ed [Dustin Cote] Update links for new consumer API
4dcffc1 [Dustin Cote] Update links for new consumer API
Pull request to update the consumer API links in the docs.
Author: Dustin Cote <dustin@dustins-mbp.attlocal.net>
Reviewers: Ismael Juma <ismael@juma.me.uk>
Closes#1331 from cotedm/KAFKA-3666
This PR fixes 8 typos in HTML files of `docs` module. I wrote explicitly here since Github sometimes does not highlight the corrections on long lines correctly.
- docs/api.html: compatability => compatibility
- docs/connect.html: simultaneoulsy => simultaneously
- docs/implementation.html: LATIEST_TIME => LATEST_TIME, nPartions => nPartitions
- docs/migration.html: Decomission => Decommission
- docs/ops.html: stoping => stopping, ConumserGroupCommand => ConsumerGroupCommand, youre => you're
Author: Dongjoon Hyun <dongjoon@apache.org>
Reviewers: Ismael Juma
Closes#1138 from dongjoon-hyun/KAFKA-3461
This fixes an issue with long topic names by considering, during topic
validation, the '-' and the partition id that is appended to the log
folder created for each topic partition.
Author: Vahid Hashemian <vahidhashemian@us.ibm.com>
Reviewers: Gwen Shapira, Grant Henke
Closes#898 from vahidhashemian/KAFKA-3219
https://issues.apache.org/jira/browse/KAFKA-1476
Let me know if these kind of contributions should have their own requisite JIRA opened in advance.
Cheers..
Author: Christian Posta <christian.posta@gmail.com>
Reviewers: Gwen Shapira
Closes#945 from christian-posta/ceposta-tidy-up-consumer-groups-describe
…list is not supported for new consumers
Author: Ashish Singh <asingh@cloudera.com>
Reviewers: Grant Henke <granthenke@gmail.com>, Ewen Cheslack-Postava <ewen@confluent.io>
Closes#923 from SinghAsDev/KAFKA-3243
Correct the text that said that you can't change the replication factor of a topic.
Author: James Cheng <jylcheng@yahoo.com>
Reviewers: Ismael Juma
Closes#881 from wushujames/KAFKA-3216
Vahid Hashemian
Rajini Sivaram
Manikumar Reddy
Randall Hauch
James Cheng
Eno Thereska
Dong Lin
Jeff Klukas
huxihx
Guozhang Wang
Eno Thereska
Alex Loddengaard
huxi
Scott Ferguson
Derrick Or
Ismael Juma
Ben Stopford
Andrew Otto
Kaufman Ng
Samuel Taylor
Todd Palino
Thanasis Katsadas
Filipe Azevedo
Dustin Cote
Dustin Cote
Dongjoon Hyun
Flavio Junqueira
Christian Posta
Ashish Singh