This fixes test_producer_throughput with compression_type=snappy.
Also: added heap dump on out of memory error to `producer_performance.py` and corrected the upgrade note related to the change in buffer size for compression streams.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Gwen Shapira
Closes#1385 from ijuma/kafka-3713-test_producer_throughput-snappy-fail and squashes the following commits:
54c7962 [Ismael Juma] Correct upgrade note about buffer size for compression stream
515040b [Ismael Juma] Call `compressor.close()` to fix memory leak
5311e5b [Ismael Juma] Dump heap on out of memory error when running `producer_performance.py`
This is a follow up of KAKFA-3421 to update the connect developer guide to include the configuration validation. Also includes a couple of minor fixes.
Author: Liquan Pei <liquanpei@gmail.com>
Reviewers: Jason Gustafson <jason@confluent.io>, Ewen Cheslack-Postava <ewen@confluent.io>
Closes#1366 from Ishiihara/connect-dev-doc
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Guozhang Wang <wangguoz@gmail.com>, Jun Rao <junrao@gmail.com>
Closes#1373 from ijuma/fix-producer-buffer-size-upgrade-note
Author: Jason Gustafson <jason@confluent.io>
Reviewers: Roger Hoover <roger.hoover@gmail.com>, Ismael Juma <ismael@juma.me.uk>, Ewen Cheslack-Postava <ewen@confluent.io>
Closes#1358 from hachikuji/KAFKA-3583
Author: Guozhang Wang <wangguoz@gmail.com>
Reviewers: Michael G. Noll <michael@confluent.io>, Ismael Juma <ismael@juma.me.uk>
Closes#1321 from guozhangwang/KStreamsJavaDoc
This contribution is my original work and I license the work under Apache 2.0.
Author: Dana Powers <dana.powers@gmail.com>
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Jun Rao <junrao@gmail.com>, Ismael Juma <ismael@juma.me.uk>
Closes#1212 from dpkp/KAFKA-3160
Pull request to update the consumer API links in the docs.
Author: Dustin Cote <dustin@dustins-mbp.attlocal.net>
Reviewers: Ismael Juma <ismael@juma.me.uk>
Closes#1331 from cotedm/KAFKA-3666
Documentation corresponding to KIP-43 - SASL/PLAIN and multiple mechanism support.
Author: Rajini Sivaram <rajinisivaram@googlemail.com>
Reviewers: Magnus Edenhill <apache_m@edenhill.se>, Jun Rao <junrao@gmail.com>
Closes#1232 from rajinisivaram/KAFKA-3517
This patch fixes all occurances of two consecutive 'the's in the code comments.
Author: Ishita Mandhan (imandhaus.ibm.com)
Author: Ishita Mandhan <imandha@us.ibm.com>
Reviewers: Guozhang Wang <wangguoz@gmail.com>
Closes#1240 from imandhan/typofixes
ewencp gwenshap Docs. I also tried to clean up some typos. However, it seems that the we don't have two words without space in between in the source yet they showed up as no space in between in the generated doc.
Author: Liquan Pei <liquanpei@gmail.com>
Reviewers: Ewen Cheslack-Postava <ewen@confluent.io>
Closes#1227 from Ishiihara/config-doc
This PR fixes 8 typos in HTML files of `docs` module. I wrote explicitly here since Github sometimes does not highlight the corrections on long lines correctly.
- docs/api.html: compatability => compatibility
- docs/connect.html: simultaneoulsy => simultaneously
- docs/implementation.html: LATIEST_TIME => LATEST_TIME, nPartions => nPartitions
- docs/migration.html: Decomission => Decommission
- docs/ops.html: stoping => stopping, ConumserGroupCommand => ConsumerGroupCommand, youre => you're
Author: Dongjoon Hyun <dongjoon@apache.org>
Reviewers: Ismael Juma
Closes#1138 from dongjoon-hyun/KAFKA-3461
Sentence was missing "as", minor grammar clean up.
Author: Paul Cavallaro <paulcavallaro@gmail.com>
Reviewers: Ewen Cheslack-Postava <ewen@confluent.io>
Closes#1151 from paulcavallaro/docs-fix
This fixes an issue with long topic names by considering, during topic
validation, the '-' and the partition id that is appended to the log
folder created for each topic partition.
Author: Vahid Hashemian <vahidhashemian@us.ibm.com>
Reviewers: Gwen Shapira, Grant Henke
Closes#898 from vahidhashemian/KAFKA-3219
This will need to be double-committed.
Author: Gwen Shapira <cshapi@gmail.com>
Reviewers: Ismael Juma <ismael@juma.me.uk>, Ewen Cheslack-Postava <ewen@confluent.io>
Closes#1107 from gwenshap/fix-doc-version
Please see https://cwiki.apache.org/confluence/display/KAFKA/KIP-36+Rack+aware+replica+assignment for the overall design.
The update to TopicMetadataRequest/TopicMetadataResponse will be done in a different PR.
Author: Allen Wang <awang@netflix.com>
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Ismael Juma <ismael@juma.me.uk>, Jason Gustafson <jason@confluent.io>, Grant Henke <granthenke@gmail.com>, Jun Rao <junrao@gmail.com>
Closes#132 from allenxwang/KAFKA-1215
Author: Manikumar reddy O <manikumar.reddy@gmail.com>
Reviewers: Ismael Juma <ismael@juma.me.uk>, Sriharsha Chintalapani <mail@harsha.io>
Closes#1054 from omkreddy/KAFKA-2551
Because protocol.html is going to be in its own page it needs the header and footer included.
Author: Grant Henke <granthenke@gmail.com>
Reviewers: Gwen Shapira
Closes#1043 from granthenke/protocol-docs-style
- Moves all generated docs under /docs/generated
- Generates docs for Protocol, Errors, and ApiKeys
- Adds new protocol.html page
Author: Grant Henke <granthenke@gmail.com>
Reviewers: Gwen Shapira
Closes#970 from granthenke/protocol-doc-wip
https://issues.apache.org/jira/browse/KAFKA-1476
Let me know if these kind of contributions should have their own requisite JIRA opened in advance.
Cheers..
Author: Christian Posta <christian.posta@gmail.com>
Reviewers: Gwen Shapira
Closes#945 from christian-posta/ceposta-tidy-up-consumer-groups-describe
Adjust the listeners property rather than the port. Following the original instructions would result in all of the brokers being started with the same listeners setting, and so fail to work.
Author: Duncan Sands <baldrick@free.fr>
Reviewers: Ismael Juma, Gwen Shapira
Closes#1002 from CunningBaldrick/quickstart
* Change `MessageFormat.writeTo` to take a `ConsumerRecord`
* Change `MessageReader.readMessage()` to use `ProducerRecord`
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Jun Rao <junrao@gmail.com>
Closes#972 from ijuma/kafka-3273-message-formatter-and-reader-resilient
…list is not supported for new consumers
Author: Ashish Singh <asingh@cloudera.com>
Reviewers: Grant Henke <granthenke@gmail.com>, Ewen Cheslack-Postava <ewen@confluent.io>
Closes#923 from SinghAsDev/KAFKA-3243
This PR includes a number of clean-ups:
* Code style
* Documentation wording improvements
* Efficiency improvements
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Jun Rao <junrao@gmail.com>
Closes#943 from ijuma/kafka-3259-kip-31-32-clean-ups
See KIP-31 and KIP-32 for details.
A few notes on the patch:
1. This patch implements KIP-31 and KIP-32. The patch includes features in both KAFKA-3025, KAFKA-3026 and KAFKA-3036
2. All unit tests passed.
3. The unit tests were run with new and old message format.
4. When message format conversion occurs during consumption, the consumer will not be able to detect the message size too large situation. I did not try to fix this because the situation seems rare and only happen during migration phase.
Author: Jiangjie Qin <becket.qin@gmail.com>
Author: Ismael Juma <ismael@juma.me.uk>
Author: Jiangjie (Becket) Qin <becket.qin@gmail.com>
Reviewers: Jason Gustafson <jason@confluent.io>, Anna Povzner <anna@confluent.io>, Ismael Juma <ismael@juma.me.uk>, Guozhang Wang <wangguoz@gmail.com>, Jun Rao <junrao@gmail.com>
Closes#764 from becketqin/KAFKA-3025
There are multi-byte characters In quickstart.html and security.html.
This PR will fix it.
Author: Sasaki Toru <sasakitoa@nttdata.co.jp>
Reviewers: Grant Henke
Closes#897 from sasakitoa/remove_multi_byte_character
Correct the text that said that you can't change the replication factor of a topic.
Author: James Cheng <jylcheng@yahoo.com>
Reviewers: Ismael Juma
Closes#881 from wushujames/KAFKA-3216
… is not specified correctly
Author: Grant Henke <granthenke@gmail.com>
Reviewers: Ewen Cheslack-Postava <ewen@confluent.io>
Closes#876 from granthenke/rebalance-doc
Set default to 1 instead of 1024, this matches the existing doc and feels like a better default value. Have run the unit tests with the change.
Author: Rajini Sivaram <rajinisivaram@googlemail.com>
Reviewers: Jason Gustafson, Ismael Juma, Grant Henke, Guozhang Wang
Closes#832 from rajinisivaram/KAFKA-3170
And added info about the krb5.conf file as we don't appear to mention that in the current docs
Author: Ben Stopford <benstopford@gmail.com>
Reviewers: Ismael Juma
Closes#625 from benstopford/security_docs
Simple fix, but important because the incorrect syntax causes the server not to start.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Grant Henke, Guozhang Wang
Closes#819 from ijuma/fix-jaas-comment-syntax
Log compaction is supported on compressed topics as of 0.9.0, so update the docs to reflect that.
Author: James Cheng <jylcheng@yahoo.com>
Reviewers: Ewen Cheslack-Postava <ewen@confluent.io>
Closes#807 from wushujames/KAFKA-3138
Ismael Juma
Liquan Pei
Jason Gustafson
Guozhang Wang
Ben Stopford
Vahid Hashemian
Ashish Singh
Ewen Cheslack-Postava
Kaufman Ng
Dana Powers
Dustin Cote
Rajini Sivaram
Dionysis Grigoropoulos
Ishita Mandhan
Dongjoon Hyun
Paul Cavallaro
Flavio Junqueira
Grant Henke
Gwen Shapira
Allen Wang
Jiangjie Qin
Manikumar reddy O
Christian Posta
Duncan Sands
Sasaki Toru
Jun Rao
James Cheng
Jakub Nowak