src-kafka

Commit Graph

Author	SHA1	Message	Date
Ismael Juma	c1694833d5	KAFKA-3490; Multiple version support for ducktape performance tests Author: Ismael Juma <ismael@juma.me.uk> Author: Geoff Anderson <geoff@confluent.io> Reviewers: Geoff Anderson <geoff@confluent.io>, Ewen Cheslack-Postava <ewen@confluent.io> Closes #1173 from ijuma/kafka-3490-multiple-version-support-perf-tests	9 years ago
Anna Povzner	f6e35dec9b	KAFKA-3201: Added rolling upgrade system tests from 0.8 and 0.9 to 0.10 Three main tests: 1. Setup: Producer (0.8) → Kafka Cluster → Consumer (0.8) First rolling bounce: Set inter.broker.protocol.version = 0.8 and message.format.version = 0.8 Second rolling bonus, use latest (default) inter.broker.protocol.version and message.format.version 2. Setup: Producer (0.9) → Kafka Cluster → Consumer (0.9) First rolling bounce: Set inter.broker.protocol.version = 0.9 and message.format.version = 0.9 Second rolling bonus, use latest (default) inter.broker.protocol.version and message.format.version 3. Setup: Producer (0.9) → Kafka Cluster → Consumer (0.9) First rolling bounce: Set inter.broker.protocol.version = 0.9 and message.format.version = 0.9 Second rolling bonus: use inter.broker.protocol.version = 0.10 and message.format.version = 0.9 Plus couple of variations of these tests using old/new consumer and no compression / snappy compression. Author: Anna Povzner <anna@confluent.io> Reviewers: Ewen Cheslack-Postava <ewen@confluent.io> Closes #980 from apovzner/kafka-3201-02	9 years ago
Geoff Anderson	7073fa7efe	KAFKA-2258: add failover to mirrormaker test This PR adds failover to simple end to end mirror maker test Marked as WIP for 2 reasons: - We may want to add a couple more test cases where kafka is being used to store offsets - There appears to be a test failure in the hard failover case Author: Geoff Anderson <geoff@confluent.io> Reviewers: Ewen Cheslack-Postava Closes #427 from granders/KAFKA-2258-mirrormaker-test	9 years ago
Rajini Sivaram	98db5ea94f	KAFKA-2644; Run relevant ducktape tests with SASL_PLAINTEXT and SASL_SSL Run sanity check, replication tests and benchmarks with SASL/Kerberos using MiniKdc. Author: Rajini Sivaram <rajinisivaram@googlemail.com> Reviewers: Geoff Anderson <geoff@confluent.io>, Jun Rao <junrao@gmail.com> Closes #358 from rajinisivaram/KAFKA-2644	9 years ago
Geoff Anderson	e6b343302f	KAFKA-1888: rolling upgrade test ewencp gwenshap This needs some refactoring to avoid the duplicated code between replication test and upgrade test, but in shape for initial feedback. I'm interested in feedback on the added `KafkaConfig` class and `kafka_props` file. This addition makes it: - easier to attach different configs to different nodes (e.g. during broker upgrade process) - easier to reason about the configuration of a particular node Notes: - in the default values in the KafkaConfig class, I removed many properties which were in kafka.properties before. This is because most of those properties were set to what is already the default value. - when running non-trunk VerifiableProducer, I append the trunk tools jar to the classpath, and run it with the non-trunk kafka-run-class.sh script Author: Geoff Anderson <geoff@confluent.io> Reviewers: Dong Lin, Ewen Cheslack-Postava Closes #229 from granders/KAFKA-1888-upgrade-test	9 years ago
Rajini Sivaram	dd514b2bb8	KAFKA-2581: Run some existing ducktape tests with SSL Parametrize console consumer sanity test, replication tests and benchmarks tests to run with both PLAINTEXT and SSL. Author: Rajini Sivaram <rajinisivaram@googlemail.com> Reviewers: Geoff Anderson, Ewen Cheslack-Postava, Guozhang Wang Closes #271 from rajinisivaram/KAFKA-2581	9 years ago
Ashish Singh	a0ca8f6422	KAFKA-2573: Mirror maker system test hangs and eventually fails Author: Ashish Singh <asingh@cloudera.com> Reviewers: Geoff Anderson, Guozhang Wang Closes #234 from SinghAsDev/KAFKA-2573	9 years ago
Ashish Singh	b62f8ea43b	KAFKA-2531: Add Ducktape based tests for KafkaLog4jAppender Author: Ashish Singh <asingh@cloudera.com> Reviewers: Geoff Anderson, Edwerd Ribeiro, Ewen Cheslack-Postava, Gwen Shapira Closes #235 from SinghAsDev/KAFKA-2531	9 years ago
Ewen Cheslack-Postava	2fa8c74853	KAFKA-2377: Add basic system test for copycat using source and sink file connectors. Tests standalone mode by running separate source and sink connectors, catting data into the source file, and validating the output in the sink file. Restarts the service to verify that clean restarts will result in tasks resuming where they left off. Author: Ewen Cheslack-Postava <me@ewencp.org> Reviewers: Geoff Andreson, Gwen Shapira Closes #150 from ewencp/kafka-2377-copycat-system-test	9 years ago
Geoff Anderson	1d2ae89c5a	KAFKA-2439; Add MirrorMaker service class for system tests Added MirrorMaker service and a few corresponding sanity checks, as well as necessary config template files. A few additional updates to accomodate the change in wait_until from ducktape0.2.0->0.3.0 Author: Geoff Anderson <geoff@confluent.io> Reviewers: Ewen Cheslack-Postava, Gwen Shapira Closes #148 from granders/KAFKA-2439 and squashes the following commits: `c7c3ebd` [Geoff Anderson] MirrorMaker now can run as multi-node service. Added kill -9 to various clean_node methods. `1e806f2` [Geoff Anderson] Various cleanups per review. `1b4b049` [Geoff Anderson] Added MirrorMaker service and a few corresponding sanity checks, as well as necessary config template files. A few additional updates to accomodate the change in wait_until from ducktape0.2.0->0.3.0	9 years ago
Geoff Anderson	be633a713e	KAFKA-2408: ConsoleConsumerService direct log output to file console consumer writes to System.out, while (some) log4j loggers operate in other threads. This occasionally led to funky interleaved output which disrupted parsing of consumed messages by ConsoleConsumerService, leading to spurious test failures. This fix directs log output to a separate file. Author: Geoff Anderson <geoff@confluent.io> Reviewers: Ewen Cheslack-Postava Closes #123 from granders/KAFKA-2408 and squashes the following commits: `247b0e0` [Geoff Anderson] Updated line counting to use wc -l `66d6f4f` [Geoff Anderson] lower -> uperrcase constants `e67f554` [Geoff Anderson] Changed incorrect license header `af67e01` [Geoff Anderson] Merged in upstream trunk `8f89044` [Geoff Anderson] Added another lifecycle check. Wait for log file to exist before exmaning contents. `521a84b` [Geoff Anderson] Updated console consumer to directo log output directly to file rather than stdout	9 years ago

11 Commits (346df72736d03acb4a6f5681e705ce3e330900a8)