src-kafka

Commit Graph

Author	SHA1	Message	Date
Max Zheng	50ec82940d	MINOR: Switch to use AWS spot instances Pricing for m3.xlarge: On-Demand is at $0.266. Reserved is at about $0.16 (40% discount). And Spot is at $0.0627 (76% discount relative to On-Demand, or 60% discount relative to Reserved). Insignificant fluctuation in the past 3 months. Ran on branch builder and works as expected -- each worker is created using spot instances (https://jenkins.confluent.io/job/system-test-kafka-branch-builder/1982/console) This can be safely backported to 0.10.2 (tested using https://jenkins.confluent.io/job/system-test-kafka-branch-builder/1983/) Author: Max Zheng <maxzheng.os@gmail.com> Reviewers: Ewen Cheslack-Postava <ewen@confluent.io> Closes #5707 from maxzheng/minor-switch@trunk	6 years ago
Matthias J. Sax	ac53979647	MINOR: update AWS test setup guide Author: Matthias J. Sax <matthias@confluent.io> Reviewers: Joseph Rea <jrea@users.noreply.github.com>, Ewen Cheslack-Postava <ewen@confluent.io> Closes #2575 from mjsax/minor-update-system-test-readme	8 years ago
Matthias J. Sax	b62cccd078	MINOR: improve test README Author: Matthias J. Sax <matthias@confluent.io> Reviewers: Ewen Cheslack-Postava <ewen@confluent.io> Closes #3416 from mjsax/minor-aws	8 years ago
Geoff Anderson	801fee89d8	MINOR: cleanup apache license in python files ijuma As discussed in https://github.com/apache/kafka/pull/1645, this patch removes an extraneous line from several __init__.py files, and a few others as well Author: Geoff Anderson <geoff@confluent.io> Reviewers: Ismael Juma <ismael@juma.me.uk> Closes #1659 from granders/minor-cleanup-init-files	8 years ago
Shikhar Bhushan	5a97304170	MINOR: fix Bash shebang on vagrant/ scripts Author: Shikhar Bhushan <shikhar@confluent.io> Reviewers: Geoff Anderson <geoff@confluent.io>, Ewen Cheslack-Postava <ewen@confluent.io> Closes #1567 from shikhar/vagrant-scripts-shebang	8 years ago
Roger Hoover	694303a3da	MINOR: Fixes to AWS init script for testing A path was wrong in the script and in the documentation. Author: Roger Hoover <roger.hoover@gmail.com> Reviewers: Geoff Anderson <geoff@confluent.io>, Ewen Cheslack-Postava <ewen@confluent.io> Closes #1309 from theduderog/fix_aws_init	9 years ago
Roger Hoover	bf578f987b	KAFKA-3650: Fix vagrant download URL Author: Roger Hoover <roger.hoover@gmail.com> Reviewers: Ewen Cheslack-Postava <ewen@confluent.io> Closes #1308 from theduderog/fix_vagrant_dl	9 years ago
Geoff Anderson	c188a68e2b	MINOR: Add vagrant up wrapper for simple parallel bringup on aws The main impediment to bringing up aws machines in parallel using vagrant was the interaction between `vagrant-hostmanager` and `vagrant-aws`. If you disable hostmanager during the `up` phase, and run it after the cluster is up, parallel bringup is possible. The only caveat is that machines must be brought up in small-ish batches to prevent rate limit errors from AWS since `vagrant-aws` doesn't seem to have mechanisms to This PR: - disables `vagrant-hostmanager` during bringup - adds a wrapper script to make it convenient to bring machines up in batches on aws Author: Geoff Anderson <geoff@confluent.io> Reviewers: Ewen Cheslack-Postava <ewen@confluent.io> Closes #982 from granders/vagrant-disable-hostmanager	9 years ago
Eno Thereska	04ef9c3547	MINOR: Increased default EC2 instance size AWS instance size increased to m3.xlarge to allow all system tests to pass. ijuma ewencp have a look please. Author: Eno Thereska <eno.thereska@gmail.com> Reviewers: Gwen Shapira Closes #1046 from enothereska/minor-aws	9 years ago
Ewen Cheslack-Postava	8ee9846e01	MINOR: Fix typo in sample Vagrantfile.local for AWS system tests Author: Ewen Cheslack-Postava <me@ewencp.org> Reviewers: Gwen Shapira Closes #565 from ewencp/fix-aws-vagrantfile-local-example	9 years ago
Geoff Anderson	e43c9aff92	KAFKA-2276; KIP-25 initial patch Initial patch for KIP-25 Note that to install ducktape, do not use pip to install ducktape. Instead: ``` $ git clone gitgithub.com:confluentinc/ducktape.git $ cd ducktape $ python setup.py install ``` Author: Geoff Anderson <geoff@confluent.io> Author: Geoff <granders@gmail.com> Author: Liquan Pei <liquanpei@gmail.com> Reviewers: Ewen, Gwen, Jun, Guozhang Closes #70 from granders/KAFKA-2276 and squashes the following commits: `a62fb6c` [Geoff Anderson] fixed checkstyle errors `a70f0f8` [Geoff Anderson] Merged in upstream trunk. `8b62019` [Geoff Anderson] Merged in upstream trunk. `47b7b64` [Geoff Anderson] Created separate tools jar so that the clients package does not pull in dependencies on the Jackson JSON tools or argparse4j. `a9e6a14` [Geoff Anderson] Merged in upstream changes `d18db7b` [Geoff Anderson] fixed :rat errors (needed to add licenses) `321fdf8` [Geoff Anderson] Ignore tests/ and vagrant/ directories when running rat build task `795fc75` [Geoff Anderson] Merged in changes from upstream trunk. `1d93f06` [Geoff Anderson] Updated provisioning to use java 7 in light of KAFKA-2316 `2ea4e29` [Geoff Anderson] Tweaked README, changed default log collection behavior on VerifiableProducer `0eb6fdc` [Geoff Anderson] Merged in system-tests `69dd7be` [Geoff Anderson] Merged in trunk `4034dd6` [Geoff Anderson] Merged in upstream trunk `ede6450` [Geoff] Merge pull request #4 from confluentinc/move_muckrake `7751545` [Geoff Anderson] Corrected license headers `e6d532f` [Geoff Anderson] java 7 -> java 6 `8c61e2d` [Geoff Anderson] Reverted jdk back to 6 `f14c507` [Geoff Anderson] Removed mode = "test" from Vagrantfile and Vagrantfile.local examples. Updated testing README to clarify aws setup. `98b7253` [Geoff Anderson] Updated consumer tests to pre-populate kafka logs `e6a41f1` [Geoff Anderson] removed stray println `b15b24f` [Geoff Anderson] leftover KafkaBenchmark in super call `0f75187` [Geoff Anderson] Rmoved stray allow_fail. kafka_benchmark_test -> benchmark_test `f469f84` [Geoff Anderson] Tweaked readme, added example Vagrantfile.local `3d73857` [Geoff Anderson] Merged downstream changes `42dcdb1` [Geoff Anderson] Tweaked behavior of stop_node, clean_node to generally fail fast `7f7c3e0` [Geoff Anderson] Updated setup.py for kafkatest `c60125c` [Geoff Anderson] TestEndToEndLatency -> EndToEndLatency `4f476fe` [Geoff Anderson] Moved aws scripts to vagrant directory `5af88fc` [Geoff Anderson] Updated README to include aws quickstart `e5edf03` [Geoff Anderson] Updated example aws Vagrantfile.local `96533c3` [Geoff] Update aws-access-keys-commands `25a413d` [Geoff] Update aws-example-Vagrantfile.local `884b20e` [Geoff Anderson] Moved a bunch of files to kafkatest directory `fc7c81c` [Geoff Anderson] added setup.py `632be12` [Geoff] Merge pull request #3 from confluentinc/verbose-client `51a94fd` [Geoff Anderson] Use argparse4j instead of joptsimple. ThroughputThrottler now has more intuitive behavior when targetThroughput is 0. `a80a428` [Geoff Anderson] Added shell program for VerifiableProducer. `d586fb0` [Geoff Anderson] Updated comments to reflect that throttler is not message-specific `6842ed1` [Geoff Anderson] left out a file from last commit `1228eef` [Geoff Anderson] Renamed throttler `9100417` [Geoff Anderson] Updated command-line options for VerifiableProducer. Extracted throughput logic to make it reusable. `0a5de8e` [Geoff Anderson] Fixed checkstyle errors. Changed name to VerifiableProducer. Added synchronization for thread safety on println statements. `475423b` [Geoff Anderson] Convert class to string before adding to json object. `bc009f2` [Geoff Anderson] Got rid of VerboseProducer in core (moved to clients) `c0526fe` [Geoff Anderson] Updates per review comments. `8b4b1f2` [Geoff Anderson] Minor updates to VerboseProducer `2777712` [Geoff Anderson] Added some metadata to producer output. `da94b8c` [Geoff Anderson] Added number of messages option. `07cd1c6` [Geoff Anderson] Added simple producer which prints status of produced messages to stdout. `a278988` [Geoff Anderson] fixed typos `f1914c3` [Liquan Pei] Merge pull request #2 from confluentinc/system_tests `81e4156` [Liquan Pei] Bootstrap Kafka system tests	9 years ago

11 Commits (da332f2241f1781aca64d1007c0c209b50ec5ff0)