tchekai704
/
src-kafka
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
Mirror of Apache Kafka
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
217 Commits
52 Branches
226 Tags
413 MiB
 
 
 
 
Tree: e42c8d5f4e
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from 'e42c8d5f4e'
${ noResults }
Jun Rao e42c8d5f4e
revisit broker config in 0.8; patched by Swapnil Ghike; reviewed by Jun Rao; KAFKA-325 		12 years ago
bin KAFKA-424. Fix command line usage description. Patch from Tommie. 12 years ago
clients KAFKA-348 merge trunk to branch 1239902:1310937 patch by Joe Stein reviewed by Jun Rao 13 years ago
config KAFKA-489 Add metrics collection and graphs to the system test framework; patched by Neha Narkhede; reviewed by Jun Rao and John Fung 12 years ago
contrib remove errorcode from ByteBufferMessageSet; patched by Swapnil Ghike; reviewed by Jay Kreps and Jun Rao; KAFKA-458 12 years ago
core revisit broker config in 0.8; patched by Swapnil Ghike; reviewed by Jun Rao; KAFKA-325 12 years ago
examples remove ZK dependency on producer; patched by Yang Ye; reviewed by Jun Rao; KAFKA-369 12 years ago
lib KAFKA 222 Mavenize contrib;patched by nehanarkhede; reviewed jakobhoman and junrao 13 years ago
perf remove ZK dependency on producer; patched by Yang Ye; reviewed by Jun Rao; KAFKA-369 12 years ago
project KAFKA-385 Fix race condition between checkSatisfied and expire in RequestPurgatory; fixed constant expiration of follower fetch requests as checkSatisfied was not getting called; add metrics to the RequestPurgatory; add a KafkaTimer convenience class; patched by Joel Koshy; reviewed by Jun Rao and Jay Kreps. 12 years ago
system_test Improve Kafka internal metrics; patched by Jun Rao; reviewed by Joel Koshy and Neha Narkhede; KAFKA-203 12 years ago
.gitignore Restore .gitignore 13 years ago
.rat-excludes KAFKA-151 Standard .rat-excludes file and rat script;patched by nehanarkhede; reviewed by junrao 13 years ago
DISCLAIMER KAFKA-206 add DISCLAIMER, cleanup NOTICE file; patched by chrisburroughs;reviewed by junrao 13 years ago
LICENSE KAFKA-221 LICENSE and NOTICE problems in Kafka 0.7; patched by jakobhoman; reviewed by junrao and nehanarkhede 13 years ago
NOTICE KAFKA-365 change copyright in NOTICE to current year, reviewed by Jun Rao 13 years ago
README.md
…
sbt reverting r1178669 to fix KAFKA-147 kafka integration tests fail on a fresh checkout; patched by nehanarkhede; reviewed by junrao 13 years ago

README.md

Kafka is a distributed publish/subscribe messaging system

It is designed to support the following

  • Persistent messaging with O(1) disk structures that provide constant time performance even with many TB of stored messages.
  • High-throughput: even with very modest hardware Kafka can support hundreds of thousands of messages per second.
  • Explicit support for partitioning messages over Kafka servers and distributing consumption over a cluster of consumer machines while maintaining per-partition ordering semantics.
  • Support for parallel data load into Hadoop.

Kafka is aimed at providing a publish-subscribe solution that can handle all activity stream data and processing on a consumer-scale web site. This kind of activity (page views, searches, and other user actions) are a key ingredient in many of the social feature on the modern web. This data is typically handled by "logging" and ad hoc log aggregation solutions due to the throughput requirements. This kind of ad hoc solution is a viable solution to providing logging data to an offline analysis system like Hadoop, but is very limiting for building real-time processing. Kafka aims to unify offline and online processing by providing a mechanism for parallel load into Hadoop as well as the ability to partition real-time consumption over a cluster of machines.

See our web site for more details on the project.

Contribution

Kafka is a new project, and we are interested in building the community; we would welcome any thoughts or patches. You can reach us on the Apache mailing lists.

The Kafka code is available from svn or a read only git mirror:

To build:

  1. ./sbt
  2. update - This downloads all the dependencies for all sub projects
  3. package - This will compile all sub projects and creates all the jars

Here are some useful sbt commands, to be executed at the sbt command prompt (./sbt) -

actions : Lists all the sbt commands and their descriptions

clean : Deletes all generated files (the target directory).

clean-cache : Deletes the cache of artifacts downloaded for automatically managed dependencies.

clean-lib : Deletes the managed library directory.

compile : Compile all the sub projects, but not create the jars

test : Run all unit tests in all sub projects

release-zip : Create all the jars, run unit tests and create a deployable release zip

package-all: Creates jars for src, test, docs etc

projects : List all the sub projects

project sub_project_name : Switch to a particular sub-project. For example, to switch to the core kafka code, use "project core-kafka"

Following commands can be run only on a particular sub project -

test-only package.test.TestName : Runs only the specified test in the current sub project

run : Provides options to run any of the classes that have a main method. For example, you can switch to project java-examples, and run the examples there by executing "project java-examples" followed by "run"