Apache Kafka ================= See our [web site](http://kafka.apache.org) for details on the project. You need to have [Gradle](http://www.gradle.org/installation) and [Java](http://www.oracle.com/technetwork/java/javase/downloads/index.html) installed. Kafka requires Gradle 2.0 or higher. Java 7 should be used for building in order to support both Java 7 and Java 8 at runtime. ### First bootstrap and download the wrapper ### cd kafka_source_dir gradle Now everything else will work. ### Build a jar and run it ### ./gradlew jar Follow instructions in http://kafka.apache.org/documentation.html#quickstart ### Build source jar ### ./gradlew srcJar ### Build aggregated javadoc ### ./gradlew aggregatedJavadoc ### Build javadoc and scaladoc ### ./gradlew javadoc ./gradlew javadocJar # builds a javadoc jar for each module ./gradlew scaladoc ./gradlew scaladocJar # builds a scaladoc jar for each module ./gradlew docsJar # builds both (if applicable) javadoc and scaladoc jars for each module ### Run unit/integration tests ### ./gradlew test # runs both unit and integration tests ./gradlew unitTest ./gradlew integrationTest ### Force re-running tests without code change ### ./gradlew cleanTest test ./gradlew cleanTest unitTest ./gradlew cleanTest integrationTest ### Running a particular unit/integration test ### ./gradlew -Dtest.single=RequestResponseSerializationTest core:test ### Running a particular test method within a unit/integration test ### ./gradlew core:test --tests kafka.api.ProducerFailureHandlingTest.testCannotSendToInternalTopic ./gradlew clients:test --tests org.apache.kafka.clients.MetadataTest.testMetadataUpdateWaitTime ### Running a particular unit/integration test with log4j output ### Change the log4j setting in either `clients/src/test/resources/log4j.properties` or `core/src/test/resources/log4j.properties` ./gradlew -i -Dtest.single=RequestResponseSerializationTest core:test ### Generating test coverage reports ### Generate coverage reports for the whole project: ./gradlew reportCoverage Generate coverage for a single module, i.e.: ./gradlew clients:reportCoverage ### Building a binary release gzipped tar ball ### ./gradlew clean ./gradlew releaseTarGz The above command will fail if you haven't set up the signing key. To bypass signing the artifact, you can run: ./gradlew releaseTarGz -x signArchives The release file can be found inside `./core/build/distributions/`. ### Cleaning the build ### ./gradlew clean ### Running a task on a particular version of Scala (either 2.11.x or 2.12.x) ### *Note that if building the jars with a version other than 2.11.12, you need to set the `SCALA_VERSION` variable or change it in `bin/kafka-run-class.sh` to run the quick start.* You can pass either the major version (eg 2.11) or the full version (eg 2.11.12): ./gradlew -PscalaVersion=2.11 jar ./gradlew -PscalaVersion=2.11 test ./gradlew -PscalaVersion=2.11 releaseTarGz Scala 2.12.x requires Java 8. ### Running a task for a specific project ### This is for `core`, `examples` and `clients` ./gradlew core:jar ./gradlew core:test ### Listing all gradle tasks ### ./gradlew tasks ### Building IDE project #### *Note that this is not strictly necessary (IntelliJ IDEA has good built-in support for Gradle projects, for example).* ./gradlew eclipse ./gradlew idea The `eclipse` task has been configured to use `${project_dir}/build_eclipse` as Eclipse's build directory. Eclipse's default build directory (`${project_dir}/bin`) clashes with Kafka's scripts directory and we don't use Gradle's build directory to avoid known issues with this configuration. ### Building the jar for all scala versions and for all projects ### ./gradlew jarAll ### Running unit/integration tests for all scala versions and for all projects ### ./gradlew testAll ### Building a binary release gzipped tar ball for all scala versions ### ./gradlew releaseTarGzAll ### Publishing the jar for all version of Scala and for all projects to maven ### ./gradlew uploadArchivesAll Please note for this to work you should create/update `${GRADLE_USER_HOME}/gradle.properties` (typically, `~/.gradle/gradle.properties`) and assign the following variables mavenUrl= mavenUsername= mavenPassword= signing.keyId= signing.password= signing.secretKeyRingFile= ### Publishing the streams quickstart archetype artifact to maven ### For the Streams archetype project, one cannot use gradle to upload to maven; instead the `mvn deploy` command needs to be called at the quickstart folder: cd streams/quickstart mvn deploy Please note for this to work you should create/update user maven settings (typically, `${USER_HOME}/.m2/settings.xml`) to assign the following variables ... ... apache.snapshots.https ${maven_username} ${maven_password} apache.releases.https ${maven_username} ${maven_password} ... ... ### Installing the jars to the local Maven repository ### ./gradlew installAll ### Building the test jar ### ./gradlew testJar ### Determining how transitive dependencies are added ### ./gradlew core:dependencies --configuration runtime ### Determining if any dependencies could be updated ### ./gradlew dependencyUpdates ### Running code quality checks ### There are two code quality analysis tools that we regularly run, findbugs and checkstyle. #### Checkstyle #### Checkstyle enforces a consistent coding style in Kafka. You can run checkstyle using: ./gradlew checkstyleMain checkstyleTest The checkstyle warnings will be found in `reports/checkstyle/reports/main.html` and `reports/checkstyle/reports/test.html` files in the subproject build directories. They are also are printed to the console. The build will fail if Checkstyle fails. #### Findbugs #### Findbugs uses static analysis to look for bugs in the code. You can run findbugs using: ./gradlew findbugsMain findbugsTest -x test The findbugs warnings will be found in `reports/findbugs/main.html` and `reports/findbugs/test.html` files in the subproject build directories. Use -PxmlFindBugsReport=true to generate an XML report instead of an HTML one. ### Common build options ### The following options should be set with a `-P` switch, for example `./gradlew -PmaxParallelForks=1 test`. * `commitId`: sets the build commit ID as .git/HEAD might not be correct if there are local commits added for build purposes. * `mavenUrl`: sets the URL of the maven deployment repository (`file://path/to/repo` can be used to point to a local repository). * `maxParallelForks`: limits the maximum number of processes for each task. * `showStandardStreams`: shows standard out and standard error of the test JVM(s) on the console. * `skipSigning`: skips signing of artifacts. * `testLoggingEvents`: unit test events to be logged, separated by comma. For example `./gradlew -PtestLoggingEvents=started,passed,skipped,failed test`. * `xmlFindBugsReport`: enable XML reports for findBugs. This also disables HTML reports as only one can be enabled at a time. ### Running in Vagrant ### See [vagrant/README.md](vagrant/README.md). ### Contribution ### Apache Kafka is interested in building the community; we would welcome any thoughts or [patches](https://issues.apache.org/jira/browse/KAFKA). You can reach us [on the Apache mailing lists](http://kafka.apache.org/contact.html). To contribute follow the instructions here: * http://kafka.apache.org/contributing.html