* Introduce `gradlewAll` script to replace `*All` tasks since the approach
used by the latter doesn't work since Gradle 6.0 and it's unclear when,
if ever, it will work again ( see https://github.com/gradle/gradle/issues/11301).
* Update release script and README given the above.
* Update zinc to 1.3.5.
* Update gradle-versions-plugin to 0.28.0.
The major improvements in Gradle 6.0 to 6.3 are:
- Improved incremental compilation for Scala
- Support for Java 14 (although some Gradle plugins
like spotBugs may need to be updated or disabled,
will do that separately)
- Improved scalac reporting, warnings are clearly
marked as such, which is very helpful.
Tested `gradlewAll` manually for the commands listed in the README
and release script. For `uploadArchive`, I tested it with a local Maven
repository.
Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>
Disabled by default, but enabled for Jenkins PR builds (maximum of 1 retry per
test with up to 5 retries for the test run).
Reviewers: Ismael Juma <ismael@juma.me.uk>
* Adjust build and documentation.
* Use lambda syntax for SAM types in `core`, `streams-scala` and
`connect-runtime` modules.
* Remove `runnable` and `newThread` from `CoreUtils` as lambda
syntax for SAM types make them unnecessary.
* Remove stale comment in `FunctionsCompatConversions`,
`KGroupedStream`, `KGroupedTable' and `KStream` about Scala 2.11,
the conversions are needed for Scala 2.12 too.
* Deprecate `org.apache.kafka.streams.scala.kstream.Suppressed`
and use `org.apache.kafka.streams.kstream.Suppressed` instead.
* Use `Admin.create` instead of `AdminClient.create`. Static methods
in Java interfaces can be invoked since Scala 2.12. I noticed that
MirrorMaker 2 uses `AdminClient.create`, but I did not change them
as Connectors have restrictions on newer client APIs.
* Improve efficiency in a few `Gauge` implementations by avoiding
unnecessary intermediate collections.
* Remove pointless `Option.apply` in `ZookeeperClient`
`SessionState` metric.
* Fix unused import/variable and other compiler warnings.
* Reduce visibility of some vals/defs.
Reviewers: Manikumar Reddy <manikumar@confluent.io>, Guozhang Wang <wangguoz@gmail.com>, Gwen Shapira <gwen@confluent.io>
Fixed a small typo in the main README file ("They are also are printed" --> "They are also printed").
Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>
Given we need to follow the Apache rule of not checking
any binaries into the source code, Kafka has always had
a bit of a tricky Gradle bootstrap.
Using ./gradlew as users expect doesn’t work and a
local and compatible version of Gradle was required to
generate the wrapper first.
This patch changes the behavior of the wrapper task to
instead generate a gradlew script that can bootstrap the
jar itself. Additionally it adds a license, removes the bat
script, and handles retries.
The documentation in the readme was also updated.
Going forward patches that upgrade gradle should run
`gradle wrapper` before checking in the change.
With this change users using ./gradlew can be sure they
are always building with the correct version of Gradle.
Reviewers: Viktor Somogyi <viktorsomogyi@gmail.com>, Ismael Juma <ismael@juma.me.uk
Author: Arvind Thirunarayanan <athirunar@confluent.io>
Reviewers: Colin Patrick McCabe <colin@cmccabe.xyz>
Closes#7694 from arvindth/ath-debugdeps-task
- include Scala 2.13 in gradle build
- handle future milestone and RC versions of Scala in a better way
- if no Scala version is specified, default to scala 2.12 (bump from 2.11)
- include certain Xlint options (removed by Scala 2.13) for Scala 2.11/2.12 build only
- upgrade versions for dependencies:
- scalaLogging: 3.9.0 -->> 3.9.2
- scalatest: 3.0.7 -->> 3.0.8
- scoverage: 1.3.1 -->> 1.4.0
Reviewers: Ewen Cheslack-Postava <me@ewencp.org>, Ismael Juma <ismael@juma.me.uk>
Verified that the https links work.
I didn't update the license header in this PR since that touches
so many files. Will file a separate one for that.
Reviewers: Manikumar Reddy <manikumar.reddy@gmail.com>
Gradle 5.0 was released on 26 November. See the release notes for enhancements and fixes: https://docs.gradle.org/5.0/release-notes.html.
A notable bugfix ensures that Javadoc artifacts are cleaned between builds.
The upgraded wrapper was spot-checked against the commands in the README and the
README was updated not to use removed system property `-Dtest.single`.
Reviewers: Ismael Juma <ismael@juma.me.uk>
Scala 2.12 has better support for newer Java versions and includes additional
compiler warnings that are helpful during development. In addition, Scala 2.11
hasn't been supported by the Scala community for a long time, the soon to be
released Spark 2.4.0 will finally support Scala 2.12 (this was the main reason
preventing many from upgrading to Scala 2.12) and Scala 2.13 is at the RC stage.
It's time to start recommending the Scala 2.12 build as we prepare support for
Scala 2.13 and start thinking about removing support for Scala 2.11.
In the meantime, Jenkins will continue to build all supported Scala versions (including
Scala 2.11) so the PR and trunk jobs will fail if people accidentally use methods
introduced in Scala 2.12.
Reviewers: Ewen Cheslack-Postava <me@ewencp.org>
findBugs is abandoned, it doesn't work with Java 9 and the Gradle plugin will be deprecated in
Gradle 5.0: https://github.com/gradle/gradle/pull/6664
spotBugs is actively maintained and it supports Java 8, 9 and 10. Java 11 is not supported yet,
but it's likely to happen soon.
Also fixed a file leak in Connect identified by spotbugs.
Manually tested spotBugsMain, jarAll and importing kafka in IntelliJ and running
a build in the IDE.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Colin Patrick McCabe <colin@cmccabe.xyz>, Dong Lin <lindong28@gmail.com>
Closes#5625 from ijuma/kafka-5887-spotbugs
* Set --source, --target and --release to 1.8.
* Build Scala 2.12 by default.
* Remove some conditionals in the build file now that Java 8
is the minimum version.
* Bump the version of Jetty, Jersey and Checkstyle (the newer
versions require Java 8).
* Fixed issues uncovered by the new version if Checkstyle.
* A couple of minor updates to handle an incompatible source
change in the new version of Jetty.
* Add dependency to jersey-hk2 to fix failing tests caused
by Jersey upgrade.
* Update release script to use Java 8 and to take into account
that Scala 2.12 is now built by default.
* While we're at it, bump the version of Gradle, Gradle plugins,
ScalaLogging, JMH and apache directory api.
* Minor documentation updates including the readme and upgrade
notes. A number of Streams Java 7 examples can be removed
subsequently.
Missed a few places in the previous commit.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Rajini Sivaram <rajinisivaram@googlemail.com>
Closes#4200 from ijuma/scala-2.11.12-part2
0. Minor fixes on the existing examples to merge all on a single input topic; also do not use `common.utils.Exit` as it is for internal usage only.
1. Add the archetype project for the quickstart. Steps to try it out:
a. `mvn install` on the quickstart directory.
b. `mvn archetype:generate \
-DarchetypeGroupId=org.apache.kafka \
-DarchetypeArtifactId=streams-quickstart-java \
-DarchetypeVersion=1.0.0-SNAPSHOT \
-DgroupId=streams-quickstart \
-DartifactId=streams-quickstart \
-Dversion=0.1 \
-Dpackage=StreamsQuickstart \
-DinteractiveMode=false` at any directory to create the project.
c. build the streams jar with version `1.0.0-SNAPSHOT` to local maven repository with `./gradlew installAll`; `cd streams-quickstart; mvn clean package`
d. create the input / output topics, start the console producer and consumer.
e. start the program: `mvn exec:java -Dexec.mainClass=StreamsQuickstart.Pipe/LineSplit/WordCount`.
f. type data on console producer and observe data on console consumer.
Author: Guozhang Wang <wangguoz@gmail.com>
Reviewers: Damian Guy <damian.guy@gmail.com>, Bill Bejeck <bbejeck@gmail.com>, Ewen Cheslack-Postava <me@ewencp.org>, Eno Thereska <eno.thereska@gmail.com>
Closes#3630 from guozhangwang/KMinor-streams-quickstart-tutorial
This allows a build system to set the correct commit ID when .git/HEAD would be wrong
if there are local commits for build purposes.
Author: Max Zheng <mzheng@confluent.io>
Reviewers: Ismael Juma <ismael@juma.me.uk>, Ewen Cheslack-Postava <ewen@confluent.io>
Closes#3606 from maxzheng/read-commit@trunk
This uses JUnit Categories to identify integration tests. Adds 2 new build targets:
`integrationTest` and `unitTest`.
Author: Damian Guy <damian.guy@gmail.com>
Reviewers: Eno Thereska <eno@confluent.io>, Ewen Cheslack-Postava <ewen@confluent.io>, Ismael Juma <ismael@juma.me.uk>
Closes#2695 from dguy/junit-categories
Generate core project with correct source folders. In addition
set output folders same as command line build. Don't generate
unnecessary projects.
Author: Dhwani Katagade <dhwani_katagade@persistent.com>
Reviewers: Edoardo Comar <ecomar@uk.ibm.com>, Guozhang Wang <wangguoz@gmail.com>, Ismael Juma <ismael@juma.me.uk>
Closes#2382 from dhwanikatagade/gradle_eclipse_plugin_path_fix
Will allow users to set one fewer environment variable if they need to
change scala version. Still, SCALA_BINARY_VERSION can be explicitly set.
Author: Konstantine Karantasis <konstantine@confluent.io>
Reviewers: Ismael Juma <ismael@juma.me.uk>
Closes#2130 from kkonstantine/MINOR-Extract-SCALA_BINARY_VERSION-from-SCALA_VERSION
Trying to build the source and publish it to internal Maven repo, I ran into an issue that I explain in the mailing list discussion here https://www.mail-archive.com/devkafka.apache.org/msg56359.html.
The commit here updates the README.md to make a note that the GRADLE_USER_HOME environment variable plays a role in deciding which file to add the maven configs to.
Author: Jaikiran Pai <jaikiran.pai@gmail.com>
Reviewers: Ismael Juma <ismael@juma.me.uk>
Closes#1837 from jaikiran/readme-update-grade-user-home
The task is called `aggregatedJavadoc` and the generated html will be under `<project.dir>/build/docs/javadoc/`.
I also disabled javadoc for `tools` and `log4j-appender` as they are not public API.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Guozhang Wang <wangguoz@gmail.com>
Closes#1398 from ijuma/kafka-3717-aggregate-javadoc
Adds a gradle task to generate a report of outdate release dependencies:
`gradle dependencyUpdates`
Updates a few minor versions.
Author: Grant Henke <granthenke@gmail.com>
Reviewers: Ismael Juma, Gwen Shapira
Closes#973 from granthenke/outdated-deps
This is handy when debugging certain kinds of Jenkins failures.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Guozhang Wang <wangguoz@gmail.com>, Ewen Cheslack-Postava <ewen@confluent.io>
Closes#739 from ijuma/gradle-show-standard-streams
Mention Java 7 requirement, fix and improve formatting and remove obsolete reference to SVN website.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Sriharsha Chintalapani <harsha@hortonworks.com>, Jason Gustafson <jason@confluent.io>
Closes#677 from ijuma/mention-java-requirement-in-readme
We can take advantage of the fact that major Scala versions are binary compatible (since 2.10) to make the build a little more user-friendly.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Ewen Cheslack=Postava, Grant Henke
Closes#574 from ijuma/install-all-and-major-instead-of-full-version
`testAll` passed locally.
Author: Ismael Juma <ismael@juma.me.uk>
Reviewers: Harsha, Ewen, Guozhang
Closes#87 from ijuma/kafka-2348-drop-support-for-scala-2.9 and squashes the following commits:
cf9796a [Ismael Juma] KAFKA-2348; Drop support for Scala 2.9
Author: Ismael Juma <ismael@juma.me.uk>
Closes#82 from ijuma/kafka-2324 and squashes the following commits:
d71bf5c [Ismael Juma] KAFKA-2324; Update to Scala 2.11.7
Ismael Juma
Boyang Chen
Viktor Somogyi
Bryan Ji
Grant Henke
Arvind Thirunarayanan
Dejan Stojadinović
Stanislav Kozlovski
Attila Sasvari
Colin Hicks
Stanislav Kozlovski
Manikumar Reddy O
Matthias Wessendorf
Guozhang Wang
Max Zheng
Colin P. Mccabe
Damian Guy
Dhwani Katagade
Konstantine Karantasis
Jaikiran Pai
David Chen
Grant Henke
Vahid Hashemian
Gabriel Avellaneda
Ewen Cheslack-Postava
Ismael Juma
Jay Kreps
jaikiran pai