Add the metadata gradle module, which will contain the metadata record
definitions, and other metadata-related broker-side code.
Add MetadataParser, MetadataParseException, etc.
Reviewers: José Armando García Sancio <jsancio@gmail.com>, Ismael Juma <ismael@juma.me.uk>, David Arthur <mumrah@gmail.com>
Instead of listing the projects that should use JUnit 5, list the
projects that are still using JUnit 4.
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
* replace `org.junit.Assert` by `org.junit.jupiter.api.Assertions`
* replace `org.junit` by `org.junit.jupiter.api`
* replace `org.junit.runners.Parameterized` by `org.junit.jupiter.params.ParameterizedTest`
* replace `org.junit.runners.Parameterized.Parameters` by `org.junit.jupiter.params.provider.{Arguments, MethodSource}`
* replace `Before` by `BeforeEach`
* replace `After` by `AfterEach`
Reviewers: Ismael Juma <ismael@juma.me.uk>
* Use the packages/classes from JUnit 5
* Move description in `assert` methods to last parameter
* Convert parameterized tests so that they work with JUnit 5
* Remove `hamcrest`, it didn't seem to add much value
* Fix `Utils.mkEntry` to have correct `equals` implementation
* Add a missing `@Test` annotation in `SslSelectorTest` override
* Adjust regex in `SaslAuthenticatorTest` due to small change in the
assert failure string in JUnit 5
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
Avoid spamming the logs at the INFO level in a tight loop when there are no new records being polled
Reviewers: Walker Carlson <wcarlson@confluent.io>, Leah Thomas <lthomas@confluent.io>
This patch removes the periodic scheduling of a thread to send AlterISR requests and instead sends a request as soon as an ISR update is ready to send. When a response is received, the callback checks for any unsent items and will schedule another request if necessary.
This PR includes following changes.
1. replace org.junit.Assert by org.junit.jupiter.api.Assertions
2. replace org.junit by org.junit.jupiter.api
3. replace Before by BeforeEach
4. replace After by AfterEach
Reviewers: Ismael Juma <ismael@confluent.io>
Prior to this patch, envelope handling was a shared responsibility between `SocketServer` and `KafkaApis`. The former was responsible for parsing and validation, while the latter was responsible for authorization. This patch consolidates logic in `KafkaApis` so that envelope requests follow the normal request flow.
Reviewers: David Jacot <djacot@confluent.io>, Chia-Ping Tsai <chia7712@gmail.com>
If a connector, converter, etc. invokes [Configuration::setConfiguration](https://docs.oracle.com/javase/8/docs/api/javax/security/auth/login/Configuration.html#setConfiguration-javax.security.auth.login.Configuration-), it will cause the Connect basic auth filter to use that JAAS config instead of the one configured at startup with the `-Djava.security.auth.login.config` JVM property. This can cause requests to the worker to succeed initially but start to be rejected after the JVM's global JAAS config is altered.
To alleviate this the current PR instructs the Connect Worker to cache the JVM's global JAAS configuration at the beginning (as soon as the `BasicAuthSecurityRestExtension` class is loaded), and use that for all future authentication.
Existing tests for the JAAS basic auth filter are modified to work with the new internal logic. The `testEmptyCredentialsFile` test is corrected to actually operate on an empty credentials file (instead of a non-existent credentials file, which it currently operates on). A new test is added to ensure that, even if the global JAAS config is overwritten, the basic auth filter will use the first one it could find.
Reviewers: Greg Harris <gregh@confluent.io>, Konstantine Karantasis <k.karantasis@gmail.com>
This patch factors out a `RaftManager` class from `TestRaftServer` which will be needed when we integrate this layer into the server. This class encapsulates the logic to build `KafkaRaftClient` as well as its IO thread.
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
If users block the response handling thread in one call waiting for the result
of a second "nested" call then the client effectively hangs because the 2nd
call's response will never be processed.
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
This PR includes following changes.
1. @Test(expected = Exception.class) is replaced by assertThrows
2. remove reference to org.scalatest.Assertions
3. change the magic code from 1 to 2 for testAppendAtInvalidOffset to test ZSTD
4. rename testMaybeAddPartitionToTransactionXXXX to testNotReadyForSendXXX
5. increase maxBlockMs from 1s to 3s to avoid unexpected timeout from TransactionsTest#testTimeout
Reviewers: Ismael Juma <ismael@confluent.io>
When we truncate the log, the first unstable offset might become valid. On the other hand, the logic in `updateHighWatermarkMetadata` assumes that the first stable offset remains at a valid position. Since this method can be reached through either `updateLogStartOffset` or `updateLogEndOffset` in the truncation paths, we need to ensure that the first unstable offset first reflects the truncated state.
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>, Jun Rao <junrao@gmail.com>
This patch fixes a bug when overriding a reassignment in `ReassignPartitionsCommand` due to the invalid assumption that adding replicas are not included in the full replica set. It also simplifies the logic to construct the move maps and makes some stylistic improvements.
Reviewers: Jason Gustafson <jason@confluent.io>
This is similar to config change logs for topic/broker. This will be useful for debugging any acl related issues.
Author: Manikumar Reddy <manikumar.reddy@gmail.com>
Reviewers: Rajini Sivaram <rajinisivaram@googlemail.com>
Closes#9841 from omkreddy/acl-change-log
If EOS is enabled and the TX commit fails with a timeout,
we should not process more messages (what is ok for non-EOS)
because we don't really know the status of the TX.
If the commit was indeed successful, we won't have an open TX
can calling send() would fail with an fatal error.
Instead, we should retry the (idempotent) commit of the TX,
and start a new TX afterwards.
Reviewers: Boyang Chen <boyang@confluent.io>, John Roesler <john@confluent.io>
Add StickyAssignor and CooperativeStickyAssignor, and also briefly introduce the RangeAssignor and RoundRobinAssignor in the docs for the partition.assignment.strategy config
Reviewers: Anna Sophie Blee-Goldman <ableegoldman@apache.org>
This patch improves append fencing after an IO error. Previously there was a window between the time of an IO error and the time the log is taken offline in which additional appends can be attempted. This is due to the asynchronous propagation of the IO error. This patch tightens the fencing so that no additional appends will be accepted after a previous append failed with an IO error.
Reviewers: Guozhang Wang <guozhang@apache.org>, Jason Gustafson <jason@confluent.io>
Current below log statements are not useful. This PR logs readable/masked configs during alterBrokerConfigs/alterTopicConfigs method call.
`[Admin Manager on Broker 1]: Updating topic test with new configuration kafka.server.KafkaConfigc9ba35e3`
Author: Manikumar Reddy <manikumar.reddy@gmail.com>
Reviewers: Rajini Sivaram <rajinisivaram@googlemail.com>, Chia-Ping Tsai <chia7712@gmail.com>
Closes#9824 from omkreddy/admin-logs
This patch adds a few missing "entity type" tags to the schema definitions of `OffsetDeleteRequest`, `OffsetDeleteResponse`, and `TxnOffsetCommitRequest`.
Reviewers: David Arthur <mumrah@gmail.com>, Chia-Ping Tsai <chia7712@gmail.com>
Previously offline replicas were included as `null` in the array of replicas in `PartitionInfo` when populated by the `MetadataCache` for the purpose of the client quota callback. This patch instead initializes empty non-null nodes, which is consistent with how `PartitionInfo` is constructed by the clients in `MetadataResponse`.
Reviewers: Chia-Ping Tsai <chia7712@gmail.com>
Handle REBALANCE_IN_PROGRESS error in JoinGroup, which is possible if there is a replication timeout while persisting the group state after the rebalance completes.
Reviewers: Boyang Chen <boyang@confluent.io>, Jason Gustafson <jason@confluent.io>
If attempting to build the project from a directory with spaces in its name and gradle-wrapper.jar is missing, the script will fail to download a new one because the "if" condition will break because $APP_HOME will resolve to multiple strings. Protecting $APP_HOME with quotes fixes the issue. Tested by deleting gradle.wrapper and trying to build.
Author: Gwen Shapira <cshapi@gmail.com>
Reviewers: Chia-Ping Tsai
Closes#9813 from gwenshap/fix_gradlew
Colin Patrick McCabe
Ning Zhang
dengziming
Ismael Juma
Mickael Maison
Luke Chen
wenbingshen
Chia-Ping Tsai
A. Sophie Blee-Goldman
David Arthur
Bruno Cadonna
Boyang Chen
Geordie
CHUN-HAO TANG
Jason Gustafson
Chris Egerton
Walker Carlson
Tom Bentley
tang7526
David Jacot
Matthias J. Sax
Manikumar Reddy
Bill Bejeck
Gwen Shapira