You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
Tree:
d44e01be48
0.10.0
0.10.1
0.10.2
0.11.0
0.7
0.7.0
0.7.1
0.7.2
0.8
0.8.0-beta1-candidate1
0.8.1
0.8.2
0.9.0
1.0
1.1
2.0
2.1
2.2
2.3
2.4
2.5
2.6
2.7
2.8
2.x
3.0
3.1
3.2
3.3
3.4
3.5
3.6
KAFKA-14367-join-group
KAFKA-14496-fix-oauth-encoder
KAFKA-15311
add-assignor-log-generation
cmccabe_2023-04-11_improve_controller_logging
cmccabe_2023-05-10_cleanup
cmccabe_2023-06-21_some_minor_fixes
cmccabe_kip_919
hekai-study-v2.8
john-disable-12049
kafka-10867-improved-task-idling-nolog
kip-866-zk-migration-to-kraft
metashell
minor-alter-isr-scheduling
printer
repro-task-idling-problem
revert-13391-kafka-14561
temp-8436
trunk
txn1
0.10.0.0
0.10.0.0-rc1
0.10.0.0-rc2
0.10.0.0-rc3
0.10.0.0-rc4
0.10.0.0-rc5
0.10.0.0-rc6
0.10.0.1
0.10.0.1-rc0
0.10.0.1-rc1
0.10.0.1-rc2
0.10.1.0
0.10.1.0-rc0
0.10.1.0-rc1
0.10.1.0-rc2
0.10.1.0-rc3
0.10.1.1
0.10.1.1-rc0
0.10.1.1-rc1
0.10.2.0
0.10.2.0-KAFKA-5526
0.10.2.0-rc0
0.10.2.0-rc1
0.10.2.0-rc2
0.10.2.1
0.10.2.1-rc0
0.10.2.1-rc1
0.10.2.1-rc2
0.10.2.1-rc3
0.10.2.2
0.10.2.2-rc0
0.10.2.2-rc1
0.11.0.0
0.11.0.0-rc0
0.11.0.0-rc1
0.11.0.0-rc2
0.11.0.1
0.11.0.1-rc0
0.11.0.2
0.11.0.2-rc0
0.11.0.3-rc0
0.8.0
0.8.0-beta1
0.8.0-beta1-candidate1
0.8.1
0.8.1.0
0.8.1.1
0.8.2-beta
0.8.2.0
0.8.2.0-cp
0.8.2.1
0.8.2.2
0.9.0.0
0.9.0.1
1.0.0
1.0.0-rc0
1.0.0-rc1
1.0.0-rc2
1.0.0-rc3
1.0.0-rc4
1.0.1
1.0.1-rc0
1.0.1-rc1
1.0.1-rc2
1.0.2
1.0.2-rc0
1.0.2-rc1
1.1.0
1.1.0-rc0
1.1.0-rc1
1.1.0-rc2
1.1.0-rc3
1.1.0-rc4
1.1.1
1.1.1-rc0
1.1.1-rc1
1.1.1-rc2
1.1.1-rc3
2.0.0
2.0.0-rc0
2.0.0-rc1
2.0.0-rc2
2.0.0-rc3
2.0.1
2.0.1-rc0
2.1.0
2.1.0-rc0
2.1.0-rc1
2.1.1
2.1.1-rc0
2.1.1-rc1
2.1.1-rc2
2.2.0
2.2.0-rc0
2.2.0-rc1
2.2.0-rc2
2.2.1
2.2.1-rc0
2.2.1-rc1
2.2.2
2.2.2-rc1
2.2.2-rc2
2.3.0
2.3.0-rc1
2.3.0-rc2
2.3.0-rc3
2.3.1
2.3.1-rc0
2.3.1-rc1
2.3.1-rc2
2.4.0
2.4.0-rc0
2.4.0-rc1
2.4.0-rc2
2.4.0-rc3
2.4.0-rc4
2.4.1
2.4.1-rc0
2.5.0
2.5.0-rc0
2.5.0-rc1
2.5.0-rc2
2.5.0-rc3
2.5.1
2.5.1-rc0
2.6.0
2.6.0-rc0
2.6.0-rc1
2.6.0-rc2
2.6.1
2.6.1-rc0
2.6.1-rc1
2.6.1-rc2
2.6.1-rc3
2.6.2
2.6.2-rc0
2.6.2-rc1
2.6.3
2.6.3-rc0
2.7.0
2.7.0-rc0
2.7.0-rc1
2.7.0-rc2
2.7.0-rc3
2.7.0-rc4
2.7.0-rc5
2.7.0-rc6
2.7.1
2.7.1-rc0
2.7.1-rc1
2.7.1-rc2
2.7.2
2.7.2-rc0
2.8.0
2.8.0-rc0
2.8.0-rc1
2.8.0-rc2
2.8.1
2.8.1-rc0
2.8.1-rc1
2.8.2
2.8.2-rc0
3.0.0
3.0.0-rc0
3.0.0-rc1
3.0.0-rc2
3.0.1
3.0.1-rc0
3.0.2
3.0.2-rc0
3.1.0
3.1.0-rc0
3.1.0-rc1
3.1.1
3.1.1-rc0
3.1.1-rc1
3.1.2
3.1.2-rc0
3.2.0
3.2.0-rc0
3.2.0-rc1
3.2.1
3.2.1-rc2
3.2.1-rc3
3.2.2
3.2.2-rc0
3.2.3
3.2.3-rc0
3.3.0
3.3.0-rc1
3.3.0-rc2
3.3.1
3.3.1-rc0
3.3.2
3.3.2-rc0
3.3.2-rc1
3.4.0
3.4.0-rc0
3.4.0-rc1
3.4.0-rc2
3.4.1
3.4.1-rc0
3.4.1-rc1
3.4.1-rc2
3.4.1-rc3
3.5.0
3.5.0-rc0
3.5.0-rc1
3.5.1
3.5.1-rc0
3.5.1-rc1
3.6.0
3.6.0-rc0
3.6.0-rc1
3.6.0-rc2
fetch
kafka-0.7.0-incubating-candidate-9
kafka-0.7.1-incubating-candidate-1
kafka-0.7.1-incubating-candidate-2
kafka-0.7.1-incubating-candidate-3
kafka-0.7.2-incubating-candidate-1
kafka-0.7.2-incubating-candidate-2
kafka-0.7.2-incubating-candidate-3
kafka-0.7.2-incubating-candidate-4
kafka-0.7.2-incubating-candidate-5
show
${ noResults }
src-kafka/checkstyle
Colin Patrick McCabe
7bc84d6ced
Implement controller-side snapshot generation.Implement QuorumController snapshot
generation. Note that this PR does not handle KRaft integration, just the internal
snapshot record generation and consumption logic.
Reading a snapshot is relatively straightforward. When the QuorumController
starts up, it loads the most recent snapshot. This is just a series of records
that we replay, plus a log offset ("snapshot epoch") that we advance to.
Writing a snapshot is more complex. There are several components:
the SnapshotWriter which persists the snapshot, the SnapshotGenerator
which manages writing each batch of records, and the SnapshotGeneratorManager
which interfaces the preceding two classes with the event queue.
Controller snapshots are done incrementally. In order to avoid blocking the
controller thread for a long time, we pull a few record batches at a time from
our record batch iterators. These iterators are implemented by controller
manager classes such as ReplicationControlManager, ClusterControlManager, etc.
Finally, this PR adds ControllerTestUtils#deepSortRecords and
ControllerTestUtils#assertBatchIteratorContains, which make it easier to write
unit tests. Since records are often constructed from unsorted data structures,
it is often useful to sort them before comparing them.
Reviewers: David Arthur <mumrah@gmail.com>
|
4 years ago | |
|---|---|---|
| .. | ||
| .scalafmt.conf |
…
|
|
| checkstyle.xml |
…
|
|
| import-control-core.xml |
…
|
|
| import-control-jmh-benchmarks.xml |
…
|
|
| import-control.xml |
…
|
|
| java.header |
…
|
|
| suppressions.xml | KAFKA-12467: Implement QuorumController snapshot generation (#10366) | 4 years ago |