Browse Source

KAFKA-7177; Update 2.0 documentation to reflect changed quota behaviors by KIP-219

Updated the 2.0 document for changed quota behaviors.

Author: Jon Lee <jonlee@linkedin.com>

Reviewers: Ismael Juma <ismael@juma.me.uk>, Dong Lin <lindong28@gmail.com>

Closes #5384 from jonlee2/KAFKA-7177
pull/5384/merge
Jon Lee 6 years ago committed by Dong Lin
parent
commit
f9cdadfdcf
  1. 10
      docs/design.html

10
docs/design.html

@ -610,10 +610,12 @@ @@ -610,10 +610,12 @@
having a fixed cluster wide bandwidth per client because that would require a mechanism to share client quota usage among all the brokers. This can be harder to get right than the quota implementation itself!
</p>
<p>
How does a broker react when it detects a quota violation? In our solution, the broker does not return an error rather it attempts to slow down a client exceeding its quota.
It computes the amount of delay needed to bring a guilty client under its quota and delays the response for that time. This approach keeps the quota violation transparent to clients
(outside of client-side metrics). This also keeps them from having to implement any special backoff and retry behavior which can get tricky. In fact, bad client behavior (retry without backoff)
can exacerbate the very problem quotas are trying to solve.
How does a broker react when it detects a quota violation? In our solution, the broker first computes the amount of delay needed to bring the violating client under its quota
and returns a response with the delay immediately. In case of a fetch request, the response will not contain any data. Then, the broker mutes the channel to the client,
not to process requests from the client anymore, until the delay is over. Upon receiving a response with a non-zero delay duration, the Kafka client will also refrain from
sending further requests to the broker during the delay. Therefore, requests from a throttled client are effectively blocked from both sides.
Even with older client implementations that do not respect the delay response from the broker, the back pressure applied by the broker via muting its socket channel
can still handle the throttling of badly behaving clients. Those clients who sent further requests to the throttled channel will receive responses only after the delay is over.
</p>
<p>
Byte-rate and thread utilization are measured over multiple small windows (e.g. 30 windows of 1 second each) in order to detect and correct quota violations quickly. Typically, having large measurement windows

Loading…
Cancel
Save