tchekai704
/
src-kafka
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
Browse Source

KAFKA-7177; Update 2.0 documentation to reflect changed quota behaviors by KIP-219 

Updated the 2.0 document for changed quota behaviors.

Author: Jon Lee <jonlee@linkedin.com>

Reviewers: Ismael Juma <ismael@juma.me.uk>, Dong Lin <lindong28@gmail.com>

Closes #5384 from jonlee2/KAFKA-7177
pull/5384/merge
Jon Lee 6 years ago committed by Dong Lin
parent
5d2bf6328e
commit
f9cdadfdcf
1 changed files with 6 additions and 4 deletions
Whitespace
Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL
Unified View
Diff Options
Show Stats Download Patch File Download Diff File
  1. 10
      docs/design.html

10
docs/design.html
Unescape View File

@ -610,10 +610,12 @@
having a fixed cluster wide bandwidth per client because that would require a mechanism to share client quota usage among all the brokers. This can be harder to get right than the quota implementation itself! having a fixed cluster wide bandwidth per client because that would require a mechanism to share client quota usage among all the brokers. This can be harder to get right than the quota implementation itself!
</p> </p>
<p> <p>
How does a broker react when it detects a quota violation? In our solution, the broker does not return an error rather it attempts to slow down a client exceeding its quota. How does a broker react when it detects a quota violation? In our solution, the broker first computes the amount of delay needed to bring the violating client under its quota
It computes the amount of delay needed to bring a guilty client under its quota and delays the response for that time. This approach keeps the quota violation transparent to clients and returns a response with the delay immediately. In case of a fetch request, the response will not contain any data. Then, the broker mutes the channel to the client,
(outside of client-side metrics). This also keeps them from having to implement any special backoff and retry behavior which can get tricky. In fact, bad client behavior (retry without backoff) not to process requests from the client anymore, until the delay is over. Upon receiving a response with a non-zero delay duration, the Kafka client will also refrain from
can exacerbate the very problem quotas are trying to solve. sending further requests to the broker during the delay. Therefore, requests from a throttled client are effectively blocked from both sides.
Even with older client implementations that do not respect the delay response from the broker, the back pressure applied by the broker via muting its socket channel
can still handle the throttling of badly behaving clients. Those clients who sent further requests to the throttled channel will receive responses only after the delay is over.
</p> </p>
<p> <p>
Byte-rate and thread utilization are measured over multiple small windows (e.g. 30 windows of 1 second each) in order to detect and correct quota violations quickly. Typically, having large measurement windows Byte-rate and thread utilization are measured over multiple small windows (e.g. 30 windows of 1 second each) in order to detect and correct quota violations quickly. Typically, having large measurement windows

Write Preview
Loading…
Cancel
Save