MINOR: Stream metrics documentation

Author: Eno Thereska <eno@confluent.io>

Reviewers: Matthias J. Sax, Guozhang Wang

Closes #2542 from enothereska/minor-streams-metrics

docs/ops.html

@@ -14,8 +14,8 @@
  See the License for the specific language governing permissions and
  limitations under the License.
 -->
-
 <script id="ops-template" type="text/x-handlebars-template">
+  
   Here is some information on actually running Kafka as a production system based on usage and experience at LinkedIn. Please send us any additional tips you know of.
 
   <h3><a id="basic_ops" href="#basic_ops">6.1 Basic Kafka Operations</a></h3>
@@ -842,9 +842,9 @@
       </tr>
   </tbody></table>
 
-  <h4><a id="selector_monitoring" href="#selector_monitoring">Common monitoring metrics for producer/consumer/connect</a></h4>
+  <h4><a id="selector_monitoring" href="#selector_monitoring">Common monitoring metrics for producer/consumer/connect/streams</a></h4>
 
-  The following metrics are available on producer/consumer/connector instances.  For specific metrics, please see following sections.
+  The following metrics are available on producer/consumer/connector/streams instances.  For specific metrics, please see following sections.
 
   <table class="data-table">
     <tbody>
@@ -931,9 +931,9 @@
     </tbody>
   </table>
 
-  <h4><a id="common_node_monitoring" href="#common_node_monitoring">Common Per-broker metrics for producer/consumer/connect</a></h4>
+  <h4><a id="common_node_monitoring" href="#common_node_monitoring">Common Per-broker metrics for producer/consumer/connect/streams</a></h4>
 
-  The following metrics are available on producer/consumer/connector instances.  For specific metrics, please see following sections.
+  The following metrics are available on producer/consumer/connector/streams instances.  For specific metrics, please see following sections.
 
   <table class="data-table">
     <tbody>
@@ -1314,7 +1314,125 @@
     </tbody>
   </table>
 
-  <h5><a id="others_monitoring" href="#others_monitoring">Others</a></h5>
+
+
+  <h4><a id="kafka_streams_monitoring" href="#kafka_streams_monitoring">Streams Monitoring</a></h4>
+
+  A Kafka Streams instance contains all the producer and consumer metrics as well as additional metrics specific to streams. By default Kafka Streams has metrics with two recording levels: debug and info. The debug level records all metrics, while the info level records only the thread-level metrics.  Use the following configuration option to specify which metrics you want collected:
+<pre>metrics.recording.level="info"</pre>
+
+<h5><a id="kafka_streams_thread_monitoring" href="#kafka_streams_thread_monitoring">Thread Metrics</a></h5>
+All the following metrics have a recording level of ``info``:
+<table class="data-table">
+    <tbody>
+      <tr>
+        <th>Metric/Attribute name</th>
+        <th>Description</th>
+        <th>Mbean name</th>
+      </tr>
+      <tr>
+        <td>[commit | poll | process | punctuate]-latency-[avg | max]</td>
+        <td>The [average | maximum] execution time in ms, for the respective operation, across all running tasks of this thread.</td>
+        <td>kafka.streams:type=stream-metrics,thread.client-id=([-.\w]+)</td>
+      </tr>
+      <tr>
+        <td>[commit | poll | process | punctuate]-rate</td>
+        <td>The average number of respective operations per second across all tasks.</td>
+        <td>kafka.streams:type=stream-metrics,thread.client-id=([-.\w]+)</td>
+      </tr>
+      <tr>
+        <td>task-created-rate</td>
+        <td>The average number of newly created tasks per second.</td>
+        <td>kafka.streams:type=stream-metrics,thread.client-id=([-.\w]+)</td>
+      </tr>
+      <tr>
+        <td>task-closed-rate</td>
+        <td>The average number of tasks closed per second.</td>
+        <td>kafka.streams:type=stream-metrics,thread.client-id=([-.\w]+)</td>
+      </tr>
+      <tr>
+        <td>skipped-records-rate</td>
+        <td>The average number of skipped records per second. </td>
+        <td>kafka.streams:type=stream-metrics,thread.client-id=([-.\w]+)</td>
+      </tr>
+ </tbody>
+</table>
+
+<h5><a id="kafka_streams_task_monitoring" href="#kafka_streams_task_monitoring">Task Metrics</a></h5>
+All the following metrics have a recording level of ``debug``:
+ <table class="data-table">
+      <tbody>
+      <tr>
+        <th>Metric/Attribute name</th>
+        <th>Description</th>
+        <th>Mbean name</th>
+      </tr>
+      <tr>
+        <td>commit-latency-[avg | max]</td>
+        <td>The [average | maximum] commit time in ns for this task. </td>
+        <td>kafka.streams:type=stream-task-metrics,streams-task-id=([-.\w]+)</td>
+      </tr>
+      <tr>
+        <td>commit-rate</td>
+        <td>The average number of commit calls per second. </td>
+        <td>kafka.streams:type=stream-task-metrics,streams-task-id=([-.\w]+)</td>
+      </tr>
+ </tbody>
+</table>
+
+ <h5><a id="kafka_streams_node_monitoring" href="#kafka_streams_node_monitoring">Processor Node Metrics</a></h5>
+All the following metrics have a recording level of ``debug``:
+ <table class="data-table">
+      <tbody>
+      <tr>
+        <th>Metric/Attribute name</th>
+        <th>Description</th>
+        <th>Mbean name</th>
+      </tr>
+      <tr>
+        <td>[process | punctuate | create | destroy]-latency-[avg | max]</td>
+        <td>The [average | maximum] execution time in ns, for the respective operation. </td>
+        <td>kafka.streams:type=stream-processor-node-metrics, processor-node-id=([-.\w]+)</td>
+      </tr>
+      <tr>
+        <td>[process | punctuate | create | destroy]-rate</td>
+        <td>The average number of respective operations per second. </td>
+        <td>kafka.streams:type=stream-processor-node-metrics, processor-node-id=([-.\w]+)</td>
+      </tr>
+      <tr>
+        <td>forward-rate</td>
+        <td>The average rate of records being forwarded downstream, from source nodes only, per second. </td>
+        <td>kafka.streams:type=stream-processor-node-metrics, processor-node-id=([-.\w]+)</td>
+      </tr>
+ </tbody>
+ </table>
+
+ <h5><a id="kafka_streams_store_monitoring" href="#kafka_streams_store_monitoring">State Store Metrics</a></h5>
+All the following metrics have a recording level of ``debug``:
+
+ <table class="data-table">
+      <tbody>
+      <tr>
+        <th>Metric/Attribute name</th>
+        <th>Description</th>
+        <th>Mbean name</th>
+      </tr>
+      <tr>
+        <td>[put | put-if-absent | get | delete | put-all | all | range | flush | restore]-latency-[avg | max]</td>
+        <td>The average execution time in ns, for the respective operation. </td>
+        <td>kafka.streams:type=stream-[store-type]-metrics</td>
+      </tr>
+        <tr>
+        <td>[put | put-if-absent | get | delete | put-all | all | range | flush | restore]-rate</td>
+        <td>The average rate of respective operations per second for this store.</td>
+        <td>kafka.streams:type=stream-[store-type]-metrics</td>
+      </tr>
+      
+    </tbody>
+</table>
+
+
+  <h4><a id="others_monitoring" href="#others_monitoring">Others</a></h4>
 
   We recommend monitoring GC time and other stats and various server stats such as CPU utilization, I/O service time, etc.