You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
351 lines
18 KiB
351 lines
18 KiB
[[observability]] |
|
= Observability Support |
|
|
|
Micrometer defines an https://micrometer.io/docs/observation[Observation concept that enables both Metrics and Traces] in applications. |
|
Metrics support offers a way to create timers, gauges, or counters for collecting statistics about the runtime behavior of your application. |
|
Metrics can help you to track error rates, usage patterns, performance, and more. |
|
Traces provide a holistic view of an entire system, crossing application boundaries; you can zoom in on particular user requests and follow their entire completion across applications. |
|
|
|
Spring Framework instruments various parts of its own codebase to publish observations if an `ObservationRegistry` is configured. |
|
You can learn more about {docs-spring-boot}/html/actuator.html#actuator.metrics[configuring the observability infrastructure in Spring Boot]. |
|
|
|
|
|
[[observability.list]] |
|
== List of produced Observations |
|
|
|
Spring Framework instruments various features for observability. |
|
As outlined xref:integration/observability.adoc[at the beginning of this section], observations can generate timer Metrics and/or Traces depending on the configuration. |
|
|
|
.Observations produced by Spring Framework |
|
[%autowidth] |
|
|=== |
|
|Observation name |Description |
|
|
|
|xref:integration/observability.adoc#observability.http-client[`"http.client.requests"`] |
|
|Time spent for HTTP client exchanges |
|
|
|
|xref:integration/observability.adoc#observability.http-server[`"http.server.requests"`] |
|
|Processing time for HTTP server exchanges at the Framework level |
|
|
|
|xref:integration/observability.adoc#observability.jms.publish[`"jms.message.publish"`] |
|
|Time spent sending a JMS message to a destination by a message producer. |
|
|
|
|xref:integration/observability.adoc#observability.jms.process[`"jms.message.process"`] |
|
|Processing time for a JMS message that was previously received by a message consumer. |
|
|
|
|xref:integration/observability.adoc#observability.tasks-scheduled[`"tasks.scheduled.execution"`] |
|
|Processing time for an execution of a `@Scheduled` task |
|
|=== |
|
|
|
NOTE: Observations are using Micrometer's official naming convention, but Metrics names will be automatically converted |
|
https://micrometer.io/docs/concepts#_naming_meters[to the format preferred by the monitoring system backend] |
|
(Prometheus, Atlas, Graphite, InfluxDB...). |
|
|
|
|
|
[[observability.concepts]] |
|
== Micrometer Observation concepts |
|
|
|
If you are not familiar with Micrometer Observation, here's a quick summary of the concepts you should know about. |
|
|
|
* `Observation` is the actual recording of something happening in your application. This is processed by `ObservationHandler` implementations to produce metrics or traces. |
|
* Each observation has a corresponding `ObservationContext` implementation; this type holds all the relevant information for extracting metadata for it. |
|
In the case of an HTTP server observation, the context implementation could hold the HTTP request, the HTTP response, any exception thrown during processing, and so forth. |
|
* Each `Observation` holds `KeyValues` metadata. In the case of an HTTP server observation, this could be the HTTP request method, the HTTP response status, and so forth. |
|
This metadata is contributed by `ObservationConvention` implementations which should declare the type of `ObservationContext` they support. |
|
* `KeyValues` are said to be "low cardinality" if there is a low, bounded number of possible values for the `KeyValue` tuple (HTTP method is a good example). |
|
Low cardinality values are contributed to metrics only. |
|
Conversely, "high cardinality" values are unbounded (for example, HTTP request URIs) and are only contributed to traces. |
|
* An `ObservationDocumentation` documents all observations in a particular domain, listing the expected key names and their meaning. |
|
|
|
|
|
[[observability.config]] |
|
== Configuring Observations |
|
|
|
Global configuration options are available at the `ObservationRegistry#observationConfig()` level. |
|
Each instrumented component will provide two extension points: |
|
|
|
* setting the `ObservationRegistry`; if not set, observations will not be recorded and will be no-ops |
|
* providing a custom `ObservationConvention` to change the default observation name and extracted `KeyValues` |
|
|
|
|
|
[[observability.config.conventions]] |
|
=== Using custom Observation conventions |
|
|
|
Let's take the example of the Spring MVC "http.server.requests" metrics instrumentation with the `ServerHttpObservationFilter`. |
|
This observation uses a `ServerRequestObservationConvention` with a `ServerRequestObservationContext`; custom conventions can be configured on the Servlet filter. |
|
If you would like to customize the metadata produced with the observation, you can extend the `DefaultServerRequestObservationConvention` for your requirements: |
|
|
|
include-code::./ExtendedServerRequestObservationConvention[] |
|
|
|
If you want full control, you can implement the entire convention contract for the observation you're interested in: |
|
|
|
include-code::./CustomServerRequestObservationConvention[] |
|
|
|
You can also achieve similar goals using a custom `ObservationFilter` – adding or removing key values for an observation. |
|
Filters do not replace the default convention and are used as a post-processing component. |
|
|
|
include-code::./ServerRequestObservationFilter[] |
|
|
|
You can configure `ObservationFilter` instances on the `ObservationRegistry`. |
|
|
|
[[observability.tasks-scheduled]] |
|
== @Scheduled tasks instrumentation |
|
|
|
An Observation is created for xref:integration/scheduling.adoc#scheduling-enable-annotation-support[each execution of an `@Scheduled` task]. |
|
Applications need to configure the `ObservationRegistry` on the `ScheduledTaskRegistrar` to enable the recording of observations. |
|
This can be done by declaring a `SchedulingConfigurer` bean that sets the observation registry: |
|
|
|
include-code::./ObservationSchedulingConfigurer[] |
|
|
|
It is using the `org.springframework.scheduling.support.DefaultScheduledTaskObservationConvention` by default, backed by the `ScheduledTaskObservationContext`. |
|
You can configure a custom implementation on the `ObservationRegistry` directly. |
|
During the execution of the scheduled method, the current observation is restored in the `ThreadLocal` context or the Reactor context (if the scheduled method returns a `Mono` or `Flux` type). |
|
|
|
By default, the following `KeyValues` are created: |
|
|
|
.Low cardinality Keys |
|
[cols="a,a"] |
|
|=== |
|
|Name | Description |
|
|`code.function` _(required)_|Name of Java `Method` that is scheduled for execution. |
|
|`code.namespace` _(required)_|Canonical name of the class of the bean instance that holds the scheduled method. |
|
|`exception` _(required)_|Name of the exception thrown during the execution, or `"none"` if no exception happened. |
|
|`outcome` _(required)_|Outcome of the method execution. Can be `"SUCCESS"`, `"ERROR"` or `"UNKNOWN"` (if for example the operation was cancelled during execution). |
|
|=== |
|
|
|
|
|
[[observability.jms]] |
|
== JMS messaging instrumentation |
|
|
|
Spring Framework uses the Jakarta JMS instrumentation provided by Micrometer if the `io.micrometer:micrometer-core` dependency is on the classpath. |
|
The `io.micrometer.core.instrument.binder.jms.JmsInstrumentation` instruments `jakarta.jms.Session` and records the relevant observations. |
|
|
|
This instrumentation will create 2 types of observations: |
|
|
|
* `"jms.message.publish"` when a JMS message is sent to the broker, typically with `JmsTemplate`. |
|
* `"jms.message.process"` when a JMS message is processed by the application, typically with a `MessageListener` or a `@JmsListener` annotated method. |
|
|
|
NOTE: currently there is no instrumentation for `"jms.message.receive"` observations as there is little value in measuring the time spent waiting for the reception of a message. |
|
Such an integration would typically instrument `MessageConsumer#receive` method calls. But once those return, the processing time is not measured and the trace scope cannot be propagated to the application. |
|
|
|
By default, both observations share the same set of possible `KeyValues`: |
|
|
|
.Low cardinality Keys |
|
[cols="a,a"] |
|
|=== |
|
|Name | Description |
|
|`exception` |Class name of the exception thrown during the messaging operation (or "none"). |
|
|`messaging.destination.temporary` _(required)_|Whether the destination is a `TemporaryQueue` or `TemporaryTopic` (values: `"true"` or `"false"`). |
|
|`messaging.operation` _(required)_|Name of JMS operation being performed (values: `"publish"` or `"process"`). |
|
|=== |
|
|
|
.High cardinality Keys |
|
[cols="a,a"] |
|
|=== |
|
|Name | Description |
|
|`messaging.message.conversation_id` |The correlation ID of the JMS message. |
|
|`messaging.destination.name` |The name of destination the current message was sent to. |
|
|`messaging.message.id` |Value used by the messaging system as an identifier for the message. |
|
|=== |
|
|
|
[[observability.jms.publish]] |
|
=== JMS message Publication instrumentation |
|
|
|
`"jms.message.publish"` observations are recorded when a JMS message is sent to the broker. |
|
They measure the time spent sending the message and propagate the tracing information with outgoing JMS message headers. |
|
|
|
You will need to configure the `ObservationRegistry` on the `JmsTemplate` to enable observations: |
|
|
|
include-code::./JmsTemplatePublish[] |
|
|
|
It uses the `io.micrometer.core.instrument.binder.jms.DefaultJmsPublishObservationConvention` by default, backed by the `io.micrometer.core.instrument.binder.jms.JmsPublishObservationContext`. |
|
|
|
[[observability.jms.process]] |
|
=== JMS message Processing instrumentation |
|
|
|
`"jms.message.process"` observations are recorded when a JMS message is processed by the application. |
|
They measure the time spent processing the message and propagate the tracing context with incoming JMS message headers. |
|
|
|
Most applications will use the xref:integration/jms/annotated.adoc#jms-annotated[`@JmsListener` annotated methods] mechanism to process incoming messages. |
|
You will need to ensure that the `ObservationRegistry` is configured on the dedicated `JmsListenerContainerFactory`: |
|
|
|
include-code::./JmsConfiguration[] |
|
|
|
A xref:integration/jms/annotated.adoc#jms-annotated-support[default container factory is required to enable the annotation support], |
|
but note that `@JmsListener` annotations can refer to specific container factory beans for specific purposes. |
|
In all cases, Observations are only recorded if the observation registry is configured on the container factory. |
|
|
|
Similar observations are recorded with `JmsTemplate` when messages are processed by a `MessageListener`. |
|
Such listeners are set on a `MessageConsumer` within a session callback (see `JmsTemplate.execute(SessionCallback<T>)`). |
|
|
|
This observation uses the `io.micrometer.core.instrument.binder.jms.DefaultJmsProcessObservationConvention` by default, backed by the `io.micrometer.core.instrument.binder.jms.JmsProcessObservationContext`. |
|
|
|
[[observability.http-server]] |
|
== HTTP Server instrumentation |
|
|
|
HTTP server exchange observations are created with the name `"http.server.requests"` for Servlet and Reactive applications. |
|
|
|
[[observability.http-server.servlet]] |
|
=== Servlet applications |
|
|
|
Applications need to configure the `org.springframework.web.filter.ServerHttpObservationFilter` Servlet filter in their application. |
|
It uses the `org.springframework.http.server.observation.DefaultServerRequestObservationConvention` by default, backed by the `ServerRequestObservationContext`. |
|
|
|
This will only record an observation as an error if the `Exception` has not been handled by the web framework and has bubbled up to the Servlet filter. |
|
Typically, all exceptions handled by Spring MVC's `@ExceptionHandler` and xref:web/webmvc/mvc-ann-rest-exceptions.adoc[`ProblemDetail` support] will not be recorded with the observation. |
|
You can, at any point during request processing, set the error field on the `ObservationContext` yourself: |
|
|
|
include-code::./UserController[] |
|
|
|
NOTE: Because the instrumentation is done at the Servlet Filter level, the observation scope only covers the filters ordered after this one as well as the handling of the request. |
|
Typically, Servlet container error handling is performed at a lower level and won't have any active observation or span. |
|
For this use case, a container-specific implementation is required, such as a `org.apache.catalina.Valve` for Tomcat; this is outside of the scope of this project. |
|
|
|
By default, the following `KeyValues` are created: |
|
|
|
.Low cardinality Keys |
|
[cols="a,a"] |
|
|=== |
|
|Name | Description |
|
|`exception` _(required)_|Name of the exception thrown during the exchange, or `"none"` if no exception happened. |
|
|`method` _(required)_|Name of HTTP request method or `"none"` if the request was not received properly. |
|
|`outcome` _(required)_|Outcome of the HTTP server exchange. |
|
|`status` _(required)_|HTTP response raw status code, or `"UNKNOWN"` if no response was created. |
|
|`uri` _(required)_|URI pattern for the matching handler if available, falling back to `REDIRECTION` for 3xx responses, `NOT_FOUND` for 404 responses, `root` for requests with no path info, and `UNKNOWN` for all other requests. |
|
|=== |
|
|
|
.High cardinality Keys |
|
[cols="a,a"] |
|
|=== |
|
|Name | Description |
|
|`http.url` _(required)_|HTTP request URI. |
|
|=== |
|
|
|
|
|
[[observability.http-server.reactive]] |
|
=== Reactive applications |
|
|
|
Applications need to configure the `WebHttpHandlerBuilder` with a `MeterRegistry` to enable server instrumentation. |
|
This can be done on the `WebHttpHandlerBuilder`, as follows: |
|
|
|
include-code::./HttpHandlerConfiguration[] |
|
|
|
It is using the `org.springframework.http.server.reactive.observation.DefaultServerRequestObservationConvention` by default, backed by the `ServerRequestObservationContext`. |
|
|
|
This will only record an observation as an error if the `Exception` has not been handled by an application Controller. |
|
Typically, all exceptions handled by Spring WebFlux's `@ExceptionHandler` and <<web.adoc#webflux-ann-rest-exceptions,`ProblemDetail` support>> will not be recorded with the observation. |
|
You can, at any point during request processing, set the error field on the `ObservationContext` yourself: |
|
|
|
include-code::./UserController[] |
|
|
|
By default, the following `KeyValues` are created: |
|
|
|
.Low cardinality Keys |
|
[cols="a,a"] |
|
|=== |
|
|Name | Description |
|
|`exception` _(required)_|Name of the exception thrown during the exchange, or `"none"` if no exception happened. |
|
|`method` _(required)_|Name of HTTP request method or `"none"` if the request was not received properly. |
|
|`outcome` _(required)_|Outcome of the HTTP server exchange. |
|
|`status` _(required)_|HTTP response raw status code, or `"UNKNOWN"` if no response was created. |
|
|`uri` _(required)_|URI pattern for the matching handler if available, falling back to `REDIRECTION` for 3xx responses, `NOT_FOUND` for 404 responses, `root` for requests with no path info, and `UNKNOWN` for all other requests. |
|
|=== |
|
|
|
.High cardinality Keys |
|
[cols="a,a"] |
|
|=== |
|
|Name | Description |
|
|`http.url` _(required)_|HTTP request URI. |
|
|=== |
|
|
|
|
|
|
|
[[observability.http-client]] |
|
== HTTP Client Instrumentation |
|
|
|
HTTP client exchange observations are created with the name `"http.client.requests"` for blocking and reactive clients. |
|
Unlike their server counterparts, the instrumentation is implemented directly in the client so the only required step is to configure an `ObservationRegistry` on the client. |
|
|
|
[[observability.http-client.resttemplate]] |
|
=== RestTemplate |
|
|
|
Applications must configure an `ObservationRegistry` on `RestTemplate` instances to enable the instrumentation; without that, observations are "no-ops". |
|
Spring Boot will auto-configure `RestTemplateBuilder` beans with the observation registry already set. |
|
|
|
Instrumentation uses the `org.springframework.http.client.observation.ClientRequestObservationConvention` by default, backed by the `ClientRequestObservationContext`. |
|
|
|
.Low cardinality Keys |
|
[cols="a,a"] |
|
|=== |
|
|Name | Description |
|
|`method` _(required)_|Name of HTTP request method or `"none"` if the request could not be created. |
|
|`uri` _(required)_|URI template used for HTTP request, or `"none"` if none was provided. Only the path part of the URI is considered. |
|
|`client.name` _(required)_|Client name derived from the request URI host. |
|
|`status` _(required)_|HTTP response raw status code, or `"IO_ERROR"` in case of `IOException`, or `"CLIENT_ERROR"` if no response was received. |
|
|`outcome` _(required)_|Outcome of the HTTP client exchange. |
|
|`exception` _(required)_|Name of the exception thrown during the exchange, or `"none"` if no exception happened. |
|
|=== |
|
|
|
.High cardinality Keys |
|
[cols="a,a"] |
|
|=== |
|
|Name | Description |
|
|`http.url` _(required)_|HTTP request URI. |
|
|=== |
|
|
|
|
|
[[observability.http-client.restclient]] |
|
=== RestClient |
|
|
|
Applications must configure an `ObservationRegistry` on the `RestClient.Builder` to enable the instrumentation; without that, observations are "no-ops". |
|
|
|
Instrumentation uses the `org.springframework.http.client.observation.ClientRequestObservationConvention` by default, backed by the `ClientRequestObservationContext`. |
|
|
|
.Low cardinality Keys |
|
[cols="a,a"] |
|
|=== |
|
|Name | Description |
|
|`method` _(required)_|Name of HTTP request method or `"none"` if the request could not be created. |
|
|`uri` _(required)_|URI template used for HTTP request, or `"none"` if none was provided. Only the path part of the URI is considered. |
|
|`client.name` _(required)_|Client name derived from the request URI host. |
|
|`status` _(required)_|HTTP response raw status code, or `"IO_ERROR"` in case of `IOException`, or `"CLIENT_ERROR"` if no response was received. |
|
|`outcome` _(required)_|Outcome of the HTTP client exchange. |
|
|`exception` _(required)_|Name of the exception thrown during the exchange, or `"none"` if no exception happened. |
|
|=== |
|
|
|
.High cardinality Keys |
|
[cols="a,a"] |
|
|=== |
|
|Name | Description |
|
|`http.url` _(required)_|HTTP request URI. |
|
|=== |
|
|
|
|
|
[[observability.http-client.webclient]] |
|
=== WebClient |
|
|
|
Applications must configure an `ObservationRegistry` on the `WebClient` builder to enable the instrumentation; without that, observations are "no-ops". |
|
Spring Boot will auto-configure `WebClient.Builder` beans with the observation registry already set. |
|
|
|
Instrumentation uses the `org.springframework.web.reactive.function.client.ClientRequestObservationConvention` by default, backed by the `ClientRequestObservationContext`. |
|
|
|
.Low cardinality Keys |
|
[cols="a,a"] |
|
|=== |
|
|Name | Description |
|
|`method` _(required)_|Name of HTTP request method or `"none"` if the request could not be created. |
|
|`uri` _(required)_|URI template used for HTTP request, or `"none"` if none was provided. Only the path part of the URI is considered. |
|
|`client.name` _(required)_|Client name derived from the request URI host. |
|
|`status` _(required)_|HTTP response raw status code, or `"IO_ERROR"` in case of `IOException`, or `"CLIENT_ERROR"` if no response was received. |
|
|`outcome` _(required)_|Outcome of the HTTP client exchange. |
|
|`exception` _(required)_|Name of the exception thrown during the exchange, or `"none"` if no exception happened. |
|
|=== |
|
|
|
.High cardinality Keys |
|
[cols="a,a"] |
|
|=== |
|
|Name | Description |
|
|`http.url` _(required)_|HTTP request URI. |
|
|=== |
|
|
|
|
|
|