Oracle Cloud Infrastructure Documentation

APM Java Agent Configurable Options

The APM Java agent has the following configurable options on data collection:

Abridged Traces

This feature is designed to eliminate cases of excessive span count per trace. When too many spans are created in a single call, the agent may cause a higher than desired overhead, it may also increase the hourly span count, causing a higher-than-expected charge. Cases when this may be a problem include recursive calls (intentional or not), generated code resulting in high number of SQLs executed per transaction, and others.

  • The abridge feature allows users to limit the maximum number of spans per call. It also provides a way to exclude spans that may be considered “not worthy” (For example, high volume of very quick SQLs).
  • Probe specific count limits, span name based inclusion/exclusion, and JDBC thresholding are configurable per thread, per call. For details about configuration options, see ProbeConfig.acml file under the oracle-apm-agent\config directory.
  • The default setting is to limit each call to 100 spans.
  • Basic summary statistics about dropped spans can be collected per thread, per call.

Below is an example of a setting that will limit the maximum number of JDBC spans per thread call to 50, exclude a known mundane SQL, and exclude SQLs that executes in less than 2ms. Note that by enabling the summarize option, the agent will add the following metrics to the parent span: dropped span count, avg dropped span duration, min dropped span time, max dropped span time, dropped span error count, as well as log records showing the 5 longest dropped spans, and first 5 dropped errors. 

abridged_probes: 
 summarize: true 
 settings_by_probe: 
 - probe: "JDBC" 
   span_limit: 50 
   excluded_patterns: 
    - contains: "select sysdate from dual" 
      excluded: true 
    threshold: 
      duration: 2 
      start_thresholding_after: 10

Circuit Breaker

  • The circuit breaker automatically reduces agent consumption of JVM resources when heavy resource pressure is detected. The circuit breaker will automatically resume normal agent function when resource pressure has been sufficiently alleviated.
  • By default, the circuit breaker incrementally and automatically halts/resumes agent work based on heap memory usage and garbage collection time.
    Performance Factor Halt Resume Interval Targeted Components
    Heap Memory Usage 95% 85% 2 minutes Probes
          5 minutes Full agent
    Garbage Collection Time 10% 5% 2 minutes Probes
          5 minutes Full agent
  • Performance factors, halt thresholds, resume thresholds, intervals, and targeted components are configurable. For details, see CircuitBreakerConfig.acml file under the oracle-apm-agent\config\<version> directory.

Sampling

  • By default, all spans are collected. To change the sampling configuration, you need to configure custom sampling.
  • You can reduce the amount of trace data by specifying sampling configuration. For details, see Configure APM Sampling.
  • Sampling operation is subject to data collection limit of the abridged traces and circuit breaker.

Log Library Injection

The active spanId and traceId can be injected into log messages. This provides correlation between log messages and APM Traces for easier log analysis and troubleshooting.

To configure this feature, do the following:

  • Enable the LOG_LIB probe in the ProbeConfig.acml file.

  • Enable the desired log injection in the detailed LOG_LIB section in the ProbeConfig.acml.

Enabling log library injection does the following:

  • The automatic injection allows the injection of spanId and traceId without modification to existing application logging configuration.

    Log4j 1.2, log4j 2, logback and java.util.logging are supported.

  • WebLogic Server access log injection can inject the active spanId and traceId for a given HTTP request. Note that static content such as favicon.ico typically does not have an associated spanId or traceId.

  • Mapped Diagnostic Context (MDC) injection takes advantage of built-in features of log4j, log4j2, and logback to make the active spanId and traceId available to logging frameworks.

    You can specify the following keys in the appropriate MDC logging configuration for the application's logging framework:
    • oracle.apm.spanId
    • oracle.apm.traceId