Deploy the APM Java Agent

On successfully provisioning the APM Java agent, you can deploy the APM Java agent.

To deploy the APM Java agent on any Java application, you need to add the -javaagent parameter to the JVM startup script. Depending on the Java environment, an application server or a microservice, users may have shell or bat startup scripts, or another way to execute the Java command line.

See below examples on how to deploy the APM Java agent on the following Java applications:

Oracle WebLogic Server

Here's information on how to deploy the APM Java agent on the Oracle WebLogic Server.

  1. Set a variable to point to your application server destination directory. This is the directory in which the APM Java agent is provisioned.

    Set the $DOMAIN_HOME variable to point to the Oracle WebLogic Server Domain directory and confirm that the APM Java agent was provisioned in the same destination directory before you perform the next step.

    Application Server Destination Directory Variable Information
    Oracle WebLogic Server Set the $DOMAIN_HOME variable to point to the Oracle WebLogic Server Domain.
    export DOMAIN_HOME=<Oracle WebLogic Server Domain>
  2. Make a backup copy of the startWebLogic.sh file:
    cd $DOMAIN_HOME/bin
    cp startWebLogic.sh startWebLogic.sh.orig
  3. Use a text editor and edit the original startWebLogic.sh script and add the -javaagent option.
    1. If you're deploying the APM Java agent on the Oracle WebLogic Administration Server and Managed Servers, then add the following -javaagent option to the set of JAVA_OPTIONS, after the setDomainEnv.sh call:
      JAVA_OPTIONS="${JAVA_OPTIONS} -javaagent:$DOMAIN_HOME/oracle-apm-agent/bootstrap/ApmAgent.jar"
    2. If you're deploying the APM Java agent on Managed Servers only, then add the following -javaagent option to the set of JAVA_OPTIONS within an if statement after the setDomainEnv.sh call:
      if [ "$SERVER_NAME" != "AdminServer" ] ; then
              set JAVA_OPTIONS="${JAVA_OPTIONS} -javaagent:$DOMAIN_HOME/oracle-apm-agent/bootstrap/ApmAgent.jar"
      fi
  4. Stop and restart the Oracle WebLogic Server:
    cd $DOMAIN_HOME/bin
    ./stopWebLogic.sh
    cd ..
    nohup ./startWebLogic.sh >& startup.log &

    If you have Managed Servers, stop and restart them too:

    cd $DOMAIN_HOME/bin
            ./stopManagedWebLogic.sh {SERVER_NAME} {ADMIN_URL} {USER_NAME} {PASSWORD}
            nohup ./startManagedWebLogic.sh {SERVER_NAME} {ADMIN_URL} >& {SERVER_NAME}.log &
    Note

    Notice that the $DOMAIN_HOME version of startWebLogic.sh is used, even though you edited the $DOMAIN_HOME/bin version. Invoking the command from one level higher (from $DOMAIN_HOME) invokes the command from a lower level (from $DOMAIN_HOME/bin). However, the stopWebLogic.sh command will be called from the $DOMAIN_HOME/bin directory.

After the APM Java agent is successfully deployed, the Oracle APM Agent: Initialized AgentInstance message is displayed in the server startup log.

Apache Tomcat Server

Here's information on how to deploy the APM Java agent on the Apache Tomcat Server.

  1. Set a variable to point to your application server destination directory. This is the directory in which the APM Java agent is provisioned.

    Set the $CATALINA_HOME variable to point to the Apache Tomcat Server destination directory and confirm that the APM Java agent was provisioned in the same destination directory before you perform the next step.

    Application Server Destination Directory Variable Information
    Apache Tomcat Server Set the $CATALINA_HOME variable to point to the Apache Tomcat Server destination directory.
    • If you're using a Bash shell:
      export CATALINA_HOME=<Apache Tomcat Server destination directory>
    • If you're using a C shell:
      setenv CATALINA_HOME "<Apache Tomcat Server destination directory>"
  2. Make a backup copy of the catalina.sh file.
    $ cd $CATALINA_HOME/bin
    $ cp catalina.sh catalina.sh.orig
  3. Use a text editor and edit the original catalina.sh file and add the following -javaagent option to CATALINA_OPTS. Make the change outside of any if statements or code blocks that may not be executed during server startup. This will ensure that the -javaagent flag is always added to the server startup options.
    CATALINA_OPTS="${CATALINA_OPTS} -javaagent:$CATALINA_HOME/oracle-apm-agent/bootstrap/ApmAgent.jar"
  4. Stop and restart the Apache Tomcat Server:
    $ cd $CATALINA_HOME/bin                      
    $ ./shutdown.sh
    $ ./startup.sh

After the APM Java agent is successfully deployed, the Oracle APM Agent: Initialized AgentInstance message is displayed in the server startup log.

For more information, see Install an APM Java Agent on a Tomcat Application Server Tutorial.

Jetty Server

Here's information on how to deploy the APM Java agent on the Jetty Server.

  1. Set a variable to point to your application server destination directory. This is the directory in which the APM Java agent is provisioned.

    Set the JETTY_HOME variable to point to the Jetty Server destination directory (where the Jetty software was extracted) and confirm that the APM Java agent was provisioned in the same destination directory before you perform the next step.

  2. Start the Jetty server.
    java -javaagent:/<Destination_Directory>/oracle-apm-agent/bootstrap/ApmAgent.jar -jar $JETTY_HOME/start.jar

After the APM Java agent is successfully deployed, the Oracle APM Agent: Initialized AgentInstance message is displayed in the server startup log.

Spring Boot

Here's information on how to deploy an APM Java agent on a Spring Boot microservice running embedded Apache Tomcat.

It's assumed that you've completed the prerequisite tasks and provisioned the APM Java agent. Also, you must ensure that you've added the following properties to the Spring Boot application's application.properties file to enable Apache Tomcat Mbeans:
spring.jmx.enabled=true
server.tomcat.mbeanregistry.enabled=true

Alternatively, add the above properties: spring.jmx.enabled and server.tomcat.mbeanregistry.enabled as system properties on the command line.

To deploy the APM Java agent, add the following -javaagent option to the startup script of your microservice. Note that <Destination Directory> denotes the directory in which you provisioned the agent.
java -javaagent:<Destination Directory>/oracle-apm-agent/bootstrap/ApmAgent.jar -jar target/<microservice.jar>

After the APM Java agent is successfully deployed, the Oracle APM Agent: Initialized AgentInstance message is displayed in the microservice startup log.

JBoss Server

Here's information on how to deploy the APM Java agent on the JBoss Server.

The below instructions are applicable for JBoss EAP and Wildfly.

  1. Set a variable to point to your application server destination directory. This is the directory in which the APM Java agent is provisioned.

    Set the $JBOSS_HOME variable to point to the JBoss Server destination directory and confirm that the APM Java agent was provisioned in the same destination directory before you perform the next step.

    Application Server Destination Directory Variable Information
    JBoss Server

    Set the $JBOSS_HOME variable to point to the JBoss Server destination directory.

    • If you're using a Bash shell:
      export JBOSS_HOME=<JBoss Server destination directory>
    • If you're using a C shell:
      setenv JBOSS_HOME "<JBoss Server destination directory>"
  2. Make a backup copy of the standalone.conf file:
    cd $JBOSS_HOME/bin
    cp standalone.conf standalone.conf.orig
  3. Use a text editor and edit the original standalone.conf file and add the following Java options to JAVA_OPTS. Make the change outside of any if statements or code blocks that may not be executed during server startup.
    • Add -javaagent option to JAVA_OPTS.
      JAVA_OPTS="$JAVA_OPTS -javaagent:$JBOSS_HOME/oracle-apm-agent/bootstrap/ApmAgent.jar"
    • Edit Java property jboss.modules.system.pkgs to include "com.oracle.apm".

      For example:

      JAVA_OPTS="$JAVA_OPTS -Djboss.modules.system.pkgs=org.jboss.byteman,org.jboss.logmanager,com.oracle.apm"
      Note

      Each environment is different. The Jboss server's property may include different packages from what are shown in the above example.
  4. Stop and restart the JBoss Server:
    cd $JBOSS_HOME/bin                      
    ./jboss-cli.sh -c :shutdown                        
    nohup ./standalone.sh -b 0.0.0.0&> startup.log &

After the APM Java agent is successfully deployed, the Oracle APM Agent: Initialized AgentInstance message is displayed in the server startup log.

Docker and Kubernetes

Here's information on how to deploy an APM Java agent in a Docker container and Oracle Container Engine for Kubernetes (OKE).

There are different options available to deploy the APM Java Agent in Docker and Kubernetes environments. Review the below and select the scenario that meets your business needs:
  1. Deploy APM Java Agent into Docker Container Image
  2. Deploy APM Java Agent using OpenTelemetry
  3. Deploy APM Java Agent on a Mounted Volume

Deploy APM Java Agent into Docker Container Image

Here's information on how to deploy an APM Java agent into a Docker container image and Oracle Container Engine for Kubernetes (OKE).

Figure 3-1 Deploy APM Java Agent into Docker Container Image

Deploy APM Java Agent into Docker Container Image

Recommendation:

Use this option when it's feasible to make changes to the Docker container image.

For example, if you may need to change the APM agent configuration, you are able to make changes to the Docker image.

To deploy the APM Java agent, follow the below steps:

  1. Before proceeding, confirm you have completed the prerequisites and provisioned the APM Java agent.
    Note

    When provisioning the APM Java agent, it's recommended to provision it in any location of your local machine and then copy it to a Docker image.
  2. Modify your Dockerfile to copy the APM Java agent to a Docker image:
    COPY <DESTINATION_DIRECTORY>/oracle-apm-agent <Docker_Image_Directory>/oracle-apm-agent/

    Note that <DESTINATION_DIRECTORY> denotes the location on your local machine where you provisioned the APM Java agent and <Docker_Image_Directory> denotes the directory in the Docker image to which you're copying the APM Java agent. The <Docker_Image_Directory> could also be the application server destination directory in Docker, for example, $DOMAIN_HOME, if you're working with Oracle WebLogic Server.

  3. Add the following -javaagent option to the startup script of your application server:
    java -javaagent:<Docker_Image_Directory>/oracle-apm-agent/bootstrap/ApmAgent.jar -jar target/<appserver.jar>
  4. Build a new Docker image with the built-in APM Java agent and the modified startup script, and push the image to the registry.

If you use Kubernetes to manage your Docker containers, then update your Kubernetes configuration to use the new Docker image, and restart the Kubernetes pod.

Also, you can set additional dimensions to be reported from the Kubernetes pod using the Downward API, by copying the following environment and volume settings in the deployment specification (yaml file) of the Kubernetes pod. For information on the Downward API, see The Downward API in Kubernetes documentation.

Environment Settings

spec:
  containers:
  - name: <container-name>
    image: image: <your-registry>/<your-docker-image>:latest
    env:
      - name: APM_ATTRIBUTES_K8S_POD_NAME
        valueFrom:
          fieldRef:
            fieldPath: metadata.name
      - name: APM_ATTRIBUTES_K8S_NAMESPACE_NAME
        valueFrom:
          fieldRef:
            fieldPath: metadata.namespace
      - name: APM_ATTRIBUTES_K8S_NODE_NAME
        valueFrom:
          fieldRef:
            fieldPath: spec.nodeName

Volume Settings

spec:
  containers:
  - name: <container-name>
    image: image: <your-registry>/<your-docker-image>:latest
    volumeMounts:
      - name: apm-attributes-k8s
        mountPath: /etc/apm-attributes-k8s
    volumes:
      - name: apm-attributes-k8s
        downwardAPI:
         items:
           - path: "labels"
             fieldRef:
               fieldPath: metadata.labels
           - path: "annotations"
             fieldRef:
               fieldPath: metadata.annotations
Note

If the Kubernetes deployment does not have labels, annotations, or both, the Downward API for the same will cause an error when the application is deployed. In this case, you must remove the Downward API entry corresponding to metadata.labels, metadata.annotations, or both.

Deploy APM Java Agent using OpenTelemetry

Here's information on how to deploy an APM Java agent using the OpenTelemetry operator to automatically inject and configure the APM Java agent into your Java application pods running on Kubernetes (K8s) clusters.

Figure 3-2 Deploy APM Java Agent using OpenTelemetry

Deploy APM Java Agent using OpenTelemetry

Recommendation:

Use this option if it's not possible to update the Docker container image and you prefer to make APM Java agent configuration changes using Kubernetes custom resource (CR) to automatically inject the APM agent into the JVMs at startup.

Docker and Kubernetes Considerations:
  • For the Docker image vesioning, avoid using the tag :latest when deploying containers in production as it is harder to track which version of the image is running and more difficult to roll back properly. Instead, specify a meaningful tag such as v1.12.1.3.
  • For Kubernetes, take backups of the Kubernetes custom resources (CRs) and configmaps.

Prerequisite: Install the OpenTelemetry Operator into the Kubernetes cluster.

There are three different options available: Operator release manifest, Operator helm chart, or Operator Hub.

In most cases, a cert-manager should be installed. If the helm chart option is used, there is an option to generate a self-signed cert instead.

For a quick start, if the cert manager needs to be installed, run the below command:
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.14.2/cert-manager.yaml

To install the OpenTelemetry operator, run the following:

kubectl apply -f https://github.com/open-telemetry/opentelemetry-operator/releases/latest/download/opentelemetry-operator.yaml

Deploy the APM Java agent

To deploy the APM Java agent, follow these steps:

  1. Create a Kubernetes Custom Resource (CR).

    To manage automatic instrumentation, the OpenTelemetry Operator needs to be provided with information about the APM Java agent and its configuration which it's done using the Custom Resource Definition (CRD).

    This Custom Resource will be used by the operator to copy the agent into the pod and add to it the required configuration.

    To create the CR, run the following:
    kubectl apply -f - <<EOF
    apiVersion: opentelemetry.io/v1alpha1
    kind: Instrumentation
    metadata:
      name: inst-apm-java
      namespace: opentelemetry-operator-system
    spec:
      java:
        image: us-ashburn-1.ocir.io/idpyjkbcc9tk/apm-java-agent-aio:1.12.1.3
        env:
          - name: OTEL_com_oracle_apm_agent_data_upload_endpoint
            value: <data-upload-endpoint>
          - name: OTEL_com_oracle_apm_agent_private_data_key
            value: <private-data-key>
    EOF
    Where:
    The created CR can be queried by running the following command:
    kubectl get otelinst -n opentelemetry-operator-system
    All endpoints and environment variables required to be correct for auto-instrumentation to work properly.
  2. Add the Kubernetes annotation.

    The OpenTelemetry operator uses Kubernetes annotation to decide which pods should be auto-injected with the APM Java agent.

    The annotation can be added to a namespace. In that case, all the pods within that namespace will get injected. The annotation can also be added to individual PodSpec objects, available as part of Deployment, Statefulset, and other resources.

    Annotation:
    instrumentation.opentelemetry.io/inject-java: "opentelemetry-operator-system/inst-apm-java"
    To start editing your namespace, do the following:
    • Run the command:
      kubectl edit namespace <your-namespace-name>
    • Edit the namespace once the editor is opened. For example, vi editor.
    • Add the annotation to the namespace. Keep in mind indentation is very important to make it as a valid YAML file.
    Example:
    apiVersion: v1
    kind: Namespace
    metadata:
      labels:
        kubernetes.io/metadata.name: mynamespace
      annotations:
        instrumentation.opentelemetry.io/inject-java: "opentelemetry-operator-system/inst-apm-java"
      name: mynamespace
    spec:
  3. Restart the Kubernetes pod.

    To restart the pod where you want to auto-inject the APM Java agent, run the following:

    kubectl delete pod <your-pod-name> -n <your-namespace-name>
  4. Verify the Kubernetes pod.
    To verify that your pod has been auto-injected with the APM Java agent after it was restarted, run the following:
    kubectl get pod <your-pod-name> -n <your-namespace-name> -o yaml

You can now go to the next step: Verify APM Java Agent Deployment.

Note

For more information about how to deploy an APM Java agent using OpenTelemetry, check out the blog: Automatically deploy an APM Java agent in Kubernetes Environments using the OpenTelemetry Operator

Deploy APM Java Agent on a Mounted Volume

Here's information on how to deploy an APM Java agent in Oracle Container Engine for Kubernetes (OKE) using a mounted volume.

Figure 3-3 Deploy APM Java Agent on a Mounted Volume

Deploy APM Java Agent on a Mounted Volume
Recommendation:
  • Use this option when it's not possible to make changes to the Docker container image and you prefer to use a shared mounted volume when changes to the APM Java agent are needed.

    • Example 1: If you need to make frequent configuration changes to the APM agent then the container image requires to get updated, but it's not feasible to do it.
    • Example 2: If the user who is deploying the APM agent does not have the required access or permissions to rebuild a container image.
  • For the docker image versioning, back up the binary and config files.

To deploy the APM Java agent on a mounted volume, follow these steps:

  1. Confirm you completed the APM Java agent prerequisite tasks.

    Make a note of the data upload endpoint and data key when the APM Domain was created.

  2. Create a new file system that will mount the pods.

    When creating the file system, it's important to ensure to select the same Virtual Cloud Network (VCN) Compartment that Kubernetes is using. Do the same for the Subnet Compartment.

  3. Mount the file system into the pods.

    This step requires edits to the corresponding yaml files.

    Create the PersistentVolume and related entities in Kubernetes, using the below yaml file. Note the following fields you need to edit for your environment: mntTargetId, server and path.

    kind: StorageClass
    apiVersion: storage.k8s.io/v1
    metadata:
      name: oci-fss
    provisioner: oracle.com/oci-fss
    parameters:
      mntTargetId: ocid1.mounttarget.oc1.iad.xxxxxxxxxxxxxxxxxxxxxx
    ---
    apiVersion: v1
    kind: PersistentVolume
    metadata:
     name: oke-fsspv
    spec:
     storageClassName: oci-fss
     capacity:
      storage: 10Gi
     accessModes:
      - ReadWriteMany
     mountOptions:
      - nosuid
     nfs:
    # Replace this with the IP of your FSS file system in OCI
      server: 10.0.10.39
    # Replace this with the Path of your FSS file system in OCI
      path: "/fss-for-kub"
      readOnly: false
    ---
    apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
     name: oke-fsspvc
    spec:
     storageClassName: oci-fss
     accessModes:
     - ReadWriteMany
     resources:
      requests:
     # Although storage is provided here it is not used for FSS file systems
        storage: 10Gi
     volumeName: oke-fsspv

    To apply the changes, run kubectl apply -f <filename.yaml>

    Then, update the yaml file that manages the pods and add the volume and volume mount.

    To take effect, recreate the pods.

  4. Download the APM Java agent file.

    Download and copy the file to the mounted volume.

    For download instructions, see Download the APM Java Agent Software.

    After downloading, copy it to the mounted volume.

  5. Provision the APM Java Agent.

    Log in to one of the containers to provision the APM Java Agent, locate the apm-java-agent-installer jar file and run the following:

    java -jar ./apm-java-agent-installer-<version>.jar provision-agent -service-name=<Name of the Service> -destination=<Destination_Directory> -private-data-key=<Agent installation key generated during APM domain creation> -data-upload-endpoint=<dataUploadEndpoint URL generated during APM domain creation>

    For provisioning instructions, see Provision the APM Java Agent.

  6. Deploy the APM Java Agent.

    Deploy the APM Java Agent by providing the oracle-apm-agent location to the microservice in the yaml file.

    Add the -javaagent argument and the APM agent jar file location to the java command of each microservice:

    java -javaagent:<Mounted Volume>/oracle-apm-agent/bootstrap/ApmAgent.jar
  7. Restart Kubernetes.

    Recreate the pods by running: kubectl apply -f <filename.yaml>.

    You can now go to the next step: Verify APM Java Agent Deployment.

Note

For more information about how to deploy an APM Java agent on a mounted volume, watch the video: Kubernetes Spring Boot Instrumentation for Distributed Tracing or check out the blog: Application Performance Monitoring: Instrument Java on Kubernetes for Monitoring and Diagnostics.