To monitor traces:

The Trace Explorer displays trace data determined by an out-of-the-box query. This page has the following main components, which enable you to run or edit out-of-the-box queries and customize the view to display trace data pertaining to the application area that you want to monitor.

• Charts, which is closed by default on the left, displays the spark charts you select from a query including timeseries.

  With the Charts panel open, if you run a query that includes timeseries, or click the Trace Services quick pick button, which already contains timeseries in the query, you can click on the spark charts in the table to add them to the Charts panel, as seen in the image bellow. You can add multiple charts to display in the panel, which allows you to take a better look at the charts and compare them at a glance.

• Attributes, on the left, lists the dimensions (with string values) and metrics (with numeric values) available for Traces and Spans, which you can add to a query. The attributes include:
  • Out-of-the-box attributes that are available in Application Performance Monitoring and configured by data sources, such as Service Name and Operation Name. Irrespective of the selected APM domain, the same set of out-of-the-box attributes are available in the Attributes list.
  • Custom attributes that you can add based on your requirements when configuring the data sources. The set of custom attributes are specific to the selected APM domain.

  If you hover the mouse over a dimension, a link is revealed and you can click it to view the values of the dimension in the Dimension <Name> Values dialog box. Note that a link is not revealed for metrics (numeric values) such as ApdexScore. Also, based on whether the dimension is listed in the Traces tab or the Spans tab within Attributes, the dimension values differ. If you click a dimension listed in the Traces tab, only the values in the traces (root spans) are displayed in the Dimension <Name> Values dialog and if you click the same dimension in the Spans tab, the values from all the spans are displayed. The Dimension <Name> Values dialog box also has the Copy, Add to Query, and Add to Global Filter buttons, which you can click to copy a dimension value or insert it in a query or global filter.

  For information on:

  • Trace and Span Dimensions, see About Trace and Span Attributes
  • Queries, see Work with Queries in Trace Explorer
• Compare spans, on the left, displays a list of spans selected to compare between each others.

  You can add up to a maximum of 4 spans to the list. For more information, see Compare Spans.

• Query Bar contains Quick pick buttons which come with out-of-the-box queries, an Actions icon () at the right side of the Query Bar, Run to execute a query, and a dropdown menu to View background queries.

  Quick pick buttons: By default the Quick pick buttons such as Traces, Services and Operations, come with out-of-the-box queries which are based on some of the likely trace data searches. The queries in Trace Explorer follow a simplified SQL-like syntax that include clauses and dimensions to filter, categorize, and display trace data, and enables you to filter huge amounts of trace data and view the information you're interested in.

  For example, here is the out-of-the-box query in the Traces quick pick, which uses clauses and out-of-the-box dimensions to display information pertaining to traces:

  show (traces) TraceStatus as Status,
      TraceFirstSpanStartTime as "Start Time",
      ServiceName as Service,
      OperationName as Operation,
      TraceDuration as Duration,
      span_summary() as Spans,
      ErrorCount as "Span Errors"
    where (ApmrumPageUpdateType <> 'Click' or ApmrumPageUpdateType is omitted)

  Based on this query, a list of a 100 traces is displayed with the following columns for the selected time period:

  • Service: Operation: Displays the root span of the trace, namely the service and the operation where the request starts. As specified by the where clause in the query, root spans in which the user activity operation (ApmrumPageUpdateType) is Click or where the value is not defined, are not displayed.
  • Status: Displays the status of the trace:
    • Complete: Indicates that the root span is available in Trace Explorer, but information regarding errors is not.
    • Incomplete: Indicates that the root span is not available in Trace Explorer, so the trace is incomplete.
    • Success: Indicates that the root span is available in Trace Explorer, and the error code and type denote success (HTTP 200).
    • Error: Indicates that the root span has errors.
  • Start Time: Displays the time when the root span started.

    The list of traces is sorted by the Start Time column, however, you can sort the list by any of the available columns. For example, you can sort the trace list by Duration to view the traces that took the longest time on the top of the list.

  • Duration: Displays the duration of time between the beginning of the first span and the end of the last span. The bar graph in the Duration column helps identify the trace which has been running for the longest time.
  • Spans: Displays the total number of spans in the trace. If you hover the mouse over the bar graph in the Spans column, a breakdown of the number of spans in each service in the trace and the number of errors, if any, are displayed.
  • Span Errors: Displays the number of spans with errors.

  Custom query bar: From Query Bars in Administration, under Resources, or from Manage using the Actions icon () at the right of the Query Bar, you can create a custom query bar, where you can add your own quick pick buttons with custom queries.

  From the Actions icon () at the right of the Query Bar, you can open other available query bars from the Open query bar panel to be the current query bar. You can also save the current Query Bar as your default.

  Background queries: You can create queries and run them in the background. This is useful when you have queries that may take some time to get executed. The query status and results can be retrieved at a later time.

  To run a background query: Go to the dropdown menu located next to Run and select the option: Run query in background. A window is displayed to create a background query. Update it with your preferred options and click Submit.

  After creating a background query, select View background queries to view the list of queries that were initialized earlier. You can see information about the background queries such as Status, Rows and Expires. Use the Actions icon () to see more options, such as View/Edit details, Fetch query results, New query from attributes, Re-run query and Delete.

  Use View/Edit details to edit a background query. For example, if you need to extend the expiration date of the background queries, update the Expires field. The default Expires value is 30 days.

  Use New query from attributes to refine the results of a background query. For example, if you want the results to return less rows, you can add a where clause with an attribute.

• Global filter allows you to add a filter for your query results that will stay on as you work in Trace Explorer, enabling you to keep certain criteria fixed while the query is changing as you, for example, switch between Quick Picks. The Global filter, formed as a where clause, will not be shown in the query box, but work alongside the query, and can be toggled on or off using the switch on the right side of the bar.

  Using the Actions icon () at the right of the Global filter bar, similarly to queries, you can save the current filter and open the Open global filter panel, where you can search, quick edit and open saved filters. You can further edit, manage and delete saved filters from Global Filters in Administration, under Resources.

You can also click the Actions icon () for each trace or span and perform the following actions:

• Show Trace Details or Show Span Details
• Show Trace Topology
• Show Traces or Show Spans

Note that the Show Trace Topology option is displayed in the Actions menu only if the query supports the Topology view. The Show Trace Details or Show Span Details options are displayed when a list of traces or spans is displayed and the Show Traces or Show Spans options are displayed for results of aggregate queries that have the group by clause.