Oracle Cloud Infrastructure Documentation
Try Free Tier

Audit Logs in Search with OpenSearch

Audit logs capture a comprehensive record of access, operations, and changes to a cluster and the cluster's data.

This feature aids you in monitoring activities, troubleshooting issues, and ensuring compliance with security policies for a cluster. For more information, see OpenSearch audit logs.

Prerequisites

  • The OpenSearch version for the cluster must be 2.11.

Enabling Audit Logs for a Cluster

Before you can use audit logs, you need to enable the functionality from the cluster's OpenSearch Dashboard.

  1. Sign in to the cluster's OpenSearch Dashboards, and select Security.

  2. Click Audit logs in the navigation menu on the left side.

  3. Toggle Enable audit logging from Disabled to Enabled.

General Settings

The general settings are the overall configuration settings for the audit logs, including whether the feature is enabled or disabled for a setting, along with the granularity of the data logged.

Layer Settings

Audit logs can be enabled or disabled for the REST layer and the transport layer.

  • REST Layer: When enabled, any REST requests to the cluster are logged.

    The REST disabled categories are:

    • AUTHENTICATED: Requests that are authenticated but not necessarily authorized aren't logged.
    • GRANTED_PRIVILEGES: Requests that are authenticated but not necessarily authorized aren't logged.

  • Transport Layer: When enabled, node-to-node requests within a cluster are logged.

    The transport disabled categories are:

    • AUTHENTICATED: Node-to-node requests that are authenticated but not necessarily authorized aren't logged.
    • GRANTED_PRIVILEGES: Node-to-node requests that were granted certain privileges aren't logged.

Attribute Settings

Attribute settings specify what parts of a request to log.

  • Bulk requests: If enabled, bulk requests are logged.

  • Request body: If enabled, the body of requests are logged.

  • Resolve indices: If enabled, the actual names of the indices accessed in the request are resolved and logged.

  • Sensitive headers: If enabled, headers that could contain sensitive information, such as authentication tokens, are logged.

Ignore Settings

Specifies whether to exclude any users or requests from being logged.

  • Ignored users: Specifies a list of users whose actions aren't logged.

  • Ignored requests: Specifies a list of request patterns that aren't logged.

Compliance Settings

These are the overall configuration settings for the compliance logging feature. They specify whether compliance logging is enabled and the granularity of the data logged.

Compliance Mode

Specifies whether the cluster operates in a mode where OpenSearch enforces certain compliance-related behaviors. If Compliance Logging is enabled, it could enforce stricter security checks, logging, and so on, based on the other compliance settings configured.

Config

Specifies whether any OpenSearch configuration updates are logged. Includes the following:

  • Internal Config Logging: If enabled, changes to the internal OpenSearch configuration are logged. This can be useful to track changes to security configurations, roles, permissions, and so on.

  • External Config Logging: If enabled, changes to external configurations, such as configurations stored outside of OpenSearch, in a file or an external service, are logged.

Read

Specifies logging configuration for read operations.

  • Read Metadata: If enabled, only metadata of read operations is logged, the actual data being read isn't logged. This is useful to monitor access patterns without logging sensitive data.

  • Ignored Users: Specifies a list of users whose read operations aren't logged.

  • Watched Fields: Specifies the fields within indices to monitor for read operations.

Write

Specifies logging configuration for write operations.

  • Write Metadata: If enabled, only metadata of write operations is logged, the actual data being written isn't logged. This is useful to track data modification patterns without logging the content of the data.

  • Log Diffs: If enabled, the differences between the old data and new data in write operations is logged. If disabled, the differences aren't logged, only that a write operation occurred is be logged.

  • Ignored Users: Specifies a list of users whose read operations aren't logged.

  • Watched Indices: Specifies the indices to monitor for write operations.

Using Audit Logs

After you enable audit logs for a cluster, OpenSearch starts populating the audit log indices with data. To access that data, you need to create an index pattern that points to the log data. An index pattern references one or more indexes, data streams, or index aliases.

Create Index Pattern

  1. Sign in to the cluster's OpenSearch Dashboards, and select Management.

  2. Click Stack Management in the navigation menu on the left side, and then select Index Patterns.

  3. Click Create index pattern.

  4. Specify the Index pattern name to match. By default, audit log index names start with security, so you can specify security* to include all audit logs. If you're not using the default configuration, specify the index pattern that matches the custom configuration.
  5. Click Next step.
  6. Select @timestamp for Step 2: Configure settings, and then click Create index pattern.

Searching Audit Logs

  1. Sign in to the cluster's OpenSearch Dashboards, and select Discover.

  2. Select the security* index pattern or other index pattern you created to reference audit log data from the upper-left dropdown menu.

At this point, you can add, delete, or change the data fields to search the log data.

Key Features

  • Filter and search:

    • Date range - use the date picker to narrow down events by specifying a start and end date.
    • Free-text search - use the search bar to look for specific terms or actions.
    • Field-based filters - apply filters based on specific fields such as audit_category, audit_request_origin, and so on.

  • Viewing Log Details

    Audit log entries typically have the following:
    • Timestamp - when the event occurred.
    • Audit category - the type of event, such as AUTHENTICATED, FAILED_LOGIN, and so on.
    • User - details about the user who triggered the event.
    • Request origin - where the request came from, such as REST, TRANSPORT, and so on.
    • Remote IP: - the IP address from which the event was triggered.

Exporting Audit Logs

To export audit logs for external analysis or backup:

  1. Use the search and filter tools to filter the logs you want to export. See Searching the data and Filtering the data.

  2. Click Share in the upper right, and select the export format you want to use.

Best Practices

  • Retention Policy: Set up index lifecycle policies to manage retention and deletion of old audit logs.
  • Monitoring: Regularly review audit logs for unusual patterns or suspicious activities.
  • Access Control: Restrict access to audit logs to authorized personnel only.

Was this article helpful?