Ingest Logs

You can load your logs into Oracle Logging Analytics using the Oracle Management Agent, uploading logs on-demand, collecting them from OCI Object Store, or using Service Connector.

Ensure to complete the prerequisites for configuring Oracle Logging Analytics. The permissions required to ingest logs are detailed in the corresponding section depending on the method of ingestion you select.

Typically, you would select one of these methods depending on your use case and how you are able to access the logs:

Log Collection Method Description More Information

Set Up Continuous Log Collection From Your Hosts

Based on your source type, you can follow specific workflow too:

  • File
  • System Event Messages (Syslog)
  • Database
  • REST API
  • Windows Event Messages
  • Oracle Diagnostic Logs (ODL)

Install Management Agents on your on-premises hosts to continuously collect logs in Oracle Logging Analytics. This is a one-time setup.

  • File: For collecting most types of logs, such as Application, and Infrastructure logs.
  • System Event Messages (Syslog)
  • Database: Collect logs stored in tables and views of a database like Oracle Database Instance, Oracle Pluggable Database, Oracle Autonomous Database, Microsoft SQL Server Database Instance, and MySQL Database Instance.
  • REST API: PULL logs periodically from the log endpoint provided by you through REST API calls.
  • Windows Event Messages: All historic Windows Event Log entries as well as custom event channels.
  • Oracle Diagnostic Logs (ODL): These are typically the diagnostic logs for Oracle Fusion Middleware and Oracle Applications.
Set Up Continuous Log Collection From Your Hosts

Upload Logs on Demand

This is useful when you have log data from old applications that aren’t supported by the Oracle Management Agent or you want to load a log file a single time to analyze them.

Upload Logs on Demand

Upload Event Logs Using LogEvents API

Use the LogEvents API to PUSH logs from your log-emitting host to Oracle Logging Analytics through an Oracle-provided endpoint.

Upload Event Logs Using LogEvents API

Collect Logs from Your Object Storage Bucket

You can continuously collect log data that you have stored in an Oracle Cloud Object Store bucket.

Collect Logs from Your OCI Object Storage Bucket

Ingest Logs from Other OCI Services Using Service Connector

Use the Service Connector to identify your Oracle Cloud Infrastructure service as the source of the logs and Oracle Cloud Logging Analytics as the destination and enable the log collection.

Ingest Logs from Other OCI Services Using Service Connector

Ingest Custom Logs from OCI Logging Service Using Service Connector

If you have custom logs in OCI Logging service for which Oracle-defined parsers and sources are not available, then you can create your own parser and source. Create a service connector to establish connection with OCI Logging service and enable collection of your custom logs in Oracle Logging Analytics.

Ingest Custom Logs from OCI Logging Service Using Service Connector

Ingest Custom Logs from OCI Streaming Service Using Service Connector

If you want to analyze and derive insights from your high-volume custom streaming data in OCI Streaming service, then create a service connector to establish connection with OCI Streaming service and enable collection of your custom logs in Oracle Logging Analytics.

Ingest Custom Logs from OCI Streaming Service Using Service Connector

Harvest Entity Model Data from Enterprise Manager Cloud Control and Collect Logs

If you are an existing user of EMCC, then you can set up the EM Bridge and benefit from the automated set up that imports their entity model data like the entity associations, and properties information from EMCC into Oracle Logging Analytics. The entity association information is then used to construct the topology in Oracle Logging Analytics. To collect your logs from your targets in EMCC, follow the steps described.

Harvest Entity Model Data from Enterprise Manager Cloud Control and Collect Logs

Ingest Logs to OCI Logging Analytics Using Fluentd

Use the open source data collector software, Fluentd to collect log data from your source. Install the OCI Logging Analytics Output Plugin to route the collected log data to Oracle Logging Analytics.

Ingest Logs to OCI Logging Analytics Using Fluentd

Auto-Discovery of Entities and Log Collection

Initiate the discovery of entities like packaged applications, databases, middleware, and other components using the Stack Monitoring UI, collect logs from those entities, and view the entity topology.

Auto-Discovery of Entities and Log Collection

To detect and troubleshoot the processing errors during ingestion, see Troubleshoot Ingestion Pipeline.

For some of the Oracle-defined solutions that cover end-to-end process for collecting logs from different applications, platforms, and infrastructure, see table Solutions and Marketplace Apps in Technical Resources at Your Finger Tips.