Upload Event Logs Using LogEvents API

If you want to PUSH your event logs to Oracle Logging Analytics endpoint through LogEvents API, then follow the steps presented below.

You can upload one or more events using the predefined payload format which is in JSON. Additionally the API supports the GZIP and ZIP archive formats
The allowed maximum size of each JSON is 2 MB. However, an upload request may contain multiple JSON files upto a max of 100 in the form of ZIP.
Log group is a mandatory request parameter that will be applied at the scope of the request. Log set is an optional parameter and may be needed for large sets of log data, for example, TBs of data on a daily basis.

For steps to perform upload of raw log files using REST API, see Upload Logs on Demand.

Prerequisites:

Log source name of a new log source or an Oracle-defined log source that matches your log format. See Oracle-defined Sources and Create a Source.
To perform upload using REST API, the source should be of the type System Event Messages (Syslog), File, or Oracle Diagnostic Logs (ODL).
OCID of the log group which provides access control for the logs. See Create Log Groups to Store Your Logs.
Optionally, OCID of the entity, if you want to map the logs that you are uploading to an entity. See Create an Entity to Represent Your Log-Emitting Resource.

Topics:

Permissions Required to Upload Event Logs

You can enable the users to perform upload operation by selectively giving only the required permissions to perform the upload task, giving individual resource-type permissions, or giving broader aggregate level permissions. So you can select any one of the three sets of policy statements provided below, as suitable for your use-case.

The following IAM policy statement is for providing the specific permissions to the user group for the upload:

allow group <group_name> to {LOG_ANALYTICS_LOG_GROUP_UPLOAD_LOGS} in compartment <log_group_compartment>

The following IAM policy statement is for providing permissions at the level of individual resource-types for the upload:

allow group <group_name> to use loganalytics-log-group in compartment <log_group_compartment>

On the other hand, the following IAM policy statement is for providing permissions at Oracle Logging Analytics aggregate resources level for the upload:

allow group <group_name> to use loganalytics-resources-family in tenancy/compartment

group_name in all the above policy statements refers to the user group that must be given the required permissions.

API for Uploading Event Logs

For information about using the REST API and signing requests, see REST APIs and Security Credentials.

For uploadLogEventsFile API documentation, see Logging Analytics API - uploadLogEventsFile.

To perform upload using REST API, run the following command on your host:

oci log-analytics upload upload-log-events-file --file <Json_Event_File_Path> --log-group-id <Log_Group_OCID> --namespace-name <Tenancy_Namespace_Name> --payload-type <Payload_Type>

In the above curl command,

Json_Event_File_Path: The path of the JSON event file. For details about the format of the JSON Events file, and an example, see Event File Format to Upload Using LogEvents API.
Tenancy_Namespace_Name: The Oracle Logging Analytics namespace used for the request.
Log_Group_OCID: The OCID of the log group where the uploaded logs are stored in Oracle Logging Analytics.
Payload_Type: Defaults to JSON. Allowed values are JSON, GZIP, ZIP.
Log_Set: The log set that gets associated with the uploaded logs. This parameter is optional.

Event File Format to Upload Using LogEvents API

Here is the typical format of the JSON payload:

{
    "metadata":{ ----------------------------------------> Global (file level) metadata section to send additional metadata in field value pair format. The fields should be valid Logging Analytics Fields
        "Field1":"Value1",
        "Field2":"Value2",
        "FieldN":"ValueN"
    },
    "logEvents":
    [ ---------------------------------------------------> Group the events (logRecords) that belong to the same logSourceName, entityId, logPath and metadata
        {
            "metadata":{ --------------------------------> Group level metadata section
                "FieldA”:”ValueA”,
                "FieldB”:”ValueB”,
                "FieldN":"ValueN"
            },
            "entityId”:”<EntityOCID>“, -------------------> Logging Analytics Entity OCID
            "logSourceName”:”<LogSourceName>”, -----------> Logging Analytics LogSource Name (Mandatory)
            "logPath”:”<LogPath>”, -----------------------> logPath or logFileName to associate the log records with.
            "logRecords": -------------------------------> Array of logRecords/logEntries (Mandatory)
            [
                "logRecord1", ---------------------------> String representation of logRecord. In case the logRecord is of JSON type, it has to be encoded.
                "logRecord2",
                "logRecordN"
            ]
        },
        {            …            …            …       }
    ]
}

Note

The metadata section is for enriching the logs with the information which is not part of the actual log records.

Here is an example JSON payload:

{
    "metadata":{
        "Error ID":"0",
        "Environment":"dev",
        "Client Host Region":"IST"
    },
    "logEvents":
    [
        {
            "metadata":{
                "Error ID":"1",
                "Environment":"prod",
                "Client Host Region":"PST"
            },
            "entityId":"ocid1.loganalyticsentity.oc1.iad.amaaaaa...........",
            "logSourceName":"Linux Syslog Logs",
            "logPath":"/var/log/messages",
            "logRecords":
            [
                “Oct 20 2023 04:02:37 hostA syslogd 1.4.1: restart.",
                “Oct 20 2023 04:02:38 hostB syslogd 1.4.1: shutdown."
            ]
        }
    ]
}

Oracle Cloud Infrastructure Documentation

Permissions Required to Upload Event Logs

API for Uploading Event Logs

Event File Format to Upload Using LogEvents API