Creating a Connector with a Streaming Source
Create a connector in Connector Hub to transfer stream data from the Streaming service to a target service.
For more information about the Streaming service, see Streaming.
A connector that's defined with a Streaming source and (optional) function task supports the following targets: Functions, Logging Analytics, Object Storage, and Streaming. The Notifications target is supported only when no function task is used.
For an example of the Connector Hub workflow, see Overview of Connector Hub. For an example of a connector that uses Monitoring as the source, see Scenario: Sending Metrics to Object Storage.
- For stream input schema, see Message Reference.
- For a Notifications target with Streaming source, all messages are sent as raw JSON blobs.
Retention Period: Streaming Source
The retention period for the Streaming source in Connector Hub is customer-defined. See Limits on Streaming Resources. For more information about delivery, see Delivery Details.
Together with the retention period, the Streaming source's read position determines where in the stream to start moving data. You specify the read position when you specify the source connection.
The creation process begins, and its progress is displayed. On completion, the connector's details page opens. Use the oci sch service-connector create command and required parameters to create a connector with a Streaming source:
oci sch service-connector create --display-name "<display_name>" --compartment-id <compartment_OCID> --source [<stream_source_in_JSON>] --target [<target_in_JSON>]
For a complete list of parameters and values for CLI commands, see the CLI Command Reference.
Run the CreateServiceConnector operation to create a connector.
To create a connector with a Streaming source, populate
source
in the request (CreateServiceConnectorDetails) with streaming details. For an example, see StreamingSourceDetails.
Confirm That the New Connector Moves Data
After you create the connector, confirm that it's moving data.
- Enable logs for the connector to get details on data flow.
- Check for expected results at the target service.
Confirming that data is moved helps you avoid automatic deactivation, which happens when a connector fails for a long time.