Oracle Cloud Infrastructure Documentation

Managing Kafka Connect Configurations

Create, move, or delete Kafka Connect configurations in the Streaming service.

To review requirements for working with Kafka Connect see Using Kafka Connect.

  • Create a Kafka Connect Configuration

    1. Access the Streaming service in the Console.
    2. Under Analytics, click Kafka Connect Configurations to display a list of existing Kafka Connect configurations.
    3. Click Create Kafka Connect Configuration.
    4. Select a compartment from the Compartment list.
    5. In the Kafka Connect Configuration Name box, enter a name for the configuration. Avoid entering confidential information.
    6. (Optional) Assign tags to the configuration. If you have permissions to create a resource, then you also have permissions to apply free-form tags to that resource. To apply a defined tag, you must have permissions to use the tag namespace. For more information about tagging, see Resource Tags. If you're not sure whether to apply tags, skip this option or ask an administrator. You can apply tags later.
    7. Click Create Kafka Connect Configuration.
    8. The details page for the configuration opens and displays a read-only text box labeled Kafka Connect Storage Topics.

      Click Copy to copy the connector configuration so that you can paste it into the connect-distributed.properties file for your Kafka connector. For more information, see the official Kafka Connect documentation.

    Delete a Kafka Connect Configuration

    1. Access the Streaming service in the Console.
    2. Under Analytics, click Kafka Connect Configurations to display a list of existing Kafka Connect configurations.
    3. Click the name of the configuration that you want to delete.
    4. On the details page, click Delete, and then click Confirm.

    Move a Kafka Connect Configuration to a Different Compartment

    1. Access the Streaming service in the Console.
    2. Under Analytics, click Kafka Connect Configurations to display a list of existing Kafka Connect configurations.
    3. Click the name of the configuration that you want to move.
    4. On the details page, click Move Resource.
    5. In the Move resource dialog box, select the destination compartment, and then click Move resource.

  • For information about using the CLI, see Command Line Interface (CLI). For a complete list of flags and options available for CLI commands, see the Command Line Reference.

    Note

    The examples in this section use the full syntax for all parameters, for example --compartment-id. For some parameters, there are shortened versions that you can use instead, like -c. See the CLI online help for instances of a shortened parameter associated with a command.
    To get a list of Kafka Connect configurations

    oci streaming admin connect-harness list --compartment-id <compartment_OCID>

    For example:

    oci streaming admin connect-harness list --compartment-id exampleuniqueID
{
  "data": [
    {
      "compartment-id": "ocid1.tenancy.oc1..exampleuniqueID",
      "defined-tags": {},
      "freeform-tags": {},
      "id": "ocid1.connectharness.oc1.phx.exampleuniqueID",
      "lifecycle-state": "ACTIVE",
      "name": "MyKafkaConnectConfig",
      "time-created": "2020-08-31T17:26:09.640000+00:00"
    }
  ]
}

    By default, getting a list of Kafka Connect configurations returns up to the first 10 configurations in the compartment.

    To create a Kafka Connect configuration

    oci streaming admin connect-harness create --name <kafka_connect_configuration_name> --compartment-id <compartment_OCID>

    For example:

    oci streaming admin connect-harness create --name MyKafkaConnectConfig --compartment-id exampleuniqueID
{
  "data": {
    "compartment-id": "ocid1.tenancy.oc1..exampleuniqueID",
    "defined-tags": {},
    "freeform-tags": {},
    "id": "ocid1.connectharness.oc1.phx.exampleuniqueID",
    "lifecycle-state": "CREATING",
    "lifecycle-state-details": null,
    "name": "MyKafkaConnectConfig",
    "time-created": "2020-11-04T17:36:44.323000+00:00"
  },
  "etag": "\"48886b1f-333f-4142-a335-ed42e88b37f8-0e7d329d-c6df-47fc-b814-f1237b414281\""
}
    To view Kafka Connect configuration details

    oci streaming admin connect-harness get --connect-harness-id <kafka_connect_configuration_OCID>

    For example:

    oci streaming admin connect-harness get --connect-harness-id exampleuniqueID
{
  "data": {
    "compartment-id": "ocid1.tenancy.oc1..exampleuniqueID",
    "defined-tags": {},
    "freeform-tags": {},
    "id": "ocid1.connectharness.oc1.phx.exampleuniqueID",
    "lifecycle-state": "ACTIVE",
    "lifecycle-state-details": null,
    "name": "MyKafkaConnectConfig",
    "time-created": "2020-08-31T17:26:09.640000+00:00"
  },
  "etag": "\"5cca5a19-ad06-42f4-b11e-3945d1b0ece5-a3cec7b6-7b84-4056-b411-10fd9f4b1763\""
}
    To update a Kafka Connect configuration

    oci streaming admin connect-harness update --connect-harness-id <kafka_connect_configuration_OCID> --defined-tags <JSON_tags>

    For example:

    oci streaming admin connect-harness update --connect-harness-id exampleuniqueid --defined-tags "{\"tagNamespace\": {\"tagKey1\": \"tagValue1\",\"tagKey2\": \"tagValue2\"}}"
WARNING: Updates to freeform-tags and defined-tags will replace any existing values. Are you sure you want to continue? [y/N]: y

    Select y and press Enter. The Kafka Connect configuration is updated:

    {
  "data": {
    "compartment-id": "ocid1.tenancy.oc1..exampleuniqueID",
    "defined-tags": {
      "tagNamespace": {
        "tagKey1": "tagValue1",
        "tagKey2": "tagValue2"
      }
    },
    "freeform-tags": {},
    "id": "ocid1.connectharness.oc1.phx.exampleuniqueID",
    "lifecycle-state": "UPDATING",
    "lifecycle-state-details": null,
    "name": "MyKafkaConnectConfig",
    "time-created": "2020-08-31T17:26:09.640000+00:00"
  },
  "etag": "\"f3b74455-2584-4f20-a8bd-24068b1d0cba-2f66ecee-2e34-4c92-90d6-5c643b2be3fb\""
}
    To delete a Kafka Connect configuration
    Caution

    Kafka Connect configurations are deleted immediately. You cannot recover a deleted configuration.

    oci streaming admin connect-harness delete --connect-harness-id <kafka_connect_configuration_OCID>

    For example:

    oci streaming admin connect-harness delete --connect-harness-id ocid1.connectharness.oc1.phx.exampleuniqueID
Are you sure you want to delete this resource? [y/N]:

    Select y and press Enter. The Kafka Connect configuration is deleted with no further prompting.

    To move a Kafka Connect configuration to a different compartment

    oci streaming admin connect-harness change-compartment --connect-harness-id <kafka_connect_configuration_OCID> --compartment-id <target_compartment_OCID>

    For example:

    oci streaming admin connect-harness change-compartment --connect-harness-id ocid1.connectharness.oc1.phx.exampleuniqueID --compartment-id ocid1.compartment.oc1..exampleuniqueID
{
  "etag": "\"69aae3a1-e5f0-4289-bbbf-950c0fdddf2e-cbdd76e0-9a97-49be-a747-812ba28248d1\""
}

  • The Streaming API calls Kafka Connect configurations harnesses.

    For information about using the API and signing requests, see REST API documentation and Security Credentials. For information about SDKs, see SDKs and the CLI.

    Use the following API operations to manage Kafka Connector configurations: