Data Flow Common Types

DBMS_CLOUD_OCI_DATAFLOW_VARCHAR2_TBL Type

Nested table type of varchar2(32767).

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_varchar2_tbl FORCE IS TABLE OF (varchar2(32767)) NOT PERSISTABLE;

DBMS_CLOUD_OCI_DATAFLOW_APPLICATION_LOG_CONFIG_T Type

Logging details of Application logs for Data Flow Run.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_application_log_config_t FORCE AUTHID CURRENT_USER IS OBJECT (
  log_group_id varchar2(32767),
  log_id varchar2(32767),
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_application_log_config_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_application_log_config_t (
    log_group_id varchar2,
    log_id varchar2
  ) RETURN SELF AS RESULT
) NOT PERSISTABLE;

Fields

Field Description

log_group_id

(required) The log group id for where log objects will be for Data Flow Runs.

log_id

(required) The log id of the log object the Application Logs of Data Flow Run will be shipped to.

DBMS_CLOUD_OCI_DATAFLOW_SHAPE_CONFIG_T Type

This is used to configure the shape of the driver or executor if a flexible shape is used.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_shape_config_t FORCE AUTHID CURRENT_USER IS OBJECT (
  ocpus number,
  memory_in_g_bs number,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_shape_config_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_shape_config_t (
    ocpus number,
    memory_in_g_bs number
  ) RETURN SELF AS RESULT
) NOT PERSISTABLE;

Fields

Field Description

ocpus

(optional) The total number of OCPUs used for the driver or executors. See here for details.

memory_in_g_bs

(optional) The amount of memory used for the driver or executors.

DBMS_CLOUD_OCI_DATAFLOW_APPLICATION_PARAMETER_T Type

The parameter of an application.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_application_parameter_t FORCE AUTHID CURRENT_USER IS OBJECT (
  name varchar2(32767),
  value varchar2(32767),
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_application_parameter_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_application_parameter_t (
    name varchar2,
    value varchar2
  ) RETURN SELF AS RESULT
) NOT PERSISTABLE;

Fields

Field Description

name

(required) The name of the parameter. It must be a string of one or more word characters (a-z, A-Z, 0-9, _). Examples: \"iterations\", \"input_file\"

value

(required) The value of the parameter. It must be a string of 0 or more characters of any kind. Examples: \"\" (empty string), \"10\", \"mydata.xml\", \"${x}\"

DBMS_CLOUD_OCI_DATAFLOW_APPLICATION_PARAMETER_TBL Type

Nested table type of dbms_cloud_oci_dataflow_application_parameter_t.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_application_parameter_tbl FORCE IS TABLE OF (dbms_cloud_oci_dataflow_application_parameter_t) NOT PERSISTABLE;

DBMS_CLOUD_OCI_DATAFLOW_APPLICATION_T Type

A Data Flow application object.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_application_t FORCE AUTHID CURRENT_USER IS OBJECT (
  application_log_config dbms_cloud_oci_dataflow_application_log_config_t,
  archive_uri varchar2(32767),
  arguments dbms_cloud_oci_dataflow_varchar2_tbl,
  class_name varchar2(32767),
  configuration json_element_t,
  compartment_id varchar2(32767),
  defined_tags json_element_t,
  description varchar2(32767),
  display_name varchar2(32767),
  driver_shape varchar2(32767),
  driver_shape_config dbms_cloud_oci_dataflow_shape_config_t,
  execute varchar2(32767),
  executor_shape varchar2(32767),
  executor_shape_config dbms_cloud_oci_dataflow_shape_config_t,
  file_uri varchar2(32767),
  freeform_tags json_element_t,
  id varchar2(32767),
  language varchar2(32767),
  lifecycle_state varchar2(32767),
  logs_bucket_uri varchar2(32767),
  metastore_id varchar2(32767),
  num_executors number,
  owner_principal_id varchar2(32767),
  owner_user_name varchar2(32767),
  parameters dbms_cloud_oci_dataflow_application_parameter_tbl,
  pool_id varchar2(32767),
  private_endpoint_id varchar2(32767),
  spark_version varchar2(32767),
  time_created timestamp with time zone,
  time_updated timestamp with time zone,
  l_type varchar2(32767),
  warehouse_bucket_uri varchar2(32767),
  max_duration_in_minutes number,
  idle_timeout_in_minutes number,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_application_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_application_t (
    application_log_config dbms_cloud_oci_dataflow_application_log_config_t,
    archive_uri varchar2,
    arguments dbms_cloud_oci_dataflow_varchar2_tbl,
    class_name varchar2,
    configuration json_element_t,
    compartment_id varchar2,
    defined_tags json_element_t,
    description varchar2,
    display_name varchar2,
    driver_shape varchar2,
    driver_shape_config dbms_cloud_oci_dataflow_shape_config_t,
    execute varchar2,
    executor_shape varchar2,
    executor_shape_config dbms_cloud_oci_dataflow_shape_config_t,
    file_uri varchar2,
    freeform_tags json_element_t,
    id varchar2,
    language varchar2,
    lifecycle_state varchar2,
    logs_bucket_uri varchar2,
    metastore_id varchar2,
    num_executors number,
    owner_principal_id varchar2,
    owner_user_name varchar2,
    parameters dbms_cloud_oci_dataflow_application_parameter_tbl,
    pool_id varchar2,
    private_endpoint_id varchar2,
    spark_version varchar2,
    time_created timestamp with time zone,
    time_updated timestamp with time zone,
    l_type varchar2,
    warehouse_bucket_uri varchar2,
    max_duration_in_minutes number,
    idle_timeout_in_minutes number
  ) RETURN SELF AS RESULT
) NOT PERSISTABLE;

Fields

Field Description

application_log_config

(optional)

archive_uri

(optional) A comma separated list of one or more archive files as Oracle Cloud Infrastructure URIs. For example, ``oci://path/to/a.zip,oci://path/to/b.zip``. An Oracle Cloud Infrastructure URI of an archive.zip file containing custom dependencies that may be used to support the execution of a Python, Java, or Scala application. See https://docs.cloud.oracle.com/iaas/Content/API/SDKDocs/hdfsconnector.htm#uriformat.

arguments

(optional) The arguments passed to the running application as command line arguments. An argument is either a plain text or a placeholder. Placeholders are replaced using values from the parameters map. Each placeholder specified must be represented in the parameters map else the request (POST or PUT) will fail with a HTTP 400 status code. Placeholders are specified as `Service Api Spec`, where `name` is the name of the parameter. Example: `[ \"--input\", \"${input_file}\", \"--name\", \"John Doe\" ]` If \"input_file\" has a value of \"mydata.xml\", then the value above will be translated to `--input mydata.xml --name \"John Doe\"`

class_name

(optional) The class for the application.

configuration

(optional) The Spark configuration passed to the running process. See https://spark.apache.org/docs/latest/configuration.html#available-properties. Example: { \"spark.app.name\" : \"My App Name\", \"spark.shuffle.io.maxRetries\" : \"4\" } Note: Not all Spark properties are permitted to be set. Attempting to set a property that is not allowed to be overwritten will cause a 400 status to be returned.

compartment_id

(required) The OCID of a compartment.

defined_tags

(optional) Defined tags for this resource. Each key is predefined and scoped to a namespace. For more information, see Resource Tags. Example: `{\"Operations\": {\"CostCenter\": \"42\"}}`

description

(optional) A user-friendly description.

display_name

(required) A user-friendly name. This name is not necessarily unique.

driver_shape

(required) The VM shape for the driver. Sets the driver cores and memory.

driver_shape_config

(optional)

execute

(optional) The input used for spark-submit command. For more details see https://spark.apache.org/docs/latest/submitting-applications.html#launching-applications-with-spark-submit. Supported options include ``--class``, ``--file``, ``--jars``, ``--conf``, ``--py-files``, and main application file with arguments. Example: ``--jars oci://path/to/a.jar,oci://path/to/b.jar --files oci://path/to/a.json,oci://path/to/b.csv --py-files oci://path/to/a.py,oci://path/to/b.py --conf spark.sql.crossJoin.enabled=true --class org.apache.spark.examples.SparkPi oci://path/to/main.jar 10`` Note: If execute is specified together with applicationId, className, configuration, fileUri, language, arguments, parameters during application create/update, or run create/submit, Data Flow service will use derived information from execute input only.

executor_shape

(required) The VM shape for the executors. Sets the executor cores and memory.

executor_shape_config

(optional)

file_uri

(required) An Oracle Cloud Infrastructure URI of the file containing the application to execute. See https://docs.cloud.oracle.com/iaas/Content/API/SDKDocs/hdfsconnector.htm#uriformat.

freeform_tags

(optional) Free-form tags for this resource. Each tag is a simple key-value pair with no predefined name, type, or namespace. For more information, see Resource Tags. Example: `{\"Department\": \"Finance\"}`

id

(required) The application ID.

language

(required) The Spark language.

Allowed values are: 'SCALA', 'JAVA', 'PYTHON', 'SQL'

lifecycle_state

(required) The current state of this application.

Allowed values are: 'ACTIVE', 'DELETED', 'INACTIVE'

logs_bucket_uri

(optional) An Oracle Cloud Infrastructure URI of the bucket where the Spark job logs are to be uploaded. See https://docs.cloud.oracle.com/iaas/Content/API/SDKDocs/hdfsconnector.htm#uriformat.

metastore_id

(optional) The OCID of OCI Hive Metastore.

num_executors

(required) The number of executor VMs requested.

owner_principal_id

(required) The OCID of the user who created the resource.

owner_user_name

(optional) The username of the user who created the resource. If the username of the owner does not exist, `null` will be returned and the caller should refer to the ownerPrincipalId value instead.

parameters

(optional) An array of name/value pairs used to fill placeholders found in properties like `Application.arguments`. The name must be a string of one or more word characters (a-z, A-Z, 0-9, _). The value can be a string of 0 or more characters of any kind. Example: [ { name: \"iterations\", value: \"10\"}, { name: \"input_file\", value: \"mydata.xml\" }, { name: \"variable_x\", value: \"${x}\"} ]

pool_id

(optional) The OCID of a pool. Unique Id to indentify a dataflow pool resource.

private_endpoint_id

(optional) The OCID of a private endpoint.

spark_version

(required) The Spark version utilized to run the application.

time_created

(required) The date and time the resource was created, expressed in RFC 3339 timestamp format. Example: `2018-04-03T21:10:29.600Z`

time_updated

(required) The date and time the resource was updated, expressed in RFC 3339 timestamp format. Example: `2018-04-03T21:10:29.600Z`

l_type

(optional) The Spark application processing type.

Allowed values are: 'BATCH', 'STREAMING', 'SESSION'

warehouse_bucket_uri

(optional) An Oracle Cloud Infrastructure URI of the bucket to be used as default warehouse directory for BATCH SQL runs. See https://docs.cloud.oracle.com/iaas/Content/API/SDKDocs/hdfsconnector.htm#uriformat.

max_duration_in_minutes

(optional) The maximum duration in minutes for which an Application should run. Data Flow Run would be terminated once it reaches this duration from the time it transitions to `IN_PROGRESS` state.

idle_timeout_in_minutes

(optional) The timeout value in minutes used to manage Runs. A Run would be stopped after inactivity for this amount of time period. Note: This parameter is currently only applicable for Runs of type `SESSION`. Default value is 2880 minutes (2 days)

DBMS_CLOUD_OCI_DATAFLOW_APPLICATION_SUMMARY_T Type

A Data Flow application object used in bulk listings.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_application_summary_t FORCE AUTHID CURRENT_USER IS OBJECT (
  compartment_id varchar2(32767),
  defined_tags json_element_t,
  display_name varchar2(32767),
  freeform_tags json_element_t,
  id varchar2(32767),
  language varchar2(32767),
  lifecycle_state varchar2(32767),
  owner_principal_id varchar2(32767),
  owner_user_name varchar2(32767),
  pool_id varchar2(32767),
  spark_version varchar2(32767),
  time_created timestamp with time zone,
  time_updated timestamp with time zone,
  l_type varchar2(32767),
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_application_summary_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_application_summary_t (
    compartment_id varchar2,
    defined_tags json_element_t,
    display_name varchar2,
    freeform_tags json_element_t,
    id varchar2,
    language varchar2,
    lifecycle_state varchar2,
    owner_principal_id varchar2,
    owner_user_name varchar2,
    pool_id varchar2,
    spark_version varchar2,
    time_created timestamp with time zone,
    time_updated timestamp with time zone,
    l_type varchar2
  ) RETURN SELF AS RESULT
) NOT PERSISTABLE;

Fields

Field Description

compartment_id

(required) The OCID of a compartment.

defined_tags

(required) Defined tags for this resource. Each key is predefined and scoped to a namespace. For more information, see Resource Tags. Example: `{\"Operations\": {\"CostCenter\": \"42\"}}`

display_name

(required) A user-friendly name. This name is not necessarily unique.

freeform_tags

(required) Free-form tags for this resource. Each tag is a simple key-value pair with no predefined name, type, or namespace. For more information, see Resource Tags. Example: `{\"Department\": \"Finance\"}`

id

(required) The application ID.

language

(required) The Spark language.

Allowed values are: 'SCALA', 'JAVA', 'PYTHON', 'SQL'

lifecycle_state

(required) The current state of this application.

Allowed values are: 'ACTIVE', 'DELETED', 'INACTIVE'

owner_principal_id

(required) The OCID of the user who created the resource.

owner_user_name

(optional) The username of the user who created the resource. If the username of the owner does not exist, `null` will be returned and the caller should refer to the ownerPrincipalId value instead.

pool_id

(optional) The OCID of a pool. Unique Id to indentify a dataflow pool resource.

spark_version

(required) The Spark version utilized to run the application.

time_created

(required) The date and time the resource was created, expressed in RFC 3339 timestamp format. Example: `2018-04-03T21:10:29.600Z`

time_updated

(required) The date and time the resource was updated, expressed in RFC 3339 timestamp format. Example: `2018-04-03T21:10:29.600Z`

l_type

(optional) The Spark application processing type.

Allowed values are: 'BATCH', 'STREAMING', 'SESSION'

DBMS_CLOUD_OCI_DATAFLOW_CHANGE_APPLICATION_COMPARTMENT_DETAILS_T Type

The change application compartment details.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_change_application_compartment_details_t FORCE AUTHID CURRENT_USER IS OBJECT (
  compartment_id varchar2(32767),
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_change_application_compartment_details_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_change_application_compartment_details_t (
    compartment_id varchar2
  ) RETURN SELF AS RESULT
) NOT PERSISTABLE;

Fields

Field Description

compartment_id

(required) The OCID of a compartment.

DBMS_CLOUD_OCI_DATAFLOW_CHANGE_POOL_COMPARTMENT_DETAILS_T Type

The details required to change a pool compartment.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_change_pool_compartment_details_t FORCE AUTHID CURRENT_USER IS OBJECT (
  compartment_id varchar2(32767),
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_change_pool_compartment_details_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_change_pool_compartment_details_t (
    compartment_id varchar2
  ) RETURN SELF AS RESULT
) NOT PERSISTABLE;

Fields

Field Description

compartment_id

(required) The OCID of a compartment.

DBMS_CLOUD_OCI_DATAFLOW_CHANGE_PRIVATE_ENDPOINT_COMPARTMENT_DETAILS_T Type

The details required to change a private endpoint compartment.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_change_private_endpoint_compartment_details_t FORCE AUTHID CURRENT_USER IS OBJECT (
  compartment_id varchar2(32767),
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_change_private_endpoint_compartment_details_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_change_private_endpoint_compartment_details_t (
    compartment_id varchar2
  ) RETURN SELF AS RESULT
) NOT PERSISTABLE;

Fields

Field Description

compartment_id

(required) The OCID of a compartment.

DBMS_CLOUD_OCI_DATAFLOW_CHANGE_RUN_COMPARTMENT_DETAILS_T Type

The change run compartment details.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_change_run_compartment_details_t FORCE AUTHID CURRENT_USER IS OBJECT (
  compartment_id varchar2(32767),
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_change_run_compartment_details_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_change_run_compartment_details_t (
    compartment_id varchar2
  ) RETURN SELF AS RESULT
) NOT PERSISTABLE;

Fields

Field Description

compartment_id

(required) The OCID of a compartment.

DBMS_CLOUD_OCI_DATAFLOW_CHANGE_SQL_ENDPOINT_COMPARTMENT_DETAILS_T Type

Details for changing the compartment of a SQL Endpoint.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_change_sql_endpoint_compartment_details_t FORCE AUTHID CURRENT_USER IS OBJECT (
  compartment_id varchar2(32767),
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_change_sql_endpoint_compartment_details_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_change_sql_endpoint_compartment_details_t (
    compartment_id varchar2
  ) RETURN SELF AS RESULT
) NOT PERSISTABLE;

Fields

Field Description

compartment_id

(required) The OCID of the compartment into which the resource should be moved.

DBMS_CLOUD_OCI_DATAFLOW_CREATE_APPLICATION_DETAILS_T Type

The create application details.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_create_application_details_t FORCE AUTHID CURRENT_USER IS OBJECT (
  archive_uri varchar2(32767),
  arguments dbms_cloud_oci_dataflow_varchar2_tbl,
  application_log_config dbms_cloud_oci_dataflow_application_log_config_t,
  class_name varchar2(32767),
  compartment_id varchar2(32767),
  configuration json_element_t,
  defined_tags json_element_t,
  description varchar2(32767),
  display_name varchar2(32767),
  driver_shape varchar2(32767),
  driver_shape_config dbms_cloud_oci_dataflow_shape_config_t,
  execute varchar2(32767),
  executor_shape varchar2(32767),
  executor_shape_config dbms_cloud_oci_dataflow_shape_config_t,
  file_uri varchar2(32767),
  freeform_tags json_element_t,
  language varchar2(32767),
  logs_bucket_uri varchar2(32767),
  metastore_id varchar2(32767),
  num_executors number,
  parameters dbms_cloud_oci_dataflow_application_parameter_tbl,
  pool_id varchar2(32767),
  private_endpoint_id varchar2(32767),
  spark_version varchar2(32767),
  l_type varchar2(32767),
  warehouse_bucket_uri varchar2(32767),
  max_duration_in_minutes number,
  idle_timeout_in_minutes number,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_create_application_details_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_create_application_details_t (
    archive_uri varchar2,
    arguments dbms_cloud_oci_dataflow_varchar2_tbl,
    application_log_config dbms_cloud_oci_dataflow_application_log_config_t,
    class_name varchar2,
    compartment_id varchar2,
    configuration json_element_t,
    defined_tags json_element_t,
    description varchar2,
    display_name varchar2,
    driver_shape varchar2,
    driver_shape_config dbms_cloud_oci_dataflow_shape_config_t,
    execute varchar2,
    executor_shape varchar2,
    executor_shape_config dbms_cloud_oci_dataflow_shape_config_t,
    file_uri varchar2,
    freeform_tags json_element_t,
    language varchar2,
    logs_bucket_uri varchar2,
    metastore_id varchar2,
    num_executors number,
    parameters dbms_cloud_oci_dataflow_application_parameter_tbl,
    pool_id varchar2,
    private_endpoint_id varchar2,
    spark_version varchar2,
    l_type varchar2,
    warehouse_bucket_uri varchar2,
    max_duration_in_minutes number,
    idle_timeout_in_minutes number
  ) RETURN SELF AS RESULT
) NOT PERSISTABLE;

Fields

Field Description

archive_uri

(optional) A comma separated list of one or more archive files as Oracle Cloud Infrastructure URIs. For example, ``oci://path/to/a.zip,oci://path/to/b.zip``. An Oracle Cloud Infrastructure URI of an archive.zip file containing custom dependencies that may be used to support the execution of a Python, Java, or Scala application. See https://docs.cloud.oracle.com/iaas/Content/API/SDKDocs/hdfsconnector.htm#uriformat.

arguments

(optional) The arguments passed to the running application as command line arguments. An argument is either a plain text or a placeholder. Placeholders are replaced using values from the parameters map. Each placeholder specified must be represented in the parameters map else the request (POST or PUT) will fail with a HTTP 400 status code. Placeholders are specified as `Service Api Spec`, where `name` is the name of the parameter. Example: `[ \"--input\", \"${input_file}\", \"--name\", \"John Doe\" ]` If \"input_file\" has a value of \"mydata.xml\", then the value above will be translated to `--input mydata.xml --name \"John Doe\"`

application_log_config

(optional)

class_name

(optional) The class for the application.

compartment_id

(required) The OCID of a compartment.

configuration

(optional) The Spark configuration passed to the running process. See https://spark.apache.org/docs/latest/configuration.html#available-properties. Example: { \"spark.app.name\" : \"My App Name\", \"spark.shuffle.io.maxRetries\" : \"4\" } Note: Not all Spark properties are permitted to be set. Attempting to set a property that is not allowed to be overwritten will cause a 400 status to be returned.

defined_tags

(optional) Defined tags for this resource. Each key is predefined and scoped to a namespace. For more information, see Resource Tags. Example: `{\"Operations\": {\"CostCenter\": \"42\"}}`

description

(optional) A user-friendly description. Avoid entering confidential information.

display_name

(required) A user-friendly name. It does not have to be unique. Avoid entering confidential information.

driver_shape

(required) The VM shape for the driver. Sets the driver cores and memory.

driver_shape_config

(optional)

execute

(optional) The input used for spark-submit command. For more details see https://spark.apache.org/docs/latest/submitting-applications.html#launching-applications-with-spark-submit. Supported options include ``--class``, ``--file``, ``--jars``, ``--conf``, ``--py-files``, and main application file with arguments. Example: ``--jars oci://path/to/a.jar,oci://path/to/b.jar --files oci://path/to/a.json,oci://path/to/b.csv --py-files oci://path/to/a.py,oci://path/to/b.py --conf spark.sql.crossJoin.enabled=true --class org.apache.spark.examples.SparkPi oci://path/to/main.jar 10`` Note: If execute is specified together with applicationId, className, configuration, fileUri, language, arguments, parameters during application create/update, or run create/submit, Data Flow service will use derived information from execute input only.

executor_shape

(required) The VM shape for the executors. Sets the executor cores and memory.

executor_shape_config

(optional)

file_uri

(optional) An Oracle Cloud Infrastructure URI of the file containing the application to execute. See https://docs.cloud.oracle.com/iaas/Content/API/SDKDocs/hdfsconnector.htm#uriformat.

freeform_tags

(optional) Free-form tags for this resource. Each tag is a simple key-value pair with no predefined name, type, or namespace. For more information, see Resource Tags. Example: `{\"Department\": \"Finance\"}`

language

(required) The Spark language.

Allowed values are: 'SCALA', 'JAVA', 'PYTHON', 'SQL'

logs_bucket_uri

(optional) An Oracle Cloud Infrastructure URI of the bucket where the Spark job logs are to be uploaded. See https://docs.cloud.oracle.com/iaas/Content/API/SDKDocs/hdfsconnector.htm#uriformat.

metastore_id

(optional) The OCID of OCI Hive Metastore.

num_executors

(required) The number of executor VMs requested.

parameters

(optional) An array of name/value pairs used to fill placeholders found in properties like `Application.arguments`. The name must be a string of one or more word characters (a-z, A-Z, 0-9, _). The value can be a string of 0 or more characters of any kind. Example: [ { name: \"iterations\", value: \"10\"}, { name: \"input_file\", value: \"mydata.xml\" }, { name: \"variable_x\", value: \"${x}\"} ]

pool_id

(optional) The OCID of a pool. Unique Id to indentify a dataflow pool resource.

private_endpoint_id

(optional) The OCID of a private endpoint.

spark_version

(required) The Spark version utilized to run the application.

l_type

(optional) The Spark application processing type.

Allowed values are: 'BATCH', 'STREAMING', 'SESSION'

warehouse_bucket_uri

(optional) An Oracle Cloud Infrastructure URI of the bucket to be used as default warehouse directory for BATCH SQL runs. See https://docs.cloud.oracle.com/iaas/Content/API/SDKDocs/hdfsconnector.htm#uriformat.

max_duration_in_minutes

(optional) The maximum duration in minutes for which an Application should run. Data Flow Run would be terminated once it reaches this duration from the time it transitions to `IN_PROGRESS` state.

idle_timeout_in_minutes

(optional) The timeout value in minutes used to manage Runs. A Run would be stopped after inactivity for this amount of time period. Note: This parameter is currently only applicable for Runs of type `SESSION`. Default value is 2880 minutes (2 days)

DBMS_CLOUD_OCI_DATAFLOW_POOL_CONFIG_T Type

An object containing the details about the compute shapes and number of compute instances to provison.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_pool_config_t FORCE AUTHID CURRENT_USER IS OBJECT (
  shape varchar2(32767),
  shape_config dbms_cloud_oci_dataflow_shape_config_t,
  l_min number,
  l_max number,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_pool_config_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_pool_config_t (
    shape varchar2,
    shape_config dbms_cloud_oci_dataflow_shape_config_t,
    l_min number,
    l_max number
  ) RETURN SELF AS RESULT
) NOT PERSISTABLE;

Fields

Field Description

shape

(optional) The compute shape of the resources you would like to provision.

shape_config

(optional)

l_min

(optional) Minimum number of compute instances in the pool for a given compute shape.

l_max

(optional) Maximum number of compute instances in the pool for a given compute shape.

DBMS_CLOUD_OCI_DATAFLOW_POOL_SCHEDULE_T Type

Definition of when pool auto start or stop for a given day of a week.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_pool_schedule_t FORCE AUTHID CURRENT_USER IS OBJECT (
  day_of_week varchar2(32767),
  start_time number,
  stop_time number,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_pool_schedule_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_pool_schedule_t (
    day_of_week varchar2,
    start_time number,
    stop_time number
  ) RETURN SELF AS RESULT
) NOT PERSISTABLE;

Fields

Field Description

day_of_week

(optional) Day of the week SUN-SAT

Allowed values are: 'SUNDAY', 'MONDAY', 'TUESDAY', 'WEDNESDAY', 'THURSDAY', 'FRIDAY', 'SATURDAY'

start_time

(optional) Hour of the day to start or stop pool.

stop_time

(optional) Hour of the day to stop the pool.

DBMS_CLOUD_OCI_DATAFLOW_POOL_CONFIG_TBL Type

Nested table type of dbms_cloud_oci_dataflow_pool_config_t.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_pool_config_tbl FORCE IS TABLE OF (dbms_cloud_oci_dataflow_pool_config_t) NOT PERSISTABLE;

DBMS_CLOUD_OCI_DATAFLOW_POOL_SCHEDULE_TBL Type

Nested table type of dbms_cloud_oci_dataflow_pool_schedule_t.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_pool_schedule_tbl FORCE IS TABLE OF (dbms_cloud_oci_dataflow_pool_schedule_t) NOT PERSISTABLE;

DBMS_CLOUD_OCI_DATAFLOW_CREATE_POOL_DETAILS_T Type

The details required to create a pool.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_create_pool_details_t FORCE AUTHID CURRENT_USER IS OBJECT (
  compartment_id varchar2(32767),
  display_name varchar2(32767),
  description varchar2(32767),
  configurations dbms_cloud_oci_dataflow_pool_config_tbl,
  schedules dbms_cloud_oci_dataflow_pool_schedule_tbl,
  idle_timeout_in_minutes number,
  freeform_tags json_element_t,
  defined_tags json_element_t,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_create_pool_details_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_create_pool_details_t (
    compartment_id varchar2,
    display_name varchar2,
    description varchar2,
    configurations dbms_cloud_oci_dataflow_pool_config_tbl,
    schedules dbms_cloud_oci_dataflow_pool_schedule_tbl,
    idle_timeout_in_minutes number,
    freeform_tags json_element_t,
    defined_tags json_element_t
  ) RETURN SELF AS RESULT
) NOT PERSISTABLE;

Fields

Field Description

compartment_id

(required) The OCID of a compartment.

display_name

(required) A user-friendly name. It does not have to be unique. Avoid entering confidential information.

description

(optional) A user-friendly description. Avoid entering confidential information.

configurations

(required) List of PoolConfig items.

schedules

(optional) A list of schedules for pool to auto start and stop.

idle_timeout_in_minutes

(optional) Optional timeout value in minutes used to auto stop Pools. A Pool will be auto stopped after inactivity for this amount of time period. If value not set, pool will not be auto stopped auto.

freeform_tags

(optional) Free-form tags for this resource. Each tag is a simple key-value pair with no predefined name, type, or namespace. For more information, see Resource Tags. Example: `{\"Department\": \"Finance\"}`

defined_tags

(optional) Defined tags for this resource. Each key is predefined and scoped to a namespace. For more information, see Resource Tags. Example: `{\"Operations\": {\"CostCenter\": \"42\"}}`

DBMS_CLOUD_OCI_DATAFLOW_SCAN_T Type

Single Client Access Name (SCAN) is the object with a fully-qualified domain name and a port number.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_scan_t FORCE AUTHID CURRENT_USER IS OBJECT (
  fqdn varchar2(32767),
  port varchar2(32767),
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_scan_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_scan_t (
    fqdn varchar2,
    port varchar2
  ) RETURN SELF AS RESULT
) NOT PERSISTABLE;

Fields

Field Description

fqdn

(optional) A fully-qualified domain name (FQDN).

port

(optional) The port number of the FQDN

DBMS_CLOUD_OCI_DATAFLOW_SCAN_TBL Type

Nested table type of dbms_cloud_oci_dataflow_scan_t.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_scan_tbl FORCE IS TABLE OF (dbms_cloud_oci_dataflow_scan_t) NOT PERSISTABLE;

DBMS_CLOUD_OCI_DATAFLOW_CREATE_PRIVATE_ENDPOINT_DETAILS_T Type

The details required to create a private endpoint.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_create_private_endpoint_details_t FORCE AUTHID CURRENT_USER IS OBJECT (
  compartment_id varchar2(32767),
  defined_tags json_element_t,
  description varchar2(32767),
  display_name varchar2(32767),
  dns_zones dbms_cloud_oci_dataflow_varchar2_tbl,
  freeform_tags json_element_t,
  max_host_count number,
  nsg_ids dbms_cloud_oci_dataflow_varchar2_tbl,
  scan_details dbms_cloud_oci_dataflow_scan_tbl,
  subnet_id varchar2(32767),
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_create_private_endpoint_details_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_create_private_endpoint_details_t (
    compartment_id varchar2,
    defined_tags json_element_t,
    description varchar2,
    display_name varchar2,
    dns_zones dbms_cloud_oci_dataflow_varchar2_tbl,
    freeform_tags json_element_t,
    max_host_count number,
    nsg_ids dbms_cloud_oci_dataflow_varchar2_tbl,
    scan_details dbms_cloud_oci_dataflow_scan_tbl,
    subnet_id varchar2
  ) RETURN SELF AS RESULT
) NOT PERSISTABLE;

Fields

Field Description

compartment_id

(required) The OCID of a compartment.

defined_tags

(optional) Defined tags for this resource. Each key is predefined and scoped to a namespace. For more information, see Resource Tags. Example: `{\"Operations\": {\"CostCenter\": \"42\"}}`

description

(optional) A user-friendly description. Avoid entering confidential information.

display_name

(optional) A user-friendly name. It does not have to be unique. Avoid entering confidential information.

dns_zones

(required) An array of DNS zone names. Example: `[ \"app.examplecorp.com\", \"app.examplecorp2.com\" ]`

freeform_tags

(optional) Free-form tags for this resource. Each tag is a simple key-value pair with no predefined name, type, or namespace. For more information, see Resource Tags. Example: `{\"Department\": \"Finance\"}`

max_host_count

(optional) The maximum number of hosts to be accessed through the private endpoint. This value is used to calculate the relevant CIDR block and should be a multiple of 256. If the value is not a multiple of 256, it is rounded up to the next multiple of 256. For example, 300 is rounded up to 512.

nsg_ids

(optional) An array of network security group OCIDs.

scan_details

(optional) An array of fqdn/port pairs used to create private endpoint. Each object is a simple key-value pair with FQDN as key and port number as value. [ { fqdn: \"scan1.oracle.com\", port: \"1521\"}, { fqdn: \"scan2.oracle.com\", port: \"1521\" } ]

subnet_id

(required) The OCID of a subnet.

DBMS_CLOUD_OCI_DATAFLOW_CREATE_RUN_DETAILS_T Type

The create run details. The following properties are optional and override the default values set in the associated application: - applicationId - archiveUri - applicationLogConfig - arguments - configuration - definedTags - displayName - driverShape - execute - executorShape - freeformTags - logsBucketUri - metastoreId - numExecutors - parameters - sparkVersion - warehouseBucketUri It is expected that either the applicationId or the execute parameter is specified; but not both. If both or none are set, a Bad Request (HTTP 400) status will be sent as the response. If an appicationId is not specified, then a value for the execute parameter is expected. Using data parsed from the value, a new application will be created and assicated with the new run. See information on the execute parameter for details on the format of this parameter. The optional parameter spark version can only be specified when using the execute parameter. If it is not specified when using the execute parameter, the latest version will be used as default. If the execute parameter is not used, the spark version will be taken from the associated application. If displayName is not specified, it will be derived from the displayName of associated application or set by API using fileUri's application file name. Once a run is created, its properties (except for definedTags and freeformTags) cannot be changed. If the parent application's properties (including definedTags and freeformTags) are updated, the corresponding properties of the run will not update.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_create_run_details_t FORCE AUTHID CURRENT_USER IS OBJECT (
  application_log_config dbms_cloud_oci_dataflow_application_log_config_t,
  application_id varchar2(32767),
  archive_uri varchar2(32767),
  arguments dbms_cloud_oci_dataflow_varchar2_tbl,
  compartment_id varchar2(32767),
  configuration json_element_t,
  defined_tags json_element_t,
  display_name varchar2(32767),
  driver_shape varchar2(32767),
  driver_shape_config dbms_cloud_oci_dataflow_shape_config_t,
  execute varchar2(32767),
  executor_shape varchar2(32767),
  executor_shape_config dbms_cloud_oci_dataflow_shape_config_t,
  freeform_tags json_element_t,
  logs_bucket_uri varchar2(32767),
  metastore_id varchar2(32767),
  num_executors number,
  parameters dbms_cloud_oci_dataflow_application_parameter_tbl,
  pool_id varchar2(32767),
  spark_version varchar2(32767),
  l_type varchar2(32767),
  warehouse_bucket_uri varchar2(32767),
  max_duration_in_minutes number,
  idle_timeout_in_minutes number,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_create_run_details_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_create_run_details_t (
    application_log_config dbms_cloud_oci_dataflow_application_log_config_t,
    application_id varchar2,
    archive_uri varchar2,
    arguments dbms_cloud_oci_dataflow_varchar2_tbl,
    compartment_id varchar2,
    configuration json_element_t,
    defined_tags json_element_t,
    display_name varchar2,
    driver_shape varchar2,
    driver_shape_config dbms_cloud_oci_dataflow_shape_config_t,
    execute varchar2,
    executor_shape varchar2,
    executor_shape_config dbms_cloud_oci_dataflow_shape_config_t,
    freeform_tags json_element_t,
    logs_bucket_uri varchar2,
    metastore_id varchar2,
    num_executors number,
    parameters dbms_cloud_oci_dataflow_application_parameter_tbl,
    pool_id varchar2,
    spark_version varchar2,
    l_type varchar2,
    warehouse_bucket_uri varchar2,
    max_duration_in_minutes number,
    idle_timeout_in_minutes number
  ) RETURN SELF AS RESULT
) NOT PERSISTABLE;

Fields

Field Description

application_log_config

(optional)

application_id

(optional) The OCID of the associated application. If this value is set, then no value for the execute parameter is required. If this value is not set, then a value for the execute parameter is required, and a new application is created and associated with the new run.

archive_uri

(optional) A comma separated list of one or more archive files as Oracle Cloud Infrastructure URIs. For example, ``oci://path/to/a.zip,oci://path/to/b.zip``. An Oracle Cloud Infrastructure URI of an archive.zip file containing custom dependencies that may be used to support the execution of a Python, Java, or Scala application. See https://docs.cloud.oracle.com/iaas/Content/API/SDKDocs/hdfsconnector.htm#uriformat.

arguments

(optional) The arguments passed to the running application as command line arguments. An argument is either a plain text or a placeholder. Placeholders are replaced using values from the parameters map. Each placeholder specified must be represented in the parameters map else the request (POST or PUT) will fail with a HTTP 400 status code. Placeholders are specified as `Service Api Spec`, where `name` is the name of the parameter. Example: `[ \"--input\", \"${input_file}\", \"--name\", \"John Doe\" ]` If \"input_file\" has a value of \"mydata.xml\", then the value above will be translated to `--input mydata.xml --name \"John Doe\"`

compartment_id

(required) The OCID of a compartment.

configuration

(optional) The Spark configuration passed to the running process. See https://spark.apache.org/docs/latest/configuration.html#available-properties. Example: { \"spark.app.name\" : \"My App Name\", \"spark.shuffle.io.maxRetries\" : \"4\" } Note: Not all Spark properties are permitted to be set. Attempting to set a property that is not allowed to be overwritten will cause a 400 status to be returned.

defined_tags

(optional) Defined tags for this resource. Each key is predefined and scoped to a namespace. For more information, see Resource Tags. Example: `{\"Operations\": {\"CostCenter\": \"42\"}}`

display_name

(optional) A user-friendly name that does not have to be unique. Avoid entering confidential information. If this value is not specified, it will be derived from the associated application's displayName or set by API using fileUri's application file name.

driver_shape

(optional) The VM shape for the driver. Sets the driver cores and memory.

driver_shape_config

(optional)

execute

(optional) The input used for spark-submit command. For more details see https://spark.apache.org/docs/latest/submitting-applications.html#launching-applications-with-spark-submit. Supported options include ``--class``, ``--file``, ``--jars``, ``--conf``, ``--py-files``, and main application file with arguments. Example: ``--jars oci://path/to/a.jar,oci://path/to/b.jar --files oci://path/to/a.json,oci://path/to/b.csv --py-files oci://path/to/a.py,oci://path/to/b.py --conf spark.sql.crossJoin.enabled=true --class org.apache.spark.examples.SparkPi oci://path/to/main.jar 10`` Note: If execute is specified together with applicationId, className, configuration, fileUri, language, arguments, parameters during application create/update, or run create/submit, Data Flow service will use derived information from execute input only.

executor_shape

(optional) The VM shape for the executors. Sets the executor cores and memory.

executor_shape_config

(optional)

freeform_tags

(optional) Free-form tags for this resource. Each tag is a simple key-value pair with no predefined name, type, or namespace. For more information, see Resource Tags. Example: `{\"Department\": \"Finance\"}`

logs_bucket_uri

(optional) An Oracle Cloud Infrastructure URI of the bucket where the Spark job logs are to be uploaded. See https://docs.cloud.oracle.com/iaas/Content/API/SDKDocs/hdfsconnector.htm#uriformat.

metastore_id

(optional) The OCID of OCI Hive Metastore.

num_executors

(optional) The number of executor VMs requested.

parameters

(optional) An array of name/value pairs used to fill placeholders found in properties like `Application.arguments`. The name must be a string of one or more word characters (a-z, A-Z, 0-9, _). The value can be a string of 0 or more characters of any kind. Example: [ { name: \"iterations\", value: \"10\"}, { name: \"input_file\", value: \"mydata.xml\" }, { name: \"variable_x\", value: \"${x}\"} ]

pool_id

(optional) The OCID of a pool. Unique Id to indentify a dataflow pool resource.

spark_version

(optional) The Spark version utilized to run the application. This value may be set if applicationId is not since the Spark version will be taken from the associated application.

l_type

(optional) The Spark application processing type.

Allowed values are: 'BATCH', 'STREAMING', 'SESSION'

warehouse_bucket_uri

(optional) An Oracle Cloud Infrastructure URI of the bucket to be used as default warehouse directory for BATCH SQL runs. See https://docs.cloud.oracle.com/iaas/Content/API/SDKDocs/hdfsconnector.htm#uriformat.

max_duration_in_minutes

(optional) The maximum duration in minutes for which an Application should run. Data Flow Run would be terminated once it reaches this duration from the time it transitions to `IN_PROGRESS` state.

idle_timeout_in_minutes

(optional) The timeout value in minutes used to manage Runs. A Run would be stopped after inactivity for this amount of time period. Note: This parameter is currently only applicable for Runs of type `SESSION`. Default value is 2880 minutes (2 days)

DBMS_CLOUD_OCI_DATAFLOW_SQL_ENDPOINT_NETWORK_CONFIGURATION_T Type

The network configuration of a SQL Endpoint.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_sql_endpoint_network_configuration_t FORCE AUTHID CURRENT_USER IS OBJECT (
  network_type varchar2(32767),
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_sql_endpoint_network_configuration_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_sql_endpoint_network_configuration_t (
    network_type varchar2
  ) RETURN SELF AS RESULT
) NOT PERSISTABLE NOT FINAL;

Fields

Field Description

network_type

(required) The type of network configuration.

Allowed values are: 'VCN', 'SECURE_ACCESS'

DBMS_CLOUD_OCI_DATAFLOW_CREATE_SQL_ENDPOINT_DETAILS_T Type

The information about a new SQL Endpoint.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_create_sql_endpoint_details_t FORCE AUTHID CURRENT_USER IS OBJECT (
  compartment_id varchar2(32767),
  display_name varchar2(32767),
  description varchar2(32767),
  sql_endpoint_version varchar2(32767),
  driver_shape varchar2(32767),
  driver_shape_config dbms_cloud_oci_dataflow_shape_config_t,
  executor_shape varchar2(32767),
  executor_shape_config dbms_cloud_oci_dataflow_shape_config_t,
  min_executor_count number,
  max_executor_count number,
  metastore_id varchar2(32767),
  lake_id varchar2(32767),
  warehouse_bucket_uri varchar2(32767),
  freeform_tags json_element_t,
  defined_tags json_element_t,
  spark_advanced_configurations json_element_t,
  network_configuration dbms_cloud_oci_dataflow_sql_endpoint_network_configuration_t,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_create_sql_endpoint_details_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_create_sql_endpoint_details_t (
    compartment_id varchar2,
    display_name varchar2,
    description varchar2,
    sql_endpoint_version varchar2,
    driver_shape varchar2,
    driver_shape_config dbms_cloud_oci_dataflow_shape_config_t,
    executor_shape varchar2,
    executor_shape_config dbms_cloud_oci_dataflow_shape_config_t,
    min_executor_count number,
    max_executor_count number,
    metastore_id varchar2,
    lake_id varchar2,
    warehouse_bucket_uri varchar2,
    freeform_tags json_element_t,
    defined_tags json_element_t,
    spark_advanced_configurations json_element_t,
    network_configuration dbms_cloud_oci_dataflow_sql_endpoint_network_configuration_t
  ) RETURN SELF AS RESULT
) NOT PERSISTABLE;

Fields

Field Description

compartment_id

(required) The identifier of the compartment used with the SQL Endpoint.

display_name

(required) The SQL Endpoint name, which can be changed.

description

(optional) The description of CreateSQLEndpointDetails.

sql_endpoint_version

(required) The version of the SQL Endpoint.

driver_shape

(required) The shape of the SQL Endpoint driver instance.

driver_shape_config

(optional)

executor_shape

(required) The shape of the SQL Endpoint worker instance.

executor_shape_config

(optional)

min_executor_count

(required) The minimum number of executors.

max_executor_count

(required) The maximum number of executors.

metastore_id

(required) Metastore OCID

lake_id

(required) OCI lake OCID

warehouse_bucket_uri

(required) The warehouse bucket URI. It is a Oracle Cloud Infrastructure Object Storage bucket URI as defined here https://docs.oracle.com/en/cloud/paas/atp-cloud/atpud/object-storage-uris.html

freeform_tags

(optional) Free-form tags for this resource. Each tag is a simple key-value pair with no predefined name, type, or namespace. For more information, see Resource Tags. Example: `{\"Department\": \"Finance\"}`

defined_tags

(optional) Defined tags for this resource. Each key is predefined and scoped to a namespace. For more information, see Resource Tags. Example: `{\"Operations\": {\"CostCenter\": \"42\"}}`

spark_advanced_configurations

(optional) The Spark configuration passed to the running process. See https://spark.apache.org/docs/latest/configuration.html#available-properties. Example: { \"spark.app.name\" : \"My App Name\", \"spark.shuffle.io.maxRetries\" : \"4\" } Note: Not all Spark properties are permitted to be set. Attempting to set a property that is not allowed to be overwritten will cause a 400 status to be returned.

network_configuration

(required)

DBMS_CLOUD_OCI_DATAFLOW_CREATE_STATEMENT_DETAILS_T Type

The details required to create a statement.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_create_statement_details_t FORCE AUTHID CURRENT_USER IS OBJECT (
  code varchar2(32767),
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_create_statement_details_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_create_statement_details_t (
    code varchar2
  ) RETURN SELF AS RESULT
) NOT PERSISTABLE;

Fields

Field Description

code

(required) The statement code to execute. Example: `println(sc.version)`

DBMS_CLOUD_OCI_DATAFLOW_ERROR_T Type

The error object.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_error_t FORCE AUTHID CURRENT_USER IS OBJECT (
  code varchar2(32767),
  message varchar2(32767),
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_error_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_error_t (
    code varchar2,
    message varchar2
  ) RETURN SELF AS RESULT
) NOT PERSISTABLE;

Fields

Field Description

code

(required) The unique code of an error.

message

(required) The description of an error.

DBMS_CLOUD_OCI_DATAFLOW_STATEMENT_OUTPUT_DATA_T Type

An object representing execution output of a statement.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_statement_output_data_t FORCE AUTHID CURRENT_USER IS OBJECT (
  l_type varchar2(32767),
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_statement_output_data_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_statement_output_data_t (
    l_type varchar2
  ) RETURN SELF AS RESULT
) NOT PERSISTABLE NOT FINAL;

Fields

Field Description

l_type

(required) The type of the `StatementOutputData` like `TEXT_PLAIN`, `TEXT_HTML` or `IMAGE_PNG`.

Allowed values are: 'TEXT_PLAIN', 'TEXT_HTML', 'IMAGE_PNG'

DBMS_CLOUD_OCI_DATAFLOW_IMAGE_PNG_STATEMENT_OUTPUT_DATA_T Type

The statement output data in png format.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_image_png_statement_output_data_t FORCE AUTHID CURRENT_USER UNDER dbms_cloud_oci_dataflow_statement_output_data_t (
  value blob,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_image_png_statement_output_data_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_image_png_statement_output_data_t (
    l_type varchar2,
    value blob
  ) RETURN SELF AS RESULT
);

dbms_cloud_oci_dataflow_image_png_statement_output_data_t is a subtype of the dbms_cloud_oci_dataflow_statement_output_data_t type.

Fields

Field Description

value

(required) The statement code execution output in png format.

DBMS_CLOUD_OCI_DATAFLOW_NODE_COUNT_T Type

An object with a logical shape and count of the number of nodes with that shape.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_node_count_t FORCE AUTHID CURRENT_USER IS OBJECT (
  logical_shape varchar2(32767),
  l_count number,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_node_count_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_node_count_t (
    logical_shape varchar2,
    l_count number
  ) RETURN SELF AS RESULT
) NOT PERSISTABLE;

Fields

Field Description

logical_shape

(optional) The compute shape of the nodes that the count is for.

l_count

(optional) The node count of this compute shape.

DBMS_CLOUD_OCI_DATAFLOW_NODE_COUNT_TBL Type

Nested table type of dbms_cloud_oci_dataflow_node_count_t.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_node_count_tbl FORCE IS TABLE OF (dbms_cloud_oci_dataflow_node_count_t) NOT PERSISTABLE;

DBMS_CLOUD_OCI_DATAFLOW_POOL_METRICS_T Type

A collection of metrics related to a particular pool.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_pool_metrics_t FORCE AUTHID CURRENT_USER IS OBJECT (
  time_last_started timestamp with time zone,
  time_last_stopped timestamp with time zone,
  time_last_used timestamp with time zone,
  time_last_metrics_updated timestamp with time zone,
  active_runs_count number,
  actively_used_node_count dbms_cloud_oci_dataflow_node_count_tbl,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_pool_metrics_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_pool_metrics_t (
    time_last_started timestamp with time zone,
    time_last_stopped timestamp with time zone,
    time_last_used timestamp with time zone,
    time_last_metrics_updated timestamp with time zone,
    active_runs_count number,
    actively_used_node_count dbms_cloud_oci_dataflow_node_count_tbl
  ) RETURN SELF AS RESULT
) NOT PERSISTABLE;

Fields

Field Description

time_last_started

(optional) The last time this pool was started.

time_last_stopped

(optional) The last time this pool was stopped.

time_last_used

(optional) The last time a run used this pool.

time_last_metrics_updated

(optional) The last time the mertics were updated for this.

active_runs_count

(optional) The number of runs that are currently running that are using this pool.

actively_used_node_count

(optional) A count of the nodes that are currently being used for each shape in this pool.

DBMS_CLOUD_OCI_DATAFLOW_POOL_T Type

A Data Flow pool object.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_pool_t FORCE AUTHID CURRENT_USER IS OBJECT (
  compartment_id varchar2(32767),
  defined_tags json_element_t,
  description varchar2(32767),
  display_name varchar2(32767),
  freeform_tags json_element_t,
  id varchar2(32767),
  lifecycle_details varchar2(32767),
  lifecycle_state varchar2(32767),
  owner_principal_id varchar2(32767),
  owner_user_name varchar2(32767),
  pool_metrics dbms_cloud_oci_dataflow_pool_metrics_t,
  configurations dbms_cloud_oci_dataflow_pool_config_tbl,
  schedules dbms_cloud_oci_dataflow_pool_schedule_tbl,
  idle_timeout_in_minutes number,
  time_created timestamp with time zone,
  time_updated timestamp with time zone,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_pool_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_pool_t (
    compartment_id varchar2,
    defined_tags json_element_t,
    description varchar2,
    display_name varchar2,
    freeform_tags json_element_t,
    id varchar2,
    lifecycle_details varchar2,
    lifecycle_state varchar2,
    owner_principal_id varchar2,
    owner_user_name varchar2,
    pool_metrics dbms_cloud_oci_dataflow_pool_metrics_t,
    configurations dbms_cloud_oci_dataflow_pool_config_tbl,
    schedules dbms_cloud_oci_dataflow_pool_schedule_tbl,
    idle_timeout_in_minutes number,
    time_created timestamp with time zone,
    time_updated timestamp with time zone
  ) RETURN SELF AS RESULT
) NOT PERSISTABLE;

Fields

Field Description

compartment_id

(required) The OCID of a compartment.

defined_tags

(required) Defined tags for this resource. Each key is predefined and scoped to a namespace. For more information, see Resource Tags. Example: `{\"Operations\": {\"CostCenter\": \"42\"}}`

description

(optional) A user-friendly description. Avoid entering confidential information.

display_name

(required) A user-friendly name. It does not have to be unique. Avoid entering confidential information.

freeform_tags

(required) Free-form tags for this resource. Each tag is a simple key-value pair with no predefined name, type, or namespace. For more information, see Resource Tags. Example: `{\"Department\": \"Finance\"}`

id

(required) The OCID of a pool. Unique Id to indentify a dataflow pool resource.

lifecycle_details

(optional) The detailed messages about the lifecycle state.

lifecycle_state

(required) The current state of this pool.

Allowed values are: 'ACCEPTED', 'SCHEDULED', 'CREATING', 'ACTIVE', 'STOPPING', 'STOPPED', 'UPDATING', 'DELETING', 'DELETED', 'FAILED'

owner_principal_id

(required) The OCID of the user who created the resource.

owner_user_name

(optional) The username of the user who created the resource. If the username of the owner does not exist, `null` will be returned and the caller should refer to the ownerPrincipalId value instead.

pool_metrics

(optional)

configurations

(required) List of PoolConfig items.

schedules

(optional) A list of schedules for pool to auto start and stop.

idle_timeout_in_minutes

(optional) Optional timeout value in minutes used to auto stop Pools. A Pool will be auto stopped after inactivity for this amount of time period. If value not set, pool will not be auto stopped auto.

time_created

(required) The date and time the resource was created, expressed in RFC 3339 timestamp format. Example: `2018-04-03T21:10:29.600Z`

time_updated

(required) The date and time the resource was updated, expressed in RFC 3339 timestamp format. Example: `2018-04-03T21:10:29.600Z`

DBMS_CLOUD_OCI_DATAFLOW_POOL_SUMMARY_T Type

A pool object used in bulk listings.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_pool_summary_t FORCE AUTHID CURRENT_USER IS OBJECT (
  compartment_id varchar2(32767),
  defined_tags json_element_t,
  display_name varchar2(32767),
  freeform_tags json_element_t,
  id varchar2(32767),
  lifecycle_state varchar2(32767),
  owner_principal_id varchar2(32767),
  owner_user_name varchar2(32767),
  time_created timestamp with time zone,
  time_updated timestamp with time zone,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_pool_summary_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_pool_summary_t (
    compartment_id varchar2,
    defined_tags json_element_t,
    display_name varchar2,
    freeform_tags json_element_t,
    id varchar2,
    lifecycle_state varchar2,
    owner_principal_id varchar2,
    owner_user_name varchar2,
    time_created timestamp with time zone,
    time_updated timestamp with time zone
  ) RETURN SELF AS RESULT
) NOT PERSISTABLE;

Fields

Field Description

compartment_id

(required) The OCID of a compartment.

defined_tags

(required) Defined tags for this resource. Each key is predefined and scoped to a namespace. For more information, see Resource Tags. Example: `{\"Operations\": {\"CostCenter\": \"42\"}}`

display_name

(required) A user-friendly name. It does not have to be unique. Avoid entering confidential information.

freeform_tags

(required) Free-form tags for this resource. Each tag is a simple key-value pair with no predefined name, type, or namespace. For more information, see Resource Tags. Example: `{\"Department\": \"Finance\"}`

id

(required) The OCID of a pool. Unique Id to indentify a dataflow pool resource.

lifecycle_state

(required) The current state of this pool.

Allowed values are: 'ACCEPTED', 'SCHEDULED', 'CREATING', 'ACTIVE', 'STOPPING', 'STOPPED', 'UPDATING', 'DELETING', 'DELETED', 'FAILED'

owner_principal_id

(required) The OCID of the user who created the resource.

owner_user_name

(required) The username of the user who created the resource. If the username of the owner does not exist, `null` will be returned and the caller should refer to the ownerPrincipalId value instead.

time_created

(required) The date and time the resource was created, expressed in RFC 3339 timestamp format. Example: `2018-04-03T21:10:29.600Z`

time_updated

(required) The date and time the resource was updated, expressed in RFC 3339 timestamp format. Example: `2018-04-03T21:10:29.600Z`

DBMS_CLOUD_OCI_DATAFLOW_POOL_SUMMARY_TBL Type

Nested table type of dbms_cloud_oci_dataflow_pool_summary_t.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_pool_summary_tbl FORCE IS TABLE OF (dbms_cloud_oci_dataflow_pool_summary_t) NOT PERSISTABLE;

DBMS_CLOUD_OCI_DATAFLOW_POOL_COLLECTION_T Type

The results of a query for a list of pools. It contains PoolSummary items.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_pool_collection_t FORCE AUTHID CURRENT_USER IS OBJECT (
  items dbms_cloud_oci_dataflow_pool_summary_tbl,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_pool_collection_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_pool_collection_t (
    items dbms_cloud_oci_dataflow_pool_summary_tbl
  ) RETURN SELF AS RESULT
) NOT PERSISTABLE;

Fields

Field Description

items

(required) A list of pools.

DBMS_CLOUD_OCI_DATAFLOW_PRIVATE_ENDPOINT_T Type

A Data Flow private endpoint object.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_private_endpoint_t FORCE AUTHID CURRENT_USER IS OBJECT (
  compartment_id varchar2(32767),
  defined_tags json_element_t,
  description varchar2(32767),
  display_name varchar2(32767),
  dns_zones dbms_cloud_oci_dataflow_varchar2_tbl,
  freeform_tags json_element_t,
  id varchar2(32767),
  lifecycle_details varchar2(32767),
  lifecycle_state varchar2(32767),
  max_host_count number,
  nsg_ids dbms_cloud_oci_dataflow_varchar2_tbl,
  owner_principal_id varchar2(32767),
  owner_user_name varchar2(32767),
  scan_details dbms_cloud_oci_dataflow_scan_tbl,
  subnet_id varchar2(32767),
  time_created timestamp with time zone,
  time_updated timestamp with time zone,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_private_endpoint_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_private_endpoint_t (
    compartment_id varchar2,
    defined_tags json_element_t,
    description varchar2,
    display_name varchar2,
    dns_zones dbms_cloud_oci_dataflow_varchar2_tbl,
    freeform_tags json_element_t,
    id varchar2,
    lifecycle_details varchar2,
    lifecycle_state varchar2,
    max_host_count number,
    nsg_ids dbms_cloud_oci_dataflow_varchar2_tbl,
    owner_principal_id varchar2,
    owner_user_name varchar2,
    scan_details dbms_cloud_oci_dataflow_scan_tbl,
    subnet_id varchar2,
    time_created timestamp with time zone,
    time_updated timestamp with time zone
  ) RETURN SELF AS RESULT
) NOT PERSISTABLE;

Fields

Field Description

compartment_id

(required) The OCID of a compartment.

defined_tags

(required) Defined tags for this resource. Each key is predefined and scoped to a namespace. For more information, see Resource Tags. Example: `{\"Operations\": {\"CostCenter\": \"42\"}}`

description

(optional) A user-friendly description. Avoid entering confidential information.

display_name

(required) A user-friendly name. It does not have to be unique. Avoid entering confidential information.

dns_zones

(required) An array of DNS zone names. Example: `[ \"app.examplecorp.com\", \"app.examplecorp2.com\" ]`

freeform_tags

(required) Free-form tags for this resource. Each tag is a simple key-value pair with no predefined name, type, or namespace. For more information, see Resource Tags. Example: `{\"Department\": \"Finance\"}`

id

(required) The OCID of a private endpoint.

lifecycle_details

(optional) The detailed messages about the lifecycle state.

lifecycle_state

(required) The current state of this private endpoint.

Allowed values are: 'CREATING', 'ACTIVE', 'INACTIVE', 'UPDATING', 'DELETING', 'DELETED', 'FAILED'

max_host_count

(optional) The maximum number of hosts to be accessed through the private endpoint. This value is used to calculate the relevant CIDR block and should be a multiple of 256. If the value is not a multiple of 256, it is rounded up to the next multiple of 256. For example, 300 is rounded up to 512.

nsg_ids

(optional) An array of network security group OCIDs.

owner_principal_id

(required) The OCID of the user who created the resource.

owner_user_name

(optional) The username of the user who created the resource. If the username of the owner does not exist, `null` will be returned and the caller should refer to the ownerPrincipalId value instead.

scan_details

(optional) An array of fqdn/port pairs used to create private endpoint. Each object is a simple key-value pair with FQDN as key and port number as value. [ { fqdn: \"scan1.oracle.com\", port: \"1521\"}, { fqdn: \"scan2.oracle.com\", port: \"1521\" } ]

subnet_id

(required) The OCID of a subnet.

time_created

(required) The date and time the resource was created, expressed in RFC 3339 timestamp format. Example: `2018-04-03T21:10:29.600Z`

time_updated

(required) The date and time the resource was updated, expressed in RFC 3339 timestamp format. Example: `2018-04-03T21:10:29.600Z`

DBMS_CLOUD_OCI_DATAFLOW_PRIVATE_ENDPOINT_SUMMARY_T Type

A Data Flow private endpoint object used in bulk listings.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_private_endpoint_summary_t FORCE AUTHID CURRENT_USER IS OBJECT (
  compartment_id varchar2(32767),
  defined_tags json_element_t,
  display_name varchar2(32767),
  dns_zones dbms_cloud_oci_dataflow_varchar2_tbl,
  freeform_tags json_element_t,
  id varchar2(32767),
  lifecycle_state varchar2(32767),
  max_host_count number,
  nsg_ids dbms_cloud_oci_dataflow_varchar2_tbl,
  owner_principal_id varchar2(32767),
  owner_user_name varchar2(32767),
  scan_details dbms_cloud_oci_dataflow_scan_tbl,
  subnet_id varchar2(32767),
  time_created timestamp with time zone,
  time_updated timestamp with time zone,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_private_endpoint_summary_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_private_endpoint_summary_t (
    compartment_id varchar2,
    defined_tags json_element_t,
    display_name varchar2,
    dns_zones dbms_cloud_oci_dataflow_varchar2_tbl,
    freeform_tags json_element_t,
    id varchar2,
    lifecycle_state varchar2,
    max_host_count number,
    nsg_ids dbms_cloud_oci_dataflow_varchar2_tbl,
    owner_principal_id varchar2,
    owner_user_name varchar2,
    scan_details dbms_cloud_oci_dataflow_scan_tbl,
    subnet_id varchar2,
    time_created timestamp with time zone,
    time_updated timestamp with time zone
  ) RETURN SELF AS RESULT
) NOT PERSISTABLE;

Fields

Field Description

compartment_id

(required) The OCID of a compartment.

defined_tags

(required) Defined tags for this resource. Each key is predefined and scoped to a namespace. For more information, see Resource Tags. Example: `{\"Operations\": {\"CostCenter\": \"42\"}}`

display_name

(required) A user-friendly name. It does not have to be unique. Avoid entering confidential information.

dns_zones

(required) An array of DNS zone names. Example: `[ \"app.examplecorp.com\", \"app.examplecorp2.com\" ]`

freeform_tags

(required) Free-form tags for this resource. Each tag is a simple key-value pair with no predefined name, type, or namespace. For more information, see Resource Tags. Example: `{\"Department\": \"Finance\"}`

id

(required) The OCID of a private endpoint.

lifecycle_state

(required) The current state of this private endpoint.

Allowed values are: 'CREATING', 'ACTIVE', 'INACTIVE', 'UPDATING', 'DELETING', 'DELETED', 'FAILED'

max_host_count

(optional) The maximum number of hosts to be accessed through the private endpoint. This value is used to calculate the relevant CIDR block and should be a multiple of 256. If the value is not a multiple of 256, it is rounded up to the next multiple of 256. For example, 300 is rounded up to 512.

nsg_ids

(optional) An array of network security group OCIDs.

owner_principal_id

(required) The OCID of the user who created the resource.

owner_user_name

(optional) The username of the user who created the resource. If the username of the owner does not exist, `null` will be returned and the caller should refer to the ownerPrincipalId value instead.

scan_details

(optional) An array of fqdn/port pairs used to create private endpoint. Each object is a simple key-value pair with FQDN as key and port number as value. [ { fqdn: \"scan1.oracle.com\", port: \"1521\"}, { fqdn: \"scan2.oracle.com\", port: \"1521\" } ]

subnet_id

(required) The OCID of a subnet.

time_created

(required) The date and time the resource was created, expressed in RFC 3339 timestamp format. Example: `2018-04-03T21:10:29.600Z`

time_updated

(required) The date and time the resource was updated, expressed in RFC 3339 timestamp format. Example: `2018-04-03T21:10:29.600Z`

DBMS_CLOUD_OCI_DATAFLOW_PRIVATE_ENDPOINT_SUMMARY_TBL Type

Nested table type of dbms_cloud_oci_dataflow_private_endpoint_summary_t.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_private_endpoint_summary_tbl FORCE IS TABLE OF (dbms_cloud_oci_dataflow_private_endpoint_summary_t) NOT PERSISTABLE;

DBMS_CLOUD_OCI_DATAFLOW_PRIVATE_ENDPOINT_COLLECTION_T Type

The results of a query for a list of private endpoints. It contains PrivateEndpointSummary items.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_private_endpoint_collection_t FORCE AUTHID CURRENT_USER IS OBJECT (
  items dbms_cloud_oci_dataflow_private_endpoint_summary_tbl,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_private_endpoint_collection_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_private_endpoint_collection_t (
    items dbms_cloud_oci_dataflow_private_endpoint_summary_tbl
  ) RETURN SELF AS RESULT
) NOT PERSISTABLE;

Fields

Field Description

items

(required) A list of private endpoints.

DBMS_CLOUD_OCI_DATAFLOW_RUN_T Type

A run object.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_run_t FORCE AUTHID CURRENT_USER IS OBJECT (
  archive_uri varchar2(32767),
  arguments dbms_cloud_oci_dataflow_varchar2_tbl,
  application_id varchar2(32767),
  application_log_config dbms_cloud_oci_dataflow_application_log_config_t,
  class_name varchar2(32767),
  compartment_id varchar2(32767),
  configuration json_element_t,
  data_read_in_bytes number,
  data_written_in_bytes number,
  defined_tags json_element_t,
  display_name varchar2(32767),
  driver_shape varchar2(32767),
  driver_shape_config dbms_cloud_oci_dataflow_shape_config_t,
  execute varchar2(32767),
  executor_shape varchar2(32767),
  executor_shape_config dbms_cloud_oci_dataflow_shape_config_t,
  file_uri varchar2(32767),
  freeform_tags json_element_t,
  id varchar2(32767),
  language varchar2(32767),
  lifecycle_details varchar2(32767),
  lifecycle_state varchar2(32767),
  logs_bucket_uri varchar2(32767),
  metastore_id varchar2(32767),
  num_executors number,
  opc_request_id varchar2(32767),
  owner_principal_id varchar2(32767),
  owner_user_name varchar2(32767),
  parameters dbms_cloud_oci_dataflow_application_parameter_tbl,
  pool_id varchar2(32767),
  private_endpoint_dns_zones dbms_cloud_oci_dataflow_varchar2_tbl,
  private_endpoint_max_host_count number,
  private_endpoint_nsg_ids dbms_cloud_oci_dataflow_varchar2_tbl,
  private_endpoint_id varchar2(32767),
  private_endpoint_subnet_id varchar2(32767),
  run_duration_in_milliseconds number,
  spark_version varchar2(32767),
  time_created timestamp with time zone,
  time_updated timestamp with time zone,
  total_o_cpu number,
  l_type varchar2(32767),
  warehouse_bucket_uri varchar2(32767),
  max_duration_in_minutes number,
  idle_timeout_in_minutes number,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_run_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_run_t (
    archive_uri varchar2,
    arguments dbms_cloud_oci_dataflow_varchar2_tbl,
    application_id varchar2,
    application_log_config dbms_cloud_oci_dataflow_application_log_config_t,
    class_name varchar2,
    compartment_id varchar2,
    configuration json_element_t,
    data_read_in_bytes number,
    data_written_in_bytes number,
    defined_tags json_element_t,
    display_name varchar2,
    driver_shape varchar2,
    driver_shape_config dbms_cloud_oci_dataflow_shape_config_t,
    execute varchar2,
    executor_shape varchar2,
    executor_shape_config dbms_cloud_oci_dataflow_shape_config_t,
    file_uri varchar2,
    freeform_tags json_element_t,
    id varchar2,
    language varchar2,
    lifecycle_details varchar2,
    lifecycle_state varchar2,
    logs_bucket_uri varchar2,
    metastore_id varchar2,
    num_executors number,
    opc_request_id varchar2,
    owner_principal_id varchar2,
    owner_user_name varchar2,
    parameters dbms_cloud_oci_dataflow_application_parameter_tbl,
    pool_id varchar2,
    private_endpoint_dns_zones dbms_cloud_oci_dataflow_varchar2_tbl,
    private_endpoint_max_host_count number,
    private_endpoint_nsg_ids dbms_cloud_oci_dataflow_varchar2_tbl,
    private_endpoint_id varchar2,
    private_endpoint_subnet_id varchar2,
    run_duration_in_milliseconds number,
    spark_version varchar2,
    time_created timestamp with time zone,
    time_updated timestamp with time zone,
    total_o_cpu number,
    l_type varchar2,
    warehouse_bucket_uri varchar2,
    max_duration_in_minutes number,
    idle_timeout_in_minutes number
  ) RETURN SELF AS RESULT
) NOT PERSISTABLE;

Fields

Field Description

archive_uri

(optional) A comma separated list of one or more archive files as Oracle Cloud Infrastructure URIs. For example, ``oci://path/to/a.zip,oci://path/to/b.zip``. An Oracle Cloud Infrastructure URI of an archive.zip file containing custom dependencies that may be used to support the execution of a Python, Java, or Scala application. See https://docs.cloud.oracle.com/iaas/Content/API/SDKDocs/hdfsconnector.htm#uriformat.

arguments

(optional) The arguments passed to the running application as command line arguments. An argument is either a plain text or a placeholder. Placeholders are replaced using values from the parameters map. Each placeholder specified must be represented in the parameters map else the request (POST or PUT) will fail with a HTTP 400 status code. Placeholders are specified as `Service Api Spec`, where `name` is the name of the parameter. Example: `[ \"--input\", \"${input_file}\", \"--name\", \"John Doe\" ]` If \"input_file\" has a value of \"mydata.xml\", then the value above will be translated to `--input mydata.xml --name \"John Doe\"`

application_id

(required) The application ID.

application_log_config

(optional)

class_name

(optional) The class for the application.

compartment_id

(required) The OCID of a compartment.

configuration

(optional) The Spark configuration passed to the running process. See https://spark.apache.org/docs/latest/configuration.html#available-properties. Example: { \"spark.app.name\" : \"My App Name\", \"spark.shuffle.io.maxRetries\" : \"4\" } Note: Not all Spark properties are permitted to be set. Attempting to set a property that is not allowed to be overwritten will cause a 400 status to be returned.

data_read_in_bytes

(optional) The data read by the run in bytes.

data_written_in_bytes

(optional) The data written by the run in bytes.

defined_tags

(optional) Defined tags for this resource. Each key is predefined and scoped to a namespace. For more information, see Resource Tags. Example: `{\"Operations\": {\"CostCenter\": \"42\"}}`

display_name

(optional) A user-friendly name. This name is not necessarily unique.

driver_shape

(required) The VM shape for the driver. Sets the driver cores and memory.

driver_shape_config

(optional)

execute

(optional) The input used for spark-submit command. For more details see https://spark.apache.org/docs/latest/submitting-applications.html#launching-applications-with-spark-submit. Supported options include ``--class``, ``--file``, ``--jars``, ``--conf``, ``--py-files``, and main application file with arguments. Example: ``--jars oci://path/to/a.jar,oci://path/to/b.jar --files oci://path/to/a.json,oci://path/to/b.csv --py-files oci://path/to/a.py,oci://path/to/b.py --conf spark.sql.crossJoin.enabled=true --class org.apache.spark.examples.SparkPi oci://path/to/main.jar 10`` Note: If execute is specified together with applicationId, className, configuration, fileUri, language, arguments, parameters during application create/update, or run create/submit, Data Flow service will use derived information from execute input only.

executor_shape

(required) The VM shape for the executors. Sets the executor cores and memory.

executor_shape_config

(optional)

file_uri

(required) An Oracle Cloud Infrastructure URI of the file containing the application to execute. See https://docs.cloud.oracle.com/iaas/Content/API/SDKDocs/hdfsconnector.htm#uriformat.

freeform_tags

(optional) Free-form tags for this resource. Each tag is a simple key-value pair with no predefined name, type, or namespace. For more information, see Resource Tags. Example: `{\"Department\": \"Finance\"}`

id

(required) The ID of a run.

language

(required) The Spark language.

Allowed values are: 'SCALA', 'JAVA', 'PYTHON', 'SQL'

lifecycle_details

(optional) The detailed messages about the lifecycle state.

lifecycle_state

(required) The current state of this run.

Allowed values are: 'ACCEPTED', 'IN_PROGRESS', 'CANCELING', 'CANCELED', 'FAILED', 'SUCCEEDED', 'STOPPING', 'STOPPED'

logs_bucket_uri

(optional) An Oracle Cloud Infrastructure URI of the bucket where the Spark job logs are to be uploaded. See https://docs.cloud.oracle.com/iaas/Content/API/SDKDocs/hdfsconnector.htm#uriformat.

metastore_id

(optional) The OCID of OCI Hive Metastore.

num_executors

(required) The number of executor VMs requested.

opc_request_id

(optional) Unique Oracle assigned identifier for the request. If you need to contact Oracle about a particular request, please provide the request ID.

owner_principal_id

(optional) The OCID of the user who created the resource.

owner_user_name

(optional) The username of the user who created the resource. If the username of the owner does not exist, `null` will be returned and the caller should refer to the ownerPrincipalId value instead.

parameters

(optional) An array of name/value pairs used to fill placeholders found in properties like `Application.arguments`. The name must be a string of one or more word characters (a-z, A-Z, 0-9, _). The value can be a string of 0 or more characters of any kind. Example: [ { name: \"iterations\", value: \"10\"}, { name: \"input_file\", value: \"mydata.xml\" }, { name: \"variable_x\", value: \"${x}\"} ]

pool_id

(optional) The OCID of a pool. Unique Id to indentify a dataflow pool resource.

private_endpoint_dns_zones

(optional) An array of DNS zone names. Example: `[ \"app.examplecorp.com\", \"app.examplecorp2.com\" ]`

private_endpoint_max_host_count

(optional) The maximum number of hosts to be accessed through the private endpoint. This value is used to calculate the relevant CIDR block and should be a multiple of 256. If the value is not a multiple of 256, it is rounded up to the next multiple of 256. For example, 300 is rounded up to 512.

private_endpoint_nsg_ids

(optional) An array of network security group OCIDs.

private_endpoint_id

(optional) The OCID of a private endpoint.

private_endpoint_subnet_id

(optional) The OCID of a subnet.

run_duration_in_milliseconds

(optional) The duration of the run in milliseconds.

spark_version

(required) The Spark version utilized to run the application.

time_created

(required) The date and time the resource was created, expressed in RFC 3339 timestamp format. Example: `2018-04-03T21:10:29.600Z`

time_updated

(required) The date and time the resource was updated, expressed in RFC 3339 timestamp format. Example: `2018-04-03T21:10:29.600Z`

total_o_cpu

(optional) The total number of oCPU requested by the run.

l_type

(optional) The Spark application processing type.

Allowed values are: 'BATCH', 'STREAMING', 'SESSION'

warehouse_bucket_uri

(optional) An Oracle Cloud Infrastructure URI of the bucket to be used as default warehouse directory for BATCH SQL runs. See https://docs.cloud.oracle.com/iaas/Content/API/SDKDocs/hdfsconnector.htm#uriformat.

max_duration_in_minutes

(optional) The maximum duration in minutes for which an Application should run. Data Flow Run would be terminated once it reaches this duration from the time it transitions to `IN_PROGRESS` state.

idle_timeout_in_minutes

(optional) The timeout value in minutes used to manage Runs. A Run would be stopped after inactivity for this amount of time period. Note: This parameter is currently only applicable for Runs of type `SESSION`. Default value is 2880 minutes (2 days)

DBMS_CLOUD_OCI_DATAFLOW_RUN_LOG_SUMMARY_T Type

A summary of a log associated with a particular run.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_run_log_summary_t FORCE AUTHID CURRENT_USER IS OBJECT (
  name varchar2(32767),
  run_id varchar2(32767),
  size_in_bytes number,
  source varchar2(32767),
  time_created timestamp with time zone,
  l_type varchar2(32767),
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_run_log_summary_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_run_log_summary_t (
    name varchar2,
    run_id varchar2,
    size_in_bytes number,
    source varchar2,
    time_created timestamp with time zone,
    l_type varchar2
  ) RETURN SELF AS RESULT
) NOT PERSISTABLE;

Fields

Field Description

name

(required) The name of the log. Example: spark_driver_stderr_20190917T114000Z.log.gz

run_id

(required) The runId associated with the log.

size_in_bytes

(optional) The size of the object in bytes.

source

(required) The source of the log such as driver and executor.

Allowed values are: 'APPLICATION', 'DRIVER', 'EXECUTOR'

time_created

(optional) The date and time the object was created, as described in RFC 2616, section 14.29.

l_type

(required) The type of log such as stdout and stderr.

Allowed values are: 'STDERR', 'STDOUT'

DBMS_CLOUD_OCI_DATAFLOW_RUN_SUMMARY_T Type

A summary of the run.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_run_summary_t FORCE AUTHID CURRENT_USER IS OBJECT (
  application_id varchar2(32767),
  compartment_id varchar2(32767),
  data_read_in_bytes number,
  data_written_in_bytes number,
  defined_tags json_element_t,
  display_name varchar2(32767),
  freeform_tags json_element_t,
  id varchar2(32767),
  language varchar2(32767),
  lifecycle_details varchar2(32767),
  lifecycle_state varchar2(32767),
  opc_request_id varchar2(32767),
  owner_principal_id varchar2(32767),
  owner_user_name varchar2(32767),
  pool_id varchar2(32767),
  run_duration_in_milliseconds number,
  total_o_cpu number,
  time_created timestamp with time zone,
  time_updated timestamp with time zone,
  l_type varchar2(32767),
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_run_summary_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_run_summary_t (
    application_id varchar2,
    compartment_id varchar2,
    data_read_in_bytes number,
    data_written_in_bytes number,
    defined_tags json_element_t,
    display_name varchar2,
    freeform_tags json_element_t,
    id varchar2,
    language varchar2,
    lifecycle_details varchar2,
    lifecycle_state varchar2,
    opc_request_id varchar2,
    owner_principal_id varchar2,
    owner_user_name varchar2,
    pool_id varchar2,
    run_duration_in_milliseconds number,
    total_o_cpu number,
    time_created timestamp with time zone,
    time_updated timestamp with time zone,
    l_type varchar2
  ) RETURN SELF AS RESULT
) NOT PERSISTABLE;

Fields

Field Description

application_id

(required) The application ID.

compartment_id

(required) The OCID of a compartment.

data_read_in_bytes

(optional) The data read by the run in bytes.

data_written_in_bytes

(optional) The data written by the run in bytes.

defined_tags

(required) Defined tags for this resource. Each key is predefined and scoped to a namespace. For more information, see Resource Tags. Example: `{\"Operations\": {\"CostCenter\": \"42\"}}`

display_name

(optional) A user-friendly name. This name is not necessarily unique.

freeform_tags

(required) Free-form tags for this resource. Each tag is a simple key-value pair with no predefined name, type, or namespace. For more information, see Resource Tags. Example: `{\"Department\": \"Finance\"}`

id

(required) The ID of a run.

language

(required) The Spark language.

Allowed values are: 'SCALA', 'JAVA', 'PYTHON', 'SQL'

lifecycle_details

(optional) The detailed messages about the lifecycle state.

lifecycle_state

(required) The current state of this run.

Allowed values are: 'ACCEPTED', 'IN_PROGRESS', 'CANCELING', 'CANCELED', 'FAILED', 'SUCCEEDED', 'STOPPING', 'STOPPED'

opc_request_id

(optional) Unique Oracle assigned identifier for the request. If you need to contact Oracle about a particular request, please provide the request ID.

owner_principal_id

(optional) The OCID of the user who created the resource.

owner_user_name

(optional) The username of the user who created the resource. If the username of the owner does not exist, `null` will be returned and the caller should refer to the ownerPrincipalId value instead.

pool_id

(optional) The OCID of a pool. Unique Id to indentify a dataflow pool resource.

run_duration_in_milliseconds

(optional) The duration of the run in milliseconds.

total_o_cpu

(optional) The total number of oCPU requested by the run.

time_created

(required) The date and time the resource was created, expressed in RFC 3339 timestamp format. Example: `2018-04-03T21:10:29.600Z`

time_updated

(required) The date and time the resource was updated, expressed in RFC 3339 timestamp format. Example: `2018-04-03T21:10:29.600Z`

l_type

(optional) The Spark application processing type.

Allowed values are: 'BATCH', 'STREAMING', 'SESSION'

DBMS_CLOUD_OCI_DATAFLOW_SECURE_ACCESS_CONTROL_RULE_T Type

The access control rule for SECURE_ACCESS network type selection.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_secure_access_control_rule_t FORCE AUTHID CURRENT_USER IS OBJECT (
  ip_notation varchar2(32767),
  value varchar2(32767),
  vcn_ips varchar2(32767),
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_secure_access_control_rule_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_secure_access_control_rule_t (
    ip_notation varchar2,
    value varchar2,
    vcn_ips varchar2
  ) RETURN SELF AS RESULT
) NOT PERSISTABLE;

Fields

Field Description

ip_notation

(required) The type of IP notation.

Allowed values are: 'IP_ADDRESS', 'CIDR', 'VCN', 'VCN_OCID'

value

(required) The associated value of the selected IP notation.

vcn_ips

(optional) A comma-separated IP or CIDR address for VCN OCID IP notation selection.

DBMS_CLOUD_OCI_DATAFLOW_SQL_ENDPOINT_T Type

The description of a SQL Endpoint.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_sql_endpoint_t FORCE AUTHID CURRENT_USER IS OBJECT (
  id varchar2(32767),
  display_name varchar2(32767),
  compartment_id varchar2(32767),
  jdbc_endpoint_url varchar2(32767),
  time_created timestamp with time zone,
  time_updated timestamp with time zone,
  lifecycle_state varchar2(32767),
  state_message varchar2(32767),
  sql_endpoint_version varchar2(32767),
  driver_shape varchar2(32767),
  driver_shape_config dbms_cloud_oci_dataflow_shape_config_t,
  executor_shape varchar2(32767),
  executor_shape_config dbms_cloud_oci_dataflow_shape_config_t,
  min_executor_count number,
  max_executor_count number,
  metastore_id varchar2(32767),
  lake_id varchar2(32767),
  warehouse_bucket_uri varchar2(32767),
  description varchar2(32767),
  last_accepted_request_token varchar2(32767),
  freeform_tags json_element_t,
  defined_tags json_element_t,
  system_tags json_element_t,
  spark_advanced_configurations json_element_t,
  network_configuration dbms_cloud_oci_dataflow_sql_endpoint_network_configuration_t,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_sql_endpoint_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_sql_endpoint_t (
    id varchar2,
    display_name varchar2,
    compartment_id varchar2,
    jdbc_endpoint_url varchar2,
    time_created timestamp with time zone,
    time_updated timestamp with time zone,
    lifecycle_state varchar2,
    state_message varchar2,
    sql_endpoint_version varchar2,
    driver_shape varchar2,
    driver_shape_config dbms_cloud_oci_dataflow_shape_config_t,
    executor_shape varchar2,
    executor_shape_config dbms_cloud_oci_dataflow_shape_config_t,
    min_executor_count number,
    max_executor_count number,
    metastore_id varchar2,
    lake_id varchar2,
    warehouse_bucket_uri varchar2,
    description varchar2,
    last_accepted_request_token varchar2,
    freeform_tags json_element_t,
    defined_tags json_element_t,
    system_tags json_element_t,
    spark_advanced_configurations json_element_t,
    network_configuration dbms_cloud_oci_dataflow_sql_endpoint_network_configuration_t
  ) RETURN SELF AS RESULT
) NOT PERSISTABLE;

Fields

Field Description

id

(required) The provision identifier that is immutable on creation.

display_name

(required) The SQL Endpoint name, which can be changed.

compartment_id

(required) The OCID of a compartment.

jdbc_endpoint_url

(optional) The JDBC URL field. For example, jdbc:spark://{serviceFQDN}:443/default;SparkServerType=DFI

time_created

(optional) The time the Sql Endpoint was created. An RFC3339 formatted datetime string.

time_updated

(optional) The time the Sql Endpoint was updated. An RFC3339 formatted datetime string.

lifecycle_state

(optional) The current state of the Sql Endpoint.

Allowed values are: 'CREATING', 'ACTIVE', 'DELETING', 'DELETED', 'FAILED'

state_message

(optional) A message describing the reason why the resource is in it's current state. Helps bubble up errors in state changes. For example, it can be used to provide actionable information for a resource in the Failed state.

sql_endpoint_version

(required) The version of SQL Endpoint.

driver_shape

(required) The shape of the SQL Endpoint driver instance.

driver_shape_config

(optional)

executor_shape

(required) The shape of the SQL Endpoint executor instance.

executor_shape_config

(optional)

min_executor_count

(required) The minimum number of executors.

max_executor_count

(required) The maximum number of executors.

metastore_id

(required) The OCID of OCI Hive Metastore.

lake_id

(required) The OCID of OCI Lake.

warehouse_bucket_uri

(required) The warehouse bucket URI. It is a Oracle Cloud Infrastructure Object Storage bucket URI as defined here https://docs.oracle.com/en/cloud/paas/atp-cloud/atpud/object-storage-uris.html

description

(required) The description of the SQL Endpoint.

last_accepted_request_token

(optional) This token is used by Splat, and indicates that the service accepts the request, and that the request is currently being processed.

freeform_tags

(optional) Free-form tags for this resource. Each tag is a simple key-value pair with no predefined name, type, or namespace. For more information, see Resource Tags. Example: `{\"Department\": \"Finance\"}`

defined_tags

(optional) Defined tags for this resource. Each key is predefined and scoped to a namespace. For more information, see Resource Tags. Example: `{\"Operations\": {\"CostCenter\": \"42\"}}`

system_tags

(optional) The system tags associated with this resource, if any. The system tags are set by Oracle cloud infrastructure services. Each key is predefined and scoped to namespaces. For more information, see Resource Tags. Example: `{orcl-cloud: {free-tier-retain: true}}`

spark_advanced_configurations

(optional) The Spark configuration passed to the running process. See https://spark.apache.org/docs/latest/configuration.html#available-properties. Example: { \"spark.app.name\" : \"My App Name\", \"spark.shuffle.io.maxRetries\" : \"4\" } Note: Not all Spark properties are permitted to be set. Attempting to set a property that is not allowed to be overwritten will cause a 400 status to be returned.

network_configuration

(optional)

DBMS_CLOUD_OCI_DATAFLOW_SQL_ENDPOINT_SUMMARY_T Type

A summary of the Sql Endpoint.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_sql_endpoint_summary_t FORCE AUTHID CURRENT_USER IS OBJECT (
  id varchar2(32767),
  display_name varchar2(32767),
  compartment_id varchar2(32767),
  jdbc_endpoint_url varchar2(32767),
  time_created timestamp with time zone,
  time_updated timestamp with time zone,
  lifecycle_state varchar2(32767),
  state_message varchar2(32767),
  sql_endpoint_version varchar2(32767),
  driver_shape varchar2(32767),
  driver_shape_config dbms_cloud_oci_dataflow_shape_config_t,
  executor_shape varchar2(32767),
  executor_shape_config dbms_cloud_oci_dataflow_shape_config_t,
  min_executor_count number,
  max_executor_count number,
  owner_principal_id varchar2(32767),
  metastore_id varchar2(32767),
  lake_id varchar2(32767),
  warehouse_bucket_uri varchar2(32767),
  description varchar2(32767),
  last_accepted_request_token varchar2(32767),
  freeform_tags json_element_t,
  defined_tags json_element_t,
  system_tags json_element_t,
  spark_advanced_configurations json_element_t,
  network_configuration dbms_cloud_oci_dataflow_sql_endpoint_network_configuration_t,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_sql_endpoint_summary_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_sql_endpoint_summary_t (
    id varchar2,
    display_name varchar2,
    compartment_id varchar2,
    jdbc_endpoint_url varchar2,
    time_created timestamp with time zone,
    time_updated timestamp with time zone,
    lifecycle_state varchar2,
    state_message varchar2,
    sql_endpoint_version varchar2,
    driver_shape varchar2,
    driver_shape_config dbms_cloud_oci_dataflow_shape_config_t,
    executor_shape varchar2,
    executor_shape_config dbms_cloud_oci_dataflow_shape_config_t,
    min_executor_count number,
    max_executor_count number,
    owner_principal_id varchar2,
    metastore_id varchar2,
    lake_id varchar2,
    warehouse_bucket_uri varchar2,
    description varchar2,
    last_accepted_request_token varchar2,
    freeform_tags json_element_t,
    defined_tags json_element_t,
    system_tags json_element_t,
    spark_advanced_configurations json_element_t,
    network_configuration dbms_cloud_oci_dataflow_sql_endpoint_network_configuration_t
  ) RETURN SELF AS RESULT
) NOT PERSISTABLE;

Fields

Field Description

id

(required) The provision identifier that is immutable on creation.

display_name

(required) The SQL Endpoint name, which can be changed.

compartment_id

(required) The OCID of a compartment.

jdbc_endpoint_url

(optional) The JDBC URL field. For example, jdbc:spark://{serviceFQDN}:443/default;SparkServerType=DFI

time_created

(optional) The time the Sql Endpoint was created. An RFC3339 formatted datetime string.

time_updated

(optional) The time the Sql Endpoint was updated. An RFC3339 formatted datetime string.

lifecycle_state

(optional) The current state of the Sql Endpoint.

Allowed values are: 'CREATING', 'ACTIVE', 'DELETING', 'DELETED', 'FAILED'

state_message

(optional) A message describing the reason why the resource is in it's current state. Helps bubble up errors in state changes. For example, it can be used to provide actionable information for a resource in the Failed state.

sql_endpoint_version

(required) The version of SQL Endpoint.

driver_shape

(required) The shape of the SQL Endpoint driver instance.

driver_shape_config

(optional)

executor_shape

(required) The shape of the SQL Endpoint executor instance.

executor_shape_config

(optional)

min_executor_count

(required) The minimum number of executors.

max_executor_count

(required) The maximum number of executors.

owner_principal_id

(optional) The OCID of the user who created the resource.

metastore_id

(required) The OCID of OCI Hive Metastore.

lake_id

(required) The OCID of OCI Lake.

warehouse_bucket_uri

(required) The warehouse bucket URI. It is a Oracle Cloud Infrastructure Object Storage bucket URI as defined here https://docs.oracle.com/en/cloud/paas/atp-cloud/atpud/object-storage-uris.html

description

(required) The description of the SQL Endpoint.

last_accepted_request_token

(optional) This token is used by Splat, and indicates that the service accepts the request, and that the request is currently being processed.

freeform_tags

(optional) Free-form tags for this resource. Each tag is a simple key-value pair with no predefined name, type, or namespace. For more information, see Resource Tags. Example: `{\"Department\": \"Finance\"}`

defined_tags

(optional) Defined tags for this resource. Each key is predefined and scoped to a namespace. For more information, see Resource Tags. Example: `{\"Operations\": {\"CostCenter\": \"42\"}}`

system_tags

(optional) The system tags associated with this resource, if any. The system tags are set by Oracle cloud infrastructure services. Each key is predefined and scoped to namespaces. For more information, see Resource Tags. Example: `{orcl-cloud: {free-tier-retain: true}}`

spark_advanced_configurations

(optional) The Spark configuration passed to the running process. See https://spark.apache.org/docs/latest/configuration.html#available-properties. Example: { \"spark.app.name\" : \"My App Name\", \"spark.shuffle.io.maxRetries\" : \"4\" } Note: Not all Spark properties are permitted to be set. Attempting to set a property that is not allowed to be overwritten will cause a 400 status to be returned.

network_configuration

(optional)

DBMS_CLOUD_OCI_DATAFLOW_SQL_ENDPOINT_SUMMARY_TBL Type

Nested table type of dbms_cloud_oci_dataflow_sql_endpoint_summary_t.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_sql_endpoint_summary_tbl FORCE IS TABLE OF (dbms_cloud_oci_dataflow_sql_endpoint_summary_t) NOT PERSISTABLE;

DBMS_CLOUD_OCI_DATAFLOW_SQL_ENDPOINT_COLLECTION_T Type

The results of a Sql Endpoint search. It contains the objects in a SqlEndpointSummary.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_sql_endpoint_collection_t FORCE AUTHID CURRENT_USER IS OBJECT (
  items dbms_cloud_oci_dataflow_sql_endpoint_summary_tbl,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_sql_endpoint_collection_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_sql_endpoint_collection_t (
    items dbms_cloud_oci_dataflow_sql_endpoint_summary_tbl
  ) RETURN SELF AS RESULT
) NOT PERSISTABLE;

Fields

Field Description

items

(required) The collection of SqlEndpointSummary objects.

DBMS_CLOUD_OCI_DATAFLOW_SECURE_ACCESS_CONTROL_RULE_TBL Type

Nested table type of dbms_cloud_oci_dataflow_secure_access_control_rule_t.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_secure_access_control_rule_tbl FORCE IS TABLE OF (dbms_cloud_oci_dataflow_secure_access_control_rule_t) NOT PERSISTABLE;

DBMS_CLOUD_OCI_DATAFLOW_SQL_ENDPOINT_SECURE_ACCESS_CONFIG_T Type

Access control rules for secure access selection.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_sql_endpoint_secure_access_config_t FORCE AUTHID CURRENT_USER UNDER dbms_cloud_oci_dataflow_sql_endpoint_network_configuration_t (
  access_control_rules dbms_cloud_oci_dataflow_secure_access_control_rule_tbl,
  public_endpoint_ip varchar2(32767),
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_sql_endpoint_secure_access_config_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_sql_endpoint_secure_access_config_t (
    network_type varchar2,
    access_control_rules dbms_cloud_oci_dataflow_secure_access_control_rule_tbl,
    public_endpoint_ip varchar2
  ) RETURN SELF AS RESULT
);

dbms_cloud_oci_dataflow_sql_endpoint_secure_access_config_t is a subtype of the dbms_cloud_oci_dataflow_sql_endpoint_network_configuration_t type.

Fields

Field Description

access_control_rules

(optional) A list of SecureAccessControlRule's to which access is limited to

public_endpoint_ip

(optional) Ip Address of public endpoint

DBMS_CLOUD_OCI_DATAFLOW_SQL_ENDPOINT_VCN_CONFIG_T Type

The VCN configuration for VCN network type selection.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_sql_endpoint_vcn_config_t FORCE AUTHID CURRENT_USER UNDER dbms_cloud_oci_dataflow_sql_endpoint_network_configuration_t (
  vcn_id varchar2(32767),
  subnet_id varchar2(32767),
  host_name_prefix varchar2(32767),
  nsg_ids dbms_cloud_oci_dataflow_varchar2_tbl,
  private_endpoint_ip varchar2(32767),
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_sql_endpoint_vcn_config_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_sql_endpoint_vcn_config_t (
    network_type varchar2,
    vcn_id varchar2,
    subnet_id varchar2,
    host_name_prefix varchar2,
    nsg_ids dbms_cloud_oci_dataflow_varchar2_tbl,
    private_endpoint_ip varchar2
  ) RETURN SELF AS RESULT
);

dbms_cloud_oci_dataflow_sql_endpoint_vcn_config_t is a subtype of the dbms_cloud_oci_dataflow_sql_endpoint_network_configuration_t type.

Fields

Field Description

vcn_id

(required) The VCN OCID.

subnet_id

(required) The VCN Subnet OCID.

host_name_prefix

(optional) The host name prefix.

nsg_ids

(optional) The OCIDs of Network Security Groups (NSGs).

private_endpoint_ip

(optional) Ip Address of private endpoint

DBMS_CLOUD_OCI_DATAFLOW_STATEMENT_OUTPUT_T Type

The execution output of a statement.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_statement_output_t FORCE AUTHID CURRENT_USER IS OBJECT (
  data dbms_cloud_oci_dataflow_statement_output_data_t,
  status varchar2(32767),
  error_name varchar2(32767),
  error_value varchar2(32767),
  traceback dbms_cloud_oci_dataflow_varchar2_tbl,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_statement_output_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_statement_output_t (
    data dbms_cloud_oci_dataflow_statement_output_data_t,
    status varchar2,
    error_name varchar2,
    error_value varchar2,
    traceback dbms_cloud_oci_dataflow_varchar2_tbl
  ) RETURN SELF AS RESULT
) NOT PERSISTABLE;

Fields

Field Description

data

(optional)

status

(optional) Status of the statement output.

Allowed values are: 'OK', 'ERROR'

error_name

(optional) The name of the error in the statement output.

error_value

(optional) The value of the error in the statement output.

traceback

(optional) The traceback of the statement output.

DBMS_CLOUD_OCI_DATAFLOW_STATEMENT_T Type

A statement object.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_statement_t FORCE AUTHID CURRENT_USER IS OBJECT (
  id number,
  code varchar2(32767),
  lifecycle_state varchar2(32767),
  output dbms_cloud_oci_dataflow_statement_output_t,
  progress number,
  run_id varchar2(32767),
  time_created timestamp with time zone,
  time_completed timestamp with time zone,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_statement_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_statement_t (
    id number,
    code varchar2,
    lifecycle_state varchar2,
    output dbms_cloud_oci_dataflow_statement_output_t,
    progress number,
    run_id varchar2,
    time_created timestamp with time zone,
    time_completed timestamp with time zone
  ) RETURN SELF AS RESULT
) NOT PERSISTABLE;

Fields

Field Description

id

(required) The statement ID.

code

(required) The statement code to execute. Example: `println(sc.version)`

lifecycle_state

(required) The current state of this statement.

Allowed values are: 'ACCEPTED', 'CANCELLING', 'CANCELLED', 'FAILED', 'IN_PROGRESS', 'SUCCEEDED'

output

(optional)

progress

(optional) The execution progress.

run_id

(optional) The ID of a run.

time_created

(required) The date and time the resource was created, expressed in RFC 3339 timestamp format. Example: `2018-04-03T21:10:29.600Z`

time_completed

(optional) The date and time a statement execution was completed, expressed in RFC 3339 timestamp format. Example: `2022-05-31T21:10:29.600Z`

DBMS_CLOUD_OCI_DATAFLOW_STATEMENT_SUMMARY_T Type

Summary of the statement.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_statement_summary_t FORCE AUTHID CURRENT_USER IS OBJECT (
  id number,
  lifecycle_state varchar2(32767),
  run_id varchar2(32767),
  time_created timestamp with time zone,
  time_completed timestamp with time zone,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_statement_summary_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_statement_summary_t (
    id number,
    lifecycle_state varchar2,
    run_id varchar2,
    time_created timestamp with time zone,
    time_completed timestamp with time zone
  ) RETURN SELF AS RESULT
) NOT PERSISTABLE;

Fields

Field Description

id

(required) The statement ID.

lifecycle_state

(required) The current state of this statement.

Allowed values are: 'ACCEPTED', 'CANCELLING', 'CANCELLED', 'FAILED', 'IN_PROGRESS', 'SUCCEEDED'

run_id

(optional) The ID of a run.

time_created

(required) The date and time the resource was created, expressed in RFC 3339 timestamp format. Example: `2018-04-03T21:10:29.600Z`

time_completed

(optional) The date and time a statement execution was completed, expressed in RFC 3339 timestamp format. Example: `2022-05-31T21:10:29.600Z`

DBMS_CLOUD_OCI_DATAFLOW_STATEMENT_SUMMARY_TBL Type

Nested table type of dbms_cloud_oci_dataflow_statement_summary_t.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_statement_summary_tbl FORCE IS TABLE OF (dbms_cloud_oci_dataflow_statement_summary_t) NOT PERSISTABLE;

DBMS_CLOUD_OCI_DATAFLOW_STATEMENT_COLLECTION_T Type

The results of a query for a list of statements of a Session Run. It contains StatementSummary items.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_statement_collection_t FORCE AUTHID CURRENT_USER IS OBJECT (
  items dbms_cloud_oci_dataflow_statement_summary_tbl,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_statement_collection_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_statement_collection_t (
    items dbms_cloud_oci_dataflow_statement_summary_tbl
  ) RETURN SELF AS RESULT
) NOT PERSISTABLE;

Fields

Field Description

items

(required) A list of statements for a Session Run.

DBMS_CLOUD_OCI_DATAFLOW_TEXT_HTML_STATEMENT_OUTPUT_DATA_T Type

The statement output data in html format.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_text_html_statement_output_data_t FORCE AUTHID CURRENT_USER UNDER dbms_cloud_oci_dataflow_statement_output_data_t (
  value varchar2(32767),
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_text_html_statement_output_data_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_text_html_statement_output_data_t (
    l_type varchar2,
    value varchar2
  ) RETURN SELF AS RESULT
);

dbms_cloud_oci_dataflow_text_html_statement_output_data_t is a subtype of the dbms_cloud_oci_dataflow_statement_output_data_t type.

Fields

Field Description

value

(required) The statement code execution output in html format.

DBMS_CLOUD_OCI_DATAFLOW_TEXT_PLAIN_STATEMENT_OUTPUT_DATA_T Type

The statement output data in text format.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_text_plain_statement_output_data_t FORCE AUTHID CURRENT_USER UNDER dbms_cloud_oci_dataflow_statement_output_data_t (
  value varchar2(32767),
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_text_plain_statement_output_data_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_text_plain_statement_output_data_t (
    l_type varchar2,
    value varchar2
  ) RETURN SELF AS RESULT
);

dbms_cloud_oci_dataflow_text_plain_statement_output_data_t is a subtype of the dbms_cloud_oci_dataflow_statement_output_data_t type.

Fields

Field Description

value

(required) The statement code execution output in text format.

DBMS_CLOUD_OCI_DATAFLOW_UPDATE_APPLICATION_DETAILS_T Type

The update application details.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_update_application_details_t FORCE AUTHID CURRENT_USER IS OBJECT (
  class_name varchar2(32767),
  file_uri varchar2(32767),
  spark_version varchar2(32767),
  language varchar2(32767),
  application_log_config dbms_cloud_oci_dataflow_application_log_config_t,
  archive_uri varchar2(32767),
  arguments dbms_cloud_oci_dataflow_varchar2_tbl,
  configuration json_element_t,
  defined_tags json_element_t,
  description varchar2(32767),
  display_name varchar2(32767),
  driver_shape varchar2(32767),
  driver_shape_config dbms_cloud_oci_dataflow_shape_config_t,
  execute varchar2(32767),
  executor_shape varchar2(32767),
  executor_shape_config dbms_cloud_oci_dataflow_shape_config_t,
  freeform_tags json_element_t,
  logs_bucket_uri varchar2(32767),
  metastore_id varchar2(32767),
  num_executors number,
  parameters dbms_cloud_oci_dataflow_application_parameter_tbl,
  pool_id varchar2(32767),
  private_endpoint_id varchar2(32767),
  warehouse_bucket_uri varchar2(32767),
  max_duration_in_minutes number,
  idle_timeout_in_minutes number,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_update_application_details_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_update_application_details_t (
    class_name varchar2,
    file_uri varchar2,
    spark_version varchar2,
    language varchar2,
    application_log_config dbms_cloud_oci_dataflow_application_log_config_t,
    archive_uri varchar2,
    arguments dbms_cloud_oci_dataflow_varchar2_tbl,
    configuration json_element_t,
    defined_tags json_element_t,
    description varchar2,
    display_name varchar2,
    driver_shape varchar2,
    driver_shape_config dbms_cloud_oci_dataflow_shape_config_t,
    execute varchar2,
    executor_shape varchar2,
    executor_shape_config dbms_cloud_oci_dataflow_shape_config_t,
    freeform_tags json_element_t,
    logs_bucket_uri varchar2,
    metastore_id varchar2,
    num_executors number,
    parameters dbms_cloud_oci_dataflow_application_parameter_tbl,
    pool_id varchar2,
    private_endpoint_id varchar2,
    warehouse_bucket_uri varchar2,
    max_duration_in_minutes number,
    idle_timeout_in_minutes number
  ) RETURN SELF AS RESULT
) NOT PERSISTABLE;

Fields

Field Description

class_name

(optional) The class for the application.

file_uri

(optional) An Oracle Cloud Infrastructure URI of the file containing the application to execute. See https://docs.cloud.oracle.com/iaas/Content/API/SDKDocs/hdfsconnector.htm#uriformat.

spark_version

(optional) The Spark version utilized to run the application.

language

(optional) The Spark language.

Allowed values are: 'SCALA', 'JAVA', 'PYTHON', 'SQL'

application_log_config

(optional)

archive_uri

(optional) A comma separated list of one or more archive files as Oracle Cloud Infrastructure URIs. For example, ``oci://path/to/a.zip,oci://path/to/b.zip``. An Oracle Cloud Infrastructure URI of an archive.zip file containing custom dependencies that may be used to support the execution of a Python, Java, or Scala application. See https://docs.cloud.oracle.com/iaas/Content/API/SDKDocs/hdfsconnector.htm#uriformat.

arguments

(optional) The arguments passed to the running application as command line arguments. An argument is either a plain text or a placeholder. Placeholders are replaced using values from the parameters map. Each placeholder specified must be represented in the parameters map else the request (POST or PUT) will fail with a HTTP 400 status code. Placeholders are specified as `Service Api Spec`, where `name` is the name of the parameter. Example: `[ \"--input\", \"${input_file}\", \"--name\", \"John Doe\" ]` If \"input_file\" has a value of \"mydata.xml\", then the value above will be translated to `--input mydata.xml --name \"John Doe\"`

configuration

(optional) The Spark configuration passed to the running process. See https://spark.apache.org/docs/latest/configuration.html#available-properties. Example: { \"spark.app.name\" : \"My App Name\", \"spark.shuffle.io.maxRetries\" : \"4\" } Note: Not all Spark properties are permitted to be set. Attempting to set a property that is not allowed to be overwritten will cause a 400 status to be returned.

defined_tags

(optional) Defined tags for this resource. Each key is predefined and scoped to a namespace. For more information, see Resource Tags. Example: `{\"Operations\": {\"CostCenter\": \"42\"}}`

description

(optional) A user-friendly description. Avoid entering confidential information.

display_name

(optional) A user-friendly name. It does not have to be unique. Avoid entering confidential information.

driver_shape

(optional) The VM shape for the driver. Sets the driver cores and memory.

driver_shape_config

(optional)

execute

(optional) The input used for spark-submit command. For more details see https://spark.apache.org/docs/latest/submitting-applications.html#launching-applications-with-spark-submit. Supported options include ``--class``, ``--file``, ``--jars``, ``--conf``, ``--py-files``, and main application file with arguments. Example: ``--jars oci://path/to/a.jar,oci://path/to/b.jar --files oci://path/to/a.json,oci://path/to/b.csv --py-files oci://path/to/a.py,oci://path/to/b.py --conf spark.sql.crossJoin.enabled=true --class org.apache.spark.examples.SparkPi oci://path/to/main.jar 10`` Note: If execute is specified together with applicationId, className, configuration, fileUri, language, arguments, parameters during application create/update, or run create/submit, Data Flow service will use derived information from execute input only.

executor_shape

(optional) The VM shape for the executors. Sets the executor cores and memory.

executor_shape_config

(optional)

freeform_tags

(optional) Free-form tags for this resource. Each tag is a simple key-value pair with no predefined name, type, or namespace. For more information, see Resource Tags. Example: `{\"Department\": \"Finance\"}`

logs_bucket_uri

(optional) An Oracle Cloud Infrastructure URI of the bucket where the Spark job logs are to be uploaded. See https://docs.cloud.oracle.com/iaas/Content/API/SDKDocs/hdfsconnector.htm#uriformat.

metastore_id

(optional) The OCID of OCI Hive Metastore.

num_executors

(optional) The number of executor VMs requested.

parameters

(optional) An array of name/value pairs used to fill placeholders found in properties like `Application.arguments`. The name must be a string of one or more word characters (a-z, A-Z, 0-9, _). The value can be a string of 0 or more characters of any kind. Example: [ { name: \"iterations\", value: \"10\"}, { name: \"input_file\", value: \"mydata.xml\" }, { name: \"variable_x\", value: \"${x}\"} ]

pool_id

(optional) The OCID of a pool. Unique Id to indentify a dataflow pool resource.

private_endpoint_id

(optional) The OCID of a private endpoint.

warehouse_bucket_uri

(optional) An Oracle Cloud Infrastructure URI of the bucket to be used as default warehouse directory for BATCH SQL runs. See https://docs.cloud.oracle.com/iaas/Content/API/SDKDocs/hdfsconnector.htm#uriformat.

max_duration_in_minutes

(optional) The maximum duration in minutes for which an Application should run. Data Flow Run would be terminated once it reaches this duration from the time it transitions to `IN_PROGRESS` state.

idle_timeout_in_minutes

(optional) The timeout value in minutes used to manage Runs. A Run would be stopped after inactivity for this amount of time period. Note: This parameter is currently only applicable for Runs of type `SESSION`. Default value is 2880 minutes (2 days)

DBMS_CLOUD_OCI_DATAFLOW_UPDATE_POOL_DETAILS_T Type

The details required to update a given pool with ```poolId```.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_update_pool_details_t FORCE AUTHID CURRENT_USER IS OBJECT (
  display_name varchar2(32767),
  description varchar2(32767),
  configurations dbms_cloud_oci_dataflow_pool_config_tbl,
  schedules dbms_cloud_oci_dataflow_pool_schedule_tbl,
  idle_timeout_in_minutes number,
  freeform_tags json_element_t,
  defined_tags json_element_t,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_update_pool_details_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_update_pool_details_t (
    display_name varchar2,
    description varchar2,
    configurations dbms_cloud_oci_dataflow_pool_config_tbl,
    schedules dbms_cloud_oci_dataflow_pool_schedule_tbl,
    idle_timeout_in_minutes number,
    freeform_tags json_element_t,
    defined_tags json_element_t
  ) RETURN SELF AS RESULT
) NOT PERSISTABLE;

Fields

Field Description

display_name

(optional) A user-friendly name. It does not have to be unique. Avoid entering confidential information.

description

(optional) A user-friendly description. Avoid entering confidential information.

configurations

(optional) List of PoolConfig items.

schedules

(optional) A list of schedules for pool to auto start and stop.

idle_timeout_in_minutes

(optional) Optional timeout value in minutes used to auto stop Pools. A Pool will be auto stopped after inactivity for this amount of time period. If value not set, pool will not be auto stopped auto.

freeform_tags

(optional) Free-form tags for this resource. Each tag is a simple key-value pair with no predefined name, type, or namespace. For more information, see Resource Tags. Example: `{\"Department\": \"Finance\"}`

defined_tags

(optional) Defined tags for this resource. Each key is predefined and scoped to a namespace. For more information, see Resource Tags. Example: `{\"Operations\": {\"CostCenter\": \"42\"}}`

DBMS_CLOUD_OCI_DATAFLOW_UPDATE_PRIVATE_ENDPOINT_DETAILS_T Type

The details required to update a private endpoint.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_update_private_endpoint_details_t FORCE AUTHID CURRENT_USER IS OBJECT (
  defined_tags json_element_t,
  description varchar2(32767),
  display_name varchar2(32767),
  dns_zones dbms_cloud_oci_dataflow_varchar2_tbl,
  freeform_tags json_element_t,
  max_host_count number,
  nsg_ids dbms_cloud_oci_dataflow_varchar2_tbl,
  scan_details dbms_cloud_oci_dataflow_scan_tbl,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_update_private_endpoint_details_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_update_private_endpoint_details_t (
    defined_tags json_element_t,
    description varchar2,
    display_name varchar2,
    dns_zones dbms_cloud_oci_dataflow_varchar2_tbl,
    freeform_tags json_element_t,
    max_host_count number,
    nsg_ids dbms_cloud_oci_dataflow_varchar2_tbl,
    scan_details dbms_cloud_oci_dataflow_scan_tbl
  ) RETURN SELF AS RESULT
) NOT PERSISTABLE;

Fields

Field Description

defined_tags

(optional) Defined tags for this resource. Each key is predefined and scoped to a namespace. For more information, see Resource Tags. Example: `{\"Operations\": {\"CostCenter\": \"42\"}}`

description

(optional) A user-friendly description. Avoid entering confidential information.

display_name

(optional) A user-friendly name. It does not have to be unique. Avoid entering confidential information.

dns_zones

(optional) An array of DNS zone names. Example: `[ \"app.examplecorp.com\", \"app.examplecorp2.com\" ]`

freeform_tags

(optional) Free-form tags for this resource. Each tag is a simple key-value pair with no predefined name, type, or namespace. For more information, see Resource Tags. Example: `{\"Department\": \"Finance\"}`

max_host_count

(optional) The maximum number of hosts to be accessed through the private endpoint. This value is used to calculate the relevant CIDR block and should be a multiple of 256. If the value is not a multiple of 256, it is rounded up to the next multiple of 256. For example, 300 is rounded up to 512.

nsg_ids

(optional) An array of network security group OCIDs.

scan_details

(optional) An array of fqdn/port pairs used to create private endpoint. Each object is a simple key-value pair with FQDN as key and port number as value. [ { fqdn: \"scan1.oracle.com\", port: \"1521\"}, { fqdn: \"scan2.oracle.com\", port: \"1521\" } ]

DBMS_CLOUD_OCI_DATAFLOW_UPDATE_RUN_DETAILS_T Type

The update run details. Only a limited set of properties of a run can be updated.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_update_run_details_t FORCE AUTHID CURRENT_USER IS OBJECT (
  defined_tags json_element_t,
  freeform_tags json_element_t,
  max_duration_in_minutes number,
  idle_timeout_in_minutes number,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_update_run_details_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_update_run_details_t (
    defined_tags json_element_t,
    freeform_tags json_element_t,
    max_duration_in_minutes number,
    idle_timeout_in_minutes number
  ) RETURN SELF AS RESULT
) NOT PERSISTABLE;

Fields

Field Description

defined_tags

(optional) Defined tags for this resource. Each key is predefined and scoped to a namespace. For more information, see Resource Tags. Example: `{\"Operations\": {\"CostCenter\": \"42\"}}`

freeform_tags

(optional) Free-form tags for this resource. Each tag is a simple key-value pair with no predefined name, type, or namespace. For more information, see Resource Tags. Example: `{\"Department\": \"Finance\"}`

max_duration_in_minutes

(optional) The maximum duration in minutes for which an Application should run. Data Flow Run would be terminated once it reaches this duration from the time it transitions to `IN_PROGRESS` state.

idle_timeout_in_minutes

(optional) The timeout value in minutes used to manage Runs. A Run would be stopped after inactivity for this amount of time period. Note: This parameter is currently only applicable for Runs of type `SESSION`. Default value is 2880 minutes (2 days)

DBMS_CLOUD_OCI_DATAFLOW_UPDATE_SQL_ENDPOINT_DETAILS_T Type

Currently only the tags of a SQL Endpoint can be updated.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_update_sql_endpoint_details_t FORCE AUTHID CURRENT_USER IS OBJECT (
  defined_tags json_element_t,
  freeform_tags json_element_t,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_update_sql_endpoint_details_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_update_sql_endpoint_details_t (
    defined_tags json_element_t,
    freeform_tags json_element_t
  ) RETURN SELF AS RESULT
) NOT PERSISTABLE;

Fields

Field Description

defined_tags

(optional) Defined tags for this resource. Each key is predefined and scoped to a namespace. For more information, see Resource Tags. Example: `{\"Operations\": {\"CostCenter\": \"42\"}}`

freeform_tags

(optional) Free-form tags for this resource. Each tag is a simple key-value pair with no predefined name, type, or namespace. For more information, see Resource Tags. Example: `{\"Department\": \"Finance\"}`

DBMS_CLOUD_OCI_DATAFLOW_WORK_REQUEST_RESOURCE_T Type

A resource related to a Data Flow work request.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_work_request_resource_t FORCE AUTHID CURRENT_USER IS OBJECT (
  action_type varchar2(32767),
  id number,
  resource_id varchar2(32767),
  resource_type varchar2(32767),
  resource_uri varchar2(32767),
  work_requestid varchar2(32767),
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_work_request_resource_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_work_request_resource_t (
    action_type varchar2,
    id number,
    resource_id varchar2,
    resource_type varchar2,
    resource_uri varchar2,
    work_requestid varchar2
  ) RETURN SELF AS RESULT
) NOT PERSISTABLE;

Fields

Field Description

action_type

(required) The way in which this resource is affected by the work tracked in the work request.

Allowed values are: 'CREATED', 'UPDATED', 'DELETED', 'INPROGRESS', 'RELATED'

id

(optional) The id of a work request resource object.

resource_id

(required) The id of the releated resource. See resourceType to identity the specific type of resource.

resource_type

(required) The type of resource. See resourceId for the id of the specific resource.

resource_uri

(optional) The URI path that the user can use to get access to the resource metadata

work_requestid

(optional) The OCID of a work request.

DBMS_CLOUD_OCI_DATAFLOW_WORK_REQUEST_RESOURCE_TBL Type

Nested table type of dbms_cloud_oci_dataflow_work_request_resource_t.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_work_request_resource_tbl FORCE IS TABLE OF (dbms_cloud_oci_dataflow_work_request_resource_t) NOT PERSISTABLE;

DBMS_CLOUD_OCI_DATAFLOW_WORK_REQUEST_T Type

A Data Flow work request object.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_work_request_t FORCE AUTHID CURRENT_USER IS OBJECT (
  compartment_id varchar2(32767),
  id varchar2(32767),
  operation varchar2(32767),
  percent_complete number,
  resources dbms_cloud_oci_dataflow_work_request_resource_tbl,
  status varchar2(32767),
  time_accepted timestamp with time zone,
  time_started timestamp with time zone,
  time_finished timestamp with time zone,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_work_request_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_work_request_t (
    compartment_id varchar2,
    id varchar2,
    operation varchar2,
    percent_complete number,
    resources dbms_cloud_oci_dataflow_work_request_resource_tbl,
    status varchar2,
    time_accepted timestamp with time zone,
    time_started timestamp with time zone,
    time_finished timestamp with time zone
  ) RETURN SELF AS RESULT
) NOT PERSISTABLE;

Fields

Field Description

compartment_id

(required) The OCID of a compartment.

id

(required) The OCID of a work request.

operation

(required) The operation related to this work request.

Allowed values are: 'CREATE_PRIVATE_ENDPOINT', 'UPDATE_PRIVATE_ENDPOINT', 'DELETE_PRIVATE_ENDPOINT', 'MOVE_PRIVATE_ENDPOINT'

percent_complete

(required) Percentage of the request completed.

resources

(required) The resources affected by this work request.

status

(required) The status of the work request.

Allowed values are: 'ACCEPTED', 'CANCELLED', 'CANCELLING', 'FAILED', 'INPROGRESS', 'SUCCEEDED'

time_accepted

(optional) The date and time the request was created, as described in RFC 3339, section 14.29.

time_started

(optional) The date and time the request was started, as described in RFC 3339, section 14.29.

time_finished

(optional) The date and time the object was finished, as described in RFC 3339.

DBMS_CLOUD_OCI_DATAFLOW_WORK_REQUEST_SUMMARY_T Type

A Data Flow work request summary object.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_work_request_summary_t FORCE AUTHID CURRENT_USER IS OBJECT (
  compartment_id varchar2(32767),
  id varchar2(32767),
  operation varchar2(32767),
  percent_complete number,
  resources dbms_cloud_oci_dataflow_work_request_resource_tbl,
  status varchar2(32767),
  time_accepted timestamp with time zone,
  time_started timestamp with time zone,
  time_finished timestamp with time zone,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_work_request_summary_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_work_request_summary_t (
    compartment_id varchar2,
    id varchar2,
    operation varchar2,
    percent_complete number,
    resources dbms_cloud_oci_dataflow_work_request_resource_tbl,
    status varchar2,
    time_accepted timestamp with time zone,
    time_started timestamp with time zone,
    time_finished timestamp with time zone
  ) RETURN SELF AS RESULT
) NOT PERSISTABLE;

Fields

Field Description

compartment_id

(required) The OCID of a compartment.

id

(required) The OCID of a work request.

operation

(required) The operation related to this work request.

Allowed values are: 'CREATE_PRIVATE_ENDPOINT', 'UPDATE_PRIVATE_ENDPOINT', 'DELETE_PRIVATE_ENDPOINT', 'MOVE_PRIVATE_ENDPOINT'

percent_complete

(required) Percentage of the request completed.

resources

(optional) The resources affected by this work request.

status

(required) The status of the work request.

Allowed values are: 'ACCEPTED', 'CANCELLED', 'CANCELLING', 'FAILED', 'INPROGRESS', 'SUCCEEDED'

time_accepted

(optional) The date and time the request was created, as described in RFC 3339, section 14.29.

time_started

(optional) The date and time the request was started, as described in RFC 3339, section 14.29.

time_finished

(optional) The date and time the object was finished, as described in RFC 3339.

DBMS_CLOUD_OCI_DATAFLOW_WORK_REQUEST_SUMMARY_TBL Type

Nested table type of dbms_cloud_oci_dataflow_work_request_summary_t.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_work_request_summary_tbl FORCE IS TABLE OF (dbms_cloud_oci_dataflow_work_request_summary_t) NOT PERSISTABLE;

DBMS_CLOUD_OCI_DATAFLOW_WORK_REQUEST_COLLECTION_T Type

Results of a query for a list of work requests. Contains WorkRequestSummary items.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_work_request_collection_t FORCE AUTHID CURRENT_USER IS OBJECT (
  items dbms_cloud_oci_dataflow_work_request_summary_tbl,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_work_request_collection_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_work_request_collection_t (
    items dbms_cloud_oci_dataflow_work_request_summary_tbl
  ) RETURN SELF AS RESULT
) NOT PERSISTABLE;

Fields

Field Description

items

(required) A list of work requests

DBMS_CLOUD_OCI_DATAFLOW_WORK_REQUEST_ERROR_T Type

A Data Flow work request error object.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_work_request_error_t FORCE AUTHID CURRENT_USER IS OBJECT (
  code varchar2(32767),
  id number,
  message varchar2(32767),
  l_timestamp timestamp with time zone,
  work_requestid varchar2(32767),
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_work_request_error_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_work_request_error_t (
    code varchar2,
    id number,
    message varchar2,
    l_timestamp timestamp with time zone,
    work_requestid varchar2
  ) RETURN SELF AS RESULT
) NOT PERSISTABLE;

Fields

Field Description

code

(required) A Machine-usable code for the error that occured.

id

(optional) The id of a work request error.

message

(required) A human readable description of the issue encountered.

l_timestamp

(required) The time the error occured. An RFC3339 formatted datetime string.

work_requestid

(optional) The OCID of a work request.

DBMS_CLOUD_OCI_DATAFLOW_WORK_REQUEST_ERROR_TBL Type

Nested table type of dbms_cloud_oci_dataflow_work_request_error_t.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_work_request_error_tbl FORCE IS TABLE OF (dbms_cloud_oci_dataflow_work_request_error_t) NOT PERSISTABLE;

DBMS_CLOUD_OCI_DATAFLOW_WORK_REQUEST_ERROR_COLLECTION_T Type

Results of a query for a list of work request errors. Contains WorkRequestError items.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_work_request_error_collection_t FORCE AUTHID CURRENT_USER IS OBJECT (
  items dbms_cloud_oci_dataflow_work_request_error_tbl,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_work_request_error_collection_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_work_request_error_collection_t (
    items dbms_cloud_oci_dataflow_work_request_error_tbl
  ) RETURN SELF AS RESULT
) NOT PERSISTABLE;

Fields

Field Description

items

(required) A list of work request errors.

DBMS_CLOUD_OCI_DATAFLOW_WORK_REQUEST_LOG_T Type

A Data Flow work request log object.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_work_request_log_t FORCE AUTHID CURRENT_USER IS OBJECT (
  id number,
  message varchar2(32767),
  l_timestamp timestamp with time zone,
  work_requestid varchar2(32767),
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_work_request_log_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_work_request_log_t (
    id number,
    message varchar2,
    l_timestamp timestamp with time zone,
    work_requestid varchar2
  ) RETURN SELF AS RESULT
) NOT PERSISTABLE;

Fields

Field Description

id

(optional) The id of a work request log.

message

(required) A human readable log message.

l_timestamp

(required) The time the log message was written. An RFC3339 formatted datetime string.

work_requestid

(optional) The OCID of a work request.

DBMS_CLOUD_OCI_DATAFLOW_WORK_REQUEST_LOG_TBL Type

Nested table type of dbms_cloud_oci_dataflow_work_request_log_t.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_work_request_log_tbl FORCE IS TABLE OF (dbms_cloud_oci_dataflow_work_request_log_t) NOT PERSISTABLE;

DBMS_CLOUD_OCI_DATAFLOW_WORK_REQUEST_LOG_COLLECTION_T Type

Results of a query for a list of work request logs. Contains WorkRequestLog items.

Syntax

CREATE OR REPLACE NONEDITIONABLE TYPE dbms_cloud_oci_dataflow_work_request_log_collection_t FORCE AUTHID CURRENT_USER IS OBJECT (
  items dbms_cloud_oci_dataflow_work_request_log_tbl,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_work_request_log_collection_t
    RETURN SELF AS RESULT,
  CONSTRUCTOR FUNCTION dbms_cloud_oci_dataflow_work_request_log_collection_t (
    items dbms_cloud_oci_dataflow_work_request_log_tbl
  ) RETURN SELF AS RESULT
) NOT PERSISTABLE;

Fields

Field Description

items

(required) A list of work request logs.

Was this article helpful?