Creating SQL Endpoints

Follow these steps to create a Data Flow SQL endpoint.

• Console
• CLI
• API

• Follow the steps to create a Metastore and set up IAM policies.

  1. Open the navigation menu, and click Analytics and AI. Under Data Lake click Data Flow.
  2. In the left-side menu, click SQL endpoints.
  3. Under List scope, select the compartment that contains the SQL endpoint to edit.
  4. Click Create SQL endpoint.
  5. Enter the name of the SQL endpoint in Name.
  6. (Optional) Description enter a description of the SQL Endpoint that you can use to help search for it.
  7. (Optional) Select Enable autoscaling to use autoscaling.
  8. From the Driver shape list, select the shape you want.
     Select the number of OCPUs if you want a different number to the default value. The memory and block volume amounts change.
  9. From the Executor shape list, select the shape you want.
     Select the number of OCPUs if you want a different number to the default value. The memory and block volume amounts change.
  10. Select the minimum number of executors.
  11. Select the maximum number of executors.
  12. Select the metastore. If the metastore is in a different compartment, click Change compartment to select its compartment.
      Note
      
      You must create a metastore in Data Catalog if one doesn't exist.
  13. (Optional) The default managed table location is set automatically using the same default managed table location configured for the metastore.
  14. Set the network configuration, by selecting either Allow secure access from everywhere, or Virtual Cloud Network.
      • If you selected Allow secure access from everywhere and clicked Configure access control rules:
        1. Select the IP notation type.
        2. Enter value.
        3. (Optional) Click +Another entry to add more access control rules.
      • If you selected Virtual Cloud Network:
        1. Select the VCN. If the VCN is in a different compartment, click Change compartment to select its compartment.
        2. Select the subnet. If the subnet is in a different compartment, click Change compartment to select its compartment.
        3. (Optional) Enter a host name prefix.
        4. Select the network security group. If the network security group is in a different compartment, click Change compartment to select its compartment.
        5. (Optional):Click +Another Network security group to add more network security groups.
  15. To set an advanced options click Show advanced options.
      The Spark configuration properties are shown. You can change the values for:

      • spark.dynamicAllocation.executorIdleTimeout
      • spark.dynamicAllocation.schedulerBacklogTimeout
      • spark.dataflow.dynamicAllocation.quotaPolicy

      To add another Spark configuration property, click +Another property and enter the Key and value.
  16. (Optional) To add tags to the application, select a tag namespace (for defined tags) and populate then specify a tag key and value. Add more tags as needed. For more information about tagging, see Overview of Tagging.
  17. Click Create SQL endpoint to create the SQL endpoint.

• Use the create command and required parameters to create a SQL endpoint

  oci data-flow sql-endpoint create [OPTIONS]

  For a complete list of parameters and values for CLI commands, see the CLI Command Reference.

• Run the CreateSqlEndpoint operation to create a SQL endpoint.