Connecting to a Cluster
This topic covers the prerequisites and connection details required to connect to your OpenSearch cluster for OCI Search with OpenSearch. It also includes a walkthrough that covers the steps for some basic connection scenarios using a VM instance in the same VCN and subnet as the cluster.
Use the information provided here to enable your OpenSearch clients and custom search solutions to connect and ingest data from your cluster.
While Search with OpenSearch is applying patches to a cluster's instances or resizing the cluster, the cluster's search functionality is in read-only mode, write mode is disabled for a maintenance window. See Cluster Patching and Resize Operations for more information.
Private Endpoints
Search with OpenSearch provides access to your cluster and the OpenSearch Dashboards for your cluster over private endpoints. The service provides two ways to connect to the private endpoints for your cluster:
Private IP: The private IP address for the private endpoint. Use the private IP if you're connecting from a VM instance within the same VCN and subnet as your cluster. You can also use the private IP when you connect from your local machine with port forwarding configured to the VM instance.
- API endpoint: The a Fully Qualified Domain Name (FQDN) provided by the service to access cluster's private endpoints. Use the API endpoint when connecting from your OpenSearch client, such as LogStash or Beats.
See Cluster Connection Information for specific details.
With private endpoints, traffic doesn't go over the internet. The endpoints are only accessible from within your cluster's VCN. To make the API endpoint access to your OpenSearch clients, more configuration is required to make the endpoint accessible to external clients. For example, you could use API Gateway to enable access with a public IP address.
Prerequisites
Following are the list of prerequisites required to connect to your cluster
- Determine the cluster's connection information.
-
Create the required security rules.
-
(Optional) Create and connect to a VM instance.
Cluster Connection Information
The cluster's connection information includes the API endpoints and IP addresses that you use to connect to and work with your cluster and the cluster's OpenSearch Dashboards. For more information, see Private Endpoints.
You can find all the connection information on the Cluster Details page in the Console.
Connections to clusters created before September 15th, 2022 require that
you download a certificate. The API endpoint's domain for these clusters is
oci.oracleiaas.com.
oci.oraclecloud.com.-
Open the navigation menu and click Databases. Under OpenSearch, click Clusters.
-
In the Clusters list, click the name of the cluster you want to view the details for.
-
On the Cluster Details page, make note of the following:
- API endpoint: The FQDN for the cluster's private endpoint.
- Private IP: The private IP address for the cluster's private endpoint.
- OpenSearch Dashboards API endpoint: The FQDN for the cluster's OpenSearch Dashboards..
- OpenSearch Dashboards Private IP: The IP address for the private endpoint you can use to connect to the cluster's OpenSearch Dashboards.
Create Security Rules
You need to create security rules for the cluster's VCN to allow connections to the cluster and the cluster's OpenSearch Dashboards.
To make the data accessible from your cluster, you need to add stateful ingress security rules for TCP traffic for the following:
- OpenSearch on destination port 9200 from source 0.0.0.0/0 and any source port.
- OpenSearch Dashboards on destination port 5601 from source 0.0.0.0/0 and any source port.
You can create a security list with these rules, or you can add them to the default security list for the cluster's VCN.
-
Open the navigation menu and click Databases. Under OpenSearch, click Clusters.
- Click the cluster that you're interested in.
- Under Cluster details, for Virtual cloud network, click the name of the cloud network. The Virtual Cloud Network Details page opens.
- Under Resources, click Security Lists.
- Click Create Security List.
- Enter the following:
- Name: A descriptive name for the security list. The name doesn't have to be unique, and it cannot be changed later in the Console (but you can change it with the API). Avoid entering confidential information.
- Create in Compartment: The compartment where you want to create the security list, if different from the compartment you're currently working in.
- Click Another Ingress Rule
-
Enter the following values for the rule:
- Stateless: Leave the check box cleared.
- Source Type: CIDR
- Source CIDR: 0.0.0.0/0
- IP Protocol: TCP
- Source Port Range: All
- Destination Port Range: 9200
- Description: An optional description of the rule.
- Click Another Ingress Rule.
-
Enter the following values for the rule:
- Stateless: Leave the check box cleared.
- Source Type: CIDR
- Source CIDR: 0.0.0.0/0
- IP Protocol: TCP
- Source Port Range: All
- Destination Port Range: 5601
- Description: An optional description of the rule.
- Click Create Security List.
-
Open the navigation menu and click Databases. Under OpenSearch, click Clusters.
- Click the cluster that you're interested in.
- Under Cluster details, for Virtual cloud network, click the name of the cloud network. The Virtual Cloud Network Details page opens.
- Under Resources, click Security Lists.
- Click the security list named Default Security List for VCN_name.
- Click Add Ingress Rules
-
Enter the following values for the rule:
- Stateless: Leave the check box cleared.
- Source Type: CIDR
- Source CIDR: 0.0.0.0/0
- IP Protocol: TCP
- Source Port Range: All
- Destination Port Range: 9200
- Description: An optional description of the rule.
- Click Another Ingress Rule.
-
Enter the following values for the rule:
- Stateless: Leave the check box cleared.
- Source Type: CIDR
- Source CIDR: 0.0.0.0/0
- IP Protocol: TCP
- Source Port Range: All
- Destination Port Range: 5601
- Description: An optional description of the rule.
- Click Add Ingress Rules.
Create a VM Instance (Optional)
After completing the connection prerequisites, optionally you can create a VM instance to run some basic connection tests to verify that the prerequisites have been configured correctly and that you have the correct connection information. You create the VM instance in the same subnet for the VCN you created the cluster in. See one of the following topics for how to create an instance:
After the instance is created, connect to the instance from your local machine. For more information, see the following:
Test the Connection to your Cluster
After you're connected to the VM instance from your local machine, run one of the following commands to test the connection.
For clusters where the API endpoint domain is oci.oraclecloud.com (created on or
after September 15, 2022):
curl https://<cluster_API_endpoint>:9200The URL passed in this command is the value specified in the API endpoint field on the Cluster Details page. See Cluster Connection Information.
If the connection to the endpoint is successful, a response like the following is returned:
{
"name" : "opensearch-master-0",
"cluster_name" : "<cluster_name>",
"cluster_uuid" : "<cluster_UUID>",
"version" : {
"distribution" : "opensearch",
"number" : "1.2.4-SNAPSHOT",
"build_type" : "tar",
"build_hash" : "<build_hash>",
"build_date" : "2022-02-08T16:44:39.596468Z",
"build_snapshot" : true,
"lucene_version" : "8.10.1",
"minimum_wire_compatibility_version" : "6.8.0",
"minimum_index_compatibility_version" : "6.0.0-beta1"
},
"tagline" : "The OpenSearch Project: https://opensearch.org/"
}For other commands you can test, including pushing data to the cluster, and submitting search queries to the cluster, see the Search and visualize data using OCI Search Service with OpenSearch tutorial.
Test the Connection to the Cluster's OpenSearch Dashboards
One of the ways that you can connect to your cluster's OpenSearch Dashboards is by configuring port forwarding from your local machine to the VM instance you created in the same VCN and subnet as the cluster.
To configure port forwarding to connect to OpenSearch Dashboards
Create a VM instance.
Run the following command from your local machine:
ssh -C -v -t -L 127.0.0.1:5601:<OpenSearch_Dashboards_private_IP>:5601 -L 127.0.0.1:9200:<OpenSearch_cluster_private_IP>:9200 opc@<VM_instance_public_IP> -i <private_key_file>Replace <OpenSearch_Dashboards_private_IP> with the private IP for your cluster's OpenSearch Dashboards. Replace <OpenSearch_cluster_private_IP> with your cluster's private IP. To locate the private IPs, see Cluster Connection Information.
Replace <VM_instance_public_IP> with the public IP address for the VM instance. Replace <private_key_file> with the path to the private key you use for connecting to the instance. For more information about these values and how to locate them, see Connecting to an Instance.
From a browser on your local machine, open
https://localhost:5601to access the OpenSearch Dashboards.
OpenSearch Resources
Search with OpenSearch supports open source OpenSearch APIs, OpenSearch Dashboards, plugins, and other OpenSearch features. See the following external resources for additional information about working with OpenSearch.