Setting Up a Bastion for Cluster Access

VCN Administrator Steps

As the VCN administrator, follow these steps to set up egress and ingress security rules to enable a bastion to access the Kubernetes API endpoint:

1. Create a new private subnet to host the bastion, in the same VCN as the cluster to which you want to provide access.

   Note that if you've already created a subnet to host a bastion to access worker nodes, you can reuse that subnet instead of creating a new subnet. See Setting up a bastion to provide SSH access to managed nodes.

2. Add the following TCP/6443 egress security rule to the security list associated with the bastion subnet.

   This egress security rule allows traffic from the bastion subnet to the cluster's Kubernetes API endpoint subnet.

   State:	Destination	Protocol / Dest. Port	Description:
   Stateful	Kubernetes API Endpoint CIDR (for example, 10.0.0.0/29)	TCP/6443	Allow bastion to Kubernetes API endpoint communication.

3. If it's not already present, add the following TCP/6443 ingress security rule to a new or existing security list associated with the subnet hosting the Kubernetes API endpoint.

   This ingress security rule allows traffic into the Kubernetes API endpoint subnet from the bastion subnet.

   State:	Source	Protocol / Dest. Port	Description:
   Stateful	Bastion subnet CIDR	TCP/6443	Allow bastion to Kubernetes API endpoint communication.

   Note
   
   

   Although it is not Oracle's preferred design, you can use the same subnet for both the bastion and the cluster's Kubernetes API endpoint. In this case, add the TCP/6443 egress security rule to the security list associated with the Kubernetes API endpoint subnet.

Cluster Administrator Steps

As the cluster administrator, follow these steps to set up a bastion to access the Kubernetes API endpoint:

1. Create a bastion, as follows:
   1. In the Console, open the navigation menu and click Identity & Security.
   2. Click Bastion.
   3. On the Bastion page, click Create bastion.
   4. In the Create bastion dialog, set the following properties:
      • Bastion name: A name for the bastion. Avoid entering confidential information. Only alphanumeric characters are supported.
      • Target Virtual Cloud Network: Specify the VCN of the Kubernetes cluster to which you want to provide access.
      • Target Subnet: The subnet to host the bastion, with the ingress and egress security rules you've set up.
      • CIDR Block Allowlist: One or more address ranges in CIDR notation that you want to allow to connect to sessions hosted by this bastion. For example, 0.0.0.0/0 to allow access from the internet, or a more limited address range. For example, 203.0.113.0/24.
   5. (Optional) To change the maximum amount of time that any session on this bastion can remain active, click Show Advanced Options, and then enter a value for Maximum Session Time-to-Live.
   6. Click Create Bastion.
   7. When the bastion has been created, click the name of the bastion to see information about it, including its OCID.

   For more information about creating a bastion, see To create a bastion.

2. Provide cluster users with the OCID of the bastion.
3. Ensure that suitable IAM policies are in place to authorize cluster users to use the bastion. See Required IAM Policies for Creating and Using Bastions to Access Clusters.

   If desired, you can also set up IAM policies to limit the resources that cluster users can access using the bastion. See Setting up IAM policies to limit the use of bastions.

Cluster User Steps

As a cluster user, follow these steps to create a bastion session to access the Kubernetes API endpoint:

1. If you haven't done so already, create the kubeconfig file for the cluster you want to access, by running:

   oci ce cluster create-kubeconfig \
     --cluster-id <cluster OCID> \
     --file $HOME/.kube/config \
     --region <region> \
     --token-version 2.0.0
   

2. Edit the cluster's kubeconfig file and change the IP address specified for server to specify the IP address and port on which to listen for SSH traffic:
   1. Locate the line:

      server: https://x.x.x.x:6443

   2. Change the line to:

      server: https://127.0.0.1:6443

      For example:

      apiVersion: v1
      clusters:
      - cluster:
          certificate-authority-data:______
          server: https://127.0.0.1:6443
      - name: cluster-xxxxxxx
      …
      

3. Create a bastion session using the Console or the CLI as follows:

   Using the Console

   1. On the Bastion page, click the name of the bastion created by the cluster administrator.
   2. On the Sessions page, click Create session.
   3. In the Create session dialog, set the following properties:
      • Session type: Select SSH port forwarding session.
      • Session name: A display name for the new session. Avoid entering confidential information.
      • Connect to the target host by using: Select IP address.
      • IP address: Specify the IP address part of the cluster's Kubernetes API private endpoint (shown on the Cluster Details page).
      • Port: Specify the port part of the cluster's Kubernetes API private endpoint (shown on the Cluster Details page). For example, 6443.
      • SSH Key: Specify the public key file of an existing SSH key pair that you want to use for the session, or generate a new SSH key pair and save the private key.
   4. (Optional) To change the maximum amount of time that the session can remain active, click Show Advanced Options, and then enter a value for Maximum Session Time-to-Live.
   5. Click Create session.
   Using the CLI

   oci bastion session create-port-forwarding \
    --bastion-id <bastion OCID> \
    --ssh-public-key-file <ssh public key> \
    --target-private-ip <API Private IP endpoint> \
    --target-port 6443
   

   For more information about creating a bastion session, see To create a session.

4. Obtain the command to create an SSH tunnel using the Console or the CLI as follows:

   Using the Console

   On the Sessions page, select Copy SSH Command from the Actions menu () beside the session you have just created.

   Using the CLI

   oci bastion session get --session-id <session OCID> | jq '.data."ssh-metadata".command'

   The SSH tunnel command you've obtained has the following format:

   ssh -i <privateKey> -N -L <localPort>:<session-IP>:<session-port> -p 22 <session-ocid>

   where:

   • <session-IP>:<session-port> is the IP address and port number of the Kubernetes API endpoint that you specified when creating the bastion session. For example, 10.0.0.6:6443.
   • <session-ocid> is the OCID of the bastion session you created.
5. Execute the command to create the SSH tunnel on a local workstation or in Cloud Shell, as follows:

   1. Edit the SSH tunnel command you've obtained as follows:

      • Replace <privateKey> with the path to the file containing the private key of the SSH key pair you specified for the session. For example, /home/johndoe/.ssh/id_rsa
      • Replace <localPort> with 6443, or a free port on your system (either a local workstation if your network is peered with the cluster's VCN, or Cloud Shell)
      • Add & to the end of the command to make the command run in the background.
   2. Run the edited SSH tunnel command on a local workstation, or in the Cloud Shell window.

      For example:

      ssh -i /home/johndoe/.ssh/id_rsa -N -L 6443:10.0.0.6:6443 -p 22 ocid1.bastionsession_______oraclecloud.com &

You can now perform kubectl operations on the cluster until the SSH tunnel or bastion session times out.

VCN Administrator Steps

As the VCN administrator, follow these steps to set up egress and ingress security rules to enable a bastion to provide SSH access to managed nodes:

1. Create a new subnet to host the bastion, in the same VCN as the cluster to which you want to provide access.

   Note that if you've already created a subnet to host a bastion to access the Kubernetes API endpoint, you can reuse that subnet instead of creating a new subnet. See Setting up a bastion to access the Kubernetes API endpoint.

2. Add the following TCP/22 egress security rule to the security list associated with the bastion subnet.

   This egress security rule allows traffic from the bastion subnet to the worker nodes subnet.

   State:	Destination	Protocol / Dest. Port	Description:
   Stateful	Worker Nodes CIDR (for example, 10.0.1.0/24)	TCP/22	Allow bastion to worker nodes communication.

3. If it's not already present, add the following TCP/22 ingress security rule to a new or existing security list associated with the subnet hosting the worker nodes.

   This ingress security rule allows traffic into the worker node subnet from the bastion subnet.

   State:	Source	Protocol / Dest. Port	Description:
   Stateful	Bastion subnet CIDR	TCP/22	Allow bastion to worker nodes communication.

   Note
   
   

   Although it is not Oracle's preferred design, you can use the same subnet for both the bastion and the worker nodes. In this case, add the TCP/22 egress security rule to the security list associated with the worker node subnet.

Cluster Administrator Steps

As the cluster administrator, follow these steps to set up a bastion to provide SSH access to managed nodes:

1. Enable the bastion agent on the worker node to which you want to connect through SSH as follows:
   1. Open the navigation menu and click Developer Services. Under Containers & Artifacts, click Kubernetes Clusters (OKE).
   2. On the Cluster List page, click the name of the cluster containing the worker node to which you want to connect.
   3. Under Resources, click Node Pools and click the name of the node pool containing the worker node to which you want to connect.
   4. Under Resources, click Nodes and click the name of the worker node to which you want to connect.
   5. On the Instance Details page, display the Oracle Cloud Agent tab, and toggle the Enabled the Bastion plugin.

      It can take up to 10 minutes for the change to take effect.

   6. Wait until the Bastion plugin Status is shown as Running before continuing to the next step.
2. Create a bastion, as follows:
   1. Open the navigation menu and click Identity & Security.
   2. Click Bastion.
   3. On the Bastion page, click Create bastion.
   4. In the Create bastion dialog, set the following properties:
      • Bastion name: A name for the bastion. Avoid entering confidential information. Only alphanumeric characters are supported.
      • Target Virtual Cloud Network: Specify the VCN of the Kubernetes cluster to which you want to provide access.
      • Target Subnet: The subnet to host the bastion, with the ingress and egress security rules you've set up.
      • CIDR Block Allowlist: One or more address ranges in CIDR notation that you want to allow to connect to sessions hosted by this bastion. For example, 0.0.0.0/0 to allow access from the internet, or a more limited address range. For example, 203.0.113.0/24.
   5. (Optional) To change the maximum amount of time that any session on this bastion can remain active, click Show Advanced Options, and then enter a value for Maximum Session Time-to-Live.
   6. Click Create Bastion.
   7. When the bastion has been created, click the name of the bastion to see information about it, including its OCID.

   For more information about creating a bastion, see To create a bastion.

3. Provide cluster users with the OCID of the bastion.
4. Ensure that suitable IAM policies are in place to authorize cluster users to use the bastion. See Required IAM Policies for Creating and Using Bastions to Access Clusters.

   If desired, you can also set up IAM policies to limit the resources that cluster users can access using the bastion. See Setting up IAM policies to limit the use of bastions.

Cluster User Steps

As a cluster user, follow these steps to create a bastion session to provide SSH access to managed nodes:

1. Create a bastion session using the Console or the CLI as follows:

   Using the CLI

   1. On the Bastion page, click the name of the bastion created by the cluster administrator.
   2. On the Sessions page, click Create session.
   3. In the Create session dialog, set the following properties:
      • Session type: Select Managed SSH session.
      • Session name: A display name for the new session. Avoid entering confidential information.
      • Username: Enter opc.
      • Compute instance in <compartment name>: Select the name of the worker node compute instance from the list of compute instances in the compartment. If needed, change the compartment to find the instance. Only active instances are listed.
      • SSH Key: Specify the public key file of an existing SSH key pair that you want to use for the session, or generate a new SSH key pair and save the private key.
   4. (Optional) To change the maximum amount of time that the session can remain active, click Show Advanced Options, and then enter a value for Maximum Session Time-to-Live.

      Notice that you do not change the default value in the Target compute instance port field (the default port is already set to 22). Also, you do not specify IP addresses in the Target compute instance IP address field because you already selected the compute instances.

   5. Click Create session.
   Using the CLI

   oci bastion session create-managed-ssh \
     --bastion-id <bastion OCID> \
     --ssh-public-key-file <ssh public key> \
     --target-resource-id <worker node instance OCID> \
     --target-os-username <instance_username>

   For more information about creating a bastion session, see To create a session.

2. Obtain the command to create an SSH tunnel using the Console or the CLI as follows:

   Using the Console

   On the Sessions page, select Copy SSH Command from the Actions menu () beside the session you have just created.

   Using the CLI

   oci bastion session get --session-id <session OCID> | jq '.data."ssh-metadata".command'

   The SSH tunnel command you've obtained has the following format:

   ssh -i <privateKey> -o ProxyCommand="ssh -i <privateKey> -W %h:%p -p 22 <session-ocid>" -p 22 opc@<node-private-ip>

   where:

   • <session-ocid> is the OCID of the bastion session you created.
   • <node-private-ip> is the private IP address of the worker node that you specified when creating the bastion session. For example, 10.0.10.99.
3. Execute the command to create the SSH tunnel on a local workstation or in Cloud Shell, as follows:

   1. Edit the SSH tunnel command you've obtained as follows:

      • Replace <privateKey> with the path to the file containing the private key of the SSH key pair you specified for the session. For example, /home/johndoe/.ssh/id_rsa
      • Add & to the end of the command to made the command run in the background.
   2. Run the edited SSH tunnel command on a local workstation, or in the Cloud Shell window.

      For example:

      ssh -i /home/johndoe/.ssh/id_rsa -o ProxyCommand="ssh -i /home/johndoe/.ssh/id_rsa -W %h:%p -p 22 ocid1.bastionsession_______oraclecloud.com" -p 22 opc@10.0.10.99 &

You can now perform operations on the worker nodes until the SSH tunnel or bastion session times out.