Oracle Cloud Infrastructure Documentation

Prerequisites for deploying the OCI Native Ingress Controller as a Standalone Program

Find out what you have to do before you can deploy the OCI native ingress controller as a standalone program to load balance and route incoming traffic to service pods running on worker nodes in a Kubernetes cluster.

Before deploying the OCI native ingress controller as a standalone program:

Setting Up an Instance Principal, User Principal, or Workload Identity Principal to Enable Access to OCI Services and Resources

To create a load balancer and route incoming traffic, the OCI native ingress controller, whether installed as a standalone program or as a cluster add-on, performs actions on other Oracle Cloud Infrastructure service resources (including the Load Balancer service and the Certificates service). To perform those actions on OCI service resources, the OCI native ingress controller pod uses the credentials of an authorized actor (or principal). You can currently set up the following types of principal to enable the OCI native ingress controller to perform actions on OCI service resources:

Note:

  • The use of instance principals to enable the OCI native ingress controller to access OCI services and resources is not supported in clusters with virtual nodes.
  • The use of workload identity principals to enable the OCI native ingress controller to access OCI services and resources is supported with enhanced clusters, but not with basic clusters.

Using instance principals to enable access to OCI services and resources

You can set up an instance principal to enable the OCI native ingress controller to perform actions on OCI service resources. Note that you can only use instance principals with managed nodes.

To set up an instance principal:

  1. Create a new dynamic group to contain the compute instances hosting the cluster's worker nodes:
    1. Open the navigation menu and click Identity & Security. Under Identity, click Domains. Under Identity domain, click Dynamic groups.

    2. Select the compartment to which the compute instances belong.

    3. Follow the instructions in To create a dynamic group, and give the dynamic group a name (for example, acme-oke-native-ingress-controller-dyn-grp).

    4. Enter a rule that includes the compute instances in the compartment, in the format:
      ALL {instance.compartment.id = '<compartment-ocid>'}

      where <compartment-ocid> is the OCID of the compartment in which cluster node pools are created.

      For example:

      ALL {instance.compartment.id = 'ocid1.compartment.oc1..aaaaaaaa23______smwa'}
    5. Click Create Dynamic Group.

Before you deploy the OCI native ingress controller, you will:

Using user principals to enable access to OCI services and resources

You can set up a user principal to enable the OCI native ingress controller to perform actions on OCI service resources.

To set up a user principal:

  1. If a suitable user does not exist already, create a user in IAM (see To create a user).
  2. If a suitable group does not exist already, create a group in IAM (see To create a group).
  3. Add the user to the group (see To add a user to a group).

  4. Get these items:

  5. Upload the public key from the key pair in the Console. See How to Upload the Public Key.
  6. Create a config file as a .yaml file containing credential information, in the following format:
    auth:
  region: <region-identifier>
  passphrase: <passphrase>
  user: <user-ocid>
  fingerprint: <fingerprint>
  tenancy: <tenancy-ocid>
    where:
    • region: <region-identifier> is the region where the cluster resides. For example, us-ashburn-1
    • passphrase: <passphrase> specifies the passphrase used for the key if it is encrypted.
    • user: <user-ocid> is the OCID of the user that the OCI native ingress controller is to use.
    • fingerprint: <fingerprint> is the fingerprint of the public key.
    • tenancy: <tenancy-ocid> is the OCID of the tenancy containing the cluster.

    For example:

    auth:
  region: us-ashburn-1
  passphrase: examplepass
  user: ocid1.user.oc1..aaaaaaaa_example
  fingerprint: 67:d9:74:4b:21:example
  tenancy: ocid1.tenancy.oc1..aaaaaaaa_example
  7. Create a Kubernetes secret resource in the cluster by entering:
    
kubectl create secret generic <secret-name> \
--from-file=config=<config-file>.yaml \
--from-file=private-key=<private-key-file-path>.pem \
--namespace <namespace>
    where:
    • <secret-name> specifies the name of the secret to create. For example, oci-config
    • --from-file=config=<config-file>.yaml specifies the name and path of the .yaml file containing the credential information that you created previously. For example, user-auth-config.yaml
    • --from-file=private-key=./oci/oci_api_key.pem specifies the name and path of the downloaded private key file. For example, ./oci/oci_api_key.pem
    • --namespace <namespace> specifies the namespace containing the OCI native ingress controller

    For example:

    
kubectl create secret generic oci-config \
--from-file=config=user-auth-config.yaml \
--from-file=private-key=./oci/oci_api_key.pem \
--namespace acme-namespace

Before you deploy the OCI native ingress controller you will:

Using workload identity principals to enable access to OCI services and resources

You can set up a workload identity principal to enable the OCI native ingress controller to perform actions on OCI service resources. Note that you can only use workload identity principals with enhanced clusters.

To set up a workload identity principal:

  1. Obtain the OCID of the cluster (for example, using the Cluster Details tab in the Console).

  1. Before you deploy the OCI native ingress controller, you will:

Granting Permissions to the OCI Native Ingress Controller as a Standalone Program

The OCI native ingress controller requires permissions to access resources created by other Oracle Cloud Infrastructure services (such as the Load Balancer service and the Certificates service). The permissions you grant are the same, regardless of whether you install the OCI native ingress controller as a standalone program or as a cluster add-on. And the permissions are the same, regardless of whether you have set up an instance principal, a user principal, or a workload identity principal for the OCI native ingress controller. However, the way in which you grant those permissions does depend on the type of principal you have set up:

To set up permissions for the OCI native ingress controller, create a policy for the group (in the case of user principals), for the dynamic group (in the case of instance principals), or for the workload (in the case of workload identity principals), with policy statements to access OCI services and resources:

  1. Open the navigation menu and click Identity & Security. Under Identity, click Policies.
  2. Follow the instructions in To create a policy and give the policy a name (for example, acme-oke-native-ingress-controller-policy).

  3. If you are using instance principals or user principals, enter policy statements to allow access to the OCI services and resources used by the OCI native ingress controller, in the format:

    Allow <group|dynamic-group> <subject-name> to <verb> <resource> in <location>

    where:

    • <group|dynamic-group> is either group (in the case of user principals) or dynamic-group (in the case of instance principals)
    • <subject-name> is either the name of the group (in the case of user principals) or the name of the dynamic group (in the case of instance principals). For example, acme-oke-nat-ing-cont-dyn-grp. Note that if a group or dynamic group is not in the default identity domain, prefix the group or dynamic group name with the identity domain name, in the format <group|dynamic-group> '<identity-domain-name>'/'<group-name|dynamic-group-name>'. You can also specify a group or dynamic group using its OCID, in the format group id <group-ocid> or dynamic-group id <dynamic-group-ocid>.
    • <verb> <resource> is one of the following (all of these are required, in separate policy statements):
      • manage load-balancers
      • use virtual-network-family
      • manage cabundles
      • manage cabundle-associations
      • manage leaf-certificates
      • read leaf-certificate-bundles
      • manage certificate-associations
      • read certificate-authorities
      • manage certificate-authority-associations
      • read certificate-authority-bundles
      • read public-ips
      • manage floating-ips
      • manage waf-family
      • read cluster-family
    • <location> is one of:
      • tenancy, if you want the OCI native ingress controller to have access to resources in the entire tenancy.
      • compartment <compartment-name> if you only want the OCI native ingress controller to have access to resources in the compartment with the name you specify as compartment <compartment-name>.

    For example:

    Allow dynamic-group acme-oke-nat-ing-cont-dyn-grp to manage load-balancers in compartment acme-oke-cluster-compartment

    The syntax for the complete list of policy statements is as follows:

    Allow <group|dynamic-group> <subject-name> to manage load-balancers in <location>   
Allow <group|dynamic-group> <subject-name> to use virtual-network-family in <location>
Allow <group|dynamic-group> <subject-name> to manage cabundles in <location>
Allow <group|dynamic-group> <subject-name> to manage cabundle-associations in <location>
Allow <group|dynamic-group> <subject-name> to manage leaf-certificates in <location>
Allow <group|dynamic-group> <subject-name> to read leaf-certificate-bundles in <location>
Allow <group|dynamic-group> <subject-name> to manage certificate-associations in <location>
Allow <group|dynamic-group> <subject-name> to read certificate-authorities in <location>
Allow <group|dynamic-group> <subject-name> to manage certificate-authority-associations in <location>
Allow <group|dynamic-group> <subject-name> to read certificate-authority-bundles in <location>
Allow <group|dynamic-group> <subject-name> to read public-ips in <location>
Allow <group|dynamic-group> <subject-name> to manage floating-ips in <location>
Allow <group|dynamic-group> <subject-name> to manage waf-family in <location>
Allow <group|dynamic-group> <subject-name> to read cluster-family in <location>

    Example of the complete list of policy statements:

    Allow dynamic-group acme-oke-nat-ing-cont-dyn-grp to manage load-balancers in compartment acme-oke-cluster-compartment   
Allow dynamic-group acme-oke-nat-ing-cont-dyn-grp to use virtual-network-family in compartment acme-oke-cluster-compartment
Allow dynamic-group acme-oke-nat-ing-cont-dyn-grp to manage cabundles in compartment acme-oke-cluster-compartment
Allow dynamic-group acme-oke-nat-ing-cont-dyn-grp to manage cabundle-associations in compartment acme-oke-cluster-compartment
Allow dynamic-group acme-oke-nat-ing-cont-dyn-grp to manage leaf-certificates in compartment acme-oke-cluster-compartment
Allow dynamic-group acme-oke-nat-ing-cont-dyn-grp to read leaf-certificate-bundles in compartment acme-oke-cluster-compartment
Allow dynamic-group acme-oke-nat-ing-cont-dyn-grp to manage certificate-associations in compartment acme-oke-cluster-compartment
Allow dynamic-group acme-oke-nat-ing-cont-dyn-grp to read certificate-authorities in compartment acme-oke-cluster-compartment
Allow dynamic-group acme-oke-nat-ing-cont-dyn-grp to manage certificate-authority-associations in compartment acme-oke-cluster-compartment
Allow dynamic-group acme-oke-nat-ing-cont-dyn-grp to read certificate-authority-bundles in compartment acme-oke-cluster-compartment
Allow dynamic-group acme-oke-nat-ing-cont-dyn-grp to read public-ips in compartment acme-oke-cluster-compartment
Allow dynamic-group acme-oke-nat-ing-cont-dyn-grp to manage floating-ips in compartment acme-oke-cluster-compartment
Allow dynamic-group acme-oke-nat-ing-cont-dyn-grp to manage waf-family in compartment acme-oke-cluster-compartment
Allow dynamic-group acme-oke-nat-ing-cont-dyn-grp to read cluster-family in compartment acme-oke-cluster-compartment
  4. If you are using workload identity principals, enter policy statements to allow access to the OCI services and resources used by the OCI native ingress controller, in the format:
    Allow any-user to <verb> <resource> in <location> where all {request.principal.type = 'workload', request.principal.namespace = 'native-ingress-controller-system', request.principal.service_account = 'oci-native-ingress-controller', request.principal.cluster_id = '<cluster-ocid>'}

    where:

    • <verb> <resource> is one of the following (all of these are required, in separate policy statements):
      • manage load-balancers
      • use virtual-network-family
      • manage cabundles
      • manage cabundle-associations
      • manage leaf-certificates
      • read leaf-certificate-bundles
      • manage certificate-associations
      • read certificate-authorities
      • manage certificate-authority-associations
      • read certificate-authority-bundles
      • read public-ips
      • manage floating-ips
      • manage waf-family
      • read cluster-family
    • <location> is one of:
      • tenancy, if you want the OCI native ingress controller to have access to resources in the entire tenancy.
      • compartment <compartment-name> if you only want the OCI native ingress controller to have access to resources in the compartment with the name you specify as compartment <compartment-name>.
    • request.principal.namespace = 'native-ingress-controller-system' is the name of the namespace created for the OCI native ingress controller during installation.
    • request.principal.service_account = 'oci-native-ingress-controller' is the name of the service account created for the OCI native ingress controller during installation.
    • <cluster-ocid> is the cluster's OCID that you obtained previously.

    For example:

    Allow any-user to manage load-balancers in tenancy where all {request.principal.type = 'workload', request.principal.namespace = 'native-ingress-controller-system', request.principal.service_account = 'oci-native-ingress-controller', request.principal.cluster_id = 'ocid1.cluster.oc1.iad.aaaaaaaa______ska'}

    The syntax for the complete list of policy statements is:

    Allow any-user to manage load-balancers in <location> where all {request.principal.type = 'workload', request.principal.namespace = 'native-ingress-controller-system', request.principal.service_account = 'oci-native-ingress-controller', request.principal.cluster_id = '<cluster-ocid>'}
Allow any-user to use virtual-network-family in <location> where all {request.principal.type = 'workload', request.principal.namespace = 'native-ingress-controller-system', request.principal.service_account = 'oci-native-ingress-controller', request.principal.cluster_id = '<cluster-ocid>'}
Allow any-user to manage cabundles in <location> where all {request.principal.type = 'workload', request.principal.namespace = 'native-ingress-controller-system', request.principal.service_account = 'oci-native-ingress-controller', request.principal.cluster_id = '<cluster-ocid>'}
Allow any-user to manage cabundle-associations in <location> where all {request.principal.type = 'workload', request.principal.namespace = 'native-ingress-controller-system', request.principal.service_account = 'oci-native-ingress-controller', request.principal.cluster_id = '<cluster-ocid>'}
Allow any-user to manage leaf-certificates in <location> where all {request.principal.type = 'workload', request.principal.namespace = 'native-ingress-controller-system', request.principal.service_account = 'oci-native-ingress-controller', request.principal.cluster_id = '<cluster-ocid>'}
Allow any-user to read leaf-certificate-bundles in <location> where all {request.principal.type = 'workload', request.principal.namespace = 'native-ingress-controller-system', request.principal.service_account = 'oci-native-ingress-controller', request.principal.cluster_id = '<cluster-ocid>'}
Allow any-user to manage certificate-associations in <location> where all {request.principal.type = 'workload', request.principal.namespace = 'native-ingress-controller-system', request.principal.service_account = 'oci-native-ingress-controller', request.principal.cluster_id = '<cluster-ocid>'}
Allow any-user to read certificate-authorities in <location> where all {request.principal.type = 'workload', request.principal.namespace = 'native-ingress-controller-system', request.principal.service_account = 'oci-native-ingress-controller', request.principal.cluster_id = '<cluster-ocid>'}
Allow any-user to manage certificate-authority-associations in <location> where all {request.principal.type = 'workload', request.principal.namespace = 'native-ingress-controller-system', request.principal.service_account = 'oci-native-ingress-controller', request.principal.cluster_id = '<cluster-ocid>'}
Allow any-user to read certificate-authority-bundles in <location> where all {request.principal.type = 'workload', request.principal.namespace = 'native-ingress-controller-system', request.principal.service_account = 'oci-native-ingress-controller', request.principal.cluster_id = '<cluster-ocid>'}
Allow any-user to read public-ips in <location> where all {request.principal.type = 'workload', request.principal.namespace = 'native-ingress-controller-system', request.principal.service_account = 'oci-native-ingress-controller', request.principal.cluster_id = '<cluster-ocid>'}
Allow any-user to manage floating-ips in <location> where all {request.principal.type = 'workload', request.principal.namespace = 'native-ingress-controller-system', request.principal.service_account = 'oci-native-ingress-controller', request.principal.cluster_id = '<cluster-ocid>'}
Allow any-user to manage waf-family in <location> where all {request.principal.type = 'workload', request.principal.namespace = 'native-ingress-controller-system', request.principal.service_account = 'oci-native-ingress-controller', request.principal.cluster_id = '<cluster-ocid>'}
Allow any-user to read cluster-family in <location> where all {request.principal.type = 'workload', request.principal.namespace = 'native-ingress-controller-system', request.principal.service_account = 'oci-native-ingress-controller', request.principal.cluster_id = '<cluster-ocid>'}

Installing cert-manager

The OCI native ingress controller, whether installed as a standalone program or as a cluster add-on, uses webhooks to inject pod readiness gates into pod specifications. To ensure security, the webhook server has to run in HTTPS mode, which requires a pair of certificates and keys. The OCI native ingress controller uses Certificate Manager (also known as cert-manager) to generate and manage the certificates and keys for the webhook server, so you have to install cert-manager on the cluster to use pod readiness gates.

Regardless of whether you install the OCI native ingress controller as a standalone program or as a cluster add-on, you can install and run cert-manager in two ways:

  • You can install and run cert-manager as an open-source product, by entering:

    kubectl apply -f https://github.com/jetstack/cert-manager/releases/latest/download/cert-manager.yaml

  • You can install and run cert-manager as a cluster add-on. For more information about installing cert-manager as a cluster add-on, see Installing a Cluster Add-on.

For more information about cert-manager, see the cert-manager.io documentation.

Installing the Helm CLI to Install the OCI Native Ingress Controller as a Standalone Program

Helm is a package manager for Kubernetes. Helm is commonly used for creating, packaging, configuring, and deploying Kubernetes applications by combining configuration files into a single reusable package called a Helm chart. Charts are packaged and distributed in an archive format known as a chart archive. A chart contains the necessary information to install a set of Kubernetes resources on a Kubernetes cluster. A chart includes:

  • a chart.yaml file, describing the application,
  • a values.yaml file, providing default values for application parameters
  • templates of files used by the application
  • other dependencies

Helm also has a command line client, the Helm CLI (sometimes referred to as helm, all lowercase)

To install the OCI native ingress controller as a standalone program, the OCI native ingress controller is made available as a Helm chart. Before you can install the OCI native ingress controller chart, you have to first install the Helm CLI.

To install the Helm CLI, follow the instructions in the Helm installation documentation to download and extract the appropriate tar.gz or zip archive file.

Cloning the OCI Native Ingress Controller Repository and Setting Parameters in values.yaml

To clone the OCI native ingress controller and set parameters in values.yaml in readiness for installing the OCI native ingress controller as a standalone program:

  1. Clone the OCI native ingress controller repository from GitHub by entering:
    git clone https://github.com/oracle/oci-native-ingress-controller
  2. In the local Git repository, navigate to the oci-native-ingress-controller directory and open the values.yaml file in a text editor.
  3. In the values.yaml file, set the compartment_id parameter to the OCID of the compartment in which the OCI native ingress controller is to create the OCI load balancer and certificate. For example:
    compartment_id: "ocid1.compartment.oc1..aaaaaaaa______ddq"
  4. In the values.yaml file, set the subnet_id parameter to the OCID of the load balancer's subnet. For example:
    subnet_id: "ocid1.subnet.oc1.iad.aaaaaaaa______dba"
  5. In the values.yaml file, set the cluster_id parameter to the OCID of the cluster. For example:
    cluster_id: "ocid1.cluster.oc1.iad.aaaaaaaa______ska"
  6. In the values.yaml file, specify how the OCI native ingress controller is to access OCI services and resources as follows:
  7. (optional) If you want to run multiple instances of the OCI native ingress controller for availability reasons, change the value of replicaCount in the values.yaml file. For example:
    replicaCount: 3
    By default, replicaCount is set to 1, meaning one instance of the OCI native ingress controller runs as a single pod. To ensure availability, you might want to increase replicaCount (typically to 2 or 3) to run multiple OCI native ingress controller instances as different pods. One pod is nominated as the leader and acts as the OCI native ingress controller, while the remaining pods wait in a passive state. If the leader subsequently becomes unavailable, one of the other pods is nominated as the leader and acts as the OCI native ingress controller.
  8. Save the changes you have made in the values.yaml file and close the file.
  9. If you want the OCI native ingress controller to access OCI services and resources using a workload identity principal, you have to set additional environment variables as follows:
    1. In the local Git repository, navigate to the oci-native-ingress-controller/templates directory and open the deployment.yaml file in a text editor.
    2. In the deployment.yaml file, set the following environment variables as shown: 
        env:
    - name: OCI_RESOURCE_PRINCIPAL_VERSION
      value: "2.2"
    - name: OCI_RESOURCE_PRINCIPAL_REGION
      value: "us-ashburn-1"
    3. Save the changes you have made in the deployment.yaml file and close the file.