Mounting and Accessing a Lustre File System

Build a Lustre client on one system and then install the client package on each instance that mounts a Lustre file system.

1. To use a Lustre client with Oracle Linux, the kernel must be changed to Red Hat Compatible Kernel (RHCK). Switching kernel versions is described in Changing the default kernel in Oracle Linux.
2. Ensure that the system is running the RHCK kernel after reboot.

3. Install required packages using the following commands:

   release=$(cat /etc/os-release | grep ^VERSION= |cut -f2 -d\" |cut -f1 -d.)

   sudo dnf config-manager --set-enabled ol${release}_codeready_builder

   sudo dnf config-manager --enable ol${release}_developer_EPEL

   sudo yum-config-manager --enable ol${release}_developer

   sudo yum install git libtool patch pkgconfig libnl3-devel.x86_64 libblkid-devel libuuid-devel rpm-build kernel-rpm-macros kernel-devel kernel-abi-whitelists libmount libmount-devel libyaml-devel

4. Clone the Lustre client source repo and checkout the client using the following commands:

   git clone git://git.whamcloud.com/fs/lustre-release.git lustre-client

   cd lustre-client/

   git checkout tags/2.15.5

5. Build the client and RPMs. All Lustre RPMs in the current directory are built using the following commands:

   sudo sh autogen.sh
   ./configure --enable-client
   sudo make
   sudo make rpms
   sudo make dkms-rpm

6. Inspect the RPMs:

   ls -l *.rpm

7. If you plan to install the Lustre client on the same instance you built it, run the following commands:

   sudo make install

   sudo depmod

   sudo modprobe lustre

Proceed to Installing a Lustre client for Oracle Linux 8.

These steps assume that you already built a Lustre client for Oracle Linux.

In this example, the Lustre Dynamic Kernel Module Support (DKMS) client package is used for its flexibility across different kernel versions. However, if you're using the DKMS client, Lustre modules must be built when it's installed or the kernel gets upgraded. You can choose the RPMs that you want to install, such as the non-DKMS kmod-lustre-client package for the Linux modules.

1. Create a directory on the instance where you want to install the Lustre client. For example:

   mkdir lustre-rpms

2. Copy the Lustre RPMs. In this example, we use the DKMS RPMs, such as:

   lustre-client-2.15.5-1.el8.x86_64.rpm
   lustre-client-dkms-2.15.5-1.el8.noarch.rpm

3. To use a Lustre client with Oracle Linux, the kernel must be changed to Red Hat Compatible Kernel (RHCK). Switching kernel versions is described in Changing the default kernel in Oracle Linux.
4. Ensure that the system is running the RHCK kernel after reboot.

5. Enable required Linux repository for dependencies, and install dependencies using the following commands:

   release=$(cat /etc/os-release | grep ^VERSION= |cut -f2 -d\" |cut -f1 -d.)

   sudo dnf config-manager --set-enabled ol${release}_codeready_builder

   sudo dnf config-manager --enable ol${release}_developer_EPEL

   sudo yum install dkms libmount-devel libyaml-devel

6. Install the client RPMs using the following command:

   sudo yum localinstall -y ./*.rpm

7. Test the modules using the following commands:

   sudo modprobe lustre

   sudo lsmod |grep lustre

8. The Lustre client is ready. Reboot the system.

Build a Lustre client on one system and then install the client package on each instance that mounts a Lustre file system.

1. Lustre 2.15.5 isn't compatible with Linux kernel 6. The client needs a kernel version of 5.15.x. Check the OS and kernel versions using the following commands:

   cat /etc/os-release

   uname -r

   In this example, the compatible kernel version used is 5.15.0-1040-oracle.

2. If a different kernel version is installed, install the required kernel version and kernel headers using the following commands, which use the example version of 5.15.0-1040-oracle:

   sudo apt-get install linux-image-5.15.0-1040-oracle

   sudo apt-get install linux-headers-5.15.0-1040-oracle

   sudo apt-get install linux-modules-5.15.0-1040-oracle

3. Update grub using the following command:

   sudo update-grub

4. Inspect the /boot/grub/grub.cfg to get the entry for the kernel version and update the entry GRUB_DEFAULT in the file /etc/default/grub as follows:

   GRUB_DEFAULT='Advanced options for Ubuntu>Ubuntu, with Linux 5.15.0-1040-oracle'

5. Update grub using the following command:

   sudo update-grub

6. Reboot the client using sudo reboot. After the reboot, ensure that the correct kernel version is running.

7. Clone the Lustre client source repo and checkout the client using the following commands:

   git clone git://git.whamcloud.com/fs/lustre-release.git lustre-client

   cd lustre-client/

   git checkout tags/2.15.5

8. Install common packages using the following command:

   sudo apt-get install -y libreadline-dev libpython3-dev libkrb5-dev libkeyutils-dev flex bison libmount-dev quilt swig libtool make git libnl-3-dev libnl-genl-3-dev libnl-3-dev pkg-config libhwloc-dev libnl-genl-3-dev libyaml-dev libtool libyaml-dev ed libreadline-dev dpatch libsnmp-dev mpi-default-dev libncurses5-dev libncurses-dev bison flex gnupg libelf-dev gcc libssl-dev bc wget bzip2 build-essential udev kmod cpio module-assistant module-assistant debhelper dpatch libsnmp-dev mpi-default-dev libssl-dev

9. Configure the client:

   cd lustre-client

   sudo sh autogen.sh

   ./configure --enable-client

10. Edit the Debian control files to remove dependencies with latest linux-image and linux-headers packages. Without removing these dependencies, building the packages will fail. Edit the control files such as:

    vi debian/control*

    Remove all the references in the two files, such as:

    linux-headers-generic | linux-headers-amd64 | linux-headers-arm64 , linux-image | linux-image-amd64 | linux-image-arm64, linux-headers-generic | linux-headers-amd64

11. Build the packages using the following commands:

    sudo make debs

    sudo make dkms-debs

12. Inspect the packages:

    ls -l debs/*.deb

13. If you plan to install the Lustre client on the same instance you built it, run the following commands:

    sudo make install

    sudo depmod

    sudo modprobe lustre

Proceed to Installing a Lustre client for Ubuntu 22.04.

These steps assume that you already built a Lustre client for Ubuntu.

1. Lustre 2.15.5 isn't compatible with Linux kernel 6. The client needs a kernel version of 5.15.x. Check the OS and kernel versions using the following commands:

   cat /etc/os-release

   uname -r

   In this example, the compatible kernel version used is 5.15.0-1040-oracle.

2. If a different kernel version is installed, install the required kernel version and kernel headers using the following commands, which use the example version of 5.15.0-1040-oracle:

   sudo apt-get install linux-image-5.15.0-1040-oracle

   sudo apt-get install linux-headers-5.15.0-1040-oracle

   sudo apt-get install linux-modules-5.15.0-1040-oracle

3. Update grub using the following command:

   sudo update-grub

4. Update the entry GRUB_DEFAULT in the file /etc/default/grub as follows:

   GRUB_DEFAULT='Advanced options for Ubuntu>Ubuntu, with Linux 5.15.0-1040-oracle'

5. Update grub using the following command:

   sudo update-grub

6. Reboot the client using sudo reboot. After the reboot, ensure that the correct kernel version is running.

7. Copy the package files that you built to the instance where you want to install the Lustre client. In the following example debs directory, the DKMS kernel module package was selected for its flexibility, but you may choose non-DKMS module RPMs if you prefer:

   rw-r--r-- 1 ubuntu ubuntu 17058172 Apr  2 02:26 lustre-client-modules-dkms_2.15.5-1_amd64.deb
   -rw-r--r-- 1 ubuntu ubuntu   678790 Apr  2 02:26 lustre-client-utils_2.15.5-1_amd64.deb

8. Install the packages using the following commands:

   sudo apt-get update

   sudo apt install --fix-broken ./*.deb

9. Test the modules using the following commands:

   sudo modprobe lustre

   sudo lsmod |grep lustre

10. The Lustre client is ready. Reboot the system.