On-Premises Transfers to File Storage Are Slow

When a File Storage file system is directly mounted on an on-premises server, transferring files from the on-premises server to OCI File Storage is slow.

Cause: Directly mounting a File Storage file system on an on-premises instance is very slow because the NFS protocol is chatty over the internet.

Solution: Use instance-to-instance streaming for on-premises to OCI File Storage transfers using parallel rsync tools such as fpsync. For example:

1. If needed, create an instance in the same subnet as the existing File Storage mount target. Then set security rules and export options so that you can mount the file system from the instance.
2. Add the on-premises root user's SSH public key into the OCI instance's ~/.ssh/authorized_keys file. For more information, see Managing Key Pairs on Linux Instances.

3. Enable the Oracle Linux developer repository, which includes the fpsync utility, on the on-premises instance using a command such as the following:

   yum --enablerepo ol7_developer_EPEL install -y fpart

   yum --enablerepo ol8_developer_EPEL install -y fpart

   Note
   
   The command differs based on the version of Oracle Linux in use.

4. As the root user of the on-premises instance, run the transfer to OCI File Storage using the new OCI instance:

   #oci_instance=<oci_instance_privIP>
   #src=<source_directory>
   #dest=<FSS_mount_point_in_the_instance>
   #fpsync -v -n `nproc` -o "-lpgtoDxXH --progress --exclude .snapshot --log-file /tmp/fpsync_ssh_instance.log" ${src} root@${oci_instance}:${dest}

   Consider the following when using the fpsync command:

   • Ensure that the ${src} and ${dest} have a trailing / to do a content-only copy of the source.
   • When using a File Storage mount target as the source, exclude the .snapshot directory, otherwise the copy time increases significantly.
   • Because the current fpsync release doesn't accept the rsync option -a, the command uses -lpgtoD.

For more information and options, see the fpsync man page.