Many common use cases for Oracle Cloud Infrastructure (OCI) File Storage include the transfer of a large amount of data. Based on the origination, destination, and the direction of the data transfer, the best method to accomplish that transfer can vary.
The following table provides recommendations for common File Storage data transfer scenarios, including the migration of on-premises data, copying File Storage data from one region to another, copying File Storage data within a region, and copying File Storage data to Object Storage.
For general information about private connections between OCI and on-premises data, see FastConnect and Site-to-Site VPN.
File Storage Data Transfer Scenarios
Transfer Data From...
To...
Recommended Method
Prerequisites and Considerations
On-premises storage
File Storage on OCI
Linux users can use instance-to-instance streaming and the fpsync tool.
Windows users can mount the Windows on-premises file share as a Common Internet File System (CIFS) share on an OCI Oracle Linux instance and use a copy tool such as fss-parallel-tools or fpsync to transfer data.
Instance should be able to connect to the Object Storage bucket.
Transferring On-Premises Data to File Storage
The following scenarios have been proven effective for Linux and Windows users when transferring large amounts of data from on-premises storage to OCI
File Storage. They might not be applicable to all environments or meet all requirements.
For Linux Users 🔗
Use the fpsync tool to perform an initial copy of on-premises data to OCI
File Storage. Then, incremental data changes can be synchronized using rsync because fpsync can't delete files and folders in the destination that don't exist in the source.
The fpsync tool is a parallel wrapper of rsync. Linux users can download fpsync from a yum repository. The commands differ depending on the version of Linux.
Download from the repository.
Linux 8 users can download the tool using the following command:
Before beginning the data transfer, complete the following prerequisites:
Ensure that network connectivity is established between the on-premises data source and OCI. Use FastConnect or Site-to-Site VPN connection to enable fast instance-to-instance streaming over SSH.
Don't use Windows copy utilities for large data copies to File Storage. Instead, use an OCI Linux instance to perform large copies using the parallel tools that come with an OCI Linux instance. Mount the Windows share as a Common Internet File System (CIFS) share and use a copy tool such as parcp or fpsync to transfer data.
Because the SMB protocol and the NFS protocol used by File Storage aren't compatible, an instance that can mount both the NFS file system and the SMB share is used to create a bridge between them.
Identify or create a Linux instance in Oracle Cloud Infrastructure that has network access to both the File Storage file system and the Windows SMB share.
Open a terminal on the instance.
Type the following to install the Common Internet File System (CIFS) utility and verify its installation:
sudo mount -t cifs -o username=<win_share_user> //<win_IP_address/hostname>/<share_name> /mnt/<win_share_dir>
For example:
$sudo mount -t cifs -o username=opc //win2012/c /mnt/win_share
Copy the files from the CIFS share to the mounted file system using the parcp utility from the File Storage Parallel Tools suite or fpsync. For installation information about parcp, see Using File Storage Parallel Tools. For information about fpsync, visit the fpsync man page.
Using Instance-to-Instance Streaming to Transfer File Storage Data 🔗
The fpsync tool is a parallel wrapper of rsync. You can use fpsync and instance-to-instance streaming to transfer data between mounted File Storage file systems.
To install fpsync, enable the Oracle Linux developer repository, which includes the fpsync utility, on the OCI instance using a command such as the following. The command differs based on the version of Oracle Linux in use:
An example showing the performance difference between the two approaches follows:
# date; time fpsync -o "-e ssh --progress --log-file ~/speedtest.log" /fss_src_bom/test root@DR_hyd:/fss_dest_hyd/ ; date
Sun Mar 13 15:22:58 GMT 2022
real 0m1.467s
user 0m0.111s
sys 0m0.075s
Sun Mar 13 15:23:00 GMT 2022
# ls -ltrd test
drwxr-xr-x. 2 root root 1 Mar 13 15:22 test
# du -sh test
1001M test
# cp -r test test1
# date; time fpsync -o "--progress --log-file ~/speedtest1.log" /fss_src_bom/test1 /fss_dest_hyd/ ; date
Sun Mar 13 15:25:16 GMT 2022
real 1m28.847s
user 0m3.688s
sys 0m1.439s
Sun Mar 13 15:26:44 GMT 202