Oracle Cloud Infrastructure Documentation

Understanding Storage Gateway Performance

Learn about the performance characteristics of Storage Gateway and the ways you can maximize its efficiency.

Performance Characteristics

This section describes the basic performance characteristics of Storage Gateway:

  • Because a transactional overhead cost for each file exists, Storage Gateway generally exhibits better performance with large files than with small files. Storage Gateway can only upload data as fast as your connection and the storage host allows. Storage Gateway buffers the data in local disk storage while waiting to upload to Oracle Cloud Infrastructure. After a file is uploaded, the file's local copy can be removed from the cache to free up space. If file system cache space falls below 10 GB, application I/O and file/directory creation can fail.
  • Storage Gateway does not support frequently modified files such as logs, databases, or virtual disks. Storage Gateway depends on the closing of a file to trigger an upload of that file. If a file never closes or is modified frequently, the upload event cannot successfully occur.
  • Upload throughput to Object Storage (WAN) is typically slower than NFS client throughput (LAN). As a result, Storage Gateway can accumulate a large amount of data that is pending upload.
  • You can attach Storage Gateway to an existing Object Storage bucket that contains data. The service is optimized for this type of initialization. Storage Gateway can initialize about 700 thousand files per hour with a hard disk drive-based cache. It can initialize about 7 million files per hour with an NVMe SSD-based cache.
Note

Because Storage Gateway enables asynchronous movement of data to and from Object Storage, the performance level does not update in real time.

Factors That Affect Performance

To get the maximum performance benefits from Storage Gateway, follow the practices documented at Best Practices for Using Storage Gateway.

In addition to having sufficient memory and file system cache space, Oracle recommends that you use SSDs to help improve NFS ingestion rate.

Storage Gateway is tuned for maximum upload and download performance by default. No additional tuning is needed.

Performance Testing

While measuring performance is complex and open to variability, we have observed the following in performance benchmark tests with 10-Gb/s link speed:

Workload (Upload/Download) Configuration Average Upload Throughput Average Download Throughput
Single large file of 400 GB

CPU: 8 cores

Memory: 32GB

Disk Read: 340 MB/s

Disk Write: 234 MB/s

Link Speed: 10 Gb/s (1.25 GB/s)

 239 MB/s 195 MB/s
Multiple files of 10-50 GB (Total data = 400GB)

CPU: 8 cores

Memory: 32GB

Disk Read: 340 MB/s

Disk Write: 234 MB/s

Link Speed: 10 Gb/s (1.25 GB/s)

 260 MB/s 225 MB/s
Note

Using FastConnect with Storage Gateway makes optimal use of full link speed. We have customers using FastConnect with 10-Gb/s link speed and have seen each gateway achieve 400–450 MB/s uploads to Oracle Cloud Infrastructure.

See FastConnect for more information.

Testing Network Bandwidth

Storage Gateway provides a diagnostic command that you can use to test the bandwidth in your environment and ensure that you get the expected upload and download speeds. The amount of data transferred depends on these factors:

Bandwidth * Delay Product (bits) = total_available_bandwidth (bits/sec) x round_trip_time (sec)

Note

Different buckets can have different upload and download speeds.

The round_trip_time can vary by region.

To run the diagnostic command:

You need root permissions to run the diag command.

  1. Using SSH, log in to the host on which you installed Storage Gateway.

  2. Run the diag command, specifying the Storage Gateway file system name:

    [root@ocisg-ashburn opc]# sudo docker exec ocisg cat /mnt/gateway/file_system/:::diag:oci-network-speed-test

    The diag command responds with the average upload speed, for example:

    Average Upload Speed = 217 MB/s