12 minute readThe Best Metric For Measuring Storage Performance
Discover the best method for measuring your performance.
Don’t just rely on a single metric – diversify!
When measuring the performance of flash storage, the most commonly used performance benchmarks are throughput and IOPS. Despite both of these being popular methods of measuring performance, latency may be an even more important metric to consider.
Reducing latency is a major concern for users, and manufacturers have responded by moving flash storage into servers. This storage has been moved onto faster buses like PCIe, and even onto memory buses.
So, what exactly is the difference between IOPS and latency in terms of performance benchmarking? To answer this, you must first understand why latency is a better performance metric for your storage environment.
Throughput, Bandwidth, and IOPS
Throughput describes the amount of data able to flow through a point in the data path over a given time. For data that need to be streamed rapidly, such as images or video files, throughput is the metric most often used.
“Bandwidth” is a similar term, meaning the ability to support a given level of throughput. Throughput and bandwidth are typically measured in “bits per second” when using networking terminology, and “bytes per second” when using storage terminology.
IOPS, on the other hand, (which stands for Input/Output operations per second) measures the number of storage transactions processed through a system each second. IOPS are used to describe performance in cases involving smaller data objects, like web traffic logs.
Latency, the Uber Metric
Latency is the total time that it takes for a specific operation to be completed. It is the time between when the request is issued, and when the requestor receives a response. Latency includes all time that is spent on both the input and the output. It is typically used because it is a good indicator of how congested a system is.
Disk drives associate read latency as the time required for the controller to first find the correct block and place the head over the data block. It also includes the time required to spin up the disk platters to begin the process.
In regards to flash, read latency includes the time to navigate through the various network connectivity such as fibre, iSCSI, PCIe bus, and memory bus. Once that navigation path is completed, latency also includes the time needed to find the required data blocks and to prepare to transfer data.
For write operations on a flash device in a steady-state, latency can also include the time the controller takes for overhead activities such as block erase and copy in preparation for new data. For these reasons, write latency is usually greater than read latency in flash systems.
How to Reduce Latency
Latency can be a huge problem when you are trying to get work done, or your business needs to function. There are a few steps you can take if you are looking to reduce latency.
What is the difference between latency and seek time?
Often latency can be confused with seek time, but they are actually completely different things.
Seek time is the specific amount of time that a hard drive spends to read or write and find the physical location of a piece of data on the disk.
In contrast, latency is the average time for the sector to actually move into position under a head, after a completed seek.
Rather than Latency being a specific measure of performance, IOPS is a neutral measure of performance. IOPS is good to use in situations where you need to compare two systems.
By combining these two measurements with throughput, it is possible to get a more complete picture of the performance of your system.
How to Measure IOPS
Both disk performance and IOPS are based on three main factors:
Every disk in your storage system has a maximum theoretical IOPS value. This is based on the formula: Divide 1 by the sum of the average latency in ms and the average seek time in ms (1 / (average latency in ms + average seek time in ms).
IOPS and Latency
Since IOPS measures the aggregate of subsystem transactions in a given second, latency can have a direct effect on IOPS. But focusing solely on IOPS as a performance metric can be misleading. While IOPS describes the number of transactions a system can sustain per second, it does not measure the amount of data each transaction delivers, which can vary widely with block size.
To maintain high IOPS performance, storage systems need enough pending data requests so that the latencies of a few individual requests don’t adversely impact overall IOPS. With low queue depths, latency becomes a bigger factor. And with storage and flash systems, low queue depth is a common occurrence.
Having adequate transactions in the queue is necessary for top SSD performance, but large queue depths can actually hide latency in the short run. Essentially, the system can process a large number of transactions per second from the queue (high IOPS) while taking a long time to complete those transactions (high latency). So IOPS may look fine in the short-term, but the system’s ability to improve long-term performance may actually be poor.
What are RAID levels?
In regards to computer storage, RAID levels are created by an array of RAID (redundant array of independent disks). These configurations use striping, mirroring, or parity in order to create data stores from many different computer hard disk drives (HDDs). RAID aids in data loss prevention and recovery.
There are six types of RAID, with 0, 1, 5, and 6 being the most common. These six types of RAID include:
RAID levels are standardized by the Storage Networking Industry Association (SNIA) in the Common RAID Disk Drive Format (DDF) standard.
The numerical values attached to RAID types only exist as a name but do not actually tell you anything regarding the generation, performance, or other metrics.
RAID levels can also be combined to create nested RAID levels. These levels offer even more protection against hardware defects, and hard errors. Despite this, RAID offers no protection
against data loss in the event of water or fire.
Increasing storage performance
Hard drives are put into RAID arrays to get more IOPS than one single drive can provide. This makes sense, as it is similar to having one horse vs. an entire cart of horses. Eventually, as your workload increases, you will hit the maximum number of IOPS the RAID array can handle with an acceptable latency.
Many SSDs are advertised with a performance of 80,000-100,000 IOPS at some decent latency. Depending on your workload, the threshold between IOPS and latency can actually be reached even if only at 50% storage capacity in your RAID array. This is why, instead of increasing your storage, you can increase its performance.
The best plan is to buy 10,000-15,000 RPM drives specifically for the IOPS rather than capacity. Even if you need to order more hard drives to meet your latency requirements, you will still have enough space left.
Now that hard drives are pushed in the role of online archival storage. SSDs are taking over the role of performance storage and can be layered for larger capacity.
Your Results for Measuring Storage vs. Vendor Benchmark Numbers
You might be wondering why your results for measuring storage do not match vendor benchmark numbers. This is because results do not always match real-world performance levels.
Although you can look at a manufacturer’s claimed throughput as a reference, you really can only look at it as a maximum.
Essentially, this measured performance number is due to the size of the I/O under measurement. The manufacturer will always be advertising in order to display their product at its maximum offerings. This is why the actual daily expectation should always be quite lower than advertised.
The Best Metric
Storage device performance can be difficult to compare accurately since measurements vary between environments and applications. In some cases, having high throughput makes for good performance. In others, high IOPS, or supporting a high number of short processes, is preferable.
Muddying the waters even further, factors such as queue depth, block size, and degree of parallelism can increase an environment’s throughput and IOPS, without benefitting overall application performance.
This is why latency is the most dependable performance metric. As a primary factor in both IOPS and throughput calculations, latency has a fundamental impact on storage performance.
Since reducing latency will always improve performance, it should always be one of the first things to consider when measuring storage performance.
Let us help you assess the performance of your environment
Talk to one of our experts today!