System Design

Determining the performance of a disk system is an often overlooked aspect of designing systems. Because the disk system is the slowest medium in a computer, it should be one of the FIRST components whose specification is properly defined. Without doing so, one risks implementing a poorly performing system that does not meet users' expectations.

All production-class systems should have their requirements defined as best as possible so that the proper disk system can be implemented. Further, after the implementation of a system, knowing the performance requirements, the system administrator can use that information as a baseline to monitor the performance of the disk system. So, sit down with the developers, the project stakeholders, and the folks on the front line of the business and understand what they want to achieve.

IOPS

IOPS are used to define the performance of a given disk or disk array. For the purposes of general calculations (and this post), the following can be assumed to be the maximum IOPS (performance) for a given disk:

• 10K RPM Fibre Channel Disk: 130 IOPS
• 15K RPM Fibre Channel Disk: 180 IOPS

NOTE: To calculate the actual IOPS for a given disk, the following information is required:

• Average latency
• Average seek time

This information is available from the drive's manufacturer.

Calculate a Single Drive's Maximum IOPS

Assume that we have a Seagate ST3146807FCV Cheetah 146GB 10K RPM Fibre Channel hard disk. It is rated as follows:

• Average latency (avgLatency): 2.99 ms or .00299 seconds
• Average seek time (avgSeek): 4.7 ms or .0047 seconds

To calculate this disk's IOPS, use the following equation:

IOPS = 1/(avgLatency + avgSeek)

For our example disk, the equation would be (note the values of avgLatency and avgSeek are measured in seconds):

IOPS = 1/(.00299 + .0047)
IOPS = 130

Total maximum IOPS for this disk is 130.

Calculate a Disk Array's Maximum IOPS

Per the note about system design above, calculating the performance of a disk system is critical to the operation of a given system. Most systems use RAID to provide storage redundancy. This section defines how IOPS are calculated for RAID arrays.

Maximum Read IOPS

Calculating maximum read IOPS (maxReadIops) for a RAID array is simple: multiply the number of disks (numDisks) in the array by the maximum IOPS per disk (diskMaxIops):

maxReadIops = numDisks * diskMaxIops

For an array of 5 10K RPM disks, the maximum read IOPS (maxReadIops) would be calculated as follows:

maxReadIops = 5 * 130
maxReadIops = 650

Total maximum read IOPS for the above array is 650.

Maximum Write IOPS

Calculating maximum write IOPS (maxWriteIops) is a slightly different matter, with respect to RAID arrays. RAID arrays suffer a write penalty; the type of RAID array determines the severity of the write penalty. This penalty is a result of the redundancy that RAID provides. The array necessarily has to write the data to multiple disks/locations to ensure data integrity.

RAID Write Penalty

Each RAID type suffers a different write penalty. The most common RAID types and their write penalties are defined in the following table:

NOTE: RAID 5 has the worst write penalty: 4x.

To calculate the maximum write IOPS (maxWriteIops) for a given RAID array, divide the maximum read IOPS (maxReadIops) by the RAID array's write penalty (raidWritePenalty):

maxWriteIops = maxReadIops / raidWritePenalty

Using the the example above, a RAID 5 array capable of 650 maximum read IOPS (maxReadIops), the maximum write IOPS (maxWriteIOPS) is calculated as follows:

maxWriteIops = 650 / 4
maxWriteIops = 162

Total maximum write IOPS for the above array is 162.

Designing for Performance

Simply calculating the maximum read and write IOPS for an existing or future RAID array is not enough. To ensure consistent and sustained performance the systems architect needs to profile the performance requirements for a system to determine the best disk solution. The minimum required IOPS must be determined so that the proper number, and speed, of disks can be purchased.

To start, one must have known performance requirements (i.e. read and write IOPS) for a given system or application. This information can be obtained from vendor documentation or system monitoring software such as Nagios.

Calculating the Minimum Required IOPS

Let's assume we have an application that requires the following:

• 400 Read IOPS
• 200 Write IOPS

The disk array will be created using RAID 5.

To calculate the minimum number of IOPS (minReqdIops), add the number of required read IOPS (reqdReadIops) to the product of the number of required write IOPS (reqdWriteIops) and the RAID type's write penalty (raidWritePenalty):

minReqdIops = reqdReadIops + (reqdWriteIops * raidWritePenalty)

In our example, the minimum IOPS would be calculated as follows:

minReqdIops = 400 + (200 * 4)
minReqdIops = 1200

The minimum number of IOPS required to provide the level of performance for our example application is 1200.

NOTE: This calculation determines the minimum number of IOPS required to meet the performance specification. This means the disk array should NOT perform below this performance level.

Calculate the Minimum Number of Disks

Once the minimum number of required IOPS is determined, it's very easy to determine the minimum number and speed of disks required to create a RAID array to meet performance requirements.

Minimum Number of Disks by Disk Speed

The minimum number of disks needed to meet our performance requirement (minNumDiskMinPerf) is calculated as follows:

minNumDisksMinPerf = minReqdIops / maxIopsByDiskSpeed

Using the information from calculating the minimum required IOPS above, and assuming we want to create an array of 10K RPM disks (130 IOPS per disk), the calculation for the minimum number of disks that will meet our minimum performance requirement (minNumDisksMinPerf) of 1200 IOPS (minReqdIops) is as follows:

minNumDisksMinPerf = 1200 / 130
minNumDisksMinPerf = 10

The minimum number of 10K RPM disks needed to meet our performance requirement is 10.

Optionally, if we want to create an array of 15K RPM disks (180 IOPS per disk), the minimum number of disks needed to meet our performance requirement would be calculated as follows:

minNumDisksMinPerf = 1200 / 180
minNumDisksMinPerf = 7

The minimum number of 15K RPM disks needed to meet our performance requirement is 7.

Minimum Number of Disks by RAID Type

The RAID type determines the minimum number of disks to satisfy the RAID type's requirements. For example, RAID 5 always requires at least 3 disks, at a minimum; RAID 10 always requires at least 4 disks, at a minimum.

For any arrays requiring a large number of disks, use the multiplier in the table below to determine the proper number of disks to meet the requirements of the RAID type:

After calculating the number of disks required by disk speed, determine the minimum number of disks required by RAID type.

In the example where 10K RPM disks were selected to build an array, the calculation shows at least 10 drives are required. If the RAID type will be 5, 10 disks will suffice. If, however, the RAID type will be 10, the minimum number of disks required by that RAID type will be 8 (not simply 7) since the multiplier for RAID 10 is 4.