Promoting Linux Requires Advertising. It Matters to Me. TM

GnuCash!target="_top">

What is RAID?

So what exactly is RAID? Nope, it's not the bug spray I'm talking about here. It is a technique that was developed to provide speed, reliability, and increased storage capacity using multiple disks, rather than single disk solutions. RAID basically takes multiple hard drives and allows them to be used as one large hard drive with benefits depending on the scheme or level of RAID being used. Depending on your needs, there are many different RAID variations and implementations available with prices ranging from less than $100 to over $25,000. Of course, the better the RAID implementation, the more expensive it's probably going to be. There is really no one best RAID implementation. Some implementations are better than others depending on the actual application.

It used to be that RAID was only available in expensive server systems. However, with the advent of inexpensive RAID controllers, it seems it has pretty much reached the mainstream market. Performance nuts, such as myself, are always looking for the latest technology to give us that edge we need. Of course, the mainstream implementation has it's limitations, and RAID isn't really for everyone. There are many levels of RAID being used today. Before I go into the different levels, I'd like to discuss the basic concepts behind these levels in a little more detail.

RAID and Data Protection Solutions for Linux When a system administrator is first asked to provide a reliable, redundant means of protecting critical data on a server, RAID is usually the first term that comes to mind. In fact, RAID is just one part of an overall data availability architecture. RAID, and some of the complimentary storage technologies, are reviewed below.

RAID, short for Redundant Array of Inexpensive Disks, is a method whereby information is spread across several disks, using techniques such as disk striping (RAID Level 0) and disk mirroring (RAID level 1) to achieve redundancy, lower latency and/or higher bandwidth for reading and/or writing, and recoverability from hard-disk crashes. Over six different types of RAID configurations have been defined. A brief introduction can be found in Mike Neuffer's What Is RAID? page.

If you are a sysadmin contemplating the use of RAID, I strongly encourage you to use EVMS instead. Its a more flexible tool that uses RAID under-the-covers, and provides a better and more comprehensive storage solution than stand-alone RAID.

Related References

EVMS HOWTO

Software RAID HOWTO (for the current linux-2.2 and 2.4 kernels)

Software-RAID-0.4x HOWTO (for the older version-0.4x software raid for linux-2.0 and older linux-2.2 kernels)

Software RAID Installation Overview (for the older version-0.4x software raid)

Logical Volume Manager for Linux

LVM HOWTO

The sg3_utils page provides an excellent cross-reference many disk drive diagnostic and monitoring tools, including links to smartmontools, scsirastools and scu. -->

Linux High Performance SCSI & RAID

Linux RAID FAQ answers some simple questions frequently seen on RAID mailing lists.

Linux-iSCSI Project

DPT Hardware RAID HOWTO

Root RAID HOWTO cookbook

Bootable-RAID on SuSE 7.3 Step by Step

Software-RAID Boot/Root HOWTO



Linux High-Availability HOWTO

-->

Multiple Disk System Tuning (or see this newer version)

SCSI HOWTO

SCSI Programming HOWTO

Disk I/O benchmarking Software

RAID und Linux, Martin Schulze, December 1996 (German) (old/out of date)

RAID Reviews contains some product reviews and performance benchmarks, circa 1998, that were originally a part of this web page. Obsolete/unmaintained.

Types of Data LossMany users come to RAID with the expectation that using it will prevent data loss. This is expecting too much: RAID can help avoid data loss, but it can't prevent it. To understand why, and to be able to plan a better data protection strategy, it is useful to understand the different types of failures, and the way they can cause data loss.

Accidental or Intentional Erasure

One of the leading causes of data loss is the accidental or intentional erasure of files by you or another (human) user. This includes files that were erased by hackers who broke into your system, files that were erased by disgruntled employees, and files erased by you, thinking that they weren't needed any more, or due to a sense of discovery, to find out what old-timers mean when they say they fixed it for good by using the wizardly command su - root; cd /; rm -r *. RAID will not help you recover data lost in this way; to mitigate these kinds of losses, you need to perform regular backups (to archive media that aren't easily lost in a fire, stolen, or accidentally erased).

Total Disk Drive Failure

One possible disk drive failure mode is "complete and total disk failure". This can happen when a computer is dropped or kicked, although it can also happen due to old age (of the drive). Typically, the read head crashes into the disk platter, thereby trashing the head, and keeping any/everything on that platter from being readable. If the disk drive has only one platter, this means everything. Failure of the drive electronics (due to e.g. electrostatic discharge) can result in the same symptoms. This is the pre-eminent failure mode that RAID protects against. By splattering data in a redundant way across many disks, the total failure of any one disk will not cause any actual data loss. A far more common disk failure mode, however, is a slow accumulation of bad blocks: disk sectors which have become bad/unreadable. RAID does not protect against data corruption. This case is discussed in detail below.

Power Loss and Ensuing Data Corruption

Many beginners think that they can test RAID by starting a disk-access intensive job, and then unplugging the power while it is running. This is usually guaranteed to cause some kind of data corruption, and RAID does nothing to prevent it or to recover the resulting lost data. This kind of data corruption/loss can be avoided by using a journaling file system, and/or a journaling database server (to avoid data loss in a running SQL server when the system goes down). In discussions of journaling, there are typically two types of protection that can be offered: journaled meta-data, and journaled (user's) data. The term "meta-data" refers to the file name, the file owner, creation date, permissions, etc., whereas "data" is that actual contents of the file. By journaling the meta-data, a journaling file system can guarantee fast system boot times, by avoiding long integrity checks during boot. However, journaling the meta-data does not prevent the contents of the file from getting scrambled. Note that most journaling file systems journal only the meta-data, and not the data. (Ext3fs can be made to journal data, but at a tremendous performance loss). Note that databases have their own unique ways of guaranteeing data integrity in the face of power loss or system crash.

Bad Blocks on Disk Drive

The most common form of disk drive failure is a slow but steady loss of 'blocks' on the disk drive. Blocks can go bad in a number of ways: microscopic dust sticking to the platter, gouges in the platter when the head struck it, magnetic media applied too thinly at the factory, or worn off due to contact, etc. Over time, bad blocks can accumulate, and, from personal experience, as fast as one a day. Once a block is bad, data cannot be read from it. Bad blocks are not uncommon: all brand new disk drives leave the factory with hundreds (if not thousands) of bad blocks on them. The hard drive electronics can detect a bad block, and automatically reassign in its place a new, good block from elsewhere on the disk. All subsequent accesses to that block by the operating system are automatically and transparently handled by the disk drive. This feature is both good, and bad. As blocks slowly fail on the drive, they are automatically handled until one day the bad-block lookup table on the hard drive is full. At this point, bad blocks become painfully visible to the operating system: Linux grinds to a near halt, while spewing dma_intr: status=0x51 { DriveReady SeekComplete UnrecoverableError } messages.

Despite this being the most common disk failure mode, there are painfully few solutions and precious little that one can do. RAID, even in theory, does not address this problem, nor does file system journaling. At this point, I am aware of only two options: (1) run badblocks or (2) use EVMS. The first option, in the form of 'e2fsck -f -cc' is terrible: it can only be run on an unmounted file system that was built on a raw disk partition, and its painfully slow. A 5 or 10 gig partition can up to an hour, and a 160 gig partition can take a day. Furthermore, it works only on a raw disk partition: if the file system sits on top of a RAID md device, or an LVM logical volume, the exercise is pointless. I have not yet personally tried EVMS ...

(smart == Self-Monitoring Analysis and Reporting Technology System) ide-smart and smartsuite can help understand if this failure mode is about to bite you.

It is likely that md (Linux Software RAID) will gain bad-block replacement capabilities in the 2.5.x kernel series. See this (nasty) discussioni on LKML.

General System Corruption

Windows users are familiar with the vague and uneasy regression of one's system into total chaos, eventually necessitating a clean-slate reinstall of the operating system. Due to bugs in the operating system, the database server, and in other applications, there is a slow buildup of corrupted data until the system finally becomes unusable. There is little that one can do about this, other than to stay away from Windows (Win95/98 in particular), and avoid putting mission-critical services on beta software. Unfortunately, even regular data backups do little to avoid this kind of corruption: most likely, one is backing up corrupted data. The good news is that this is an uncommon phenomenon under Linux; I can't name any examples of this kind of corruption. Which is not to say that it doesn't occur: although unseen when handling ordinary files under linux/ext2fs, it may show up in some database products, or systems that do a lot of document mangling (e.g. due to an obscure bug in a word processor). While Linux won't crash if the word processor has a bug in it, this kind of a bug can lead to irretrievable data loss, which can be almost as bad. Other than file archiving, I know of no strategies for dealing with this kind of data loss.

Note that this kind of corruption can also occur due to bad hardware, cabling, or even an electrically noisy environment. A loose cable may slowly corrupt data, although it will usually show itself in other ways, which the device driver will interpret as broken hardware.

Linux RAID SolutionsThere are three types of RAID solution options available to Linux users: software RAID, outboard DASD boxes, and RAID disk controllers.

Software RAID

Pure software RAID implements the various RAID levels in the kernel disk (block device) code. Pure-software RAID offers the cheapest possible solution: not only are expensive disk controller cards or hot-swap chassis not required, but software RAID works with cheaper IDE disks as well as SCSI disks. With today's fast CPU's, software RAID performance can hold its own against hardware RAID in all but the most heavily loaded and largest systems. The current Linux Software RAID is becoming increasingly fast, feature-rich and reliable, making many of the lower-end hardware solutions uninteresting. Expensive, high-end hardware may still offer advantages, but the nature of those advantages are not entirely clear.

Note that there are currently two Linux Software RAID implementations: the md (multi-disk) driver, which has been around since the early linux-2.0.x days, and the newer EVMS driver. The EVMS driver appears to be disk-format compatible with md. Features of the md driver include:

RAID-0 (striping), 1(mirroring), 4 and 5(parity) support.

Automatic Hot Reconstruction (if the array is inconsistent due to a power outage or a replaced disk, it will be rebuilt in the background while the system is running).

Hot Spare (a standby disk will get used if another disk fails).

Hot Swap (disks can be changed in a running array).

MD does not handle bad block relocation. Note that while Linux MD is tried-and-true, reliable, robust, and does what it promises, its development is essentially at a standstill. For this reason, it seems that EVMS, with its greater activity, and strong long-term vision, is

Posted in: