File systems and hard drives have always been fascinating to me, ever since I invested way too much money in a large (at the time) 80MB drive for the computer I was using in the early 90s. There's just something neat about blasting big chunks of data back and forth across your bus, reading and writing thousands of files, and doing benchmarks.
Maybe you don't share my infatuation of hard drives and the software that keeps track of files and directories, but you're probably at least a bit interested in keeping your data safe, using your drives efficiently, and squeezing as much performance as possible from your system's meager I/O subsystem.
Yeah, I called it meager. Unless you've got an enormous budget for exotic hardware, your disk I/O subsystem has advanced very little compared to things like your CPU, RAM, and video card. Moore's famous law doesn't apply here, just micro magnetism and advanced manufacturing processes.
Of the commonly used operating systems, Linux® has the most support for different file systems. In this respect, Linux diverges from other UNIX® systems, which have traditionally supported their native file system and the ISO-9660 file system used on standard CD-ROMs. My Fedora Core 4 system has loadable kernel modules for about two dozen kinds of file systems. They exist primarily to provide compatibility. You can plug in a disk from nearly any other system and manipulate it from within Linux. But what if you're adding another disk to your Linux system and don't need to let it speak with Windows®, QNX, Mac OS X, or Minix?
You'll need to know at least a little about some of the common Linux file systems, such as ext2 (second extended file system), ext3 (third extended file system), and ReiserFS 4 (an interesting advanced file system with many exotic features that can improve your system's file handling).
If you're not lucky enough to be running a Linux distribution that comes with support for Reiser4 (such as Arch, Linspire, or SUSE), you're going to have to do something that might be painful: rebuild your kernel. Since rebuilding your kernel would be a tutorial of its own, you'll have to find a how-to document for your distribution that leads you through the steps required to rebuild the kernel.
Before you actually start compiling your brand new Linux kernel, you'll also have to visit the Reiser4 home page at Namesys (seeResources) and download the Reiser4 patches appropriate for your kernel. These patches come with instructions for applying them before you configure and compile the kernel.
So you'll be able to create and manipulate Reiser4 file systems, make sure the reiser4progs package is installed. Again, if your distribution doesn't have the resier4progs, see the Namesys site for downloads (see Resources).
If you want to experiment with Reiser4 easily, an excellent Gentoo Linux Live CD is available with Reiser4 support. See theResources section for a link.
Back in the early days of Linux (late 1993, in fact), only one file system was supported by the kernel, a port of the rather minimal Minix file system. This had several shortcomings, including 14-character filenames and a mere 64MB as the maximum file system size. It didn't even support the full set of UNIX file system attributes, specifically the full set of creation, modification, and access time stamps required for a conforming POSIX (Portable Operating System Interface) file system.
Because of the limitations of the Minix file system, work on a replacement file system began and resulted in the addition of a Virtual Filesystem (VFS) abstraction layer, which would make it easier to write file systems for Linux. Using the new VFS, the Minix file system was extended to allow longer filenames and larger file systems (up to two gigabytes). This became the extended file system (ext), but it still had several limitations.
The solution was the ext2, which is still used on many systems, and was the default Linux file system for quite a while. The ext2 file system was improved by the addition of journalling, which created the ext3 file system as its successor.
Before any of the other journaled file systems were usable under Linux, ReiserFS (also known as the Reiser3 file system) was the first advanced file system for Linux that had support for journalling and more efficient disk usage. Its successor, Reiser4, is a complete redesign and rewrite, focusing on extensibility, security, and performance, while maintaining your data safely and efficiently. Reiser4 isn't integrated with the Linux 2.6 kernel yet, which is usually a sign of possible instability or other reasons to tread carefully. Remember to always back up your important data, regardless of which file system you're using.
Let's take a quick look at the ext2, ext3, and Reiser4 file systems.
The ext2 file system, Linux's default file system, is a traditional UNIX file system (that is, based on Berkeley's Fast Filesystem, FFS). It has a maximum filename length of 255 characters and a theoretical maximum file system size of four terabytes. (Linux's block device driver limits this to "just" 2047 gigabytes; let me know when I can buy a single hard drive with that much storage space).
Because of its ubiquity, ext2 drivers have been created for Windows and Mac OS X, letting you read and write an ext2 file system directly from those operating systems, making it an excellent format for shared devices, such as portable hard drives.
The ext2 file system supports all of the standard UNIX features:
- Owner user ID and group ID.
- Mode bits controlling user, group, and other permissions and operating system flags.
- Storage for entry creation, modification, and access times (although most systems run with access time tracking turned off to improve performance at the cost of pure POSIX 1003.1 compatibility).
The major drawback for ext2 file systems is that hard drives have become much larger since it was originally designed. A system crash or power failure requires a file system check with
fsck, which takes ages on modern disks with lots of directories and files.The ext3 file system for Linux is the new default file system for most popular distributions. Compared to ext2, it adds:
- A metadata journal, to ensure that your file system structures are always in a valid state. This (mostly) removes the need for a lengthy
fscksession after a system crash or power failure. - Hashed tree directory indexes, to speed up access times for large directories.
- Online file system resizing and the ability to upgrade to ext3 from ext2 without reformatting the drive.
- Larger maximum file and file system sizes (two and 32 terabytes, respectively).
Although ext3 isn't as fast, or scalable, as its competitors (such as Reiser3 or SGI's excellent XFS), its compatibility with ext2 makes it attractive because of the large number of mature ext2 maintenance and administration utilities.
While the Reiser3 file system gained some popularity because of its speed and journalling support (it's the default file system for a number of Linux distributions, even today), its designer wasn't satisfied. The Reiser4 rewrite grew out of this and added some additional interesting features:
- Efficient journalling through wandering logs.
- Efficient support of small files, in terms of disk space and speed.
- Fast handling of very large directories with hundreds of millions of files (yes, millions of files in a single directory without affecting performance).
- Flexible plug-in infrastructure at the source level, which allows easy addition of compression and encryption at some point in the future.
- Atomic file system modification, ensuring that the on-disk file system is always in a valid state.
- Dynamically optimized disk layout through allocate-on-flush.
- Support for database-like file system transactions.
But wait, why did I call it the Batmobile option? Reiser4 supports lots of interesting features that you might never need to take advantage of and some that you can't even access, because Linux's VFS layer doesn't expose the functionality -- much like how the Batmobile has all kinds of features you're not going to need on your way to the office.
Before you can do anything interesting with the Reiser4 file system, you'll need to format a partition with it. As you can see fromFigure 1, I have a spare partition just waiting to be used for this purpose:
Figure 1. A partition waiting to be formatted
You need to create a new Reiser4 file system on this partition as well as get it mounted.
To create the file system, you'll log in as root (or use the
sudo command to masquerade as root) and use the mkfs.reiser4command:/sbin/mkfs.reiser4 -L myLabel /dev/hda1 |
This command creates a Reiser4 file system on the specified partition (/dev/hda1 in my case) with a label of "myLabel" and a random unique identifier, as seen in Figure 2.
Figure 2. Creating the Reiser4 file system
A fresh, clean Reiser4 file system awaits us! Let's mount it, so you can actually use it.
To mount the new file system, you'll log in as root (or use the
sudo command to masquerade as root) and use the mkdir andmount commands:mkdir /mnt/reiser4 mount /dev/hda1 /mnt/reiser4 |
Use
mkdir first to create a mount point, then use mount to mount the device containing the file system at the mount point you just created.You can use the
mount command without any arguments to get a list of the currently mounted file systems, which should now include your new Reiser4 file system (see Figure 3).Figure 3. Mounted file systems, including Reiser4
Now that you've got your new file system mounted, you should make sure the system can mount this file system automatically.
To let the system automatically mount your new Reiser4 volume, you need to describe it in the /etc/fstab file.
As root (or using
sudo to masquerade as root), edit /etc/fstab using your favorite text editor and add the text from the following code at the bottom:/dev/hda1 /mnt/reiser4 reiser4 defaults 0 0 |
In /etc/fstab, you'll want to use the appropriate device and mount point for your file system. After the device and mount point, you specify the file system type and file system options (defaults is usually best, unless you really know what you're doing and you can find good documentation for your file system). The last two options are for the "dump frequency" and "pass number on parallel fsck", which are needed for historical reasons.
You can test to see if you've edited /etc/fstab correctly by unmounting the file system and telling the
mount command to mount everything automatically:umount /mnt/resier4 mount -a |
Now when you use the
mount command without any arguments, it should look exactly the same as it did before (Figure 3). Your new Reiser4 file system will automatically be mounted at boot time, along with the rest of your file systems.Like most other Linux file systems, Reiser4 responds to some options that can improve its overall performance and modify its behavior. These options can be passed on the
mount command line using the -o option, as seen in the following code:mount -t reiser4 -o option1,option2,...,optionn /dev/hda1 /mnt/reiser4 |
Several file system options can be included in the command by separating them with commas.
So they're picked up at boot time, file system options can also be included in the /etc/fstab file:
/dev/hda1 /mnt/reiser4 reiser4 option1,option2,...,optionn 0 0 |
The most common options include:
defaults-- The default Linux file system settings. This is the same as specifyingrw,suid,dev,exec,auto,nouser,asyncdirectly. The file system will be mounted in read-write mode, set-UID bits will be honored, character or block special devices on the file system are treated normally, binaries can be executed, the file system can be auto-mounted, and all I/O is done asynchronously, respectively.noatime-- Don't update the access time field when files or directories are read. This isn't strictly POSIX behavior, but it can speed up file system operations significantly, especially on file systems with a large number of directories and files that are mostly read from instead of being written to.noexec-- Don't run binaries from this file system. It's treated as a data-only file system, which can be handy if you don't entirely trust the source of the files or executables found there.nosuid-- Ignore the set-user-ID and set-group-ID bits on files stored in this directory; again, a safety feature if you don't trust the source of the files.ro-- The file system is mounted read-only. Attempts to write to a file or create a new file or directory will fail.data=journal-- Instead of writing only file system metadata to the journal, all data is journaled before being written out to the file system. This ensures data integrity in the event of a disaster, but seriously reduces write performance.
Using the defaults is generally what you want, but adding
noatime to the mix can safely help speed things up. Thedata=journal option might be a good idea for an important CVS server or a file system used for making backups, because having valid data is much more important than raw throughput.Adding a file system to your Linux setup can be daunting, especially when you want to try out one of the many different file systems supported by Linux. Knowing about the features and design limitations of the popular file systems helps you to choose wisely.
Reiser4 should still be considered experimental (even though many people are using it without problems), because it hasn't been accepted into the Linux kernel yet. The folks at Namesys are working hard to make their code part of the kernel, so it's only a matter of time before distributions start appearing based on Reiser4.
After creating your new file system with the appropriate
mkfs command, mounting it with the mount command (and adding it to the /etc/fstab file) allows you to begin working with the file system. After that, remember to always back up your important data regularly instead of when a hard drive fails.Learn
- Visit the ReiserFS home at Namesys.
- Visit the Wikipedia site and read the following articles regarding:
- Read a huge article comparing dozens of different file systems.
- " In the beginning was the command line", by Neal Stephenson, is an interesting (if somewhat dated) article that also uses the Batmobile for an analogy.
- Visit the developerWorks Linux zone to expand your Linux skills.
- Stay current with developerWorks technical events and Webcasts.
Get products and technologies
- Get a Gentoo Linux Live CD with Reiser4 built-in support.
- Build your next development project with IBM trial software, available for download directly from developerWorks.
