mupuf.org // we are octopimupuf.org

Benchmarking Btrfs on Linux 2.6.37-rc3

Before reinstalling all the operating systems of my laptop, I was wondering if Btrfs would be a good fit for my unstable/development OS.

I wouldn’t have cared that much about the fs if my lappy’s hdd wasn’t slow as hell. I wondered if the built-in compression support of Btrfs could help me get a little more performance out of it. Also, as I tend to deal with loads and loads of ridiculously small files, the fact that Btrfs stores them in the extent of the directory inode should lower the seeking time.

Before reinstalling everything based on the assumption that Btrfs would actually be faster, I wanted to see some benchmarks on simple tasks like extracting, compiling a kernel and removing files. So far, I haven’t been able to find benchmarks for a recent kernel (read, at least linux 2.6.36) except the one from phoronix which doesn’t tell you how much CPU is being used and doesn’t test the compress mount option.

Benchmark Protocol

The same tests are done on two different systems, my desktop and my laptop. The file systems have been mounted using the default options unless stated otherwise.

The compress mount option of Btrfs compresses the data on the hdd using gzip. The less space it takes on the hdd, the faster you’ll read and write it. The question is will the CPU overhead kill the performance improvement? I have tested the worst case scenario (clearing all caches between every test except for the linux recompilation). Please be aware that the data written by bonnie++’ is highly compressible, so, we shouldn’t be able to reach the read/write speed in daily usage scenarios.

The space_cache mount option of Btrfs caches the available space in the extent of an inode. This should ease the kernel’s work when looking for space where to write data but increase the size of the inode header.

The kernel compilation test has not been performed on my laptop because it takes around 45 minutes to complete. Anyway, from my limited testing, btrfs+compression was still slower than the ext4.

Boonie++ is a IO tester. Please read the boonie++ official page to understand the meaning of the figures.

Pacman is the package manager of ArchLinux. I was wondering how Btrfs would affect pacman’s performance. Pacman -S base will install ArchLinux. Pacman -Ql lists all the files installed by the pacman, it is a way to test reading operations.

Benchmark

Disclaimer: Some of the test results are highly variable. Do not expect an accuracy greater than 30% except for the bonnie++ test where an accuracy of 10% seems reasonable.

Computer 1 - My desktop

Well, apart from the fact that I have 3GB of RAM, an i7 920 @ 2.67GHz (stock frequency) and a Samsung F2 EcoGreen HD502HI hdd, there is not much to say. Ask me if you need more info.

Drive speed (hdparm -Tt):

Timing cached reads:   11406 MB in  2.00 seconds = 5707.67 MB/sec
Timing buffered disk reads: 316 MB in  3.01 seconds = 104.86 MB/sec

Computer 2 - My laptop

My laptop is a ThinkPad R61i . The processor is a Core duo clocked at 1.87GHz (NFI why Lenovo doesn’t list this one on his website), 2GB of RAM and the hdd is an Hitachi, 120GB. Ask me if you need more info.

Drive speed (hdparm -Tt):

Timing cached reads:   1544 MB in  2.00 seconds = 772.06 MB/sec
Timing buffered disk reads: 166 MB in  3.02 seconds = 55.00 MB/sec

Analysis

Seriously, I have no idea on what should be the conclusion, at least, performance-wise.

First, some result have an incredible variance, even the long ones. I’ve tried to make an average but some tests just take too long to execute and my time is limited.

Anyway, it seems like ext4 is a nice choice if you don’t need the features added to Btrfs. Otherwise, I would say that the cost of Btrfs over ext4 isn’t tremendous and it shouldn’t matter in your day to day work.

I’m pretty sure ext4 is acting as if it was fast, but in fact, would return from the IOCTL before the job is fully done. That would explain why the hdd was still working even after the apparent end of an update. This makes it very hard to actually test anything.

The boonie++ figures should be right as I write 4 times more data than what my RAM can store. This basically trashes the possibility for the system to cache anything. Well, don’t take the compression figures too seriously, the author himself told what boonie++ writes is highly compressible. This should be considered as the best case scenario!

Also, these benchmarks didn’t take into account the interactivity or the performance under severe IO pressure (with multiple clients). It’ll need day to day testing in order to try to feel the difference.

The compress flag can give you a tremendous performance boost when dealing with heavy yet highly compressible files (such as human readable ones). Otherwise, it can help a bit when reading but at the price of a longer writing process. Don’t expect too much out of it.
I’ll try to benchmark the boot sequence and see if it helps.

The space_cache flag is wonderful, this makes btrfs really faster. I guess everyone should use it. Beware: This flag has been introduced in Linux 2.6.37. Do not use this flag (even once) if you want to be able to read data from an earlier kernel.

Using both the compress and the space_cache flags seem to slow the writes but makes reading faster than the simple usage of one or the other flag.

Important notes

  • I went on the #btrfs IRC channel, asking why deleting files was slow. The answer has been that is was not a critical but it can be optimised by returning after the IOCTL and doing the work in backstage. You’ve understood, this is low priority for the devs, and this is understandable as there are many other features missing right now that should be added first!
  • Also, “compressed content always uses extents of a maximum of 128kb. This probably means that in most real situations, there’s going to be more extents in a large compressed file than in the same file uncompressed (although the total size should still be significantly smaller).”. “Any write to a compressed file will end up touching 128kb on disk at a minimum (i.e., to recompress that extent), and any read may require at least that much)”.

Conclusion

According to these benchmarks, there are no clear winner between ext4 and Btrfs performance-wise. This could very well mean Btrfs’s goal is complete, that is to say, adding features without a noticeable performance drop.

To make-up my mind and support Btrfs’s development, I’ve decided to try it on my laptop (where ext4 have already proven badly with performance going down every day, probably because of fragmentation).

I’ll keep you updated with my experience.

EDIT: I tipped Phoronix about this feature and two days later they issued an article about it: Btrfs space cache.

PS: Hurray for Google docs, the embedded spreadshits rock!

Comments