ZFS-FUSE, XFS, NTFS3G and LESSFS
A PERFORMANCE COMPARISON
Introduction
Once in a while I run a set of tests to compare Lessfs with other filesystems. The results of these tests are used to determine where lessfs can be improved and how it does in general. The upcoming 1.2.2.2 release of lessfs again improves some performance aspects which made it interesting to perform a set of performance tests. This article describes the tests and shows the results.
Iozone
As usual I started with iozone. In the past lessfs would do great on writes but would be weak on reads. The latest code completely changes this as can be observed from these iozone results.
 read |
Sequential throughput test:
This is a simple test that shows how fast the filesystem can write a data stream to disk.
Test : time (dd if=/dev/sda1 of=/fs bs=1M; sync)
ntfs3g : 999292928 bytes (999 MB) copied, 33.803 s, 29.6 MB/s real 0m39.394s lessfs : 999292928 bytes (999 MB) copied, 9.2262 s, 108 MB/s real 0m17.347s xfs : 999292928 bytes (999 MB) copied, 2.55242 s, 392 MB/s real 0m12.722s zfs-fuse : 999292928 bytes (999 MB) copied, 40.9082 s, 24.4 MB/s real 0m41.451s
Extract kernel archive
Test : time tar xjf /Downloads/linux-2.6.34.tar.bz
ntfs3g : 0m32.314s lessfs : 2m51.920s xfs: 2m24.540 zfs-fuse: 1m29.172s
Test : cp -ar /etc /fs
ntfs3g: 0m2.338s lessfs: 0m8.347s xfs: 0m4.528s zfs-fuse: 0m2.088s
Conclusion
What does all this tell us? Lessfs has originally been designed for backup purposes and has therefore been tuned to store a small number of fairly large files as fast as possible. This still is a point in which lessfs excels. Although the creation and handling of many small files can still be improved the last releases come with acceptable performance, even on this point. Read and write performance are excellent. The next release will focus on further improvement of file creation / metadata performance. But all things considered it does not look bad at all.
Are these benchmarks with metadata on the same block device as the block data, or on a different, faster (SSD?), device?
All these benchmarks where done on a HP Pavillion DV7-3150 laptop.
This laptop contains a single drive TOSHIBA MK5056GS.
No fancy tuning of either lessfs or one of the other filesystems.
I hope to find time to upload the lessfs-1.2.2.2 release today.
The used lessfs.cfg file is this:
# Enable informational messages about compression.
DEBUG = 5
HASHNAME=MHASH_TIGER192
#HASHNAME=MHASH_SHA256
# The (chopped) hashlen in bytes, minimum is 20.
HASHLEN = 24
BLOCKDATA_IO_TYPE=file_io
BLOCKDATA_PATH=/data/dta/blockdata.dta
#BLOCKDATA_PATH=/data/mta
#BLOCKDATA_BS=1048576
#
BLOCKUSAGE_PATH=/data/mta
BLOCKUSAGE_BS=1048576
#
DIRENT_PATH=/data/mta
DIRENT_BS=1048576
#
FILEBLOCK_PATH=/data/mta
FILEBLOCK_BS=1048576
#
META_PATH=/data/mta
META_BS=1048576
HARDLINK_PATH=/data/mta
HARDLINK_BS=1048576
#
SYMLINK_PATH=/data/mta
SYMLINK_BS=1048576
#
# The freelist database is only used
# with the file_io backend
#
FREELIST_PATH=/data/mta
FREELIST_BS=1048576
#
# CACHESIZE in MB
CACHESIZE=512
# Flush data to disk after X seconds.
COMMIT_INTERVAL=15
#
LISTEN_IP=127.0.0.1
LISTEN_PORT=100
# Not more then 2 on most machines.
MAX_THREADS=32
#DYNAMIC_DEFRAGMENTATION on or off, default is off.
DYNAMIC_DEFRAGMENTATION=on
COREDUMPSIZE=2560000000
# Consider SYNC_RELAX=1 or SYNC_RELAX=2 when exporting lessfs with NFS.
SYNC_RELAX=0
# When BACKGROUND_DELETE=on lessfs will spawn a thread to delete
# a file as a background task. This is a recently added feature
# and is therefore disabled by default.
BACKGROUND_DELETE=on
# Requires openssl and lessfs has to be configured with –with-crypto
ENCRYPT_DATA=off
# ENCRYPT_META on or off, default is off
# Requires ENCRYPT_DATA=on and is otherwise ignored.
ENCRYPT_META=on
# You don’t like fsck?
ENABLE_TRANSACTIONS=on
# Select a blocksize to fit your needs.
BLKSIZE=131072
COMPRESSION=qlz
MIN_SPACE_FREE=10
STICKY_ON_LOCKED=on
I’m very glad to hear about change in development focus.
I’d like to provide access to past file versions to users over Samba (with native windows interface), unfortunately, for this to work, the files need not to be in a singe FS image.
As such, with a dump consisting of about a quater milion of files. Making a copy (snapshot) takes close to 16h on my hardware (and that is with data and metadata on thee different disks including one RAID0).
I’m not certain about the numbers, but in this use case, current btrfs with compression-force and native FS snapshots beats lessfs in disk usage at about 6-10 snapshots. Maybe a bit later.
Now I have over 8 months of daily rdiff-backup snapshots and I’d like to keep ‘em…
What’s more rdiff from server to lessfs machine takes about 4h, with btrfs… 17 minutes.