lessfs & btier

This version of lessfs passes all the fsx-linux tests. It has been confirmed that this release fixes the problems that people where reporting with rsync.

Open discussion about data deduplication for Linux in general.

Would you like a data deduplicating blockdevice in userspace or in the kernel?

Would you prefer lessfs-2.0 with replication and snapshots?

Your opinion counts!

This release fixes a major bug in the truncate code. This bug causes lessfs to crash with ccrypt and rsync -inline.

Lessfs-0.8.2 is mostly a bugfix release.
Fixes a bug that causes lessfsck and mklessfs to segfault when compiled
with encryption support and encryption disabled in the config.
Fixes a bug that causes lessfs to segfault on umount when compiled
with encryption support and encryption disabled in the config.
lessfsck,listdb and mklessfs are now installed in /usr/sbin
instead of /usr/bin.

This release fixes a bug that causes mklessfs to segfault when DEBUG is not set in the configuration file.

This release add the program lessfsck. lessfsck can be used  to check, optimize and repair a lessfs filesystem. lessfsck only supports the tc data store for now. Support for the file_io backend will come with release 0.8.2 

Please note that lessfsck is still under active development.

Change log:

Fixes a possible segfault when lessfs is used with lzo compression.
Fixes a problem when compiling lessfs without encryption on a system without openssl-devel.
Enhances the logging facility.
Performance has improved on higher latency storage like drbd or iscsi.
Reduces the number of fsync operations when sync_relax>0, needed for syncing many small files with for example rsync.

Thanks to : Roland Kletzing for finding and assisting in solving some of the problems mentioned.

Fixes a segfault on free after unmounting lessfs without
encryption support. Fixes a problem that could lead to a
deadlock when using file_io with NFS.
A performance improvement, changed a mutex lock for a
spinlock, to improve data throughput.

This version of lessfs introduces a new hash named Blue Midnight Whish.

This is a very fast hash that increases lessfs performance significantly. The implementation makes it easy to use any of the hashes from the NIST hash competition. BMW was choosen for lessfs because of the speed.

The length of the hash is now configurable between 20 and 32 bytes for BMW and 20 to 24 for the tiger hash.

Lessfs-0.7.2 fixes a typo in lib_tc.c that could potentially lead to data corruption.
Please upgrade a.s.a.p. when you use lessfs-0.7.1 with the tokyocabinet data store. Older releases are not effected.

Lessfs-0.7.2 also contains a minor performance enhancement.

This new release improves performance in general. It adds a new storage method for the data that is stored. Previous lessfs versions would store the data in tokyocabinet. The new file_io backend allows to store the data in a regular file or even a raw device (not recommended).

The file_io backend allows lessfs to store larger amounts of data with more modest memory requirements. The file_io backend should give similar performance when compared to tokyocabinet.