just to let you know that i still use lessfs, just updated from 1.3.3.1 to 1.3.3.8, everything run fine on my 32bit systems. The upgraded version allow me to “du” folders faster than before, even on my low performance machines.

Thank you for your huge work on this project.

DataDomain: 2,807,467,410
LessFS: 4,547,298,233

LessFs is just more near to 1,6 bigger than DD.

Many thanks Maru!!

filesys show compression /backup/
—
629766144 sdlmoc01-4k_boot-1.iso
1659176960 sdlmoc01-4k_boot-nocomp-04032011-1.iso
1165178880 sglvms52-4k_boot-03032011-1.iso
1092429824 sglvms52-4k_boot-1.iso
3985399808 sglvms52-4k_boot-nocompressione-07032011-1.iso
4615309312 sglvms52-4k_boot-nocompressione207032011-1.iso
—-
Some image are gziped and not but with some file changed inside of it (to the “offset” problem).
This is the result from DataDomain command:
Total files: 6; bytes/storage_used: 4.7
Original Bytes: 13,188,470,376
Globally Compressed: 4,026,972,574 (dedupped)
Locally Compressed: 2,807,467,410 (dedupped+compress)
Meta-data: 13,160,296
—-
So the sum of all file is like a 12GiB dedupped from DD are only 2,5 GiB!!!Very good performance!!

Now i show the same file using LessFS:
—
[root@sglvms51 mondo-img]# cat /fuse/.lessfs/lessfs_stats
INODE SIZE COMPRESSED_SIZE FILENAME
10 0 0 lessfs_stats
14 0 0 enabled
15 0 0 backlog
20 629766144 110648854 sdlmoc01-4k_boot-1.iso
21 1659176960 808317386 sdlmoc01-4k_boot-nocomp-04032011-1.iso
22 1165178880 1034221145 sglvms52-4k_boot-03032011-1.iso
23 1092429824 976047268 sglvms52-4k_boot-1.iso
24 3985399808 1378525345 sglvms52-4k_boot-nocompressione-07032011-1.iso
25 4615309312 239538235 sglvms52-4k_boot-nocompressione207032011-1.iso
—–

I think “sliding windows” is very useful for many file type, for example as backup storage for Bacula or other Backup software or for simple tar backup.
Please ask me other info if you need it.
Many thanks for your great work!!

It’s always interesting to hear about what other dedup solutions are doing. Datadomain uses sliding windows deduplication where lessfs is using fixed blocksize deduplication. I have been hesitating to add sliding windows to lessfs due to patent issues.

Can you give an example on the dedup ratios that you see on Datadomain and lessfs and what type of data you are testing with?

Lessfs is known to work very well with raw disk images and when you copy regular files to it. When you use Lessfs to store tar or zip archives then the compression will be low or none since the offsets in the tar archive will differ each time.