Comments on: Lessfs-1.1.0-beta6 has been released.

By: wxp

wxp — Fri, 02 Jul 2010 02:19:01 +0000

Hi, Hans
Could you mind to give me some manual reference on lessfs?

These days I try to use lessfs, but hard becuase of help-less

Thank you!

By: Hans

Hans — Thu, 01 Jul 2010 17:23:12 +0000

The first vm is cloned and lessfs 1.1.0 is still running :). du reports the 2T ietd sparce file as being 64k. Thats unlikle since there is a 80G vm in it.

I can’t get esx 3.5 to work with iscsi blocksizes above 512 bytes. I don’t think any higher makes sense. When a vm writes 512 bytes to disk, esx should then fetch a 64k block over iscsi modify the 64k block with the 512 bytes and send it back. Thats a lot of network traffic, and does not improve the lessfs performance in any way.

The only way out is if there would be a 64k block cache between ietd and lessfs, and if the dirty blocks would not be synced to disk/lessfs immedeately, so the vm can put more 512 byte writes in to it before its deduped.

By: wxp

wxp — Thu, 01 Jul 2010 00:14:08 +0000

Hi, I am so sorry to disturb you!
today, I have download lessfs software and compiled, tested it. I find it hard to run because there is little description on how to start and use it.
Can you tell or send me some documents on how to use this software.
Thank you!
Wxp

By: wxp

wxp — Thu, 01 Jul 2010 00:13:44 +0000

Thank you very much for your help!
I have another question: Besides lessfs, there are opendedup (sdfs)、Bacula (SIS)、Zmanda and Nexenta (ZFS) on open source deduplication, I want to know whether the compared data on “iozone – ZFS/SDFS/Les” is about these companies or software?
Thanks again!

By: maru

maru — Wed, 30 Jun 2010 20:41:41 +0000

Please try lessfs-1.1.0
When you want maximum performance of lessfs+ietd you will need to patch ietd to write with a larger block size. ietd will normally use 4k blocks which degrades lessfs performance. This is also true for other fuse based filesystems.

By: Hans

Hans — Tue, 29 Jun 2010 21:23:21 +0000

beta6 ietd and esx 3.5 ends in a lessfs crash after just formating 6 2TB luns. beta3 was stable enough to clone en boot some vm’s

By: Pete

Pete — Tue, 29 Jun 2010 14:33:22 +0000

I tested it with full SQL dumps from different days. It worked exceptionally. The total size of 3 backups was marginally more than the original size of one. Since the DBs are re-indexed occasionally, I expected the deduplication to not be that great.

The first file was compressed down to 50% of the original size. The 2nd file was compressed to 50% of the compressed size of the 1st file. And the 3rd file was just a fraction of the 2nd (on disk). All three files were between 500 and 600MB, and the total size of the dedup folder (outside lessfs) was just over 600MB. I was impressed.

By: wxp

wxp — Tue, 29 Jun 2010 07:04:18 +0000

By: wxp

wxp — Tue, 29 Jun 2010 01:51:28 +0000

By: Mark

Mark — Mon, 28 Jun 2010 20:40:00 +0000

Hi Pete,

You are most likely using lessfs with lot’s of small files?
In that case you are certainly right. Compression has improved a lot for that workload.

Please note the lessfs-1.1.0-beta6 contains a nasty bug. Lessfs-1.1.0-beta4 does not contain the improved read cache code but is pretty stable as far as I can tell.