Comments on: Lessfs-1.5.7 has been released

By: Mark

Mark — Thu, 19 Apr 2012 09:01:40 +0000

Hi Mark,

I tested lessfs a couple of years ago when it was pre 1.0 and couldn’t continue with it because it was pre-release.

So now it’s at 1.5.9 I thought I’d give it a go again and hopefully use it in a production environment, but I’m not too sure where I’m going wrong with performance.
I am trying Berkelely DB 4.8 as you recommend and lzo. It is sitting on a fairly fast xfs partition which a dd from /dev/zero to at 850+MB/s.

Howevere, lessfs 1.5.9 using BLOCKDATA_IO_TYPE=file_io only writes at 30MB\s and gets slower and slower. With only 200GB of completely random data to test a worst case scenario, it slows down to 7MB\s. Is this to be expected?
There are 8 cores and 16GB RAM

I would like to get about 5TB of data in there, but at the rate it is getting slower and slower it is just not a viable option.

Hope you can help where I am going wrong.

Thanks,

Mark

By: maru

maru — Tue, 03 Apr 2012 20:17:24 +0000

Fuse file systems in general have been known to deadlock with transparent huge pages enabled. I have seen this happen with Lessfs as well. I did not recently check if this is still the case. The problem is not limited to Lessfs. All fuse filesystems are effected:

http://zfs-fuse.net/issues/123
http://us.generation-nt.com/answer/deadlocks-transparent-huge-pages-userspace-fs-daemons-help-200924121.html

By: Vins

Vins — Tue, 03 Apr 2012 10:22:02 +0000

Could you please tell us, why it is needed to disable transparent huge pages with RH6 when using lessfs?

By: Laika

Laika — Wed, 26 Oct 2011 09:42:36 +0000

Hello,

Could you show me how should i set the various paramters for the highest performance? (lessfs.cfg and DB_CONFIG)

Actually, when i ran the filesystem, i only get 25MB/s even in a ramdisk.
I guess this is due to the miss configuration of those files.

By: Richard

Richard — Thu, 20 Oct 2011 01:45:15 +0000

please not this is two dashes for – -with-snappy
(Two dashes in front of “with”)
it looks like only one dash in my comments

it’s also two dashes in ./configure –help

By: Richard

Richard — Thu, 20 Oct 2011 01:43:28 +0000

When you do your
./configure –with-snappy

I found this by doing ./configure –help.

FYI, I built mine doing rpmbuild and I don’t think I had to specify this.

By: leepfrog

leepfrog — Tue, 11 Oct 2011 16:42:10 +0000

Hi Richard,

no I did not enable it manually, I thought it would be enabled by default like the other compression algos.
Is there a documentation with all the switches available for ./configure?
What would be the parameter to enable snappy support?

By: Richard

Richard — Tue, 11 Oct 2011 15:37:46 +0000

did you make sure you enabled snappy when you compiled? Did you get any warnings in your compile?

By: leepfrog

leepfrog — Tue, 11 Oct 2011 11:57:42 +0000

Did anyone else have problems getting snappy to run? I’ve compiled 1.5.8 without problems, but if COMPRESSION=snappy is specified the mklessfs command does not create the .dta/mta folders/files.
The command runs and does not return any error or information, but the following lessfs command does not mount anything as there is nothing to mount.

If I change the configuration to another compression or no compression everything works fine.

Any ideas?

By: Fajar Priyanto

Fajar Priyanto — Fri, 23 Sep 2011 12:59:46 +0000

LessFS crashed

Hi all,
Ubuntu 10.04.2 LTS
LessFS lessfs-1.3.3.1

During copying process LessFS crashed.
And rebooting the server doesn’t help.
When I ls -l the mount point it just hang.
Any workaround? Thank you.

The log only shows this:
Sep 23 20:38:17 server1 lessfs[1185]: The selected data store is tokyocabinet.
Sep 23 20:38:17 server1 lessfs[1185]: Lessfs transaction support is enabled.
Sep 23 20:38:17 server1 lessfs[1185]: Hash MHASH_TIGER192 has been selected
Sep 23 20:38:17 server1 lessfs[1185]: Lessfs uses a 24 bytes long hash.
Sep 23 20:38:17 server1 lessfs[1185]: Automatic defragmentation is enabled.
Sep 23 20:38:17 server1 lessfs[1185]: cache 8192 data blocks
Sep 23 20:38:17 server1 lessfs[1185]: Tuning the bucketsize for /data/db/mta/fileblock.tch to 1048576
Sep 23 20:38:18 server1 lessfs[1185]: Tuning the bucketsize for /data/db/mta/blockusage.tch to 1048576
Sep 23 20:38:18 server1 lessfs[1185]: Tuning the bucketsize for /data/db/mta/metadata.tcb to 1048576
Sep 23 20:38:19 server1 lessfs[1185]: Tuning the bucketsize for /data/db/dta/blockdata.tch to 1048576
Sep 23 20:38:20 server1 lessfs[1185]: Tuning the bucketsize for /data/db/mta/symlink.tch to 1048576
Sep 23 20:38:22 server1 lessfs[1187]: Lessfs has not been unmounted cleanly.
Sep 23 20:38:22 server1 lessfs[1187]: Rollback to : Fri Sep 23 19:27:37 2011

Thank you.