Comments on: Lessfs-1.1.3 is available.

By: maru

maru — Thu, 22 Jul 2010 08:39:20 +0000

For now lessfs reports the actual filesize to du and not the compressed filesize. I am thinking about changing du to report the compressed space instead.

By: wxp

wxp — Thu, 22 Jul 2010 02:38:58 +0000

First, Thank you very much!
Alex, I think you are right and thank you for your help.
I tested with a directory which contains so many small files, as what you said, if I store a 129K file, this will use 2×128K block, so, at least, about 256K for 1 129K .

When I set the BlockSize = 4K (4096), and I copy a dircetory ,which also contains so many small files, to /fuse, the result are: 8.3M—->8.2M;38M—–>37M

Now, I have another question: When I read lessfs_stats file, I find that the size of each file is about the half of the source file, but the whole size of the directory almost the same as source directory.

why ?

By: maru

maru — Wed, 21 Jul 2010 11:28:16 +0000

Did you check .lessfs/lessfs_stats to see the real space that these files consume?
The du statistics are something that I am working on. lessfs_stats will give you better insight for now. You can also check the size of the actual lessfs databases.

By: Alex

Alex — Wed, 21 Jul 2010 11:13:01 +0000

Hi wxp,

i guess your different sizes on the little files is due to the blocksize you choose to store your datas.

If you use 128k blocksize then, a single little file will be stored in 1 block, which is at least 128k because of your settings. So as that won’t happen on bigger files, because it will feat in multiple block store.

So as, to be clear, if you store a 129K file, this will use 2x128K block, so, at least, about 256K for 1 129K file. That’s why you have to choose the best settings to fit your needs, as you may loose efficiency on really little files.

By the way, on the 1.0.8 lessfs, storing files lower than block size make the FS become so slow near unusable.

Hope i’m right in the way i explain it, and hope this helps.

By: wxp

wxp — Wed, 21 Jul 2010 09:26:48 +0000

Hi, I first do by following:
#mklessfs /etc/lessfs.cfg /fuse/
#lessfs /etc/lessfs.cfg /fuse/

then:

# cd /home/wxp/test
# ls
test1 test4

cd ..
# du -sh test
12K test

I copy test/ in /home/wxp to /fuse directory:

# cp -r test/ /fuse/test_lessfs/

# cd /fuse/test_lessfs/test
# ls
test1 test4

#cd ..
# du -sh test
257K test

However when I tested with a single file:

# tar -cvf linux_ker.bak.tar linux-2.6.33
# du -sh linux_kernel.bak.tar
1.4G linux_kernel.bak.tar

—copy to /fuse
# cp linux_kernel.bak.tar /fuse/tar
# du -sh linux_kernel.bak.tar
1.4G linux_kernel.bak.tar

who can tell me why, and thank you!

By: wxp

wxp — Tue, 20 Jul 2010 02:49:54 +0000

When I use the lessFS 1.1.4 , I don’t why to happen this:
#mklessfs /etc/lessfs.cfg /fuse/
#lessfs /etc/lessfs.cfg /fuse/ -d

FUSE library version: 2.8.1
nullpath_ok: 0
unique: 1, opcode: INIT (26), nodeid: 0, insize: 56
INIT: 7.12
flags=0x0000007b
max_readahead=0x00020000
INIT: 7.12
flags=0x00000031
max_readahead=0x00020000
max_write=0x00020000
unique: 1, success, outsize: 40

and then I cope my /home directory to /fuse:(open another terminal)
#cp -r /home/ /fuse

It takes so many time and /home size is 586M,but in /fuse, the size of home/ is 2.0G

I don’t why?

By: Pete

Pete — Mon, 19 Jul 2010 16:29:18 +0000

One more update… when I restarted lessfs, this time it said that it had to rollback. It did that, and was able to start it again. It could not open fileblock.tch, or blockusage.tch until it rolled back. The number after “failed to delete” kept decreasing sequentially, from 17-15804 until it reached 17-0. After it hit zero, the messages stopped, I killed the lessfs process and then restarted. That’s when it did the rollback and then started back up in a usable state again.

By: Pete

Pete — Mon, 19 Jul 2010 16:06:22 +0000

Wow, now my syslog is filling up with dozens and dozens of lines like this one:

Jul 19 12:01:05 localhost lessfs[11708]: delete_dbb: failed to delete 17-15160 reason : no record found

By: Pete

Pete — Mon, 19 Jul 2010 16:03:05 +0000

I was able to reproduce it. I started copying a 15GB file onto a lessfs share via samba. 25% through the transfer, I clicked cancel. The cancel dialog froze, and everything became unresponsive. I am unable to access lessfs, even through the console. When I tried to restart the lessfs service, it crashed on me. I do have a core dump, now. And I also noticed the following in my syslog:

Jul 19 11:53:49 localhost lessfs[12415]: The selected data store is tokyocabinet.
Jul 19 11:53:49 localhost lessfs[12415]: Lessfs transaction support is enabled.
Jul 19 11:53:49 localhost lessfs[12415]: Hash MHASH_TIGER192 has been selected
Jul 19 11:53:49 localhost lessfs[12415]: Lessfs uses a 24 bytes long hash.
Jul 19 11:53:49 localhost lessfs[12415]: Lessfs fsync does not sync the databases to the disk when fsync is called on an inode
Jul 19 11:53:49 localhost lessfs[12415]: Automatic defragmentation is enabled.
Jul 19 11:53:49 localhost lessfs[12415]: cache 400 data blocks
Jul 19 11:55:23 localhost lessfs[12415]: segfault at 0 ip b7f254dd sp bfb79140 error 4 in libtokyocabinet.so.9.8.0[b7eef000+7e000]
Jul 19 11:55:23 localhost lessfs[12415]: Exit signal received, exitting

By: Pete

Pete — Mon, 19 Jul 2010 15:36:34 +0000

I did not have the syslog service running at the time. I started it now and am trying to reproduce the issue.