This version add one new feature to lessfs. When configured with –enable-filelog lessfs will log to a file instead of using syslog.
-
Archives
- January 2015
- August 2014
- February 2014
- December 2013
- May 2013
- March 2013
- January 2013
- December 2012
- July 2012
- June 2012
- April 2012
- January 2012
- October 2011
- September 2011
- August 2011
- June 2011
- May 2011
- April 2011
- March 2011
- February 2011
- January 2011
- December 2010
- October 2010
- September 2010
- August 2010
- July 2010
- June 2010
- April 2010
- March 2010
- February 2010
- January 2010
- December 2009
- November 2009
- October 2009
- September 2009
- August 2009
- June 2009
- March 2009
-
Meta
Using the spec file, I built an rpm. Lessfs and mklessfs get installed to /usr/local/sbin instead of /usr/sbin. The lessfs_example in /etc/init.d look for less in /usr/sbin so depending on your distro you will have to edit it before running service lessfs start
Hi Marcin,
Attached is lessfs-1.4.10 which should solve the problems that you are reporting.
It also adds a warning that using tc as datastore is deprecated.
I would like to advise you to :
A. Use file_io instead of TC
BLOCKDATA_IO_TYPE=file_io
BLOCKDATA_PATH=/data/dta/blockdata.dta
B. Use a much larger BLKSIZE. At least 64k or even better 128k.
BLKSIZE=131072
This only works with a recent >=2.6.26 kernel since fuse otherwise does not support big_writes.
C. Use at least 4 threads, especially if you would ever when want to enable replication since this allocates 1 thread.
MAX_THREADS=4
Can you confirm that mklessfs now works for you?
Thanks,
Mark
Now it’s better, i’m not getting segfault. I have a question/suggestion. Would it be possible to lessfs creates needed subdirectory by self? Now it only throws “User defined signal 1″ which isn’t much helpfull.
I didn’t get any notification about using tokyocabinet while doing mklessfs.
Ad.B. Sometimes i try corner case values to check how software deal with it.
Thanks!
Hello!
I’ve got problem with mklessfs, i’m getting segmentation fault. I filled bug at gentoo bugzilla ( https://bugs.gentoo.org/371923 ). Here is config file:
DEBUG = 1
HASHNAME=MHASH_TIGER192
HASHLEN = 24
BLOCKDATA_PATH=/tmp/lessfsdb/dta
BLOCKDATA_BS=1048576
BLOCKUSAGE_PATH=/tmp/lessfsdb/mta
BLOCKUSAGE_BS=1048576
DIRENT_PATH=/tmp/lessfsdb/mta
DIRENT_BS=1048576
FILEBLOCK_PATH=/tmp/lessfsdb/mta
FILEBLOCK_BS=1048576
META_PATH=/tmp/lessfsdb/mta
META_BS=1048576
HARDLINK_PATH=/tmp/lessfsdb/mta
HARDLINK_BS=1048576
SYMLINK_PATH=/tmp/lessfsdb/mta
SYMLINK_BS=1048576
FREELIST_PATH=/tmp/lessfsdb/mta
FREELIST_BS=1048576
CACHESIZE=256
COMMIT_INTERVAL=10
LISTEN_IP=127.0.0.1
LISTEN_PORT=100
MAX_THREADS=2
DYNAMIC_DEFRAGMENTATION=on
COREDUMPSIZE=2560000000
SYNC_RELAX=0
ENCRYPT_DATA=off
ENCRYPT_META=on
ENABLE_TRANSACTIONS=on
BLKSIZE=4096
COMPRESSION=qlz
Test case:
# ls /tmp|grep lessfsdb
# mklessfs -c /etc/lessfs.cfg
User defined signal 1
# mkdir /tmp/lessfsdb
# mklessfs -c /etc/lessfs.cfg
# ls /tmp/lessfsdb/
replog.dta
Meseems there should be more files and dirs. Let’s go on:
# rm /tmp/lessfsdb/replog.dta
# mkdir /tmp/lessfsdb/dta
# mkdir /tmp/lessfsdb/mta
# mklessfs -c /etc/lessfs.cfg
Segmentation fault
# ls -R /tmp/lessfsdb/
/tmp/lessfsdb/:
dta mta replog.dta
/tmp/lessfsdb/dta:
blockdata.tch blockdata.tch.wal
/tmp/lessfsdb/mta:
blockusage.tch dirent.tcb fileblock.tch freelist.tcb hardlink.tcb metadata.tcb symlink.tch
blockusage.tch.wal dirent.tcb.wal fileblock.tch.wal freelist.tcb.wal hardlink.tcb.wal metadata.tcb.wal symlink.tch.wal
Here is gdb backtrace:
# gdb mklessfs
GNU gdb (Gentoo 7.2 p1) 7.2
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type “show copying”
and “show warranty” for details.
This GDB was configured as “i686-pc-linux-gnu”.
For bug reporting instructions, please see:
…
Reading symbols from /usr/sbin/mklessfs…Reading symbols from /usr/lib/debug/usr/sbin/mklessfs.debug…done.
done.
(gdb) run -c /etc/lessfs.cfg
Starting program: /usr/sbin/mklessfs -c /etc/lessfs.cfg
[Thread debugging using libthread_db enabled]
Program received signal SIGSEGV, Segmentation fault.
0x080515f9 in formatfs () at lib_common.c:1365
1365 lib_common.c: No such file or directory.
in lib_common.c
(gdb) thread apply all bt full
Thread 1 (Thread 0xb7b496c0 (LWP 1904)):
#0 0x080515f9 in formatfs () at lib_common.c:1365
stbuf = {st_dev = 13258579423029886975, __pad1 = 60214, __st_ino = 3086606272, st_mode = 1, st_nlink = 3086938193,
st_uid = 3086869904, st_gid = 3082131464, st_rdev = 13245790139998797825, __pad2 = 37792, st_size = -4611699109352850648,
st_blksize = -1212020095, st_blocks = -5207481559359298160, st_atim = {tv_sec = -1208135692, tv_nsec = 134855320}, st_mtim = {
tv_sec = 134855464, tv_nsec = -1073744872}, st_ctim = {tv_sec = -1208000448, tv_nsec = 134855464},
st_ino = 13257840790522169872}
nextinode = 0
stiger = 0x809bc60 “▒3▒Ks▒25T▒7T▒▒▒▒a32l▒u▒\211\222″
blockdatadir =
crypto = {
passwd = “\230▒\t\b(▒\t\bP\v▒\230▒\t\b@`▒▒(▒\t\b0\v▒▒\t\b6\v▒▒O▒▒02▒\230▒\t\b(▒\t\b27000000\20002▒(▒\t\b”,
iv = “▒O▒\230▒\t\b”}
hashstr =
inuse = {offset = 579199192178230068, size = 134855464, allocated_size = 13256911686020500368, inuse = 578378338028834804}
__PRETTY_FUNCTION__ = “formatfs”
#1 0x08059949 in main (argc=3, argv=0xbffff4b4) at mklessfs.c:183
dbg =
rnd =
ckpasswd =
p =
Thanks for help.
I received a segfault at first, too. Then I realized that the database had been changed from BTREE to hash, I believe, in version 1.4.5. I had previously created my databases using 1.4.0, so I ended up having to rollback to 1.4.0, copying my data off, re-creating my DBs, and then copying the data back. Maybe yours is a similar problem?
I wanted to create database, destination dir was empty.
Hello,
I am testing Lessfs 1.4.9 and have two question:
1. When I copied data on Lessfs and then delete it, status shown by df -h command is incorrect, because it shows that there is used space on disk but there are no files on Lessfs volume.
Is there any posibility to fix it ?
2. Which filesystem works best with Lessfs (currently I am using XFS) ?
@Peter
depending on the storage that you have chosen lessfs will free the space after defragmentation or never.
lessfs 1.4.9 Crashes:
Jul 12 11:48:37 virustest kernel: lessfs[24889]: segfault at 0 ip 080599f9 sp b2a79120 error 4 in lessfs[8048000+29000]
FILE lib_safe.c – LINE 211 – PID 8449 : Tue Jul 12 11:57:35 2011 – Entering function : mark_dirty : 1310464655:433840
seems to be the last entry
i wonder if lessfs could be used together with levelDB: http://google-opensource.blogspot.com/2011/07/leveldb-fast-persistent-key-value-store.html
Hi,
I’ve compiled and used BerkelyDB but how do I “know” that lessfs is using BerekelyDB.
Here is my lessfs.cfg
# Enable informational messages about compression.
DEBUG = 5
HASHNAME=MHASH_TIGER192
#HASHNAME=MHASH_SHA256
# The (chopped) hashlen in bytes, minimum is 20.
HASHLEN = 24
BLOCKDATA_IO_TYPE=file_io
BLOCKDATA_PATH=/data/dta/blockdata.dta
#
BLOCKUSAGE_PATH=/data/mta
BLOCKUSAGE_BS=1048576
#
META_PATH=/data/mta
META_BS=1048576
#
DIRENT_PATH=/data/mta
DIRENT_BS=1048576
#
TUNEFORSPEED=yes
#
FREELIST_PATH=/data/dta
FREELIST_BS=1048576
#
FILEBLOCK_PATH=/data/mta
FILEBLOCK_BS=1048576
#
HARDLINK_PATH=/data/mta
HARDLINK_BS=1048576
#
SYMLINK_PATH=/data/mta
SYMLINK_BS=1048576
#
CACHESIZE=512
# Flush data to disk after X seconds.
COMMIT_INTERVAL=10
#
LISTEN_IP=127.0.0.1
LISTEN_PORT=100
# Not more then 2 on most machines.
MAX_THREADS=16
#DYNAMIC_DEFRAGMENTATION on or off, default is off.
DYNAMIC_DEFRAGMENTATION=on
COREDUMPSIZE=2560000000
# Consider SYNC_RELAX=1 or SYNC_RELAX=2 when exporting lessfs with NFS.
SYNC_RELAX=0
# When BACKGROUND_DELETE=on lessfs will spawn a thread to delete
# a file as a background task. This is a recently added feature
# and is therefore disabled by default.
BACKGROUND_DELETE=on
# Requires openssl and lessfs has to be configured with –with-crypto
ENCRYPT_DATA=off
# ENCRYPT_META on or off, default is off
# Requires ENCRYPT_DATA=on and is otherwise ignored.
ENCRYPT_META=off
# You don’t like fsck?
ENABLE_TRANSACTIONS=on
# Select a blocksize to fit your needs.
BLKSIZE=131072
COMPRESSION=qlz
MIN_SPACE_FREE=2
MIN_SPACE_CLEAN=5
Also, deleting files does not seem to free up space in the lessfs filesytem. I’ve hit the limit in space and need to free up data. Is there something I’m missing in deletions?
Thanks,
David