Lessfs-1.1.6 is available for download.

This versions adds two new features to lessfs.
A. Synchronous replication.
Lessfs now supports a master / slave configuration. In this case the master will synchronously replicate the data to the slave. The slave is read-only.
The master will freeze for ever when the slave becomes unreachable. Many options will be added in the future, including maintaining a replication backlog and async replication. The replication feature should be considered alpha code.
B. Background delete.
When enabled lessfs will spawn a background thread that will take care of the truncation/delete task. Lessfs will therefore return immediately after a delete or truncate operation. This is a new feature and is therefore not enabled by default.

Enjoy,

Mark Ruijter

This entry was posted in Uncategorized. Bookmark the permalink.

13 Responses to Lessfs-1.1.6 is available for download.

  1. Brad says:

    Mark –

    I’d love to contribute back to this project, but I can’t seem to get this installed. Is there a install doc out there for Ubuntu / Debian based systems?

    Brad

  2. Stuart Harper says:

    Yeah this link was good when installing the older version (1.0.x)
    http://ubuntuforums.org/showthread.php?t=1390085

    You’ll need to review the Google Group forum for info on correctlky setting up the 1.1.x config file. (http://groups.google.com/group/lessfs)

  3. Morten says:

    Hi

    Would like to run a manual defrag on one of my lessfs volumes, but getting the following error:

    Aug 30 14:35:50 backup2 kernel: defrag_lessfs[27760]: segfault at ffffffffffffffff ip 00007f6863900a5c sp 00007fff4b60dec8 error 4 in libtokyocabinet.so.9.6.0[7f68638d3000+82000]
    Segmentation fault

    Any ideas ?

    • maru says:

      Not without a stack trace and a peek at your lessfs.cfg file.
      I just tested it on a rather large volume myself and it works for me. ;-(

      xgtest01:~ # /etc/init.d/ietd stop
      Stopping iSCSI Target: Terminated
      xgtest01:~ # cd /usr/src/lessfs-1.1.6
      xgtest01:/usr/src/lessfs-1.1.6 # umount /fuse/
      xgtest01:/usr/src/lessfs-1.1.6 # ./defrag_lessfs /etc/lessfs.cfg

      Is there a core file present?
      If so can you do a backtrace?
      gdb ./defrag_lessfs /core
      bt

      P.S.

      Did you know that for defragmentation you can leave lessfs mounted and just do:
      telnet localhost 100
      > defrag

  4. Morten says:

    No core file, turned back all values to default in .config file, but still crashes. (have enabled background delete, previously)

    Tried telnet localhost 100, and it is working.

    Strange ……..

    • maru says:

      I’ll make a note to check defrag_lessfs.
      There must be something wrong with it. We don’t need it anymore, but it should work anyway.

  5. Morten says:

    Having a new small issue with iscsi on just one of my lessfs hosts. Running esxi 4.1 with iscsi connection to dedicated hardware box, running lessfs 1.1.6.

    It seems that under heavy loads I get a lot of “Abort Task” on the iscsi-target i.e :
    iscsi_trgt: Abort Task (01) issued on tid:1 lun:0 by sid:4224323677913600 (Function Complete)

    And after a while lessfs terminates, telling me that mount point is not connected. A restart of lessfs and iscsi-target fixes all again, but just for a while.

    Wondering what can cause lessfs to terminate ? I run other hosts on the exact same configuration and the exact same load, without problems like this.

    • Mark says:

      When Lessfs terminates, does it log something in /var/log/messages? Or when it does not, is there a coredump that you can backtrace? Can you email your lessfs.cfg?
      How much memory do you have in this system and what is the cache size in lessfs.cfg? Can it be that the system runs out of memory?

      Which iSCSI target are you using? IETD/SCST/STGT or LIO?

      • Morten says:

        Got a couple of core file today. Have also done the following:

        Changed from ietd to tgtd
        installed 8GB of more memory, total is 12GB
        Changed TC cache from 1GB to 4GB

        Systems seems more stable, running stress test now. Will be happy to send conf and core to someone.

        Morten

  6. Alex says:

    Hi mark,

    just to let you know that 1.1.6 is working great now :)

    Well, at beginning i’ve been disappointed with some crashes using a big cache setting (2048) resulting in crashes on the test storage server (out of memory thing).

    Well, it’s a dual Xeon HT 3Ghz with 6Go Ram x86 (32bit so as i have to compile TC with the good options). So settings cachesize to 256 (Mo) fixed the issue.

    It is Xeon HT, i’ve got 4 cpu unit seen by the system, so i’ve set the MAX_THREADS to 20 (5x No of cpus).

    Using rsync through vpn from another “normal storage server” (without lessfs), i get transfer rates between 5 to 10Mb/sec writing on lessfs storage. Which is quite good and expected i guess :D

    So, just as an example, here is the config file :
    # Automatically generated by lessfsconfig.sh
    # Enable informational messages about compression.
    DEBUG
    # The (chopped) hashlen in bytes, minimum is 20.
    HASHLEN = 24
    #BLOCKDATA_IO_TYPE=file_io
    BLOCKDATA_PATH=/home/replication_data/dta
    BLOCKDATA_BS=13762560
    #
    BLOCKUSAGE_PATH=/home/replication_data/mta
    BLOCKUSAGE_BS=13762560
    #
    DIRENT_PATH=/home/replication_data/mta
    DIRENT_BS=30000000
    #
    FILEBLOCK_PATH=/home/replication_data/mta
    FILEBLOCK_BS=132644864
    #
    META_PATH=/home/replication_data/mta
    META_BS=30000000
    #
    HARDLINK_PATH=/home/replication_data/mta
    HARDLINK_BS=30000000
    #
    SYMLINK_PATH=/home/replication_data/mta
    SYMLINK_BS=30000000
    #
    # The freelist database is only used
    # with the file_io backend
    #
    FREELIST_PATH=/home/replication_data/mta
    FREELIST_BS=13762560
    #
    # CACHESIZE in MB
    CACHESIZE=256
    # Flush data to disk after X seconds.
    COMMIT_INTERVAL=30
    #
    LISTEN_IP=127.0.0.1
    LISTEN_PORT=100
    # Not more then 2 on most machines.
    MAX_THREADS=20
    #DYNAMIC_DEFRAGMENTATION on or off, default is off.
    DYNAMIC_DEFRAGMENTATION=on
    COREDUMPSIZE=1000000000
    # Consider SYNC_RELAX=1 or SYNC_RELAX=2 when exporting lessfs with NFS.
    SYNC_RELAX=1
    # Requires openssl and lessfs has to be configured with –with-crypto
    ENCRYPT_DATA=off
    # ENCRYPT_META on or off, default is off
    # Requires ENCRYPT_DATA=on and is otherwise ignored.
    ENCRYPT_META=on
    #PASSWORD=1234
    MIN_SPACE_FREE=10
    #MIN_SPACE_CLEAN=15
    #CLEAN_PROGRAM=/tmp/clean.sh
    # Highly recommended.
    ENABLE_TRANSACTIONS=on
    # Select a blocksize to fit your needs.
    # Most people should select 65536
    BLKSIZE=65536

    This is just a feedback (positive for once), in case of ;)

    System is :
    Linux 2.6.32-21-generic-pae #32-Ubuntu SMP Fri Apr 16 09:39:35 UTC 2010 i686 GNU/Linux
    Ubuntu 10.04.1 LTS

  7. wxp says:

    Hi,Who can tell me which functions in lessfs or others achieve the function of data dedup, thanks very much!

  8. Ok found the problem….
    The filesystem was almost full. Lessfs freeze the IO.
    It runs again after I set MIN_SPACE_FREE = 5
    The bad news is it rolls back to Aug 24 data …..

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>