Lessfs-1.0.0 stable released.

Lessfs has finally made it to 1.0.0 And even better, sourceforge tells me that it’s now stable. ;-)

Only minor changes this release:

Added license information at the top of every file. Updated the license information for QuickLZ after contacting the author. And finally solved a bug where lessfs would crash upon defragmentation when the file_io backend had been selected.

This entry was posted in Uncategorized. Bookmark the permalink.

18 Responses to Lessfs-1.0.0 stable released.

  1. Alex says:

    hip hip hip hoera gefeliciteerd ! :P

    Well, welcome to lessfs 1.0, gonna update right now ! Nice to see the first “official” stable version !

  2. Alex says:

    Updated – everything seem to work well. Had revisited my way of using lessfs. At the beginning i was creating multiple instances of lessfs, but it caused crashes because i ran out of memory, killed program, full data corruption.
    Don’t know why, instead of swapping, the program got killed. So, i recently modified my data structure in order to include all the datas in one container.

    Advantage : more efficienty because deduplication apply to all the stored datas
    Problem : if lessfs crashes, i loose ALL the datas (that was my fear)

    Since i’ve put a replication system of my datas on another server, i changed this (the first server) to one lessfs container (mount point) with one instance, setting more memory, and more thread to the only lessfs instance.

    Performances got enhanced of course. For my peace of mind, i’ll keep replication of all the datas on a 2nd server, also storing all of those in 1 container. (in fact, i started to try the “one lessfs instance for all” on the replication server, which had no problem at all, no lack of memory, no crashes etc).

    That’s the feed back for now :)
    Have a happy new year !

  3. MasterTH says:

    hmm..

    get this error message “mklessfs /root/backup/lessfs.cfg
    mklessfs: /usr/lib/libfuse.so.2: version `FUSE_2.8′ not found (required by mklessfs)”

    fusermount -V –> “2.8.0”

  4. Bas Bleeker says:

    Great news. I’m planning to use it when my new backup disk arrives.
    One question though: I’m planning to use LVM to create a RAID 0 JBOD system.
    With this i’m flexible to add extra disks easily when needed.
    Can a lessfs volume or container easily be enlaged, ideally on-line/mouted?
    And as last thank you for this great FS.

    • maru says:

      Since Lessfs is basically an overlay filesystem it relies on an other filesystem to hold it’s databases.
      If you use for example reiserfs to hold the databases the you can resize reiserfs online while lessfs maintains operational.
      It would not hurt to freeze IO on lessfs during the resize of course but it should not be necessary.

      Mark

  5. Ludo says:

    Hello
    have you somewhere a document with all parameter optimisation ?
    for exemple number of thread for computer with 2 4 6 8 or 16 cpu ?
    size of bs for different type of usage , benchmark etc etc ?
    i try it and i have 7% of deduplication , with the same data i have 50% of deduplication on a commercial product .

    My computer
    Dual Quad core , 16To of hardrive , 16Go of ram

    Thanks

    • MasterTH says:

      That would be great!!!

    • MasterTH says:

      compiling options (configure) would be also really great

    • maru says:

      Ludo,

      What type of data are you storing on lessfs? For example tar or vtl files?
      Lessfs compression should be equal to that of commercial vendors.

      The number of threads only effect the performance and should be 30% less then the number of cores in the system with a minimum of 1. ;-)

      Use a 32k block size for small files.
      Use a 64k block size for most workloads.
      Use a 128k block size for max performance.

      Make sure that the _BS values are tuned.
      For max performance compile with –with-sha3 and put the FILEBLOCK and BLOCKUSAGE databases each on there own raid one disk set. The faster these disks are the better. BLOCKDATA can be on raid5/6.

  6. MasterTH says:

    Now i have another question. I’m planning to use lessfs for large backup files. The Files are 40GB and bigger….

    The transfer to the lessfs folder starts really fast, but after ~10GB onle 180kb/s are written.

    What can i do??

    • maru says:

      Tune the databases. I assume you didn’t change the default _BS settings?
      BLOCKDATA_BS=1048576
      BLOCKUSAGE_BS=1048576
      FILEBLOCK_BS=1048576

      You should increase the numbers you see above. These settings are usable for testing small datasets. For production you need at least 2MB or ram for every 1 million records stored.

      Mark.

      • MasterTH says:

        size 39338331033 bytes : lzo compressed bytes 39334224938 and 426 duplicate blocks

        sounds not bad, but 52hours for transfer, argh… ;)
        do i have to make another mklessfs after changing the bs??

        At this time this is not the problem, because its only testing, but when the system is online, there should be a way to increase the size of the bs without destroying any data.

      • MasterTH says:

        which size would you set für a backup partition with 1TB?

        • admin says:

          For most purposes it is best to use a blocksize of 64k. I am afraid that today this is a one time choice. You can not change the blocksize later.
          As for your _BS settings, just add two zero’s : BLOCKDATA_BS=104857600
          If you ever need to change the _BS settings, just unmount lessfs, change the settings, mount lessfs and run defrag.

          • MasterTH says:

            ok, thanks,

            i already made the filesys with this bs-size, and will see how its going, but looks great.

            Which mount-options would you choose if you’re in my position?

            i tried with max_write/read=131072 but filesys wasn’t mounted, so i decided to mount with 4096 is this to low?

            At this moment 20GB are transfered and the speed is still at 2-3MB/s (its transfered over internet and i set the transfer-limit to 5mb/s)

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>