Lessfs-1.1.0-beta6 has been released.

Lessfs-1.1.0-beta6 is now available for download. This release brings much improved read performance. Lessfs-1.1.0-beta6 should be the last beta release. When no bugs are reported the coming week this code will be released as 1.1.0. People who would like to download a full comparison between zfs,sdfs and lessfs can download the iozone graphs from this location:

iozone – ZFS/SDFS/LessFS

Enjoy,

Mark Ruijter

This entry was posted in Uncategorized. Bookmark the permalink.

15 Responses to Lessfs-1.1.0-beta6 has been released.

  1. Marc says:

    Talk about timing…
    I had just found a fatal error when accessing .lessfs/lessfs_stats and a performance problem in beta4. Both are still in beta6. Patch comes via mail.

    The error: The memcpy in lessfs_read() might read past the end of config->lfsstats leading to a core dump when reading .lessfs/lessfs_stats. Using strncpy removes the error.

    The performance problem: I created a filesystem where lessfs_stats had nearly 26000 lines. After mounting it and without any writes to it, each call to lessfs_stats() in end_flush_commit() took 30 seconds (I added LINFO() calls before and after the call). I modified lessfs_stats to reduce the number of malloc/free calls, now lessfs_stats takes <1s.

    Marc

    • maru says:

      Hi Marc,

      Nice patch. I have included it for the next release.
      There is more work to do because I do not yet update the statistic when we truncate a file. ;-)

      I’ll see when I have time to fix that.

      Mark.

  2. wxp says:

    Hi, I cannot open the “iozone – ZFS/SDFS/LessFS”
    I donnot know why, thank you, can you send me a packet aout ” full comparison between zfs,sdfs and lessfs”?
    Thank you very much!

    • maru says:

      I just changed the incorrect link. You should now be able to access the iozone results.

      • wxp says:

        Thank you very much for your help!
        I have another question: Besides lessfs, there are opendedup (sdfs)、Bacula (SIS)、Zmanda and Nexenta (ZFS) on open source deduplication, I want to know whether the compared data on “iozone – ZFS/SDFS/Les” is about these companies or software?
        Thanks again!

        • wxp says:

          Hi, I am so sorry to disturb you!
          today, I have download lessfs software and compiled, tested it. I find it hard to run because there is little description on how to start and use it.
          Can you tell or send me some documents on how to use this software.
          Thank you!
          Wxp

  3. Pete says:

    Wow! I have not updated since version 1.0.8. It seems like there is a big improvement in deduplication. Maybe it’s just that now I see the stats. This latest version looks really great!

    • Mark says:

      Hi Pete,

      You are most likely using lessfs with lot’s of small files?
      In that case you are certainly right. Compression has improved a lot for that workload.

      Please note the lessfs-1.1.0-beta6 contains a nasty bug. Lessfs-1.1.0-beta4 does not contain the improved read cache code but is pretty stable as far as I can tell.

      • Pete says:

        I tested it with full SQL dumps from different days. It worked exceptionally. The total size of 3 backups was marginally more than the original size of one. Since the DBs are re-indexed occasionally, I expected the deduplication to not be that great.

        The first file was compressed down to 50% of the original size. The 2nd file was compressed to 50% of the compressed size of the 1st file. And the 3rd file was just a fraction of the 2nd (on disk). All three files were between 500 and 600MB, and the total size of the dedup folder (outside lessfs) was just over 600MB. I was impressed.

  4. Hans says:

    beta6 ietd and esx 3.5 ends in a lessfs crash after just formating 6 2TB luns. beta3 was stable enough to clone en boot some vm’s

    • maru says:

      Please try lessfs-1.1.0 :-)
      When you want maximum performance of lessfs+ietd you will need to patch ietd to write with a larger block size. ietd will normally use 4k blocks which degrades lessfs performance. This is also true for other fuse based filesystems.

      • Hans says:

        The first vm is cloned and lessfs 1.1.0 is still running :). du reports the 2T ietd sparce file as being 64k. Thats unlikle since there is a 80G vm in it.

        I can’t get esx 3.5 to work with iscsi blocksizes above 512 bytes. I don’t think any higher makes sense. When a vm writes 512 bytes to disk, esx should then fetch a 64k block over iscsi modify the 64k block with the 512 bytes and send it back. Thats a lot of network traffic, and does not improve the lessfs performance in any way.

        The only way out is if there would be a 64k block cache between ietd and lessfs, and if the dirty blocks would not be synced to disk/lessfs immedeately, so the vm can put more 512 byte writes in to it before its deduped.

        • wxp says:

          Hi, Hans
          Could you mind to give me some manual reference on lessfs?

          These days I try to use lessfs, but hard becuase of help-less

          Thank you!

  5. wxp says:

    Thank you very much for your help!
    I have another question: Besides lessfs, there are opendedup (sdfs)、Bacula (SIS)、Zmanda and Nexenta (ZFS) on open source deduplication, I want to know whether the compared data on “iozone – ZFS/SDFS/Les” is about these companies or software?
    Thanks again!

  6. wxp says:

    Hi, I am so sorry to disturb you!
    today, I have download lessfs software and compiled, tested it. I find it hard to run because there is little description on how to start and use it.
    Can you tell or send me some documents on how to use this software.
    Thank you!
    Wxp

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>