Lessfs-1.2.0 has been released.

15 Responses to Lessfs-1.2.0 has been released.

1. Alex says:

   November 3, 2010 at 2:37 pm

   Updated, running on slow servers, but working well, and it seems faster than before

   Nice to get updates and enhancement.

   I use it for “slow/static” storage…and quite happy with lessfs, which allow me to store a lot of versions of archived datas on little servers

   For replication, as it’s the fastest way for me between slow comps, i shut down lessfs, copy databases (through rsync), et mount it back. Works nice.(i know about drbd, but too complicated for my case of use )

   Reply
   • Mark says:

     November 3, 2010 at 9:26 pm

     Did you try lessfs built-in replication?
     It might be more convenient then copying the files over.

     Reply
     • wxp says:

       November 8, 2010 at 3:24 am

       How to use lessfs built-in replication?
       Thank you

       Reply
       • Mark says:

         November 8, 2010 at 8:30 am

         Take a look in the examples directory.
         In short one configures one lessfs server as master and one as slave. The lessfs manual explains the directives.

         Reply
         • wxp says:

           November 10, 2010 at 8:13 am

           Thank you Mark
         • wxp says:

           November 10, 2010 at 8:53 am

           BTW, Mark, Would you mind telling me where the lessfs manual is? So for, What I can do is only by the direction of “man lessfs” command.
           Thank you!
   • John says:

     November 8, 2010 at 4:28 pm

     Any thoughts of adding an “ultra paranoid mode”. Where all committed blocks are double checked directly against the block they are being deduplicated against in case of a hash collision?

     I understand that the likelihood of a hash collision is remote, but in some cases being absolutely sure may be worth the extra performance hit of the read. Using a less demanding hash may help offset some of the cost in this case.

     Reply
2. Emmanuel Florac says:

   November 9, 2010 at 1:08 pm

   Hello, just to keep you inform of my test progress :

   My lessfs test system currently hold 12.64 TB of data in 588 thousand files. These actually consume 8.2 TB. Not a single hiccup to report from lessfs so far, good job I’m eager to see what happens when the FS will fill up ( 19 TB, still 4 or 5 weeks to go).

   Reply
3. wxp says:

   November 11, 2010 at 5:00 am

   Hi, Mark
   Are there some commands about lessfs to take snapshot?
   can you tell me something about how to take snapshot on lessfs

   Thank you

   Reply
4. wiloyee says:

   November 20, 2010 at 5:41 pm

   lessfschk is now faster —-
   Version 1.1.9 lessfschk -c /etc/lessfs.cfg takes 4 days
   Version 1.2.0 lessfschk -c /etc/lessfs.cfg takes 4 minutes

   very nice

   Reply
5. wxp says:

   November 26, 2010 at 11:03 am

   hi,Mark:
   When i use lessfs 1.1.9.6 and lessfs 1.2.0, I have meet this phenomenon:

   # ./lessfs example/lessfs-master.conf /sharedir / -d
   FUSE library version 2.8.4
   nullpath_ok: 0
   unique: 1, opcode: INIT (26), nodeid:0 , insize: 56
   INIT: 7.13
   flags=0x000007b
   max_readahead=0x0020000
   Bus error (core dumped)
   #

   What can I do for it, thank you!

   Reply
6. AJ says:

   December 3, 2010 at 4:26 pm

   I really like the idea of this, and have tried installing it on my CentOS box for testing.

   I’m looking for additional documentation on how to configure and test it. The first time I tried starting lessfs it dumped a few messages to the /var/log/messages file about not being able to recover the database (which didn’t exist yet), and there’s no “lessfs” proc running.

   I can copy files to the /fuse dir, but I don’t know if they’re just being copied to the ext3 fs or what’s going on.

   Is there supposed to be a lessfs proc running as a daemon? What does lessfsck do? What are the switches? How can I test deduplication/space savings?

   Reply
   • AJ says:

     December 3, 2010 at 4:48 pm

     OK, so I was missing the mklessfs step. Need documentation.

     See the /fuse fs now mounted as fuse. Would like to understand how to test further. For example, I copied the FAQ document three times to the /fuse dir (with three different names). Theoretically there should only be one copy of those blocks/bytes, right? An ls -l shows them all (and it probably should), so how do I check what disk space is actually being used and any other possible metrics???

     Thanks again,
     AJ

     Reply
     • Gianni says:

       December 4, 2010 at 11:25 am

       you can find stats info in /fuse/.lessfs/lessfs_stats

       i create this little script to make some number from that file:

       gianni@laptop:~$ cat lessfs-stat.sh
       #!/bin/bash
       let linee=$(wc -l $1|cut -d” ” -f1)-1

       echo “Numero file:” $linee;
       tail -n $linee $1|
       for i in `seq 1 $linee`
       do
       read a b c d
       # echo b $b c $c
       let ORI=$ORI+b
       let DEDUP=$DEDUP+c

       #echo $i ORI $ORI DEDUP $DEDUP

       if [ $i -eq “$linee” ]; then
       # echo|awk ‘{ print ($c/$b*100)}’
       echo “Valori finali:” deduplicati=$DEDUP originale=$ORI;
       echo -n “Percentuale: “;
       # echo PERC=$((${DEDUP}/${ORI}*100));
       echo scale\=2\;$DEDUP/$ORI*100|bc;
       # echo perc $PERC;
       #echo |awk ‘{ print expr ${DEDUP}/${ORI}*100 } ‘
       fi;
       done
       #echo $ORI
       #echo $DEDUP

       To use

       ./lessfs-stat.sh /fuse/.lessfs/lessfs_stats

       i keep that script in/usr/local/bin

       hope usefull

       bye

       G

       Reply
7. Hubert Kario says:

   December 5, 2010 at 7:45 pm

   little benchmarks (Thinkpad X61s with 500G 5400rpm drive), config file is the sample.cfg one, lessfs files (/data) reside on btrfs volume on LVM, /home is xfs. Overall the results are quite nice (speedup is noticeable) but I really miss a snapshot tool…

   # hdparm -tT /dev/sda

   /dev/sda:
   Timing cached reads: 5302 MB in 2.00 seconds = 2656.97 MB/sec
   Timing buffered disk reads: 222 MB in 3.02 seconds = 73.48 MB/sec

   # dft-seeker 1 /dev/sda
   Seeding PRNGs.done.
   Performing random access timing… done.
   Cleaning up.
   thread 0: 3221 reads, 18.938218 ms random access time
   1 threads: 3221 reads, 18.938218 ms random access time, 18.938218 ms mean random access time

   # dft-seeker 8 /dev/sda
   Seeding PRNGs……..done.
   Performing random access timing… done.
   Cleaning up……..
   thread 0: 587 reads, 103.918228 ms random access time
   thread 1: 583 reads, 104.631218 ms random access time
   thread 2: 613 reads, 99.510604 ms random access time
   thread 3: 602 reads, 101.328904 ms random access time
   thread 4: 566 reads, 107.773852 ms random access time
   thread 5: 599 reads, 101.836394 ms random access time
   thread 6: 571 reads, 106.830123 ms random access time
   thread 7: 598 reads, 102.006689 ms random access time
   8 threads: 4719 reads, 12.926467 ms random access time, 103.479501 ms mean random access time

   # du -hs /home/test/ /home/tomato/workspace/
   5.7G /home/test/
   1.3G /home/tomato/workspace/
   # find /home/test/ -type f | wc -l
   2412
   # find /home/tomato/workspace/ -type f | wc -l
   42593
   # time cp -a /home/test/ /mnt/lessfs/home/

   real 13m33.465s
   7.18 MiB/s
   2.967 files/s

   # time cp -a home/ snapshots/home-2010-12-03-18-37-10

   real 6m54.212s
   14.1 MiB/s
   5.826 files/s

   # mkdir /mnt/lessfs/home/tomato
   # time cp -a /home/tomato/workspace/ /mnt/lessfs/home/tomato/

   real 15m26.656s
   1.43 MiB/s
   45.997 files/s

   # time cp -a home/ snapshots/home-2010-12-03-19-09-17

   real 18m27.182s
   6.47 MiB/s
   40.655 files/s

   # uname -a
   Linux localhost 2.6.36-ARCH #1 SMP PREEMPT Wed Nov 24 06:44:11 UTC 2010 i686 Intel(R) Core(TM)2 Duo CPU L7500 @ 1.60GHz GenuineIntel GNU/Linux
   # cat /proc/mounts | grep lessfs
   lessfs /mnt/lessfs fuse.lessfs rw,nosuid,nodev,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072

   Reply