This version of Lessfs comes with a significant number of changes.
The multifile_io backend is now fully functional with replication.
By default Lessfs will now compile with berkeleydb instead of tokyocabinet. Lessfs requires berkeleydb >= 4.8.
Batch replication has been extensively tested and improved. Some nasty problems that could occur when either the master of the slave suffered an unclean shutdown have been solved.
In the SPEC files snappy compression is now enabled by default.
Lessfs-1.6.0 will be the last of the 1.x serie releases that introduces new features. From now on the Lessfs-1.x series will remain frozen and new releases will only contain bug fixes.
Have you had time to test LZ4 and compare it to Snappy ?
Thanks for making me aware that there even is something like LZ4.
I just looked into it and it appears to be promising. On average the speeds appear to be 48% faster then snappy.
This is amazing since snappy has proved to be 30% faster then QuickLZ which once was the fastest compression protocol.
http://fastcompression.blogspot.com/
https://issues.apache.org/jira/browse/HADOOP-7657?focusedCommentId=13170733&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13170733
Porting a new compression algorithm to Lessfs is very simple.
So when I have time I’ll give it a go so we can see if LZ4 lives up to it’s promise.
Thanks,
Mark.
Great ! Sounds promising…
“By default Lessfs will now compile with berkeleydb instead of tokyocabinet.” – Could you be more specific, what were the reasons for such decision? (Or link to related materials, like mailinglist discussions?)
Currently there is successor of Tokyo Cabinet , which is Kyoto Cabinet. Have you considered it ?
What are practical implication of switching LessFS from Tokyo Cabinet to berkeleydb ? I am most curious about performance.
Thanks in advance for all info and pointing to right places. I’d love to read more about details related under the cover of this stuff.
Best,
Grzegorz