Posted 2011-08-21 13:13:00 GMT
BTRFS is a next generation filesystem for Linux that has some pleasant features which set it apart from ext4. For example, checksums, snapshots, and online defragmentation. In many respects it is inspired by ZFS.
I was converting a filesystem from ext4 to btrfs when I came upon a problem: it seems that BTRFS supports only 256 hard-links for files in the same directory (that is, up to 256 names for the same inode in one directory) though across directories you can have up to 232 names for the same inode.
This restriction causes btrfs-convert 0.19 to crash out with a segfault and no helpful message: something like btrfs-convert: segfault at ffffffffcfb25fb9 ip 000000000040f9f1 sp 00007fffddefb398 error 6 in btrfs-convert[400000+21000].
It seems a priori that there should not be any need for more than 256 names for the same file in the same directory. However, the GNUS mailreader's nnmaildir backend uses hardlinks to mark email messages read, and instead of creating a separate inode for each marked message, uses a hardlink to a single markfile. This means that there maybe thousands of hardlinks to the same inode in a single directory.
You can use the command find -type f -links +255 to locate such troublesome files, however this will not indicate whether the links are in the same directory or not, so there can be false positives.
The reason designing file-systems and other system infrastructure is hard is that the range of at first glance nonsensical configurations that people will expect to work seamlessly. I hope that as BTRFS matures it will be able to handle these use-cases — the nnmaildir link hack to use the filesystem as a mark database is not altogether unreasonable.
Post a comment
Hi,
There are more real world applications that can not work with BTRFS due to the hardlink limit.
I use storeBackup which makes hardlinks to files between backups and machines to deduplicate. I want to move to BTRFS for my disk based backup so that it can benefit from the checksumming.
But BTRFS cannot handle the amount of hardlinks.
Shame.
Posted 2011-09-09 10:23:00 GMT by Anonymous from 212.238.246.219
I *hope* that someone is reporting this as a bug against btrfs. Simply redesigning apps is not the answer since a lot of users depend on legacy apps and conversion would be a very big deal. When a new file system lacks support for something that extant file systems currently broadly support, that is a bug, and it should get fixed.
Posted 2011-09-17 16:10:24 GMT by Anonymous from 50.0.98.68
More real-world cases causing upgrade failures:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=642603
"git - too many hardlinks in /usr/lib/git-core"
Posted 2011-09-24 12:54:23 GMT by Anonymous from 188.29.76.225
It's really a shame btrfs can't handle such many hard links, I'm unable to test it on my servers because of that
Posted 2012-02-12 19:45:58 GMT by Anonymous
To anyone who is still facing the problem , I can confirm that the problem has been fixed as of kernel 3.7-rc6(at least) . The fix is in the btrfs changelist for 3.7 .
Check the extended backref patch @ http://thread.gmane.org/gmane.comp.file-systems.btrfs/19113/focus=19222 for details.
However to actually use the "extended iref"'s you have to use the patch for btrfs-progs available at the above URL and then REFORMAT your btrfs partition with the patched mkfs.btrfs . Apparently , since this fix requires changes to the disk format the only way is to mkfs again.
Posted 2012-11-26 15:35:56 GMT by Anonymous from 122.165.6.161
I only just recently ran into this issue. You can fix it with "btrfs -r /dev/sdxn" instead of a reformat, having checked first that you're using Linux 3.7 or higher. It can't be done online though, so you may need a live CD.
Posted 2017-11-19 18:40:17 GMT by Ross