John Fremlin's blog: BTRFS hard links in the same directory limit is too small

Posted 2011-08-21 13:13:00 GMT

BTRFS is a next generation filesystem for Linux that has some pleasant features which set it apart from ext4. For example, checksums, snapshots, and online defragmentation. In many respects it is inspired by ZFS.

I was converting a filesystem from ext4 to btrfs when I came upon a problem: it seems that BTRFS supports only 256 hard-links for files in the same directory (that is, up to 256 names for the same inode in one directory) though across directories you can have up to 232 names for the same inode.

This restriction causes btrfs-convert 0.19 to crash out with a segfault and no helpful message: something like btrfs-convert: segfault at ffffffffcfb25fb9 ip 000000000040f9f1 sp 00007fffddefb398 error 6 in btrfs-convert[400000+21000].

It seems a priori that there should not be any need for more than 256 names for the same file in the same directory. However, the GNUS mailreader's nnmaildir backend uses hardlinks to mark email messages read, and instead of creating a separate inode for each marked message, uses a hardlink to a single markfile. This means that there maybe thousands of hardlinks to the same inode in a single directory.

You can use the command find -type f -links +255 to locate such troublesome files, however this will not indicate whether the links are in the same directory or not, so there can be false positives.

The reason designing file-systems and other system infrastructure is hard is that the range of at first glance nonsensical configurations that people will expect to work seamlessly. I hope that as BTRFS matures it will be able to handle these use-cases — the nnmaildir link hack to use the filesystem as a mark database is not altogether unreasonable.

Post a comment