[Lustre-discuss] Swap over lustre

[John Hanks]

Hi,

I've been trying to get swap on lustre to work with not much success
using blockdev_attach and the resulting lloop0 device and using
losetup and the resulting loop device. This thread
(http://www.mail-archive.com/lustre-discuss@lists.lustre.org/msg00856.html)
claims that it works, but in all my attempts almost as soon as swap is
used (testing with memhog), the host hangs. In some cases it hangs
hard, but on occasion if I'm patient enough the OOM will eventually
kill something and the node will become responsive again. If I
carefully increase memory with each successive memhog run I can get
some pages to swap, but any real pressure always results in a hang.
I'm attempting this on Redhat EL 5.6 with lustre 1.8.4 patchless
client over IB.

DIgging around search results for "swap over NFS" I've found a lot fo
discussion about race conditions and different patches to address
this, but CONFIG_NFS_SWAP seems to be missing from the redhat kernel.
And upon trying swap to an NFS server, I see the same behavior. Is
swap to a network device doomed to always fail on Redhat EL 5 and if
not, does anyone have a recipe for getting swap on lustre to work?

I've also fiddled with min_free_kbytes and swappiness in an attempt to
induce swapping before the node's memory is actually all gone but all
this results in is an earlier hang with less memory having been used.

Thanks,

jbh