[Lustre-discuss] Swap on Lustre (was: Client is not accesible when OSS/OST server is down)

Andreas Dilger adilger at sun.com
Fri May 2 11:25:03 PDT 2008


On Apr 29, 2008  09:05 -0700, Kilian CAVALOTTI wrote:
> On Tuesday 29 April 2008 07:53:01 am Brian J. Murrell wrote:
> > Unless you are using Lustre for your root and/or usr filesystem
> > and/or for swap, Lustre should not hang a machine completely.
> 
> I was precisely wondering if it was possible to use a file residing on a 
> Lustre filesystem as a swap file. I tried the basic steps without any 
> success.
> 
> On a regular ext3 fs, no problem:
> 
> /tmp # dd if=/dev/zero of=./swapfile  bs=1024 count=1024
> 10240+0 records in
> 10240+0 records out
> /tmp # mkswap ./swapfile
> Setting up swapspace version 1, size = 104853 kB
> /tmp # swapon -a ./swapfile
> /tmp # swapon -s
> Filename                       Type            Size    Used    Priority
> /dev/sda3                      partition       4096564 204     -1
> /tmp/swapfile                  file            102392  0       -2
> 
> But on a Lustre mount:
> 
> # cd /scratch
> /scratch # grep /scratch /proc/mounts
> 10.10.50.2 at o2ib:/scratch /scratch lustre rw 0 0
> /scratch # dd if=/dev/zero of=./swapfile  bs=1024 count=1024
> 10240+0 records in
> 10240+0 records out
> /scratch # mkswap ./swapfile
> Setting up swapspace version 1, size = 104853 kB
> /scratch # swapon -a ./swapfile
> swapon: ./swapfile: Invalid argument

Note that in 1.6.4+ there is an interface to export a block device
more directly from Lustre instead of using the loopback driver on top
of the client filesystem.  This is the "llite_loop" module and is
configured like:

	lctl blockdev_attach {loopback_filename} {blockdev_filename}

where {loopback_filename} is the file that should be turned into a
block device (it can be sparse if desired) and {blockdev_filename}
is the full filename that the new block device should be created at.
To clean up the device use:

	lctl blockdev_detach {blockdev_filename}

Note that using this block device for swap hasn't been very successful
in our testing so far, but we also haven't done a great deal of real
world testing only "allocate a ton of RAM and dirty it all as fast
as possible", which isn't a very realistic usage.  Feedback would
be welcome.

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.




More information about the lustre-discuss mailing list