[Lustre-discuss] Unable to write to filesystem (device full)

[Nick Jennings]

Hi Everyone,

 RHEL5.3 - Lustre 1.6.7.2 - filesystem mounted as /data.

 I've got another odd problem this morning. Suddenly the file system was
reporting "no space left on device" failing every write attempt. A df
showed still 1.7TB free. 

 I first tried unmounting and remounting the /data filesystem. However
on remount I got the error:

# mount /data 
mount.lustre: mount idbn1 at tcp1:/clients at /data failed: Cannot send
after transport endpoint shutdown

 I then realized top was reporting one of the Lustre threads on the MDT
at 100% CPU. So I unmounted and remounted the MDT partition on that
server, and could then remount the /data filesystem on the client node. 

 Now everything seems to be back to normal operation. Any idea what was
going on there? I've got absolutely nothing from any of the log files. 

Thanks for any help,
-Nick


-  
Nick Jennings
Technical Director
Creative Motion Design
www.creativemotiondesign.com 

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20090929/f616a210/attachment.pgp>