[Lustre-discuss] lustre no longer allows reads/writes (stopped working)?

[Robert Minvielle]

I have setup a lustre system for testing consisting of four OST's and one
MDT. It seems to work fine for about a day. At the end of about 24 hours,
the clients can no longer read or write the mount point (although a file
listing (ls) works). For example, a mkdir yields a "cannot create directory
'/datafs/temp': Identifier removed", and the temp dir does not exist. 
A file listing of the /datafs directory comes back complete and correct,
but if I try to ls a subdirectory it gives me the erorr "ls: /datafs/test2:
Identifier removed". 

The client is mounting the dir to /datafs. This worked fine eariler, I left
for the day, came back in and this error is occurring on all clients (albeit
I only have three clients for testing). All clients/servers are running
RHEL5, and the lustre was installed via rpms as per the manual. 

Out of curiosity, if I go to the server and do an ls on /mnt/data/mdt or
to the OST server and do an ls on /mnt/data/ost1, I get an error that
it is not a directory (although that could be normal, I am not sure). 

A cat of /proc/fs/lustre/devices on the mdt does not show anything out of place
(or at least, it is the same as when I started the lustre and mounted
the servers/clients) 

I have configured it all according to 
http://manual.lustre.org/manual/LustreManual16_HTML/ConfiguringLustreExamples.html#50548848_pgfId-1286919
as per section 6.1.1.2 Configuration Generation and Application, using one server
for the MGT and MDS, and I have four OSTs, just like the example. 

Has anyone seen this before? 

Robert