[Lustre-discuss] Read-only file system

[Mag Gam]

We have been experiecing problems recently, where our Lustre
filesystem is becoming read-only (we can't even see our data).

For example when I invoke 'ls' or 'find'

ls: .: Read-only file system
find: cannot get current directory: Read-only file system

The client version is:

lustre: 1.6.6
kernel: patchless
build:  1.6.6-19691231190000-PRISTINE-.usr.src.linux-2.6.18-92.1.10.el5


The MDS/OST version is:
lustre: 1.6.4.3
kernel: patchless

On the MDS:
 lctl dl
-bash-3.1$ lctl dl
  0 UP mgs MGS MGS 23
  1 UP mgc MGC141.128.90.153 at tcp caf62255-4f22-53a3-25d9-c7f9e6c31277 5
  2 UP mdt MDS MDS_uuid 3
  3 UP lov lfs002-mdtlov lfs002-mdtlov_UUID 4
  4 UP mds lfs002-MDT0000 lfs002-MDT0000_UUID 21
  5 UP osc lfs002-OST0000-osc lfs002-mdtlov_UUID 5
  6 UP osc lfs002-OST0001-osc lfs002-mdtlov_UUID 5
  7 UP ost OSS OSS_uuid 3
  8 UP obdfilter lfs002-OST0001 lfs002-OST0001_UUID 23


It interesting because we have another lustre filesystem where we can
see things and everything is working properly.  Is there something I
am missing?
TIA