[Lustre-discuss] ost_write operation failed with -28 in 1.8.5 lustre client

[Rajendra prasad]

Hi All,

I am running lustre servers on 1.8.5 (recently upgraded from 1.8.2).
Clients are still on 1.8.2 .

I am getting the error "ost_write operation failed with -28" in the clients.
Due to this i am getting error message as "No space left on the device"
oftenly. As per lfs df -h output all the OSTs are occupied around 55% only.

lfs df -h
UUID                       bytes        Used   Available Use% Mounted on
lustre-MDT0000_UUID        52.3G        4.2G       48.1G   8%
/opt/lustre[MDT:0]
lustre-OST0000_UUID       442.9G      245.6G      197.3G  55%
/opt/lustre[OST:0]
lustre-OST0001_UUID       442.9G      238.7G      204.3G  53%
/opt/lustre[OST:1]
lustre-OST0002_UUID       442.9G      243.2G      199.7G  54%
/opt/lustre[OST:2]
lustre-OST0003_UUID       442.9G      236.5G      206.5G  53%
/opt/lustre[OST:3]
lustre-OST0004_UUID       442.9G      234.8G      208.1G  53%
/opt/lustre[OST:4]
lustre-OST0005_UUID       442.9G      239.7G      203.3G  54%
/opt/lustre[OST:5]
lustre-OST0006_UUID       442.9G      237.2G      205.7G  53%
/opt/lustre[OST:6]
lustre-OST0007_UUID       442.9G      227.9G      215.0G  51%
/opt/lustre[OST:7]
filesystem summary:         3.5T        1.9T        1.6T  53% /opt/lustre
As per the below bugzilla, i have upgraded one of the lustre client verstion
to 1.8.5 but still the issue persist in that client.

                https://bugzilla.lustre.org/show_bug.cgi?id=22755

Lustre clients are on Suse linux 10.1 . In order to install lustre client
packages of 1.8.5, i have upgraded the Suse kernel also.

I have also checked and found that no quota are enabled in the clients.

lfs quota -u 36401 /opt/lustre
Disk quotas for user 36401 (uid 36401):
     Filesystem  kbytes   quota   limit   grace   files   quota   limit
grace
    /opt/lustre 127315748       0       0       - 1001083       0
0       -
Below are the lustre client packages i have installed.


lustre-client-modules-1.8.5-2.6.16_60_0.69.1_lustre.1.8.5_smp

lustre-client-1.8.5-2.6.16_60_0.69.1_lustre.1.8.5_smp



Suse kernel packages installed:



kernel-default-2.6.16.60-0.69.1

kernel-source-2.6.16.60-0.69.1

kernel-smp-2.6.16.60-0.69.1

kernel-syms-2.6.16.60-0.69.1



Error:

Apr 29 15:35:55 hostname kernel: LustreError: 11-0: an error occurred while
communicating with 172.16.x.x at tcp. The ost_write operation failed with -28

Apr 29 15:35:55 hostname kernel: LustreError: Skipped 9657 previous similar
messages

Apr 29 15:38:03 hostname kernel: LustreError: 11-0: an error occurred while
communicating with 172.16.x.x at tcp. The ost_write operation failed with -28



Kindly suggest.



Regards,

Prasad
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20110429/d570c715/attachment.htm>