[Lustre-discuss] ost_write operation failed with -28 in 1.8.5 lustre client

[David Noriega]

We are running lustre 1.8.4 and I can confirm that I see this message
on one of our clients, the 'file server.' It serves up the lustre fs
to machines outside our network via samba and nfs. On other
clients(nodes in our compute cluster), I see the same message on a few
times, though it says "-19" or in one case "-107" as the error number.
Though just as they reported, we've had a few users say they have
gotten a message saying the filesystem is full, even though its not.

On Fri, Apr 29, 2011 at 10:04 AM, Rajendra prasad <rajendra.dn at gmail.com> wrote:
> Hi All,
>
> I am running lustre servers on 1.8.5 (recently upgraded from 1.8.2).
> Clients are still on 1.8.2 .
>
> I am getting the error "ost_write operation failed with -28" in the clients.
> Due to this i am getting error message as "No space left on the device"
> oftenly. As per lfs df -h output all the OSTs are occupied around 55% only.
>
> lfs df -h
> UUID                       bytes        Used   Available Use% Mounted on
> lustre-MDT0000_UUID        52.3G        4.2G       48.1G   8%
> /opt/lustre[MDT:0]
> lustre-OST0000_UUID       442.9G      245.6G      197.3G  55%
> /opt/lustre[OST:0]
> lustre-OST0001_UUID       442.9G      238.7G      204.3G  53%
> /opt/lustre[OST:1]
> lustre-OST0002_UUID       442.9G      243.2G      199.7G  54%
> /opt/lustre[OST:2]
> lustre-OST0003_UUID       442.9G      236.5G      206.5G  53%
> /opt/lustre[OST:3]
> lustre-OST0004_UUID       442.9G      234.8G      208.1G  53%
> /opt/lustre[OST:4]
> lustre-OST0005_UUID       442.9G      239.7G      203.3G  54%
> /opt/lustre[OST:5]
> lustre-OST0006_UUID       442.9G      237.2G      205.7G  53%
> /opt/lustre[OST:6]
> lustre-OST0007_UUID       442.9G      227.9G      215.0G  51%
> /opt/lustre[OST:7]
> filesystem summary:         3.5T        1.9T        1.6T  53% /opt/lustre
> As per the below bugzilla, i have upgraded one of the lustre client verstion
> to 1.8.5 but still the issue persist in that client.
>
>                 https://bugzilla.lustre.org/show_bug.cgi?id=22755
>
> Lustre clients are on Suse linux 10.1 . In order to install lustre client
> packages of 1.8.5, i have upgraded the Suse kernel also.
>
> I have also checked and found that no quota are enabled in the clients.
>
> lfs quota -u 36401 /opt/lustre
> Disk quotas for user 36401 (uid 36401):
>      Filesystem  kbytes   quota   limit   grace   files   quota   limit
> grace
>     /opt/lustre 127315748       0       0       - 1001083       0
> 0       -
> Below are the lustre client packages i have installed.
>
>
> lustre-client-modules-1.8.5-2.6.16_60_0.69.1_lustre.1.8.5_smp
>
> lustre-client-1.8.5-2.6.16_60_0.69.1_lustre.1.8.5_smp
>
>
>
> Suse kernel packages installed:
>
>
>
> kernel-default-2.6.16.60-0.69.1
>
> kernel-source-2.6.16.60-0.69.1
>
> kernel-smp-2.6.16.60-0.69.1
>
> kernel-syms-2.6.16.60-0.69.1
>
>
>
> Error:
>
> Apr 29 15:35:55 hostname kernel: LustreError: 11-0: an error occurred while
> communicating with 172.16.x.x at tcp. The ost_write operation failed with -28
>
> Apr 29 15:35:55 hostname kernel: LustreError: Skipped 9657 previous similar
> messages
>
> Apr 29 15:38:03 hostname kernel: LustreError: 11-0: an error occurred while
> communicating with 172.16.x.x at tcp. The ost_write operation failed with -28
>
>
>
> Kindly suggest.
>
>
>
> Regards,
>
> Prasad
>
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>
>



-- 
Personally, I liked the university. They gave us money and facilities,
we didn't have to produce anything! You've never been out of college!
You don't know what it's like out there! I've worked in the private
sector. They expect results. -Ray Ghostbusters