[Lustre-discuss] missing ost's?

[Mag Gam]

do you have many small files?



On Tue, Jun 16, 2009 at 8:58 PM, Michael Di
Domenico<mdidomenico4 at gmail.com> wrote:
> On Tue, Jun 16, 2009 at 8:25 PM, Michael Di
> Domenico<mdidomenico4 at gmail.com> wrote:
>> I have a small lustre test cluster with eight OST's running.  The
>> servers were shut off over the weekend, upon turning them back on and
>> trying to startup lustre I seem to have lost my OST's.
>>
>> [root at node1 ~]$ lctl dl
>>  0 UP mgs MGS MGS 19
>>  1 UP mgc MGC192.168.1.254 at tcp 8acd9bf1-d1ca-8e26-1fad-bd2cf88a2957 5
>>  2 UP mdt MDS MDS_uuid 3
>>  3 UP lov lustre-mdtlov lustre-mdtlov_UUID 4
>>  4 UP mds lustre-MDT0000 lustre-MDT0000_UUID 3
>>  5 UP ost OSS OSS_uuid 3
>>  6 UP obdfilter lustre-OST0000 lustre-OST0000_UUID 3
>>
>> Everything in the messages log appears to be fine as if it was just a
>> normal startup of lustre, except for the below message.  I'm not sure
>> what logfile the error is referring to, and the message gives little
>> detail on where i should start looking for an error.
>>
>> Jun 16 20:13:55 node1-eth0 kernel: LustreError:
>> 3106:0:(llog_lvfs.c:577:llog_filp_open()) logfile creation
>> CONFIGS/lustre-MDT0000T: -28
>> Jun 16 20:13:55 node1-eth0 kernel: LustreError:
>> 3106:0:(mgc_request.c:1086:mgc_copy_llog()) Failed to copy remote log
>> lustre-MDT0000 (-28)
>
> Apparently from the lustre manual the -28 at the end of the line is an
> error code, which points to
>
> -28 -ENOSPC The file system is out-of-space or out of inodes. Use lfs df
> (query the amount of file system space) or lfs df -i
> (query the number of inodes).
>
> verified by
>
> [root at node1 ~]$ df -i
> Filesystem            Inodes   IUsed   IFree IUse% Mounted on
> /dev/md2             1280000   42132 1237868    4% /
> /dev/md0              255232      45  255187    1% /boot
> tmpfs                 124645       1  124644    1% /dev/shm
> /dev/md3               63872      24   63848    1% /mgs
> /dev/md4              255040  255040       0  100% /mdt
> /dev/md5             29892608   28726 29863882    1% /ost
>
> I only put 500k files in the filesystem i would not have thought the
> mdt would have used up the inodes that fast
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>