Hi,<br><br>Looks like your file system is running out of inodes. Please do lfs df -i on client and copy the output here.<br><br>best regards,<br><br>Wojciech<br><br><div class="gmail_quote">On 8 July 2010 13:47, Katya Tutlyaeva <span dir="ltr"><<a href="mailto:ordi@xgl.pereslavl.ru">ordi@xgl.pereslavl.ru</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">Hello,<br>
<br>
We have running lustre 1.8.1 and have met "No space lest on device"<br>
error when uploading 500 Gb small files (less then 100 Kb each).<br>
The problem seems to depends on the number of files. If we remove one<br>
file, we can create one new file, even with Gb size; but if we haven't<br>
remove something we can't create even very little file, as an example<br>
using touch command:<br>
[client]$ touch newfile<br>
touch: cannot touch `newfile': No space left on device<br>
[client]$ rm oldfile<br>
rm: remove regular file `qqq'? y<br>
[client]$ dd bs=100M count=10 < /dev/zero > qqq<br>
10+0 records in<br>
10+0 records out<br>
1048576000 bytes (1.0 GB) copied, 15.9882 s, 65.6 MB/s<br>
beda:/mnt/lustre# dd bs=100M count=10 < /dev/zero > qqq.tmp<br>
qqq.tmp: No space left on device.<br>
beda:/mnt/lustre# dd bs=100M count=10 < /dev/zero > qqq<br>
^C5+0 records in<br>
5+0 records out<br>
524288000 bytes (524 MB) copied, 7.66803 s, 68.4 MB/s<br>
<br>
But there are a lot of free space both on OSTs and MDS:<br>
[client]$ lfs df -h<br>
UUID bytes Used Available Use% Mounted on<br>
lustre-MDT0000_UUID 43.8G 813.4M 40.5G 1% /mnt/lustre[MDT:0]<br>
lustre-OST0000_UUID 867.7G 45.8G 777.8G 5% /mnt/lustre[OST:0]<br>
lustre-OST0001_UUID 916.9G 44.9G 825.4G 4% /mnt/lustre[OST:1]<br>
lustre-OST0002_UUID 916.9G 44.9G 825.5G 4% /mnt/lustre[OST:2]<br>
lustre-OST0003_UUID 916.9G 44.9G 825.4G 4% /mnt/lustre[OST:3]<br>
lustre-OST0004_UUID 916.9G 44.9G 825.4G 4% /mnt/lustre[OST:4]<br>
lustre-OST0005_UUID 916.9G 44.9G 825.4G 4% /mnt/lustre[OST:5]<br>
lustre-OST0006_UUID 916.9G 44.5G 825.9G 4% /mnt/lustre[OST:6]<br>
lustre-OST0007_UUID 916.9G 44.6G 825.8G 4% /mnt/lustre[OST:7]<br>
lustre-OST0008_UUID 916.9G 44.5G 825.8G 4% /mnt/lustre[OST:8]<br>
<br>
filesystem summary: 8.0T 403.8G 7.2T 4% /mnt/lustre<br>
<br>
There are not additional information in dmesg, no new messeges in<br>
/tmp/lustre-log also.<br>
<br>
Would you be kind enough to help me, to give some advice?<br>
Thank you!<br>
<br>
Best regards,<br>
Katya<br>
<br>
<br>
<br>
_______________________________________________<br>
Lustre-discuss mailing list<br>
<a href="mailto:Lustre-discuss@lists.lustre.org">Lustre-discuss@lists.lustre.org</a><br>
<a href="http://lists.lustre.org/mailman/listinfo/lustre-discuss" target="_blank">http://lists.lustre.org/mailman/listinfo/lustre-discuss</a><br>
</blockquote></div><br><br clear="all"><br>-- <br>--<br>Wojciech Turek<br><br>Assistant System Manager<br><br>High Performance Computing Service<br>University of Cambridge<br><br>