Dear All,
After some tests in these days, now I want to report what I have found
about "moving data error" more detailly.
As long as the Lustre file system was upgraded from the very old version
1.8.8 to 2.12.6, the problem appears, where MDT is ldiskfs based. Although
probably no body care about the very old version like 1.8.8, but in case
some people might encounter similar scenario, then probably this message
could provide some information.
The problem I have found is: For any directories A/ and B/ created under
Lustre-1.8.8, then after upgrading to Lustre-2.12.6, running the following
"mv" command:
mv A/file B/
i.e., moving a file from A/ to B/, there is an error message and file
moving failed:
mv: cannot move 'A/file' to 'B/file': No data available
I tested the following upgrade procedures:
1. Lustre-1.8.8 -> Lustre-2.10.7 -> Lustre-2.12.6 (has problem)
- Lustre file system created with Lustre-1.8.8, and directoies A/ and
B/ are stored in the Lustre file system (A/ and B/ have some files).
- Lustre-1.8.8 -> Lustre-2.10.7:
After installing 2.10.7 of Lustre software and corresponding e2fsprogs:
$ tunefs.lustre --writeconf /dev/sda1 (the MDT partition)
$ tunefs.lustre --writeconf /dev/sda2 (the OST partition)
$ tune2fs -O dirdata /dev/sda1
$ tune2fs -O dirdata /dev/sda2
Then mounting Lustre file system in the client, no problem at all.
- Lustre-2.10.7 -> Lustre-2.12.6:
After installing 2.12.6 of Lustre software and corresponding e2fsprogs:
$ tunefs.lustre --writeconf /dev/sda1 (the MDT partition)
$ tunefs.lustre --writeconf /dev/sda2 (the OST partition)
Then mounting Lustre file system in the client, the "mv" problem appeared.
2. Lustre-1.8.8 -> Lustre-2.12.6 (has problem)
- Lustre file system created with Lustre-1.8.8, and directoies A/ and
B/ are stored in the Lustre file system (A/ and B/ have some files).
- Lustre-1.8.8 -> Lustre-2.12.6:
After installing 2.12.6 of Lustre software and corresponding e2fsprogs:
$ tunefs.lustre --writeconf /dev/sda1 (the MDT partition)
$ tunefs.lustre --writeconf /dev/sda2 (the OST partition)
$ tune2fs -O dirdata /dev/sda1
$ tune2fs -O dirdata /dev/sda2
Then mounting Lustre file system in the client, the "mv" problem appeared.
3. Lustre-2.10.7 -> Lustre-2.12.6 (no problem)
- Lustre file system created with Lustre-2.10.7, and directoies A/ and
B/ are stored in the Lustre file system.
- Lustre-2.10.7 -> Lustre-2.12.6:
After installing 2.12.6 of Lustre software and corresponding e2fsprogs:
$ tunefs.lustre --writeconf /dev/sda1 (the MDT partition)
$ tunefs.lustre --writeconf /dev/sda2 (the OST partition)
Then mounting Lustre file system in the client, no problem at all.
So, something is missing when upgrading from 1.8.8, which does not cause
problem in 2.10.7, but caused "mv" problem in 2.12.6. But so far I cannot
figure out what has been missed.
The way to cure this problem is simple. We only need to rename the directories
created in 1.8.8, i.e.,
mv A A.tmp
mv A.tmp A
mv B B.tmp
mv B.tmp B
Then the "mv" problem between A/ and B/ goes away. But that means we need to
rename the whole Lustre directories tree in order to cure the problem of old
directories created in 1.8.8. That would be a huge task. So far I did not
find better way to resolve this problem.
Any comments is very welcome.
Best Regards,
T.H.Hsieh
On Fri, Feb 19, 2021 at 03:53:41AM +0800, Tung-Han Hsieh wrote:
> Dear All,
>
> Recently we found a strange problem of the upgraded Lustre file system.
>
> We have several very old Lustre file systems with version 1.8.8. We first
> upgraded them to 2.10.6. It seems ok. Then we upgraded them to 2.12.6.
> Now we encouter a problem of moving file from directory A to directory B:
>
> mv A/file B/
>
> where both A/ and B/ are existing directories since version 1.8.8. The
> error message says:
>
> mv: cannot move 'A/file' to 'B/file': No data available
>
> But we have no problem of opening A/file, copying A/file to B/file, and
> no problem of running "mv A/file A/file1".
>
> More strange is, if we rename both directories, then the problem went away:
>
> mv A A1
> mv B B1
> mv A1/file B1/ # no problem at all
>
> But if we rename only one of them, the problem still remains.
>
> It seems that there is something missing during the upgrade. Could anyone
> know how to fix it ?
>
> Thank you very much.
>
> T.H.Hsieh
> _______________________________________________
> lustre-discuss mailing list
> lustre-discuss at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org