<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="EN-US" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal">Hello, T.H.Hsieh.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Your report sounds familiar to me. Although you are concerned about upgrades from 1.8.x, there were some other troubles reported when updating from earlier 2.x. You might want to take a closer look at
<a href="https://jira.whamcloud.com/browse/LU-13392">https://jira.whamcloud.com/browse/LU-13392</a>. I didnt review it deeply and maybe it isnt even closely related to your trouble, but you may find it helpful. In any case since you seem so willing to experiment,
Im curious what happens if you run LFSCK. LFSCK ought to be able to add and check FID-in-dirent and linkEA entries, both of which wont exist in a 1.8.x filesystem. I think Xyratex even released an upgrade tool to make these sorts of updates prior to mounting
under 2.x for the first time.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">-Cory<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal" style="margin-left:.5in">On 2/22/21, 1:22 AM, "lustre-discuss" <lustre-discuss-bounces@lists.lustre.org> wrote:<o:p></o:p></p>
</div>
<p class="MsoNormal" style="margin-left:.5in"><br>
<br>
<o:p></o:p></p>
<div>
<p class="MsoNormal" style="margin-left:.5in">Dear All,<br>
<br>
After some tests in these days, now I want to report what I have found<br>
about "moving data error" more detailly.<br>
<br>
As long as the Lustre file system was upgraded from the very old version<br>
1.8.8 to 2.12.6, the problem appears, where MDT is ldiskfs based. Although<br>
probably no body care about the very old version like 1.8.8, but in case<br>
some people might encounter similar scenario, then probably this message<br>
could provide some information.<br>
<br>
The problem I have found is: For any directories A/ and B/ created under<br>
Lustre-1.8.8, then after upgrading to Lustre-2.12.6, running the following<br>
"mv" command:<br>
<br>
mv A/file B/<br>
<br>
i.e., moving a file from A/ to B/, there is an error message and file<br>
moving failed:<br>
<br>
mv: cannot move 'A/file' to 'B/file': No data available<br>
<br>
<br>
I tested the following upgrade procedures:<br>
<br>
1. Lustre-1.8.8 -> Lustre-2.10.7 -> Lustre-2.12.6 (has problem)<br>
- Lustre file system created with Lustre-1.8.8, and directoies A/ and<br>
B/ are stored in the Lustre file system (A/ and B/ have some files).<br>
<br>
- Lustre-1.8.8 -> Lustre-2.10.7:<br>
After installing 2.10.7 of Lustre software and corresponding e2fsprogs:<br>
$ tunefs.lustre --writeconf /dev/sda1 (the MDT partition)<br>
$ tunefs.lustre --writeconf /dev/sda2 (the OST partition)<br>
$ tune2fs -O dirdata /dev/sda1<br>
$ tune2fs -O dirdata /dev/sda2<br>
<br>
Then mounting Lustre file system in the client, no problem at all.<br>
<br>
- Lustre-2.10.7 -> Lustre-2.12.6:<br>
After installing 2.12.6 of Lustre software and corresponding e2fsprogs:<br>
$ tunefs.lustre --writeconf /dev/sda1 (the MDT partition)<br>
$ tunefs.lustre --writeconf /dev/sda2 (the OST partition)<br>
<br>
Then mounting Lustre file system in the client, the "mv" problem appeared.<br>
<br>
2. Lustre-1.8.8 -> Lustre-2.12.6 (has problem)<br>
- Lustre file system created with Lustre-1.8.8, and directoies A/ and<br>
B/ are stored in the Lustre file system (A/ and B/ have some files).<br>
<br>
- Lustre-1.8.8 -> Lustre-2.12.6:<br>
After installing 2.12.6 of Lustre software and corresponding e2fsprogs:<br>
$ tunefs.lustre --writeconf /dev/sda1 (the MDT partition)<br>
$ tunefs.lustre --writeconf /dev/sda2 (the OST partition)<br>
$ tune2fs -O dirdata /dev/sda1<br>
$ tune2fs -O dirdata /dev/sda2<br>
<br>
Then mounting Lustre file system in the client, the "mv" problem appeared.<br>
<br>
3. Lustre-2.10.7 -> Lustre-2.12.6 (no problem)<br>
- Lustre file system created with Lustre-2.10.7, and directoies A/ and<br>
B/ are stored in the Lustre file system.<br>
<br>
- Lustre-2.10.7 -> Lustre-2.12.6:<br>
After installing 2.12.6 of Lustre software and corresponding e2fsprogs:<br>
$ tunefs.lustre --writeconf /dev/sda1 (the MDT partition)<br>
$ tunefs.lustre --writeconf /dev/sda2 (the OST partition)<br>
<br>
Then mounting Lustre file system in the client, no problem at all.<br>
<br>
<br>
So, something is missing when upgrading from 1.8.8, which does not cause<br>
problem in 2.10.7, but caused "mv" problem in 2.12.6. But so far I cannot<br>
figure out what has been missed.<br>
<br>
The way to cure this problem is simple. We only need to rename the directories<br>
created in 1.8.8, i.e.,<br>
<br>
mv A A.tmp<br>
mv A.tmp A<br>
mv B B.tmp<br>
mv B.tmp B<br>
<br>
Then the "mv" problem between A/ and B/ goes away. But that means we need to<br>
rename the whole Lustre directories tree in order to cure the problem of old<br>
directories created in 1.8.8. That would be a huge task. So far I did not<br>
find better way to resolve this problem.<br>
<br>
Any comments is very welcome.<br>
<br>
Best Regards,<br>
<br>
T.H.Hsieh<br>
<br>
On Fri, Feb 19, 2021 at 03:53:41AM +0800, Tung-Han Hsieh wrote:<br>
> Dear All,<br>
> <br>
> Recently we found a strange problem of the upgraded Lustre file system.<br>
> <br>
> We have several very old Lustre file systems with version 1.8.8. We first<br>
> upgraded them to 2.10.6. It seems ok. Then we upgraded them to 2.12.6.<br>
> Now we encouter a problem of moving file from directory A to directory B:<br>
> <br>
> mv A/file B/<br>
> <br>
> where both A/ and B/ are existing directories since version 1.8.8. The<br>
> error message says:<br>
> <br>
> mv: cannot move 'A/file' to 'B/file': No data available<br>
> <br>
> But we have no problem of opening A/file, copying A/file to B/file, and<br>
> no problem of running "mv A/file A/file1".<br>
> <br>
> More strange is, if we rename both directories, then the problem went away:<br>
> <br>
> mv A A1<br>
> mv B B1<br>
> mv A1/file B1/ # no problem at all<br>
> <br>
> But if we rename only one of them, the problem still remains.<br>
> <br>
> It seems that there is something missing during the upgrade. Could anyone<br>
> know how to fix it ?<br>
> <br>
> Thank you very much.<br>
> <br>
> T.H.Hsieh<br>
> _______________________________________________<br>
> lustre-discuss mailing list<br>
> lustre-discuss@lists.lustre.org<br>
> <a href="http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org">http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org</a>
<br>
_______________________________________________<br>
lustre-discuss mailing list<br>
lustre-discuss@lists.lustre.org<br>
<a href="http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org">http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org</a>
<o:p></o:p></p>
</div>
</div>
</body>
</html>