<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
It seems possible that some automation has accidentally reformatted or at least repartitioned your MDT device? Most Lustre installations do *not* configure a DOS/GPT partition on the device, because this offsets allocations in the filesystem from the underlying
RAID stripe alignment.
<div class=""><br class="">
</div>
<div class="">Like I wrote, if you run "blkid" on the node it should tell you if it can find the MDT volume on one of the devices attached to the system.</div>
<div class=""><br class="">
</div>
<div class="">Failing that, there is a "findsuper" tool in the e2fsprogs *source tree* that can be built to scan the whole device looking for ext4 filesystem superblock signatures, and if they are found it would tell you what device offset that filesystem should
start at, so you might be able to rewrite/remove the partition table to get the MDT filesystem back (depending on what else was written to that disk in the meantime).<br class="">
<div><br class="">
<blockquote type="cite" class="">
<div class="">On Nov 30, 2021, at 06:48, STEPHENS, DEAN - US <<a href="mailto:dean.stephens@caci.com" class="">dean.stephens@caci.com</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div class="WordSection1" style="page: WordSection1; caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
You are right I am new to setting up Lustre and the system that we are working on has been around for a while.<o:p class=""></o:p></div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
I have verified that the LUN that is attached to the VM is the correct one using the UUID and WWID as reference. The interesting thing is that when I attach the LUN as a disk to the VM it immediately creates the partition of /dev/sdb1. When I attach the OSS
LUNs to the OSS servers they do not create the partition they are just /dev/sdX. We are running puppet on these servers so I am not sure if that is what is creating the /dev/sdb1 partition or not.<o:p class=""></o:p></div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Dean<o:p class=""></o:p></div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
<div class="">
<div style="border-style: solid none none; border-top-width: 1pt; border-top-color: rgb(225, 225, 225); padding: 3pt 0in 0in;" class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<b class="">From:</b><span class="Apple-converted-space"> </span>Andreas Dilger <<a href="mailto:adilger@whamcloud.com" style="color: purple; text-decoration: underline;" class="">adilger@whamcloud.com</a>><span class="Apple-converted-space"> </span><br class="">
<b class="">Sent:</b><span class="Apple-converted-space"> </span>Monday, November 29, 2021 2:04 PM<br class="">
<b class="">To:</b><span class="Apple-converted-space"> </span>STEPHENS, DEAN - US <<a href="mailto:dean.stephens@caci.com" style="color: purple; text-decoration: underline;" class="">dean.stephens@caci.com</a>><br class="">
<b class="">Cc:</b><span class="Apple-converted-space"> </span>Colin Faber <<a href="mailto:cfaber@gmail.com" style="color: purple; text-decoration: underline;" class="">cfaber@gmail.com</a>>;<span class="Apple-converted-space"> </span><a href="mailto:lustre-discuss@lists.lustre.org" style="color: purple; text-decoration: underline;" class="">lustre-discuss@lists.lustre.org</a><br class="">
<b class="">Subject:</b><span class="Apple-converted-space"> </span>Re: [lustre-discuss] Lustre and server upgrade<o:p class=""></o:p></div>
</div>
</div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
On Nov 29, 2021, at 13:10, STEPHENS, DEAN - US <<a href="mailto:dean.stephens@caci.com" style="color: purple; text-decoration: underline;" class="">dean.stephens@caci.com</a>> wrote:<o:p class=""></o:p></div>
<div class="">
<blockquote style="margin-top: 5pt; margin-bottom: 5pt;" class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Unfortunately I do not have that as that was done years ago before I got involved. Let me ask a few questions here:<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<ol start="1" type="1" style="margin-bottom: 0in; margin-top: 0in;" class="">
<li class="MsoListParagraph" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;">
If I were to do a mkfs.lustre on the MDS nodes, that will destroy all existing data. How do the MDS nodes know how to see the OSS nodes and the lustre filesystem to derive the meta data?<o:p class=""></o:p></li><li class="MsoListParagraph" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;">
Since the lustre meta data LUN is 1.1TB, any idea how long it will take to do the mkfs and rebuild the lustre metadata?<o:p class=""></o:p></li><li class="MsoListParagraph" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;">
Do you think that this will resolve the error that I am seeing in the OSS nodes, “Invalid argument” after seeing the lustre filesystem per the output from before?<o:p class=""></o:p></li></ol>
</div>
</blockquote>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
</div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
I think you misunderstand how Lustre is working. If you reformat the MDT then your filesystem will be totally lost, so definitely don't do that. There is no "rebuild the metadata from the OSTs" functionality available, the same as if you reformat a local
filesystem there is no way it will rebuild itself, unless you restore it from backup.<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
It is possible that you are not checking the correct block device? The output below shows that /dev/sdb1 is an LVM volume, so it would appear that the MDT is an LV like /dev/vg<something>/lv<something>? Running "blkid" on the system might tell you what the
volume name is.<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div class="">
<blockquote style="margin-top: 5pt; margin-bottom: 5pt;" class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div style="border-style: solid none none; border-top-width: 1pt; border-top-color: rgb(225, 225, 225); padding: 3pt 0in 0in;" class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<b class="">From:</b><span class="apple-converted-space"> </span>Colin Faber <<a href="mailto:cfaber@gmail.com" style="color: purple; text-decoration: underline;" class="">cfaber@gmail.com</a>><span class="apple-converted-space"> </span><br class="">
<b class="">Sent:</b><span class="apple-converted-space"> </span>Monday, November 29, 2021 12:12 PM<br class="">
<b class="">To:</b><span class="apple-converted-space"> </span>STEPHENS, DEAN - US <<a href="mailto:dean.stephens@caci.com" style="color: purple; text-decoration: underline;" class="">dean.stephens@caci.com</a>><br class="">
<b class="">Cc:</b><span class="apple-converted-space"> </span>Andreas Dilger <<a href="mailto:adilger@whamcloud.com" style="color: purple; text-decoration: underline;" class="">adilger@whamcloud.com</a>>;<span class="apple-converted-space"> </span><a href="mailto:lustre-discuss@lists.lustre.org" style="color: purple; text-decoration: underline;" class="">lustre-discuss@lists.lustre.org</a><br class="">
<b class="">Subject:</b><span class="apple-converted-space"> </span>Re: [lustre-discuss] Lustre and server upgrade<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Well, all signs indicate that this target has not been prepared for lustre. Can you post the output of your original formatting command?<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div class="">
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
On Mon, Nov 29, 2021 at 8:26 AM STEPHENS, DEAN - US <<a href="mailto:dean.stephens@caci.com" style="color: purple; text-decoration: underline;" class="">dean.stephens@caci.com</a>> wrote:<o:p class=""></o:p></div>
</div>
</div>
<blockquote style="border-style: none none none solid; border-left-width: 1pt; border-left-color: rgb(204, 204, 204); padding: 0in 0in 0in 6pt; margin: 5pt 0in 5pt 4.8pt;" class="">
<div class="">
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
That was my fault. I did not use the correct command.<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
The output of the lsblk command showing the attached storage (I did not created the /dev/sdb1 partition and the OSS servers do not have /dev/sdX1 partitions, they are just the /dev/sbX):<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Sdb 1.1TB 0 disk<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Sdb1 1.1TB 0 part<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Here is the output of the tune2fs -l /dev/sdb and /dev/sdb1 as they are different:<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Tune2fs -l /dev/sdb<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Tune2fs: Bad magic number in super-block while trying to open /dev/sdb<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Found a gpt partition table in /dev/sdb<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Tune2fs -l /dev/sdb1<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Tune2fs: Bad magic number in super-block while trying to open /dev/sdb1<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
/dev/sdb1 contains a LVM2_member file system<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<br class="">
Dean<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div style="border-style: solid none none; border-top-width: 1pt; padding: 3pt 0in 0in; border-color: currentcolor;" class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<b class="">From:</b><span class="apple-converted-space"> </span>Colin Faber <<a href="mailto:cfaber@gmail.com" target="_blank" style="color: purple; text-decoration: underline;" class="">cfaber@gmail.com</a>><span class="apple-converted-space"> </span><br class="">
<b class="">Sent:</b><span class="apple-converted-space"> </span>Monday, November 29, 2021 8:02 AM<br class="">
<b class="">To:</b><span class="apple-converted-space"> </span>STEPHENS, DEAN - US <<a href="mailto:dean.stephens@caci.com" target="_blank" style="color: purple; text-decoration: underline;" class="">dean.stephens@caci.com</a>><br class="">
<b class="">Cc:</b><span class="apple-converted-space"> </span>Andreas Dilger <<a href="mailto:adilger@whamcloud.com" target="_blank" style="color: purple; text-decoration: underline;" class="">adilger@whamcloud.com</a>>;<span class="apple-converted-space"> </span><a href="mailto:lustre-discuss@lists.lustre.org" target="_blank" style="color: purple; text-decoration: underline;" class="">lustre-discuss@lists.lustre.org</a><br class="">
<b class="">Subject:</b><span class="apple-converted-space"> </span>Re: [lustre-discuss] Lustre and server upgrade<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div class="">
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Hi, tune2fs and tunefs.lustre are different tools which yield different information about the block device. I'd like to be sure that we're working with the right type of device here and basic ext4/ldiskfs data is present (nevermind if the lustre configuration
data is present)<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
-cf<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div class="">
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
On Mon, Nov 29, 2021 at 6:22 AM STEPHENS, DEAN - US <<a href="mailto:dean.stephens@caci.com" target="_blank" style="color: purple; text-decoration: underline;" class="">dean.stephens@caci.com</a>> wrote:<o:p class=""></o:p></div>
</div>
</div>
<blockquote style="border-style: none none none solid; border-left-width: 1pt; padding: 0in 0in 0in 6pt; margin: 5pt 0in 5pt 4.8pt; border-color: currentcolor currentcolor currentcolor rgb(204, 204, 204);" class="">
<div class="">
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
No worries. I was out last week with Thanksgiving and was not able to respond to your email.<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
The tunefs.lustre /dev/sdb says this:<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Checking for existing lustre data: not found<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Tunefs.lustre FATAL: Device /dev/sdb has not been formatted with mkfs.lustre<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Tunefs.lustre: existing with 19 (no such device)<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
I think that this is a bit weird as the “disk” /dev/sdb is a LUN from a SAN that is attached to the VM and is the same LUN that was attached to the RHEL6 VM (verified the WWID is the same). The LUNs that are attached to the OSS servers have do not seem to have
that same issue as the tunefs.lustre command comes back seeing lustre data.<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Any idea on what the “Invalid argument” means from the OSS tunefs.lustre command is?<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Dean<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div style="border-style: solid none none; border-top-width: 1pt; padding: 3pt 0in 0in; border-color: currentcolor;" class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<b class="">From:</b><span class="apple-converted-space"> </span>Colin Faber <<a href="mailto:cfaber@gmail.com" target="_blank" style="color: purple; text-decoration: underline;" class="">cfaber@gmail.com</a>><span class="apple-converted-space"> </span><br class="">
<b class="">Sent:</b><span class="apple-converted-space"> </span>Wednesday, November 24, 2021 8:35 PM<br class="">
<b class="">To:</b><span class="apple-converted-space"> </span>STEPHENS, DEAN - US <<a href="mailto:dean.stephens@caci.com" target="_blank" style="color: purple; text-decoration: underline;" class="">dean.stephens@caci.com</a>><br class="">
<b class="">Cc:</b><span class="apple-converted-space"> </span>Andreas Dilger <<a href="mailto:adilger@whamcloud.com" target="_blank" style="color: purple; text-decoration: underline;" class="">adilger@whamcloud.com</a>>;<span class="apple-converted-space"> </span><a href="mailto:lustre-discuss@lists.lustre.org" target="_blank" style="color: purple; text-decoration: underline;" class="">lustre-discuss@lists.lustre.org</a><br class="">
<b class="">Subject:</b><span class="apple-converted-space"> </span>Re: [lustre-discuss] Lustre and server upgrade<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div class="">
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
what does tune2fs report for /dev/sdb on the MDS?<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
(Also sorry, this somehow got lost in my inbox)<o:p class=""></o:p></div>
</div>
</div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div class="">
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
On Mon, Nov 22, 2021 at 8:57 AM STEPHENS, DEAN - US <<a href="mailto:dean.stephens@caci.com" target="_blank" style="color: purple; text-decoration: underline;" class="">dean.stephens@caci.com</a>> wrote:<o:p class=""></o:p></div>
</div>
</div>
<blockquote style="border-style: none none none solid; border-left-width: 1pt; padding: 0in 0in 0in 6pt; margin: 5pt 0in 5pt 4.8pt; border-color: currentcolor currentcolor currentcolor rgb(204, 204, 204);" class="">
<div class="">
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Colin and Andreas, so to clarify some points for you, This is what I am seeing:<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; text-indent: 0.5in;" class="">
Rpm -qa | grep lustre<o:p class=""></o:p></div>
</div>
<div style="margin-left: 0.5in;" class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Kmod_lustre-2.12.6-1.el7.x86_64<o:p class=""></o:p></div>
</div>
<div style="margin-left: 0.5in;" class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Lustre-iokit-2.12.6-1.el7.x86_64<o:p class=""></o:p></div>
</div>
<div style="margin-left: 0.5in;" class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Lustre_test-2.12.6-1.el7.x86_64<o:p class=""></o:p></div>
</div>
<div style="margin-left: 0.5in;" class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Kernel-devel-3.10.0-1160.2.el7_lustre.x86_64<o:p class=""></o:p></div>
</div>
<div style="margin-left: 0.5in;" class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Lustre-osd-ldiskfs-2.12.6-1.el7.x86_64<o:p class=""></o:p></div>
</div>
<div style="margin-left: 0.5in;" class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Kmod-lustre-osd-ldiskfs-2.12.6-1.el7.x86_64<o:p class=""></o:p></div>
</div>
<div style="margin-left: 0.5in;" class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Kmod-lustre-tests-2.12.6-1.el7.x86_64<o:p class=""></o:p></div>
</div>
<div style="margin-left: 0.5in;" class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
lustre-resource-agents-2.12.6-1.el7.x86_64<o:p class=""></o:p></div>
</div>
<div style="margin-left: 0.5in;" class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
kernel-3.10.0-1160.2.el7_lustre.x86_64<o:p class=""></o:p></div>
</div>
<div style="margin-left: 0.5in;" class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
lustre-2.12.6-1.el7.x86_64<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; text-indent: 0.5in;" class="">
rpm -qa | grep e2fs<o:p class=""></o:p></div>
</div>
<div style="margin-left: 0.5in;" class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
e2fsprogs-libs-1.45.6.wc1-0.el7.x86_64<o:p class=""></o:p></div>
</div>
<div style="margin-left: 0.5in;" class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
e2fsprogs-1.45.6.wc1-0.el7.x86_64<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
With all of that installed and the successful running and clean up of the llmount.sh and llmountcleanup.sh I am still getting the errors:<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
“Unable to mount /dev/sdb: Invalid argument”<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
“tunefs.luster: FATAL: failed to write local files and tunefs.luster: exiting with 22 (Invalid argument)”<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
When I use the command tunefs.lustre /dev/sdb (which is one of the lustre LUNs that is attached as a “disk” to the VM)<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Full output of the tunefs.luster /dev/sdb command (as mush as I can show anyway):<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div style="margin-left: 0.5in;" class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Tunefs.lustre /dev/sdb<o:p class=""></o:p></div>
</div>
<div style="margin-left: 0.5in;" class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Checking for existing lustre data: found<o:p class=""></o:p></div>
</div>
<div style="margin-left: 0.5in;" class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Reading CONFIGS/mountdata<o:p class=""></o:p></div>
</div>
<div style="margin-left: 0.5in;" class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div style="margin-left: 0.5in;" class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Read previous values:<o:p class=""></o:p></div>
</div>
<div style="margin-left: 0.5in;" class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Target: <name>-OST0009<o:p class=""></o:p></div>
</div>
<div style="margin-left: 0.5in;" class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Index: 9<o:p class=""></o:p></div>
</div>
<div style="margin-left: 0.5in;" class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Luster FS: <name><o:p class=""></o:p></div>
</div>
<div style="margin-left: 0.5in;" class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Mount type: ldiskfs<o:p class=""></o:p></div>
</div>
<div style="margin-left: 0.5in;" class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Flags: 0x1002<o:p class=""></o:p></div>
</div>
<div style="margin-left: 0.5in;" class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
(OST no_primmode )<o:p class=""></o:p></div>
</div>
<div style="margin-left: 0.5in;" class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Persistent mount opts: errors=remount-ro<o:p class=""></o:p></div>
</div>
<div style="margin-left: 0.5in;" class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Parameters: mgsnode=<IP of the 1<sup class="">st</sup><span class="apple-converted-space"> </span>MGS node>@tcp mgsnode=<IP of the 2<sup class="">nd</sup><span class="apple-converted-space"> </span>MGS node>@tcp failover.node=<IP of the 1<sup class="">st</sup><span class="apple-converted-space"> </span>OSS
node>@tcp failover.node=<IP of the 2<sup class="">nd</sup><span class="apple-converted-space"> </span>OSS node>@tcp<o:p class=""></o:p></div>
</div>
<div style="margin-left: 0.5in;" class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div style="margin-left: 0.5in;" class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Permanent disk data:<o:p class=""></o:p></div>
</div>
<div style="margin-left: 0.5in;" class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Target: <name>-OST0009<o:p class=""></o:p></div>
</div>
<div style="margin-left: 0.5in;" class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Index: 9<o:p class=""></o:p></div>
</div>
<div style="margin-left: 0.5in;" class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Luster FS: <name><o:p class=""></o:p></div>
</div>
<div style="margin-left: 0.5in;" class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Mount type: ldiskfs<o:p class=""></o:p></div>
</div>
<div style="margin-left: 0.5in;" class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Flags: 0x1002<o:p class=""></o:p></div>
</div>
<div style="margin-left: 0.5in;" class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
(OST no_primmode )<o:p class=""></o:p></div>
</div>
<div style="margin-left: 0.5in;" class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Persistent mount opts: errors=remount-ro<o:p class=""></o:p></div>
</div>
<div style="margin-left: 0.5in;" class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Parameters: mgsnode=<IP of the 1<sup class="">st</sup><span class="apple-converted-space"> </span>MGS node>@tcp mgsnode=<IP of the 2<sup class="">nd</sup><span class="apple-converted-space"> </span>MGS node>@tcp failover.node=<IP of the 1<sup class="">st</sup><span class="apple-converted-space"> </span>OSS
node>@tcp failover.node=<IP of the 2<sup class="">nd</sup><span class="apple-converted-space"> </span>OSS node>@tcp<o:p class=""></o:p></div>
</div>
<div style="margin-left: 0.5in;" class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div style="margin-left: 0.5in;" class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
tunefs.luster: Unable to mount /dev/sdb: Invalid argument<o:p class=""></o:p></div>
</div>
<div style="margin-left: 0.5in;" class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div style="margin-left: 0.5in;" class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
tunefs.luster: FATAL: failed to write local files<o:p class=""></o:p></div>
</div>
<div style="margin-left: 0.5in;" class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
tunefs.luster: exiting with 22 (Invalid argument<b class="">)</b><o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<b class=""> </b><o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Now to be clear the MDS nodes are not working correctly as I am not able to mount /dev/sdb on them where the existing meta data is served out from. To this point I have been concentrating on the OSS nodes as that is where the lustre data is coming from. I have
installed the lustre kernel and the same software on the MDS nodes in the same way that I have on the OSS nodes. When I try to use tunefs.lustre /dev/sdb on the MDS nodes I get an error saying:<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div style="margin-left: 0.5in;" class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Checking for existing lustre data: not found<o:p class=""></o:p></div>
</div>
<div style="margin-left: 0.5in;" class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div style="margin-left: 0.5in;" class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
tunefs.luster: FATAL: device /dev/sdb has not been formatted with mkfs.lustre<o:p class=""></o:p></div>
</div>
<div style="margin-left: 0.5in;" class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
tunefs.luster: exiting with 19 (no such device<b class="">)</b><o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<b class=""> </b><o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
I am assuming that this is correct as that attached LUN does not need to have lustre data on it as it is the meta data server. Is there anything that I can/need to check on the MDS nodes to see what is running/working correctly?<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
I know that this is a lot and I appreciate any help that you can give me to troubleshoot this.<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<br class="">
Dean<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div class="">
<div style="border-style: solid none none; border-top-width: 1pt; padding: 3pt 0in 0in; border-color: currentcolor;" class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<b class="">From:</b><span class="apple-converted-space"> </span>STEPHENS, DEAN - US<span class="apple-converted-space"> </span><br class="">
<b class="">Sent:</b><span class="apple-converted-space"> </span>Monday, November 22, 2021 5:58 AM<br class="">
<b class="">To:</b><span class="apple-converted-space"> </span>Andreas Dilger <<a href="mailto:adilger@whamcloud.com" target="_blank" style="color: purple; text-decoration: underline;" class="">adilger@whamcloud.com</a>><br class="">
<b class="">Cc:</b><span class="apple-converted-space"> </span>Colin Faber <<a href="mailto:cfaber@gmail.com" target="_blank" style="color: purple; text-decoration: underline;" class="">cfaber@gmail.com</a>>;<span class="apple-converted-space"> </span><a href="mailto:lustre-discuss@lists.lustre.org" target="_blank" style="color: purple; text-decoration: underline;" class="">lustre-discuss@lists.lustre.org</a><br class="">
<b class="">Subject:</b><span class="apple-converted-space"> </span>RE: [lustre-discuss] Lustre and server upgrade<o:p class=""></o:p></div>
</div>
</div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Thanks for the clarification. I am using llmount.sh to test the install of the OST and MDT not run in production. I hope to have more done today and will reach out to let you all know what I find.<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Dean<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div class="">
<div style="border-style: solid none none; border-top-width: 1pt; padding: 3pt 0in 0in; border-color: currentcolor;" class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<b class="">From:</b><span class="apple-converted-space"> </span>Andreas Dilger <<a href="mailto:adilger@whamcloud.com" target="_blank" style="color: purple; text-decoration: underline;" class="">adilger@whamcloud.com</a>><span class="apple-converted-space"> </span><br class="">
<b class="">Sent:</b><span class="apple-converted-space"> </span>Friday, November 19, 2021 5:25 PM<br class="">
<b class="">To:</b><span class="apple-converted-space"> </span>STEPHENS, DEAN - US <<a href="mailto:dean.stephens@caci.com" target="_blank" style="color: purple; text-decoration: underline;" class="">dean.stephens@caci.com</a>><br class="">
<b class="">Cc:</b><span class="apple-converted-space"> </span>Colin Faber <<a href="mailto:cfaber@gmail.com" target="_blank" style="color: purple; text-decoration: underline;" class="">cfaber@gmail.com</a>>;<span class="apple-converted-space"> </span><a href="mailto:lustre-discuss@lists.lustre.org" target="_blank" style="color: purple; text-decoration: underline;" class="">lustre-discuss@lists.lustre.org</a><br class="">
<b class="">Subject:</b><span class="apple-converted-space"> </span>Re: [lustre-discuss] Lustre and server upgrade<o:p class=""></o:p></div>
</div>
</div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Dean,<o:p class=""></o:p></div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
it should be emphasized that "llmount.sh" and "llmountcleanup.sh" are for quickly formatting and mounting *TEST* filesystems. They only create a few small (400MB) loopback files in /tmp and format them as OSTs and MDTs. This should *NOT* be used on a production
system, or you will be very sad when the files in /tmp disappear after the server is rebooted and/or they reformat your real filesystem devices.<o:p class=""></o:p></div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
I mention this here because it isn't clear to me whether you are using them for testing, or trying to get a real filesystem mounted.<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Cheers, Andreas<o:p class=""></o:p></div>
</div>
<div class="">
<p class="MsoNormal" style="margin: 0in 0in 12pt; font-size: 11pt; font-family: Calibri, sans-serif;">
<o:p class=""></o:p></p>
<blockquote style="margin-top: 5pt; margin-bottom: 5pt;" class="">
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
On Nov 19, 2021, at 13:25, STEPHENS, DEAN - US via lustre-discuss <<a href="mailto:lustre-discuss@lists.lustre.org" target="_blank" style="color: purple; text-decoration: underline;" class="">lustre-discuss@lists.lustre.org</a>> wrote:<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div class="">
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
I also figure out how to clean up after the llmount.sh script is run. There is a llmountcleanup.sh that will do that.<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Dean<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div style="border-style: solid none none; border-top-width: 1pt; padding: 3pt 0in 0in; border-color: currentcolor;" class="">
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<b class="">From:</b><span class="gmail-m8274011436611286623gmail-m842532262387312469gmail-m-1734618617407989705apple-converted-space"> </span>STEPHENS, DEAN - US<span class="gmail-m8274011436611286623gmail-m842532262387312469gmail-m-1734618617407989705apple-converted-space"> </span><br class="">
<b class="">Sent:</b><span class="gmail-m8274011436611286623gmail-m842532262387312469gmail-m-1734618617407989705apple-converted-space"> </span>Friday, November 19, 2021 1:08 PM<br class="">
<b class="">To:</b><span class="gmail-m8274011436611286623gmail-m842532262387312469gmail-m-1734618617407989705apple-converted-space"> </span>Colin Faber <<a href="mailto:cfaber@gmail.com" target="_blank" style="color: purple; text-decoration: underline;" class="">cfaber@gmail.com</a>><br class="">
<b class="">Cc:</b><span class="gmail-m8274011436611286623gmail-m842532262387312469gmail-m-1734618617407989705apple-converted-space"> </span><a href="mailto:lustre-discuss@lists.lustre.org" target="_blank" style="color: purple; text-decoration: underline;" class="">lustre-discuss@lists.lustre.org</a><br class="">
<b class="">Subject:</b><span class="gmail-m8274011436611286623gmail-m842532262387312469gmail-m-1734618617407989705apple-converted-space"> </span>RE: [lustre-discuss] Lustre and server upgrade<o:p class=""></o:p></div>
</div>
</div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
One more thing that I have noticed using the llmount.sh script, the directories that were created by the script under /mnt have 000 set for the permissions. The ones that I have configure under /mnt/lustre are set to 750 permissions.<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Is this something that needs to be fixed. I have these server being configure via puppet and that is how the /mnt/lustre directories are being created and the permissions set.<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Dean<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div style="border-style: solid none none; border-top-width: 1pt; padding: 3pt 0in 0in; border-color: currentcolor;" class="">
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<b class="">From:</b><span class="gmail-m8274011436611286623gmail-m842532262387312469gmail-m-1734618617407989705apple-converted-space"> </span>STEPHENS, DEAN - US<span class="gmail-m8274011436611286623gmail-m842532262387312469gmail-m-1734618617407989705apple-converted-space"> </span><br class="">
<b class="">Sent:</b><span class="gmail-m8274011436611286623gmail-m842532262387312469gmail-m-1734618617407989705apple-converted-space"> </span>Friday, November 19, 2021 7:14 AM<br class="">
<b class="">To:</b><span class="gmail-m8274011436611286623gmail-m842532262387312469gmail-m-1734618617407989705apple-converted-space"> </span>Colin Faber <<a href="mailto:cfaber@gmail.com" target="_blank" style="color: purple; text-decoration: underline;" class="">cfaber@gmail.com</a>><br class="">
<b class="">Cc:</b><span class="gmail-m8274011436611286623gmail-m842532262387312469gmail-m-1734618617407989705apple-converted-space"> </span><a href="mailto:lustre-discuss@lists.lustre.org" target="_blank" style="color: purple; text-decoration: underline;" class="">lustre-discuss@lists.lustre.org</a><br class="">
<b class="">Subject:</b><span class="gmail-m8274011436611286623gmail-m842532262387312469gmail-m-1734618617407989705apple-converted-space"> </span>RE: [lustre-discuss] Lustre and server upgrade<o:p class=""></o:p></div>
</div>
</div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
The other question that I have is how to clean up after the llmount.sh has been run? If I do a df on the server I see that mds1, osd1, and ost2 are still mounted to /mnt. Do I need to manually umount them since the llmount.sh completed successfully?<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Also I have not done anything to my MDS node so some direction on what to do there will be helpful as well.<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Dean<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div style="border-style: solid none none; border-top-width: 1pt; padding: 3pt 0in 0in; border-color: currentcolor;" class="">
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<b class="">From:</b><span class="gmail-m8274011436611286623gmail-m842532262387312469gmail-m-1734618617407989705apple-converted-space"> </span>STEPHENS, DEAN - US<span class="gmail-m8274011436611286623gmail-m842532262387312469gmail-m-1734618617407989705apple-converted-space"> </span><br class="">
<b class="">Sent:</b><span class="gmail-m8274011436611286623gmail-m842532262387312469gmail-m-1734618617407989705apple-converted-space"> </span>Friday, November 19, 2021 7:00 AM<br class="">
<b class="">To:</b><span class="gmail-m8274011436611286623gmail-m842532262387312469gmail-m-1734618617407989705apple-converted-space"> </span>Colin Faber <<a href="mailto:cfaber@gmail.com" target="_blank" style="color: purple; text-decoration: underline;" class="">cfaber@gmail.com</a>><br class="">
<b class="">Cc:</b><span class="gmail-m8274011436611286623gmail-m842532262387312469gmail-m-1734618617407989705apple-converted-space"> </span><a href="mailto:lustre-discuss@lists.lustre.org" target="_blank" style="color: purple; text-decoration: underline;" class="">lustre-discuss@lists.lustre.org</a><br class="">
<b class="">Subject:</b><span class="gmail-m8274011436611286623gmail-m842532262387312469gmail-m-1734618617407989705apple-converted-space"> </span>RE: [lustre-discuss] Lustre and server upgrade<o:p class=""></o:p></div>
</div>
</div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Thanks for the help yesterday and I was able to install the Lustre kernel and software on a VM to include the test RPM.<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
This is what I did following<span class="gmail-m8274011436611286623gmail-m842532262387312469gmail-m-1734618617407989705apple-converted-space"> </span><a href="https://secure-web.cisco.com/1T2T_OiJFIk8v4JxFhEYWwUsMjPnEix0viV4ITX_qAff2qOe5Y5bc8HXbCHnm_oN4TMy_mwmSZkorQNpI_PQtb5LvIOyYn3RAh0FT5GB8sJvzPiTND9-wtdZUPW3XR08cZtbQiqgI61w83zZinHXISR4_2k9JUPfxL9-3Siesjd9TcB7D4z4foS2ct5s4tjHC_xb5rYg_El-EeJvxMQJgyq9TbxwP0JFB_oZ1GaYrytnG-gEEIfUFw11dWxHSVqhx9BEuqgqrBF8K2fXb7phsIcisLcmfrPIh1-y9z2T9ewtqKL2PtjMHKUWr1ySRHhubCgmuwIAl0jMGOfa36pkegGEtC2FbeK_7_vm4I6cCN18/https%3A%2F%2Fwiki.lustre.org%2FInstalling_the_Lustre_Software%23Lustre_Servers_with_LDISKFS_OSD_Support" target="_blank" style="color: purple; text-decoration: underline;" class="">these
directions</a>:<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Installed the Lustre kernel and kernel-devel (the other RPMs listed were not in my luster-server repo)<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Rebooted the VM<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Installed kmod-lustre kmod-lustre-osd-ldiskfs lustre-osd-ldiskfs-mount lustre lustre-resource-agents lustre-tests<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Ran modprobe -v lustre (did not show that it loaded kernel modules as it has done in the past)<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Ran lustre_rmmod (got an error Module Luster in use)<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Rebooted again<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Ran llmount.sh and it looked like it completed successfully<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Ran tunefs.lustre /dev/sdb (at the bottom of the output I am seeing tunefs.luster: Unable to mount /dev/sdb: Invalid argument and tunefs.luster: FATAL: failed to write local files and tunefs.luster: exiting with 22 (Invalid argument))<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Any idea what the “invalid argument” is talking about?<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Dean<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div style="border-style: solid none none; border-top-width: 1pt; padding: 3pt 0in 0in; border-color: currentcolor;" class="">
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<b class="">From:</b><span class="gmail-m8274011436611286623gmail-m842532262387312469gmail-m-1734618617407989705apple-converted-space"> </span>Colin Faber <<a href="mailto:cfaber@gmail.com" target="_blank" style="color: purple; text-decoration: underline;" class="">cfaber@gmail.com</a>><span class="gmail-m8274011436611286623gmail-m842532262387312469gmail-m-1734618617407989705apple-converted-space"> </span><br class="">
<b class="">Sent:</b><span class="gmail-m8274011436611286623gmail-m842532262387312469gmail-m-1734618617407989705apple-converted-space"> </span>Thursday, November 18, 2021 3:34 PM<br class="">
<b class="">To:</b><span class="gmail-m8274011436611286623gmail-m842532262387312469gmail-m-1734618617407989705apple-converted-space"> </span>STEPHENS, DEAN - US <<a href="mailto:dean.stephens@caci.com" target="_blank" style="color: purple; text-decoration: underline;" class="">dean.stephens@caci.com</a>><br class="">
<b class="">Cc:</b><span class="gmail-m8274011436611286623gmail-m842532262387312469gmail-m-1734618617407989705apple-converted-space"> </span><a href="mailto:lustre-discuss@lists.lustre.org" target="_blank" style="color: purple; text-decoration: underline;" class="">lustre-discuss@lists.lustre.org</a><br class="">
<b class="">Subject:</b><span class="gmail-m8274011436611286623gmail-m842532262387312469gmail-m-1734618617407989705apple-converted-space"> </span>Re: [lustre-discuss] Lustre and server upgrade<o:p class=""></o:p></div>
</div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
The VM will need a full install of all server packages, as well as the tests package to allow for this test.<o:p class=""></o:p></div>
</div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
On Thu, Nov 18, 2021 at 2:26 PM STEPHENS, DEAN - US <<a href="mailto:dean.stephens@caci.com" target="_blank" style="color: purple; text-decoration: underline;" class="">dean.stephens@caci.com</a>> wrote:<o:p class=""></o:p></div>
</div>
</div>
</div>
<blockquote style="border-style: none none none solid; border-left-width: 1pt; padding: 0in 0in 0in 6pt; margin: 5pt 0in 5pt 4.8pt; border-color: currentcolor currentcolor currentcolor rgb(204, 204, 204);" class="">
<div class="">
<div class="">
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
I have not tried that but I can do that on a new VM that I can create. I assume that is all that I need is the lustre-tests RPM and associated dependencies and not the full blown lustre install?<span class="gmail-m8274011436611286623gmail-m842532262387312469gmail-m-1734618617407989705apple-converted-space"> </span><o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Dean<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div style="border-style: solid none none; border-top-width: 1pt; padding: 3pt 0in 0in; border-color: currentcolor;" class="">
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<b class="">From:</b><span class="gmail-m8274011436611286623gmail-m842532262387312469gmail-m-1734618617407989705apple-converted-space"> </span>Colin Faber <<a href="mailto:cfaber@gmail.com" target="_blank" style="color: purple; text-decoration: underline;" class="">cfaber@gmail.com</a>><span class="gmail-m8274011436611286623gmail-m842532262387312469gmail-m-1734618617407989705apple-converted-space"> </span><br class="">
<b class="">Sent:</b><span class="gmail-m8274011436611286623gmail-m842532262387312469gmail-m-1734618617407989705apple-converted-space"> </span>Thursday, November 18, 2021 2:22 PM<br class="">
<b class="">To:</b><span class="gmail-m8274011436611286623gmail-m842532262387312469gmail-m-1734618617407989705apple-converted-space"> </span>STEPHENS, DEAN - US <<a href="mailto:dean.stephens@caci.com" target="_blank" style="color: purple; text-decoration: underline;" class="">dean.stephens@caci.com</a>><br class="">
<b class="">Cc:</b><span class="gmail-m8274011436611286623gmail-m842532262387312469gmail-m-1734618617407989705apple-converted-space"> </span><a href="mailto:lustre-discuss@lists.lustre.org" target="_blank" style="color: purple; text-decoration: underline;" class="">lustre-discuss@lists.lustre.org</a><br class="">
<b class="">Subject:</b><span class="gmail-m8274011436611286623gmail-m842532262387312469gmail-m-1734618617407989705apple-converted-space"> </span>Re: [lustre-discuss] Lustre and server upgrade<o:p class=""></o:p></div>
</div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
So that indicates that your installation is incomplete or something else is preventing lustre, ldiskfs, and possibly other modules from loading. Have you been able to reproduce this behavior on a fresh rhel install with lustre 2.12.7? (i.e. llmount.sh failing)?<o:p class=""></o:p></div>
</div>
</div>
</div>
<div class="">
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
</div>
<div class="">
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
-cf<o:p class=""></o:p></div>
</div>
</div>
</div>
<div class="">
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
On Thu, Nov 18, 2021 at 2:20 PM STEPHENS, DEAN - US <<a href="mailto:dean.stephens@caci.com" target="_blank" style="color: purple; text-decoration: underline;" class="">dean.stephens@caci.com</a>> wrote:<o:p class=""></o:p></div>
</div>
</div>
</div>
<blockquote style="border-style: none none none solid; border-left-width: 1pt; padding: 0in 0in 0in 6pt; margin: 5pt 0in 5pt 4.8pt; border-color: currentcolor currentcolor currentcolor rgb(204, 204, 204);" class="">
<div class="">
<div class="">
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Thanks for the direction. I found it and installed lustre-tests.x86_64 and now I have the llmount but it was defaulted to /usr/lib64/lustre/tests/llmount.sh and when I ran it but it failed with:<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Stopping clients: <hostname> /mnt/lustre (opts: -f)<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Stopping clients: <hostname> /mnt/lustre2 (opts: -f)<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Loading modules from /usr/lib64/lustre/tests/..<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Detected 2 online CPUs by sysfs<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Force libcfs to create 2 CPU partitions<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Formatting mgs, mds, osts<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Format mds1: /tmp/lustre-mdt1<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Mkfs.lustre: Unable to mount /dev/loop0: No such device (even though /dev/loop0 is a thing)<br class="">
Is the ldiskfs module loaded?<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Mkfs.lustre FATAL: failed to write local files<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Mkfs.lustre: exiting with 19 (no such device)<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div style="border-style: solid none none; border-top-width: 1pt; padding: 3pt 0in 0in; border-color: currentcolor;" class="">
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<b class="">From:</b><span class="gmail-m8274011436611286623gmail-m842532262387312469gmail-m-1734618617407989705apple-converted-space"> </span>Colin Faber <<a href="mailto:cfaber@gmail.com" target="_blank" style="color: purple; text-decoration: underline;" class="">cfaber@gmail.com</a>><span class="gmail-m8274011436611286623gmail-m842532262387312469gmail-m-1734618617407989705apple-converted-space"> </span><br class="">
<b class="">Sent:</b><span class="gmail-m8274011436611286623gmail-m842532262387312469gmail-m-1734618617407989705apple-converted-space"> </span>Thursday, November 18, 2021 2:03 PM<br class="">
<b class="">To:</b><span class="gmail-m8274011436611286623gmail-m842532262387312469gmail-m-1734618617407989705apple-converted-space"> </span>STEPHENS, DEAN - US <<a href="mailto:dean.stephens@caci.com" target="_blank" style="color: purple; text-decoration: underline;" class="">dean.stephens@caci.com</a>><br class="">
<b class="">Cc:</b><span class="gmail-m8274011436611286623gmail-m842532262387312469gmail-m-1734618617407989705apple-converted-space"> </span><a href="mailto:lustre-discuss@lists.lustre.org" target="_blank" style="color: purple; text-decoration: underline;" class="">lustre-discuss@lists.lustre.org</a><br class="">
<b class="">Subject:</b><span class="gmail-m8274011436611286623gmail-m842532262387312469gmail-m-1734618617407989705apple-converted-space"> </span>Re: [lustre-discuss] Lustre and server upgrade<o:p class=""></o:p></div>
</div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
This would be part of the lustre-tests RPM package and will install llmount.sh to /usr/lib/lustre/tests/llmount.sh I believe.<o:p class=""></o:p></div>
</div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
On Thu, Nov 18, 2021 at 1:45 PM STEPHENS, DEAN - US <<a href="mailto:dean.stephens@caci.com" target="_blank" style="color: purple; text-decoration: underline;" class="">dean.stephens@caci.com</a>> wrote:<o:p class=""></o:p></div>
</div>
</div>
</div>
<blockquote style="border-style: none none none solid; border-left-width: 1pt; padding: 0in 0in 0in 6pt; margin: 5pt 0in 5pt 4.8pt; border-color: currentcolor currentcolor currentcolor rgb(204, 204, 204);" class="">
<div class="">
<div class="">
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Not sure what you mean by “If you install the test suite”. I am not seeing a llmount.sh file on the server using “locate llmount.sh” at this point. What are the steps to install the test suite?<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Dean<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div style="border-style: solid none none; border-top-width: 1pt; padding: 3pt 0in 0in; border-color: currentcolor;" class="">
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<b class="">From:</b><span class="gmail-m8274011436611286623gmail-m842532262387312469gmail-m-1734618617407989705apple-converted-space"> </span>Colin Faber <<a href="mailto:cfaber@gmail.com" target="_blank" style="color: purple; text-decoration: underline;" class="">cfaber@gmail.com</a>><span class="gmail-m8274011436611286623gmail-m842532262387312469gmail-m-1734618617407989705apple-converted-space"> </span><br class="">
<b class="">Sent:</b><span class="gmail-m8274011436611286623gmail-m842532262387312469gmail-m-1734618617407989705apple-converted-space"> </span>Thursday, November 18, 2021 1:34 PM<br class="">
<b class="">To:</b><span class="gmail-m8274011436611286623gmail-m842532262387312469gmail-m-1734618617407989705apple-converted-space"> </span>STEPHENS, DEAN - US <<a href="mailto:dean.stephens@caci.com" target="_blank" style="color: purple; text-decoration: underline;" class="">dean.stephens@caci.com</a>><br class="">
<b class="">Cc:</b><span class="gmail-m8274011436611286623gmail-m842532262387312469gmail-m-1734618617407989705apple-converted-space"> </span><a href="mailto:lustre-discuss@lists.lustre.org" target="_blank" style="color: purple; text-decoration: underline;" class="">lustre-discuss@lists.lustre.org</a><br class="">
<b class="">Subject:</b><span class="gmail-m8274011436611286623gmail-m842532262387312469gmail-m-1734618617407989705apple-converted-space"> </span>Re: [lustre-discuss] Lustre and server upgrade<o:p class=""></o:p></div>
</div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Hm.. If you install the test suite does llmount.sh succeed? This should setup a single node cluster on whatever node you're running lustre on, I believe it will load modules as needed (IIRC), if this test succeeds, then you know that lustre is installed correctly
(or correctly enough), if not, I'd focus on the installation as the target issue may be a redheirring<o:p class=""></o:p></div>
</div>
</div>
</div>
<div class="">
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
</div>
<div class="">
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
-cf<o:p class=""></o:p></div>
</div>
</div>
</div>
<div class="">
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
On Thu, Nov 18, 2021 at 1:01 PM STEPHENS, DEAN - US <<a href="mailto:dean.stephens@caci.com" target="_blank" style="color: purple; text-decoration: underline;" class="">dean.stephens@caci.com</a>> wrote:<o:p class=""></o:p></div>
</div>
</div>
</div>
<blockquote style="border-style: none none none solid; border-left-width: 1pt; padding: 0in 0in 0in 6pt; margin: 5pt 0in 5pt 4.8pt; border-color: currentcolor currentcolor currentcolor rgb(204, 204, 204);" class="">
<div class="">
<div class="">
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Thanks for the fast reply.<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
When I do the tunefs.lustre /dev/sdX command I get:<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Target: <name>-OST0009<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Index: 9<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Target: <name>-OST0008<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Index: 8<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
I spot checked some others and they seem to be good with the exception of one. It shows:<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Target: <name>-OST000a<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Index: 10<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
But since there are 11 LUNs attached that make sense to me.<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
As far as the upgrade it was a fresh install using the legacy targets as the OSS and MDS nodes are virtual machine with the LUN disks attached to them so that Red Hat sees them as /dev/sdX devices.<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
When I loaded Lustre on the server I did a yum install lustre and since we were pointed at the lustre-2.12 repo in our environment it picked up the following RPMs to install:<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Luster-resource-agents-2.12.6-1.el7.x86_64<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Kmod-lustre-2.12.6-1.el7.x86_64<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Kmod-zfs-3.10.0-1160.2.1.el7_lustre.x86_64-09.7.13-1.el7.x86_64<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Kmod-lustre-osd-zfs-2.12.6-1.el7.x86_64<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Lustre-2.12.6-1.el7.x86_64<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Kmod-spl-3.10.0-1160.2.1.el7_lustre.x86_64-09.7.13-1.el7.x86_64<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Lustre-osd-zfs-mount-2.12.6-1.el7.x86_64<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Lustre-osd-ldiskfs-mount-2.12.6-1.el7.x86_64<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Dean<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div style="border-style: solid none none; border-top-width: 1pt; padding: 3pt 0in 0in; border-color: currentcolor;" class="">
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<b class="">From:</b><span class="gmail-m8274011436611286623gmail-m842532262387312469gmail-m-1734618617407989705apple-converted-space"> </span>Colin Faber <<a href="mailto:cfaber@gmail.com" target="_blank" style="color: purple; text-decoration: underline;" class="">cfaber@gmail.com</a>><span class="gmail-m8274011436611286623gmail-m842532262387312469gmail-m-1734618617407989705apple-converted-space"> </span><br class="">
<b class="">Sent:</b><span class="gmail-m8274011436611286623gmail-m842532262387312469gmail-m-1734618617407989705apple-converted-space"> </span>Thursday, November 18, 2021 12:35 PM<br class="">
<b class="">To:</b><span class="gmail-m8274011436611286623gmail-m842532262387312469gmail-m-1734618617407989705apple-converted-space"> </span>STEPHENS, DEAN - US <<a href="mailto:dean.stephens@caci.com" target="_blank" style="color: purple; text-decoration: underline;" class="">dean.stephens@caci.com</a>><br class="">
<b class="">Cc:</b><span class="gmail-m8274011436611286623gmail-m842532262387312469gmail-m-1734618617407989705apple-converted-space"> </span><a href="mailto:lustre-discuss@lists.lustre.org" target="_blank" style="color: purple; text-decoration: underline;" class="">lustre-discuss@lists.lustre.org</a><br class="">
<b class="">Subject:</b><span class="gmail-m8274011436611286623gmail-m842532262387312469gmail-m-1734618617407989705apple-converted-space"> </span>Re: [lustre-discuss] Lustre and server upgrade<o:p class=""></o:p></div>
</div>
</div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<p class="gmail-m8274011436611286623gmail-m842532262387312469gmail-m-1734618617407989705gmail-m-5415300209567740660gmail-m-1795078219693914619gmail-m-1374272708024244046gmail-m-8166945065793112345msonospacing" style="margin-right: 0in; margin-left: 0in; font-size: 11pt; font-family: Calibri, sans-serif;">
<span style="font-size: 12pt; color: red;" class="">EXTERNAL EMAIL - This email originated from outside of CACI. Do not click any links or attachments unless you recognize and trust the sender.</span><o:p class=""></o:p></p>
<p class="gmail-m8274011436611286623gmail-m842532262387312469gmail-m-1734618617407989705gmail-m-5415300209567740660gmail-m-1795078219693914619gmail-m-1374272708024244046gmail-m-8166945065793112345msonospacing" style="margin-right: 0in; margin-left: 0in; font-size: 11pt; font-family: Calibri, sans-serif;">
<span style="font-size: 12pt; color: red;" class=""> </span><o:p class=""></o:p></p>
<p class="gmail-m8274011436611286623gmail-m842532262387312469gmail-m-1734618617407989705gmail-m-5415300209567740660gmail-m-1795078219693914619gmail-m-1374272708024244046gmail-m-8166945065793112345msonospacing" style="margin-right: 0in; margin-left: 0in; font-size: 11pt; font-family: Calibri, sans-serif;">
<span style="font-size: 12pt;" class=""> </span><o:p class=""></o:p></p>
<div class="">
<div class="">
<div class="">
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Hi,<o:p class=""></o:p></div>
</div>
</div>
</div>
<div class="">
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
</div>
<div class="">
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
I believe in 2.10 sometime (someone correct me if I'm wrong) that the index parameter was required and needs to be specified. On an existing system this should already be set, but can you check the parameters line with tunefs.lustre for correct index=N values
across your storage nodes?<o:p class=""></o:p></div>
</div>
</div>
</div>
<div class="">
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
</div>
<div class="">
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Also, with your "upgrade", was this a fresh install utilizing legacy targets?<o:p class=""></o:p></div>
</div>
</div>
</div>
<div class="">
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
</div>
<div class="">
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
The last thing I can think of IIRC, there was on-disk format changes between 2.5 and 2.12, these should be transparent to you, but it may be some other issue is preventing successful upgrade, though the missing module error really speaks to possible issues
around how lustre was installed and loaded on the system.<o:p class=""></o:p></div>
</div>
</div>
</div>
<div class="">
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
</div>
<div class="">
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Cheers!<o:p class=""></o:p></div>
</div>
</div>
</div>
<div class="">
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
</div>
<div class="">
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
-cf<o:p class=""></o:p></div>
</div>
</div>
</div>
<div class="">
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
On Thu, Nov 18, 2021 at 12:24 PM STEPHENS, DEAN - US via lustre-discuss <<a href="mailto:lustre-discuss@lists.lustre.org" target="_blank" style="color: purple; text-decoration: underline;" class="">lustre-discuss@lists.lustre.org</a>> wrote:<o:p class=""></o:p></div>
</div>
</div>
</div>
<blockquote style="border-style: none none none solid; border-left-width: 1pt; padding: 0in 0in 0in 6pt; margin: 5pt 0in 5pt 4.8pt; border-color: currentcolor currentcolor currentcolor rgb(204, 204, 204);" class="">
<div class="">
<div class="">
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
I am by no means a Lustre expert and am seeking some help with our system. I am not able to get log file to post as the servers are in the closed area with no access to the Internet.<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Here is a bit of history of our system:<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
The OSS and MDS nodes were RHEL6 and running a Luster server the kernel 2.6.32-431.23.3.el6_lustre.x86_64 and the Lustre version of 2.5.3. the client version was 2.10. That was in a working state.<span class="gmail-m8274011436611286623gmail-m842532262387312469gmail-m-1734618617407989705apple-converted-space"> </span><o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
We upgraded the OSS ad MDS nodes to RHEL7 and installed Lustre server 2.12 software and kernel.<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
The attached 11 LUNs are showing up as /dev/sdb - /dev/sdl<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Right now, on the OSS nodes, if I use the command tunefs.luster /dev/sdb I get some data back saying that Lustre data has been found but at the bottom of the out put it shows “tunefs.lustre: Unable to mount /dev/sdb: No such device” and “Is the ldiskfs module
available”<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
When I do a “modprobe -v lustre” I do not see ldiskfs.ko as being loaded even though there is a ldiskfs.ko file in /lib/modules/3.10.0-1160.2.1.el7_lustre.x86_64/extra/lustre/fs directory. I am not sure how to get it to load in the modprobe command.<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
I used “insmod /lib/modules/3.10.0-1160.2.1.el7_lustre.x86_64/extra/lustre/fs/ ldiskfs.ko” and re-ran the “tunefs.luster /dev/sdb” command with the same result.<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
If I use the same command on the MDS nodes I get “no Lustre data found and /dev/sdb has not been formatted with mkfs.lustre”. I am not sure that is what is needed here as the MDS nodes do not really have the lustre data as it is the meta data server.<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
I tried to use the command “tunefs.lustre --mgs --erase_params --mgsnode=<IP address>@tcp --writeconf --dryrun /dev/sdb” and get the error “/dev/sdb has not been formatted with mkfs.lustre”.<span class="gmail-m8274011436611286623gmail-m842532262387312469gmail-m-1734618617407989705apple-converted-space"> </span><o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
I need some help and guidance and I can provide what may be needed though it will need to be typed out as I am not able to get actual log files from the system.<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Dean Stephens<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
CACI<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Linux System Admin<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
<div class="MsoNormal" align="center" style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif; text-align: center;">
<hr size="2" width="100%" align="center" class="">
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span style="font-size: 7.5pt; font-family: Arial, sans-serif; color: gray;" class=""><br class="">
This electronic message contains information from CACI International Inc or subsidiary companies, which may be company sensitive, proprietary, privileged or otherwise protected from disclosure. The information is intended to be used solely by the recipient(s)
named above. If you are not an intended recipient, be aware that any review, disclosure, copying, distribution or use of this transmission or its contents is prohibited. If you have received this transmission in error, please notify the sender immediately.</span><o:p class=""></o:p></div>
</div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
_______________________________________________<br class="">
lustre-discuss mailing list<br class="">
<a href="mailto:lustre-discuss@lists.lustre.org" target="_blank" style="color: purple; text-decoration: underline;" class="">lustre-discuss@lists.lustre.org</a><br class="">
<a href="http://secure-web.cisco.com/1W1muQD4c3W9jlz3ATBTpimQFCy3uxbyFeFRH8b0Giilvxexdseo0_vO0INboN2XMxnE0zr07jC5NKwiGPByP0BJWmjwsQR-b9rIb33vipV7lnU1Opbt_tXVS15PfodllSOvtx2dic-GEkcsxax3_OkBmIBoxMQJ7fgsvElG-xEzd6w10X9A6Y1pzqXXw6nYUhcnXKpam0qScIdg_sLMHDoj58svaBZT7XOYtHASwV04oHjhSRpluqldfGo-Pjkcnf5ukqA83bz16lMvJ2Gldgg_4L1y9r8KDerupJSTxiZ5nTexQWP3EPHJIQnAyKb-g9v-u_VLCjolzOcvZk0vpyQxjZVt_asbpPaJeUWVaDxs/http%3A%2F%2Flists.lustre.org%2Flistinfo.cgi%2Flustre-discuss-lustre.org" target="_blank" style="color: purple; text-decoration: underline;" class="">http://secure-web.cisco.com/1W1muQD4c3W9jlz3ATBTpimQFCy3uxbyFeFRH8b0Giilvxexdseo0_vO0INboN2XMxnE0zr07jC5NKwiGPByP0BJWmjwsQR-b9rIb33vipV7lnU1Opbt_tXVS15PfodllSOvtx2dic-GEkcsxax3_OkBmIBoxMQJ7fgsvElG-xEzd6w10X9A6Y1pzqXXw6nYUhcnXKpam0qScIdg_sLMHDoj58svaBZT7XOYtHASwV04oHjhSRpluqldfGo-Pjkcnf5ukqA83bz16lMvJ2Gldgg_4L1y9r8KDerupJSTxiZ5nTexQWP3EPHJIQnAyKb-g9v-u_VLCjolzOcvZk0vpyQxjZVt_asbpPaJeUWVaDxs/http%3A%2F%2Flists.lustre.org%2Flistinfo.cgi%2Flustre-discuss-lustre.org</a><o:p class=""></o:p></div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</blockquote>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span style="font-size: 9pt; font-family: Helvetica, sans-serif;" class="">_______________________________________________<br class="">
lustre-discuss mailing list<br class="">
<a href="mailto:lustre-discuss@lists.lustre.org" target="_blank" style="color: purple; text-decoration: underline;" class="">lustre-discuss@lists.lustre.org</a><br class="">
<a href="http://secure-web.cisco.com/1W1muQD4c3W9jlz3ATBTpimQFCy3uxbyFeFRH8b0Giilvxexdseo0_vO0INboN2XMxnE0zr07jC5NKwiGPByP0BJWmjwsQR-b9rIb33vipV7lnU1Opbt_tXVS15PfodllSOvtx2dic-GEkcsxax3_OkBmIBoxMQJ7fgsvElG-xEzd6w10X9A6Y1pzqXXw6nYUhcnXKpam0qScIdg_sLMHDoj58svaBZT7XOYtHASwV04oHjhSRpluqldfGo-Pjkcnf5ukqA83bz16lMvJ2Gldgg_4L1y9r8KDerupJSTxiZ5nTexQWP3EPHJIQnAyKb-g9v-u_VLCjolzOcvZk0vpyQxjZVt_asbpPaJeUWVaDxs/http%3A%2F%2Flists.lustre.org%2Flistinfo.cgi%2Flustre-discuss-lustre.org" target="_blank" style="color: purple; text-decoration: underline;" class="">http://secure-web.cisco.com/1W1muQD4c3W9jlz3ATBTpimQFCy3uxbyFeFRH8b0Giilvxexdseo0_vO0INboN2XMxnE0zr07jC5NKwiGPByP0BJWmjwsQR-b9rIb33vipV7lnU1Opbt_tXVS15PfodllSOvtx2dic-GEkcsxax3_OkBmIBoxMQJ7fgsvElG-xEzd6w10X9A6Y1pzqXXw6nYUhcnXKpam0qScIdg_sLMHDoj58svaBZT7XOYtHASwV04oHjhSRpluqldfGo-Pjkcnf5ukqA83bz16lMvJ2Gldgg_4L1y9r8KDerupJSTxiZ5nTexQWP3EPHJIQnAyKb-g9v-u_VLCjolzOcvZk0vpyQxjZVt_asbpPaJeUWVaDxs/http%3A%2F%2Flists.lustre.org%2Flistinfo.cgi%2Flustre-discuss-lustre.org</a></span><o:p class=""></o:p></div>
</div>
</div>
</blockquote>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div class="">
<div class="">
<div class="">
<div class="">
<div class="">
<div class="">
<div class="">
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Cheers, Andreas<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
--<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Andreas Dilger<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Lustre Principal Architect<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Whamcloud<o:p class=""></o:p></div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</blockquote>
</div>
</blockquote>
</div>
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
<div class="">
<div class="">
<div class="">
<div class="">
<div class="">
<div class="">
<div class="">
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span style="" class="">Cheers, Andreas<o:p class=""></o:p></span></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span style="" class="">--<o:p class=""></o:p></span></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span style="" class="">Andreas Dilger<o:p class=""></o:p></span></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span style="" class="">Lustre Principal Architect<o:p class=""></o:p></span></div>
</div>
<div class="">
<div style="margin: 0in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span style="" class="">Whamcloud</span></div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br class="">
<div class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div dir="auto" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div>Cheers, Andreas</div>
<div>--</div>
<div>Andreas Dilger</div>
<div>Lustre Principal Architect</div>
<div>Whamcloud</div>
<div><br class="">
</div>
<div><br class="">
</div>
<div><br class="">
</div>
</div>
</div>
</div>
</div>
</div>
<br class="Apple-interchange-newline">
</div>
<br class="Apple-interchange-newline">
<br class="Apple-interchange-newline">
</div>
<br class="">
</div>
</body>
</html>