<div dir="ltr">
<p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US">Hello, lustre development team</span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><br></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)">background: we have two postgresql instances running as a primary and standby and they share the same xlog file and data files (we
change PG code to achieve this) which located in the mounted lustre file system, and we want to have a try with the lustre
file system, we used gfs2 before, and we expect the lustre will show a
much better performance, but ...<br></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US"><span></span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US"><span> </span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US">We
meet a read/write concurrent access problem related with Lustre. Would
you like to give us some suggestions? Any advices are appreciated, and
thank you in advanced : )<span></span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US"><span>note:
we are very sure the standy instance will not write any data to disk.
(to be sure of this, we also shutdown the standby end, and use
pg_xlogdump tool to read the xlog file, the problem still happened, and
pg_xlogdump to is just a query to with any write operation) <br></span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US"><span><br></span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US"><span><br></span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US">Scenario Description:<span></span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US">There</span>’<span lang="EN-US">re 4 nodes(CentOS Linux release 7.4) connected with infiniband network(driven by MLNX_OFED_LINUX-4.4):<span></span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US">10.0.0.106 acts as MDS with a local PCI-E 800GB SSD that used as MDT.<span></span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US">10.0.0.101 acts as OSS with a same local PCI-E 800GB SSD that used as OST.<span></span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US">10.0.0.104 and 10.0.0.105 act as Lustre client and mount the Lustre file system at the directory of<span class="gmail-m_-4615751837339015999gmail-Apple-converted-space"> </span></span>“<span lang="EN-US">/lustre</span>”<span lang="EN-US">.<span></span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US">The Lustre related packages are compiled from official lustre-2.10.5-1.src.rpm.<span></span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US"><span> </span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US">The simplest verification(i.e. dd command) passed without errors.<span></span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US"><span> </span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US">Error:<span></span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US">Then
start our customized PostgreSQL service at 104 and 105. 104 runs as the
primary PostgreSQL server, and 105 runs as the secondary PostgreSQL
server. All the two PostgreSQL nodes read/write the shared directory of<span class="gmail-m_-4615751837339015999gmail-Apple-converted-space"> </span></span>“<span lang="EN-US">/lustre</span>”<span lang="EN-US"><span class="gmail-m_-4615751837339015999gmail-Apple-converted-space"> </span>provided by Lustre. The primary PostgreSQL server will open files with *<b>RW</b>* mode and write something into the files; at the *<b>meantime</b>* the second PostgreSQL server will open the same files with *<b>R</b>* mode and read the data written by the primary PostgreSQL server, and it gets the *<b>wrong</b>*
data (the flushed data by primary in disk is error i.e. write wrong
data into disk). This will happen when we run a benchmark tool of
PostgreSQL.<span></span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US"><span> </span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US"><span> </span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US">PS 1.<span></span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US">We tried different options to mount the Lustre:<span></span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US">mount -t lustre -o flock<span class="gmail-m_-4615751837339015999gmail-Apple-converted-space"> </span><a href="mailto:10.0.0.106@o2ib0:/birdfs" style="text-decoration:underline;color:rgb(149,79,114)" target="_blank">10.0.0.106@o2ib0:/birdfs</a><span class="gmail-m_-4615751837339015999gmail-Apple-converted-space"> </span>/lustre<span></span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US">mount -t lustre -o flock -o ro<span class="gmail-m_-4615751837339015999gmail-Apple-converted-space"> </span><a href="mailto:10.0.0.106@o2ib0:/birdfs" style="text-decoration:underline;color:rgb(149,79,114)" target="_blank">10.0.0.106@o2ib0:/birdfs</a><span class="gmail-m_-4615751837339015999gmail-Apple-converted-space"> </span>/lustre<span></span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US">but the error always exists.<span></span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US"><span> </span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US">PS 2.<span></span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US">Attach the initial information, maybe helpful.<span></span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US">[root@106 ~]# mkfs.lustre --fsname=birdfs --mgs --mdt --index=0 --reformat /dev/nvme0n1<span></span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US"><span> </span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US"> Permanent disk data:<span></span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US">Target: birdfs:MDT0000<span></span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US">Index: 0<span></span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US">Lustre FS: birdfs<span></span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US">Mount type: ldiskfs<span></span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US">Flags: 0x65<span></span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US"> (MDT MGS first_time update )<span></span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US">Persistent mount opts: user_xattr,errors=remount-ro<span></span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US">Parameters:<span></span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US"><span> </span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US">device size = 763097MB<span></span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US">formatting backing filesystem ldiskfs on /dev/nvme0n1<span></span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US"> target name birdfs:MDT0000<span></span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US"> 4k blocks 195353046<span></span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US"> options -J size=4096 -I 1024 -i 2560 -q -O dirdata,uninit_bg,^extents,dir_nlink,quota,huge_file,flex_bg -E lazy_journal_init -F<span></span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US">mkfs_cmd = mke2fs -j -b 4096 -L birdfs:MDT0000 -J size=4096 -I 1024 -i 2560 -q -O dirdata,uninit_bg,^extents,dir_nlink,quota,huge_file,flex_bg -E lazy_journal_init -F /dev/nvme0n1 195353046<span></span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US"><span> </span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US">Writing CONFIGS/mountdata<span></span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US"><span> </span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US">[root@101 ~]# mkfs.lustre --fsname=birdfs --ost --reformat --index=0<span class="gmail-m_-4615751837339015999gmail-Apple-converted-space"> </span><a href="mailto:--mgsnode=10.0.0.106@o2ib0" style="text-decoration:underline;color:rgb(149,79,114)" target="_blank">--mgsnode=10.0.0.106@o2ib0</a><span class="gmail-m_-4615751837339015999gmail-Apple-converted-space"> </span>/dev/nvme0n1<span></span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US"><span> </span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US"> Permanent disk data:<span></span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US">Target: birdfs:OST0000<span></span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US">Index: 0<span></span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US">Lustre FS: birdfs<span></span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US">Mount type: ldiskfs<span></span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US">Flags: 0x62<span></span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US"> (OST first_time update )<span></span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US">Persistent mount opts: ,errors=remount-ro<span></span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US">Parameters:<span class="gmail-m_-4615751837339015999gmail-Apple-converted-space"> </span><a href="mailto:mgsnode=10.0.0.106@o2ib" style="text-decoration:underline;color:rgb(149,79,114)" target="_blank">mgsnode=10.0.0.106@o2ib</a><span></span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US"><span> </span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US">device size = 763097MB<span></span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US">formatting backing filesystem ldiskfs on /dev/nvme0n1<span></span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US"> target name birdfs:OST0000<span></span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US"> 4k blocks 195353046<span></span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US"> options -J size=400 -I 512 -i 69905 -q -O extents,uninit_bg,dir_nlink,quota,huge_file,flex_bg -G 256 -E resize="4290772992",lazy_journal_init -F<span></span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US">mkfs_cmd = mke2fs -j -b 4096 -L birdfs:OST0000 -J size=400 -I 512 -i 69905 -q -O extents,uninit_bg,dir_nlink,quota,huge_file,flex_bg -G 256 -E resize="4290772992",lazy_journal_init -F /dev/nvme0n1 195353046<span></span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US">Writing CONFIGS/mountdata<span></span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US"><span> </span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US"><span> </span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US">Looking forward to any replies.<span></span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US"><span> </span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US">Regards,<span></span></span></p><p class="MsoNormal" style="margin:0cm 0cm 0.0001pt;text-align:justify;font-size:14px;font-family:等线;color:rgb(0,0,0);font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;background-color:rgb(255,255,255)"><span lang="EN-US">Bird</span></p>
<br clear="all"><br>-- <br><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr">regards<div>denny</div></div></div></div>