<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; color: rgb(0, 0, 0); font-size: 14px; font-family: Calibri, sans-serif; "><div>Hi,</div><div><br></div><div>I suffered an oss crash where my oss server had a cpu fault. I have it running again, but I am trying to decommission it. I am migrating the data off of it onto other ost's using the lfs find command with lfs_migrate.</div><div><br></div><div>It's been nearly 36 hours and about 2 terabytes have been moved. This means I am about halfway. Is this a decent rate? </div><div><br></div><div>Here are the particulars, which basically are snags. I know they affect things, I just am not certain to what degree:</div><ol><li>I am running lfs_migrate on two systems, migrating different subdirectories of the same mount point.</li><li>All systems are running using ip over infiniband.</li><li>None of my client-only systems have lfs or lfs_migrate. I think this is because they are ubuntu and only the lustre kernel modules are installed. Thus I can't run it there.</li><li> Oh, and that also means that the lustre filesytem is mounted on the oss's too.</li><li>lfs_migrate and lfs did not seem to operate correctly on the oss's that are 1.8.6. Works ok on 1.8.8 though.</li><li>AND the two systems I am running lfs_migrate on are probably the very systems with free ost space on them. In other words, file blocks are being written to the very systems that lfs_migrate is being run on and/or there is a lot of block write traffic between the two. </li></ol><div><br></div><div><br></div><div>Lustre versions:</div><div>Mds/mgs: 1.8.6 </div><div>5 of 7 OSS's: 1.8.6</div><div>2 of 7 oss's: 1.8.8</div><div><br></div><div>Clients: 1.8.6, ubuntu.</div><div><br></div><div><br></div></body></html>