<html><head><base href="x-msg://926/"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><div><div>On Nov 2, 2011, at 1:48 PM, Charland, Denis wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><span class="Apple-style-span" style="border-collapse: separate; font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "><div lang="FR-CA" link="blue" vlink="purple"><div class="WordSection1" style="page: WordSection1; "><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-US" style="font-family: Arial, sans-serif; color: rgb(0, 32, 96); ">I read in the Lustre Operations Manual that there is an OST size limitation of 16 TB on RHEL and<o:p></o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-US" style="font-family: Arial, sans-serif; color: rgb(0, 32, 96); ">8 TB on other distributions because of the ext3 file system limitation. I have a few questions about that.<o:p></o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-US" style="font-family: Arial, sans-serif; color: rgb(0, 32, 96); "><o:p> </o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-US" style="font-family: Arial, sans-serif; color: rgb(0, 32, 96); ">Why is the limitation 16 TB on RHEL?</span></div></div></div></span></blockquote><div><br></div><div><br></div><div>16TB is the maximum size RedHat supports. See <a href="http://www.redhat.com/rhel/compare/">http://www.redhat.com/rhel/compare/</a></div><div>Larger than that requires bigger changes.</div><div><br></div><div>Note that whamcloud's 1.8.6-wc1 claimed support for 24TB LUNs (but see <a href="http://jira.whamcloud.com/browse/LU-419">http://jira.whamcloud.com/browse/LU-419</a> ).</div><div><br></div><div>Whamcloud's Lustre 2.1 (not sure you'd want to use it) claims support for 128TB LUNs.</div><div><br></div><br><blockquote type="cite"><span class="Apple-style-span" style="border-collapse: separate; font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "><div lang="FR-CA" link="blue" vlink="purple"><div class="WordSection1" style="page: WordSection1; "><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-US" style="font-family: Arial, sans-serif; color: rgb(0, 32, 96); "><o:p> </o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-US" style="font-family: Arial, sans-serif; color: rgb(0, 32, 96); ">I plan to use Lustre 1.8.5 on Fedora 12 for a new Lustre file system. What will be the OST size limitation?<o:p></o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-US" style="font-family: Arial, sans-serif; color: rgb(0, 32, 96); "><o:p> </o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-US" style="font-family: Arial, sans-serif; color: rgb(0, 32, 96); ">What is the OST size limitation when using ext4?</span></div></div></div></span></blockquote><div><br></div><div>16TB with the Lustre-patched RHEL kernel.</div><br><blockquote type="cite"><span class="Apple-style-span" style="border-collapse: separate; font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "><div lang="FR-CA" link="blue" vlink="purple"><div class="WordSection1" style="page: WordSection1; "><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-US" style="font-family: Arial, sans-serif; color: rgb(0, 32, 96); "><o:p></o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-US" style="font-family: Arial, sans-serif; color: rgb(0, 32, 96); "><o:p> </o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-US" style="font-family: Arial, sans-serif; color: rgb(0, 32, 96); ">Is it preferable to use ext4 instead of ext3?<o:p></o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-US" style="font-family: Arial, sans-serif; color: rgb(0, 32, 96); "><o:p> </o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-US" style="font-family: Arial, sans-serif; color: rgb(0, 32, 96); ">If the block device has more than 8 TB or 16 TB, it must be partitioned. Is there a performance degradation<o:p></o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-US" style="font-family: Arial, sans-serif; color: rgb(0, 32, 96); ">when a device has multiple partitions compared to a single partition? In other words, is it better to have three<o:p></o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-US" style="font-family: Arial, sans-serif; color: rgb(0, 32, 96); ">8 TB devices with one partition per device than to have one 24 TB device with three partitions?</span></div></div></div></span></blockquote><div><br></div><div><br></div><div>Better to have 3 separate 8TB LUNs. Different OSTs forcing the same drive heads to move to opposite parts of the disk does degrade performance (with a single OST moving the drive heads, the block allocator tries to minimize movement).</div><div><br></div><div><br></div><blockquote type="cite"><span class="Apple-style-span" style="border-collapse: separate; font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "><div lang="FR-CA" link="blue" vlink="purple"><div class="WordSection1" style="page: WordSection1; "><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-US" style="font-family: Arial, sans-serif; color: rgb(0, 32, 96); "><o:p></o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-US" style="font-family: Arial, sans-serif; color: rgb(0, 32, 96); "><o:p> </o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-US" style="font-family: Arial, sans-serif; color: rgb(0, 32, 96); ">Denis Charland</span><b><span lang="EN-US" style="font-size: 10pt; font-family: Arial, sans-serif; color: navy; "><o:p></o:p></span></b></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-US" style="font-size: 9pt; font-family: Arial, sans-serif; color: navy; ">UNIX Systems Administrator<o:p></o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-US" style="font-size: 9pt; font-family: Arial, sans-serif; color: navy; ">National Research Council Canada<o:p></o:p></span></div><div style="margin-top: 0cm; margin-right: 0cm; margin-left: 0cm; margin-bottom: 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; "><span lang="EN-US" style="font-family: Arial, sans-serif; color: rgb(0, 32, 96); "><o:p> </o:p></span></div></div>_______________________________________________<br>Lustre-discuss mailing list<br><a href="mailto:Lustre-discuss@lists.lustre.org" style="color: blue; text-decoration: underline; ">Lustre-discuss@lists.lustre.org</a><br><a href="http://lists.lustre.org/mailman/listinfo/lustre-discuss" style="color: blue; text-decoration: underline; ">http://lists.lustre.org/mailman/listinfo/lustre-discuss</a><br></div></span></blockquote></div><br><br><br>
Confidentiality Notice: This e-mail message, its contents and any attachments to it are confidential to the intended recipient, and may contain information that is privileged and/or exempt from disclosure under applicable law. If you are not the intended recipient, please immediately notify the sender and destroy the original e-mail message and any attachments (and any copies that may have been made) from your system or otherwise. Any unauthorized use, copying, disclosure or distribution of this information is strictly prohibited.
<br>
</body></html>