<div dir="ltr">Greetings,<div><br></div><div>We have experienced some type of issue that is causing both of our MDS servers to only be able to mount the mdt device in read only mode.  Here are some of the error messages we are seeing in the log files below.   We lost our Lustre expert a while back and we are not sure how to proceed to troubleshoot this issue.   Can anybody provide us guidance on how to proceed?</div><div><br></div><div>Thanks,</div><div><br></div><div>Mike</div><div><br></div><div><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Jun 20 15:12:14 hyd-mds1 kernel: INFO: task mount.lustre:4123 blocked for more than 120 seconds.</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Jun 20 15:12:14 hyd-mds1 kernel: &quot;echo 0 &gt; /proc/sys/kernel/hung_task_timeout_secs&quot; disables this message.</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Jun 20 15:12:14 hyd-mds1 kernel: mount.lustre    D ffff9f27a3bc5230     0  4123      1 0x00000086</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Jun 20 15:12:14 hyd-mds1 kernel: Call Trace:</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Jun 20 15:12:14 hyd-mds1 kernel: [&lt;ffffffffbb585da9&gt;] schedule+0x29/0x70</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Jun 20 15:12:14 hyd-mds1 kernel: [&lt;ffffffffbb5838b1&gt;] schedule_timeout+0x221/0x2d0</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Jun 20 15:12:14 hyd-mds1 kernel: [&lt;ffffffffbaf6b8e5&gt;] ? tracing_is_on+0x15/0x30</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Jun 20 15:12:14 hyd-mds1 kernel: [&lt;ffffffffbaf6f5bd&gt;] ? tracing_record_cmdline+0x1d/0x120</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Jun 20 15:12:14 hyd-mds1 kernel: [&lt;ffffffffbaf77d9b&gt;] ? probe_sched_wakeup+0x2b/0xa0</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Jun 20 15:12:14 hyd-mds1 kernel: [&lt;ffffffffbaed7d15&gt;] ? ttwu_do_wakeup+0xb5/0xe0</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Jun 20 15:12:14 hyd-mds1 kernel: [&lt;ffffffffbb58615d&gt;] wait_for_completion+0xfd/0x140</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Jun 20 15:12:14 hyd-mds1 kernel: [&lt;ffffffffbaedb990&gt;] ? wake_up_state+0x20/0x20</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Jun 20 15:12:14 hyd-mds1 kernel: [&lt;ffffffffc0f529a4&gt;] llog_process_or_fork+0x244/0x450 [obdclass]</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Jun 20 15:12:14 hyd-mds1 kernel: [&lt;ffffffffc0f52bc4&gt;] llog_process+0x14/0x20 [obdclass]</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Jun 20 15:12:14 hyd-mds1 kernel: [&lt;ffffffffc0f85d05&gt;] class_config_parse_llog+0x125/0x350 [obdclass]</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Jun 20 15:12:14 hyd-mds1 kernel: [&lt;ffffffffc0a69fc0&gt;] mgc_process_cfg_log+0x790/0xc40 [mgc]</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Jun 20 15:12:14 hyd-mds1 kernel: [&lt;ffffffffc0a6d4cc&gt;] mgc_process_log+0x3dc/0x8f0 [mgc]</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Jun 20 15:12:14 hyd-mds1 kernel: [&lt;ffffffffc0a6e15f&gt;] ? config_recover_log_add+0x13f/0x280 [mgc]</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Jun 20 15:12:14 hyd-mds1 kernel: [&lt;ffffffffc0f8df40&gt;] ? class_config_dump_handler+0x7e0/0x7e0 [obdclass]</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Jun 20 15:12:14 hyd-mds1 kernel: [&lt;ffffffffc0a6eb2b&gt;] mgc_process_config+0x88b/0x13f0 [mgc]</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Jun 20 15:12:14 hyd-mds1 kernel: [&lt;ffffffffc0f91b58&gt;] lustre_process_log+0x2d8/0xad0 [obdclass]</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Jun 20 15:12:14 hyd-mds1 kernel: [&lt;ffffffffc0e5a177&gt;] ? libcfs_debug_msg+0x57/0x80 [libcfs]</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Jun 20 15:12:14 hyd-mds1 kernel: [&lt;ffffffffc0f7c8b9&gt;] ? lprocfs_counter_add+0xf9/0x160 [obdclass]</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Jun 20 15:12:14 hyd-mds1 kernel: [&lt;ffffffffc0fc08f4&gt;] server_start_targets+0x13a4/0x2a20 [obdclass]</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Jun 20 15:12:14 hyd-mds1 kernel: [&lt;ffffffffc0f94bb0&gt;] ? lustre_start_mgc+0x260/0x2510 [obdclass]</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Jun 20 15:12:14 hyd-mds1 kernel: [&lt;ffffffffc0f8df40&gt;] ? class_config_dump_handler+0x7e0/0x7e0 [obdclass]</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Jun 20 15:12:14 hyd-mds1 kernel: [&lt;ffffffffc0fc303c&gt;] server_fill_super+0x10cc/0x1890 [obdclass]</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Jun 20 15:12:14 hyd-mds1 kernel: [&lt;ffffffffc0f97a08&gt;] lustre_fill_super+0x468/0x960 [obdclass]</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Jun 20 15:12:14 hyd-mds1 kernel: [&lt;ffffffffc0f975a0&gt;] ? lustre_common_put_super+0x270/0x270 [obdclass]</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Jun 20 15:12:14 hyd-mds1 kernel: [&lt;ffffffffbb0510cf&gt;] mount_nodev+0x4f/0xb0</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Jun 20 15:12:14 hyd-mds1 kernel: [&lt;ffffffffc0f8f9a8&gt;] lustre_mount+0x38/0x60 [obdclass]</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Jun 20 15:12:14 hyd-mds1 kernel: [&lt;ffffffffbb051c4e&gt;] mount_fs+0x3e/0x1b0</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Jun 20 15:12:14 hyd-mds1 kernel: [&lt;ffffffffbb0707a7&gt;] vfs_kern_mount+0x67/0x110</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Jun 20 15:12:14 hyd-mds1 kernel: [&lt;ffffffffbb072edf&gt;] do_mount+0x1ef/0xd00</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Jun 20 15:12:14 hyd-mds1 kernel: [&lt;ffffffffbb049d7a&gt;] ? __check_object_size+0x1ca/0x250</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Jun 20 15:12:14 hyd-mds1 kernel: [&lt;ffffffffbb0288ec&gt;] ? kmem_cache_alloc_trace+0x3c/0x200</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Jun 20 15:12:14 hyd-mds1 kernel: [&lt;ffffffffbb073d33&gt;] SyS_mount+0x83/0xd0</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Jun 20 15:12:14 hyd-mds1 kernel: [&lt;ffffffffbb592ed2&gt;] system_call_fastpath+0x25/0x2a</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Jun 20 15:13:14 hyd-mds1 kernel: LNet: 4458:0:(o2iblnd_cb.c:3397:kiblnd_check_conns()) Timed out tx for 172.16.100.4@o2ib: 9 seconds</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Jun 20 15:13:14 hyd-mds1 kernel: LNet: 4458:0:(o2iblnd_cb.c:3397:kiblnd_check_conns()) Skipped 239 previous similar messages</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Jun 20 15:14:14 hyd-mds1 kernel: INFO: task mount.lustre:4123 blocked for more than 120 seconds.</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Jun 20 15:14:14 hyd-mds1 kernel: &quot;echo 0 &gt; /proc/sys/kernel/hung_task_timeout_secs&quot; disables this message.</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Jun 20 15:14:14 hyd-mds1 kernel: mount.lustre    D ffff9f27a3bc5230     0  4123      1 0x00000086</span></p></div><div><br></div><div>dumpe2fs seems to show that the file systems are clean i.e.</div><div><br></div><div><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">dumpe2fs 1.45.6.wc1 (20-Mar-2020)</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Filesystem volume name:   hydra-MDT0000</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Last mounted on:          /</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Filesystem UUID:          3ae09231-7f2a-43b3-a4ee-7f36080b5a66</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Filesystem magic number:  0xEF53</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Filesystem revision #:    1 (dynamic)</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Filesystem features:      has_journal ext_attr resize_inode dir_index filetype mmp flex_bg dirdata sparse_super large_file huge_file uninit_bg dir_nlink quota</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Filesystem flags:         signed_directory_hash </span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Default mount options:    user_xattr acl</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Filesystem state:         clean</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Errors behavior:          Continue</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Filesystem OS type:       Linux</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Inode count:              2247671504</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Block count:              1404931944</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Reserved block count:     70246597</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Free blocks:              807627552</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Free inodes:              2100036536</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">First block:              0</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Block size:               4096</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Fragment size:            4096</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Reserved GDT blocks:      1024</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Blocks per group:         20472</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Fragments per group:      20472</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Inodes per group:         32752</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Inode blocks per group:   8188</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Flex block group size:    16</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Filesystem created:       Thu Aug  8 14:21:01 2019</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Last mount time:          Tue Jun 20 15:19:03 2023</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Last write time:          Wed Jun 21 10:43:51 2023</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Mount count:              38</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Maximum mount count:      -1</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Last checked:             Thu Aug  8 14:21:01 2019</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Check interval:           0 (&lt;none&gt;)</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Lifetime writes:          219 TB</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Reserved blocks uid:      0 (user root)</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Reserved blocks gid:      0 (group root)</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">First inode:              11</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Inode size:<span style="white-space-collapse: preserve;">    </span>          1024</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Required extra isize:     32</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Desired extra isize:      32</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Journal inode:            8</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Default directory hash:   half_md4</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Directory Hash Seed:      2e518531-82d9-4652-9acd-9cf9ca09c399</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Journal backup:           inode blocks</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">MMP block number:         1851467</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">MMP update interval:      5</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">User quota inode:         3</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Group quota inode:        4</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Journal features:         journal_incompat_revoke</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Journal size:             4096M</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Journal length:           1048576</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Journal sequence:         0x0a280713</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">Journal start:            0</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">MMP_block:</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">    mmp_magic: 0x4d4d50</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">    mmp_check_interval: 6</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">    mmp_sequence: 0xff4d4d50</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">    mmp_update_date: Wed Jun 21 10:43:51 2023</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">    mmp_update_time: 1687358631</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">    mmp_node_name: <a href="http://hyd-mds1.uncc.edu/" target="_blank">hyd-mds1.uncc.edu</a></span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;font-size:14px;line-height:normal;font-family:Menlo;color:rgb(16,8,182)"><span style="font-variant-ligatures:no-common-ligatures">    mmp_device_name: dm-0</span></p></div></div>