[lustre-discuss] problems to mount MDS and MDT

Abdeslam Tahari abeslam at gmail.com
Mon May 17 16:32:23 PDT 2021


here is an attachment of the output of debugging file

Le mar. 18 mai 2021 à 01:19, Abdeslam Tahari <abeslam at gmail.com> a écrit :

> Will do.
>  i join an attachment of the debug output
> i hope that will help
>
> Le lun. 17 mai 2021 à 21:50, Colin Faber <cfaber at gmail.com> a écrit :
>
>> It appears part of the debug data is missing (the part before you posted
>> it), Can you try again, lctl dk > /dev/null to clear it then try your mount
>> and grab the debug again?
>>
>> On Mon, May 17, 2021 at 1:35 PM Abdeslam Tahari <abeslam at gmail.com>
>> wrote:
>>
>>> Thank you Colin
>>>
>>> No i don't have iptables or rules
>>>
>>> firewalled is stopped selinux disabled as well
>>>  iptables -L
>>> Chain INPUT (policy ACCEPT)
>>> target     prot opt source               destination
>>>
>>> Chain FORWARD (policy ACCEPT)
>>> target     prot opt source               destination
>>>
>>> Chain OUTPUT (policy ACCEPT)
>>> target     prot opt source               destination
>>>
>>>
>>> Regards
>>>
>>>
>>> Regards
>>>
>>> Le lun. 17 mai 2021 à 21:29, Colin Faber <cfaber at gmail.com> a écrit :
>>>
>>>> Firewall rules dealing with localhost?
>>>>
>>>> On Mon, May 17, 2021 at 11:33 AM Abdeslam Tahari via lustre-discuss <
>>>> lustre-discuss at lists.lustre.org> wrote:
>>>>
>>>>> Hello
>>>>>
>>>>> i have a problem to mount the mds/mdt luster, it wont mount at all and
>>>>> there is no message errors at the console
>>>>>
>>>>> -it does not show errors or messages while mounting it
>>>>>
>>>>> here are some debug file logs
>>>>>
>>>>>
>>>>> i specify it is a new project that i am doing.
>>>>>
>>>>> the version and packages of luter installed:
>>>>> kmod-lustre-2.12.5-1.el7.x86_64
>>>>> kernel-devel-3.10.0-1127.8.2.el7_lustre.x86_64
>>>>> lustre-2.12.5-1.el7.x86_64
>>>>> lustre-resource-agents-2.12.5-1.el7.x86_64
>>>>> kernel-3.10.0-1160.2.1.el7_lustre.x86_64
>>>>> kernel-debuginfo-common-x86_64-3.10.0-1160.2.1.el7_lustre.x86_64
>>>>> kmod-lustre-osd-ldiskfs-2.12.5-1.el7.x86_64
>>>>> kernel-3.10.0-1127.8.2.el7_lustre.x86_64
>>>>> lustre-osd-ldiskfs-mount-2.12.5-1.el7.x86_64
>>>>>
>>>>>
>>>>>
>>>>> the system(os) Centos 7
>>>>>
>>>>> the kernel
>>>>> Linux lustre-mds1 3.10.0-1127.8.2.el7_lustre.x86_64
>>>>>  cat /etc/redhat-release
>>>>>
>>>>>
>>>>> when i mount the luster file-system it wont show up and no errors
>>>>>
>>>>> mount -t lustre /dev/sda /mds
>>>>>
>>>>> lctl dl  does not show up
>>>>>
>>>>> df -h   no mount point for /dev/sda
>>>>>
>>>>>
>>>>> lctl dl
>>>>>
>>>>> shows this:
>>>>> lctl dl
>>>>>   0 UP osd-ldiskfs lustre-MDT0000-osd lustre-MDT0000-osd_UUID 3
>>>>>   2 UP mgc MGC10.0.1.70 at tcp 57e06c2d-5294-f034-fd95-460cee4f92b7 4
>>>>>   3 UP mds MDS MDS_uuid 2
>>>>>
>>>>>
>>>>> but unfortunately it disappears after 03 seconds
>>>>>
>>>>> lctl  dl shows nothing
>>>>>
>>>>> lctl dk
>>>>>
>>>>> shows this debug output
>>>>>
>>>>>
>>>>> 00000020:00000080:18.0:1621276062.004338:0:13403:0:(obd_config.c:1128:class_process_config())
>>>>> processing cmd: cf006
>>>>> 00000020:00000080:18.0:1621276062.004341:0:13403:0:(obd_config.c:1147:class_process_config())
>>>>> removing mappings for uuid MGC10.0.1.70 at tcp_0
>>>>> 00000020:01000004:18.0:1621276062.004346:0:13403:0:(obd_mount.c:661:lustre_put_lsi())
>>>>> put ffff9bbbf91d5800 1
>>>>> 00000020:00000080:18.0:1621276062.004351:0:13403:0:(genops.c:1501:class_disconnect())
>>>>> disconnect: cookie 0x256dd92fc5bf929c
>>>>> 00000020:00000080:18.0:1621276062.004354:0:13403:0:(genops.c:1024:class_export_put())
>>>>> final put ffff9bbf3e66a400/lustre-MDT0000-osd_UUID
>>>>> 00000020:01000000:18.0:1621276062.004361:0:13403:0:(obd_config.c:2100:class_manual_cleanup())
>>>>> Manual cleanup of lustre-MDT0000-osd (flags='')
>>>>> 00000020:00000080:18.0:1621276062.004368:0:821:0:(genops.c:974:class_export_destroy())
>>>>> destroying export ffff9bbf3e66a400/lustre-MDT0000-osd_UUID for
>>>>> lustre-MDT0000-osd
>>>>> 00000020:00000080:18.0:1621276062.004376:0:13403:0:(obd_config.c:1128:class_process_config())
>>>>> processing cmd: cf004
>>>>> 00000020:00000080:18.0:1621276062.004379:0:13403:0:(obd_config.c:659:class_cleanup())
>>>>> lustre-MDT0000-osd: forcing exports to disconnect: 0/0
>>>>> 00000020:00080000:18.0:1621276062.004382:0:13403:0:(genops.c:1590:class_disconnect_exports())
>>>>> OBD device 0 (ffff9bbf47141080) has no exports
>>>>> 00000020:00000080:18.0:1621276062.004788:0:13403:0:(obd_config.c:1128:class_process_config())
>>>>> processing cmd: cf002
>>>>> 00000020:00000080:18.0:1621276062.004791:0:13403:0:(obd_config.c:589:class_detach())
>>>>> detach on obd lustre-MDT0000-osd (uuid lustre-MDT0000-osd_UUID)
>>>>> 00000020:00000080:18.0:1621276062.004794:0:13403:0:(genops.c:1024:class_export_put())
>>>>> final put ffff9bbf48800c00/lustre-MDT0000-osd_UUID
>>>>> 00000020:00000080:18.0:1621276062.004796:0:13403:0:(genops.c:974:class_export_destroy())
>>>>> destroying export ffff9bbf48800c00/lustre-MDT0000-osd_UUID for
>>>>> lustre-MDT0000-osd
>>>>> 00000020:01000000:18.0:1621276062.004799:0:13403:0:(genops.c:481:class_free_dev())
>>>>> finishing cleanup of obd lustre-MDT0000-osd (lustre-MDT0000-osd_UUID)
>>>>> 00000020:01000004:18.0:1621276062.450759:0:13403:0:(obd_mount.c:605:lustre_free_lsi())
>>>>> Freeing lsi ffff9bbbf91d6800
>>>>> 00000020:01000000:18.0:1621276062.450805:0:13403:0:(obd_config.c:2100:class_manual_cleanup())
>>>>> Manual cleanup of MDS (flags='F')
>>>>> 00000020:00000080:18.0:1621276062.450806:0:13403:0:(obd_config.c:1128:class_process_config())
>>>>> processing cmd: cf004
>>>>> 00000020:00000080:18.0:1621276062.450807:0:13403:0:(obd_config.c:659:class_cleanup())
>>>>> MDS: forcing exports to disconnect: 0/0
>>>>> 00000020:00080000:18.0:1621276062.450809:0:13403:0:(genops.c:1590:class_disconnect_exports())
>>>>> OBD device 3 (ffff9bbf43fdd280) has no exports
>>>>> 00000020:00000080:58.0F:1621276062.490781:0:13403:0:(obd_config.c:1128:class_process_config())
>>>>> processing cmd: cf002
>>>>> 00000020:00000080:58.0:1621276062.490787:0:13403:0:(obd_config.c:589:class_detach())
>>>>> detach on obd MDS (uuid MDS_uuid)
>>>>> 00000020:00000080:58.0:1621276062.490788:0:13403:0:(genops.c:1024:class_export_put())
>>>>> final put ffff9bbf3e668800/MDS_uuid
>>>>> 00000020:00000080:58.0:1621276062.490790:0:13403:0:(genops.c:974:class_export_destroy())
>>>>> destroying export ffff9bbf3e668800/MDS_uuid for MDS
>>>>> 00000020:01000000:58.0:1621276062.490791:0:13403:0:(genops.c:481:class_free_dev())
>>>>> finishing cleanup of obd MDS (MDS_uuid)
>>>>> 00000020:02000400:58.0:1621276062.490877:0:13403:0:(obd_mount_server.c:1642:server_put_super())
>>>>> server umount lustre-MDT0000 complete
>>>>> 00000400:02020000:42.0:1621276086.284109:0:5400:0:(acceptor.c:321:lnet_accept())
>>>>> 120-3: Refusing connection from 127.0.0.1 for 127.0.0.1 at tcp: No
>>>>> matching NI
>>>>> 00000800:00020000:6.0:1621276086.284152:0:5383:0:(socklnd_cb.c:1817:ksocknal_recv_hello())
>>>>> Error -104 reading HELLO from 127.0.0.1
>>>>> 00000400:02020000:6.0:1621276086.284174:0:5383:0:(acceptor.c:127:lnet_connect_console_error())
>>>>> 11b-b: Connection to 127.0.0.1 at tcp at host 127.0.0.1 on port 988 was
>>>>> reset: is it running a compatible version of Lustre and is 127.0.0.1 at tcp
>>>>> one of its NIDs?
>>>>> 00000800:00000100:6.0:1621276086.284189:0:5383:0:(socklnd_cb.c:438:ksocknal_txlist_done())
>>>>> Deleting packet type 2 len 0 10.0.1.70 at tcp->127.0.0.1 at tcp
>>>>> 00000800:00000100:34.0:1621276136.363882:0:5401:0:(socklnd_cb.c:979:ksocknal_launch_packet())
>>>>> No usable routes to 12345-127.0.0.1 at tcp
>>>>> 00000400:02020000:42.0:1621276186.440095:0:5400:0:(acceptor.c:321:lnet_accept())
>>>>> 120-3: Refusing connection from 127.0.0.1 for 127.0.0.1 at tcp: No
>>>>> matching NI
>>>>> 00000800:00020000:44.0:1621276186.446533:0:5386:0:(socklnd_cb.c:1817:ksocknal_recv_hello())
>>>>> Error -104 reading HELLO from 127.0.0.1
>>>>> 00000400:02020000:44.0:1621276186.452996:0:5386:0:(acceptor.c:127:lnet_connect_console_error())
>>>>> 11b-b: Connection to 127.0.0.1 at tcp at host 127.0.0.1 on port 988 was
>>>>> reset: is it running a compatible version of Lustre and is 127.0.0.1 at tcp
>>>>> one of its NIDs?
>>>>> 00000800:00000100:44.0:1621276186.461433:0:5386:0:(socklnd_cb.c:438:ksocknal_txlist_done())
>>>>> Deleting packet type 2 len 0 10.0.1.70 at tcp->127.0.0.1 at tcp
>>>>> Debug log: 872 lines, 872 kept, 0 dropped, 0 bad.
>>>>>
>>>>>
>>>>>
>>>>> I just cant find out any help would be very appreciated
>>>>>
>>>>>
>>>>> Thanks all
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Tahari.Abdeslam
>>>>> _______________________________________________
>>>>> lustre-discuss mailing list
>>>>> lustre-discuss at lists.lustre.org
>>>>> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>>>>>
>>>>
>>>
>>> --
>>> Tahari.Abdeslam
>>>
>>
>
> --
> Tahari.Abdeslam
>


-- 
Tahari.Abdeslam
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20210518/21ea2c42/attachment-0001.html>
-------------- next part --------------

00000020:02000400:57.0:1621282930.338345:0:9961:0:(obd_mount_server.c:1642:server_put_super()) server umount lustre-MDT0000 complete
00000020:01200004:4.0:1621283828.945143:0:10966:0:(obd_mount.c:1542:lustre_fill_super()) VFS Op: sb ffff8958fc45e800
00000020:01000004:4.0:1621283828.945234:0:10966:0:(obd_mount.c:930:lmd_print())   mount data:
00000020:01000004:4.0:1621283828.945235:0:10966:0:(obd_mount.c:933:lmd_print()) device:  /dev/sda
00000020:01000004:4.0:1621283828.945237:0:10966:0:(obd_mount.c:934:lmd_print()) flags:   200
00000020:01000004:4.0:1621283828.945238:0:10966:0:(obd_mount.c:937:lmd_print()) options: user_xattr,errors=remount-ro
00000020:01000004:4.0:1621283828.945240:0:10966:0:(obd_mount.c:1589:lustre_fill_super()) Mounting server from /dev/sda
00000020:01000004:4.0:1621283828.945244:0:10966:0:(obd_mount_server.c:1881:osd_start()) Attempting to start lustre-MDT0000, type=osd-ldiskfs, lsifl=200005, mountfl=0
00000020:01000004:4.0:1621283828.945305:0:10966:0:(obd_mount.c:193:lustre_start_simple()) Starting obd lustre-MDT0000-osd (typ=osd-ldiskfs)
00000020:00000080:4.0:1621283828.945308:0:10966:0:(obd_config.c:1128:class_process_config()) processing cmd: cf001
00000020:00000080:4.0:1621283828.945317:0:10966:0:(genops.c:451:class_newdev()) Allocate new device lustre-MDT0000-osd (ffff8954fe90b180)
00000020:00000080:4.0:1621283828.945379:0:10966:0:(obd_config.c:431:class_attach()) OBD: dev 0 attached type osd-ldiskfs with refcount 1
00000020:00000080:4.0:1621283828.945384:0:10966:0:(obd_config.c:1128:class_process_config()) processing cmd: cf003
00100000:10000000:6.0:1621283834.068537:0:10966:0:(osd_scrub.c:1855:osd_ios_index_register()) lustre-MDT0000: index object [0x200000003:0x13:0x0] (16/1) registered
00100000:10000000:6.0:1621283834.069416:0:10966:0:(osd_scrub.c:1855:osd_ios_index_register()) lustre-MDT0000: index object [0x200000001:0x3:0x0] (8/24) registered
00100000:10000000:6.0:1621283834.071214:0:10966:0:(osd_scrub.c:1855:osd_ios_index_register()) lustre-MDT0000: index object [0x200000003:0x18:0x0] (16/1) registered
00100000:10000000:6.0:1621283834.071861:0:10966:0:(osd_scrub.c:1855:osd_ios_index_register()) lustre-MDT0000: index object [0x200000003:0x30:0x0] (24/16) registered
00100000:10000000:6.0:1621283834.072000:0:10966:0:(osd_scrub.c:1855:osd_ios_index_register()) lustre-MDT0000: index object [0x200000003:0x15:0x0] (16/1) registered
00100000:10000000:6.0:1621283834.072169:0:10966:0:(osd_scrub.c:1855:osd_ios_index_register()) lustre-MDT0000: index object [0x200000003:0x2a:0x0] (24/16) registered
00100000:10000000:6.0:1621283834.072350:0:10966:0:(osd_scrub.c:1855:osd_ios_index_register()) lustre-MDT0000: index object [0x200000003:0x2c:0x0] (24/16) registered
00100000:10000000:6.0:1621283834.072578:0:10966:0:(osd_scrub.c:1855:osd_ios_index_register()) lustre-MDT0000: index object [0x200000003:0x2d:0x0] (24/16) registered
00100000:10000000:6.0:1621283834.074246:0:10966:0:(osd_scrub.c:1855:osd_ios_index_register()) lustre-MDT0000: index object [0x200000003:0x32:0x0] (24/16) registered
00100000:10000000:6.0:1621283834.074420:0:10966:0:(osd_scrub.c:1855:osd_ios_index_register()) lustre-MDT0000: index object [0x200000003:0x33:0x0] (24/16) registered
00100000:10000000:6.0:1621283834.074571:0:10966:0:(osd_scrub.c:1855:osd_ios_index_register()) lustre-MDT0000: index object [0x200000003:0x19:0x0] (16/1) registered
00100000:10000000:6.0:1621283834.074726:0:10966:0:(osd_scrub.c:1855:osd_ios_index_register()) lustre-MDT0000: index object [0x200000003:0x22:0x0] (16/1) registered
00100000:10000000:6.0:1621283834.074881:0:10966:0:(osd_scrub.c:1855:osd_ios_index_register()) lustre-MDT0000: index object [0x200000003:0x21:0x0] (16/1) registered
00100000:10000000:6.0:1621283834.075027:0:10966:0:(osd_scrub.c:1855:osd_ios_index_register()) lustre-MDT0000: index object [0x200000003:0x2f:0x0] (24/16) registered
00100000:10000000:6.0:1621283834.075109:0:10966:0:(osd_scrub.c:1855:osd_ios_index_register()) lustre-MDT0000: index object [0x200000003:0x34:0x0] (24/16) registered
00100000:10000000:6.0:1621283834.075241:0:10966:0:(osd_scrub.c:1855:osd_ios_index_register()) lustre-MDT0000: index object [0x200000003:0x20:0x0] (16/1) registered
00100000:10000000:6.0:1621283834.075302:0:10966:0:(osd_scrub.c:1855:osd_ios_index_register()) lustre-MDT0000: index object [0x200000003:0x1c:0x0] (16/1) registered
00100000:10000000:6.0:1621283834.075436:0:10966:0:(osd_scrub.c:1855:osd_ios_index_register()) lustre-MDT0000: index object [0x200000003:0x2b:0x0] (24/16) registered
00100000:10000000:6.0:1621283834.075497:0:10966:0:(osd_scrub.c:1855:osd_ios_index_register()) lustre-MDT0000: index object [0x200000003:0x1d:0x0] (16/1) registered
00100000:10000000:6.0:1621283834.075641:0:10966:0:(osd_scrub.c:1855:osd_ios_index_register()) lustre-MDT0000: index object [0x200000003:0x17:0x0] (16/1) registered
00100000:10000000:6.0:1621283834.075719:0:10966:0:(osd_scrub.c:1855:osd_ios_index_register()) lustre-MDT0000: index object [0x200000003:0x2e:0x0] (24/16) registered
00100000:10000000:6.0:1621283834.075851:0:10966:0:(osd_scrub.c:1855:osd_ios_index_register()) lustre-MDT0000: index object [0x200000003:0x1e:0x0] (16/1) registered
00100000:10000000:6.0:1621283834.076022:0:10966:0:(osd_scrub.c:1855:osd_ios_index_register()) lustre-MDT0000: index object [0x200000003:0x27:0x0] (24/16) registered
00100000:10000000:6.0:1621283834.076099:0:10966:0:(osd_scrub.c:1855:osd_ios_index_register()) lustre-MDT0000: index object [0x200000003:0x1b:0x0] (16/1) registered
00100000:10000000:6.0:1621283834.076237:0:10966:0:(osd_scrub.c:1855:osd_ios_index_register()) lustre-MDT0000: index object [0x200000003:0x1f:0x0] (16/1) registered
00100000:10000000:6.0:1621283834.076392:0:10966:0:(osd_scrub.c:1855:osd_ios_index_register()) lustre-MDT0000: index object [0x200000003:0x25:0x0] (24/16) registered
00100000:10000000:6.0:1621283834.076536:0:10966:0:(osd_scrub.c:1855:osd_ios_index_register()) lustre-MDT0000: index object [0x200000003:0x14:0x0] (16/1) registered
00100000:10000000:6.0:1621283834.076698:0:10966:0:(osd_scrub.c:1855:osd_ios_index_register()) lustre-MDT0000: index object [0x200000003:0x16:0x0] (16/1) registered
00100000:10000000:6.0:1621283834.076885:0:10966:0:(osd_scrub.c:1855:osd_ios_index_register()) lustre-MDT0000: index object [0x200000003:0x29:0x0] (24/16) registered
00100000:10000000:6.0:1621283834.077053:0:10966:0:(osd_scrub.c:1855:osd_ios_index_register()) lustre-MDT0000: index object [0x200000003:0x1a:0x0] (16/1) registered
00100000:10000000:6.0:1621283834.077132:0:10966:0:(osd_scrub.c:1855:osd_ios_index_register()) lustre-MDT0000: index object [0x200000003:0x31:0x0] (24/16) registered
00100000:10000000:6.0:1621283834.077193:0:10966:0:(osd_scrub.c:1855:osd_ios_index_register()) lustre-MDT0000: index object [0x200000003:0x23:0x0] (16/1) registered
00100000:10000000:6.0:1621283834.077360:0:10966:0:(osd_scrub.c:1855:osd_ios_index_register()) lustre-MDT0000: index object [0x200000003:0x26:0x0] (24/16) registered
00100000:10000000:6.0:1621283834.077547:0:10966:0:(osd_scrub.c:1855:osd_ios_index_register()) lustre-MDT0000: index object [0x200000003:0x28:0x0] (24/16) registered
00100000:10000000:6.0:1621283834.078252:0:10966:0:(osd_scrub.c:1855:osd_ios_index_register()) lustre-MDT0000: index object [0x200000003:0x2:0x0] (8/32) registered
00100000:10000000:6.0:1621283834.078919:0:10966:0:(osd_scrub.c:1855:osd_ios_index_register()) lustre-MDT0000: index object [0x200000005:0x20001:0x0] (8/8) registered
00100000:10000000:6.0:1621283834.079997:0:10966:0:(osd_scrub.c:1855:osd_ios_index_register()) lustre-MDT0000: index object [0x200000005:0x2:0x0] (8/8) registered
00100000:10000000:6.0:1621283834.080162:0:10966:0:(osd_scrub.c:1855:osd_ios_index_register()) lustre-MDT0000: index object [0x200000003:0xd:0x0] (8/32) registered
00100000:10000000:6.0:1621283834.080385:0:10966:0:(osd_scrub.c:1855:osd_ios_index_register()) lustre-MDT0000: index object [0x200000003:0xb:0x0] (8/32) registered
00100000:10000000:6.0:1621283834.080462:0:10966:0:(osd_scrub.c:1855:osd_ios_index_register()) lustre-MDT0000: index object [0x200000005:0x20002:0x0] (8/8) registered
00100000:10000000:6.0:1621283834.080628:0:10966:0:(osd_scrub.c:1855:osd_ios_index_register()) lustre-MDT0000: index object [0x200000005:0x3:0x0] (8/8) registered
00100000:10000000:6.0:1621283834.080720:0:10966:0:(osd_scrub.c:1855:osd_ios_index_register()) lustre-MDT0000: index object [0x200000003:0xc:0x0] (8/32) registered
00100000:10000000:6.0:1621283834.080882:0:10966:0:(osd_scrub.c:1855:osd_ios_index_register()) lustre-MDT0000: index object [0x200000005:0x20003:0x0] (8/8) registered
00100000:10000000:6.0:1621283834.081088:0:10966:0:(osd_scrub.c:1855:osd_ios_index_register()) lustre-MDT0000: index object [0x200000003:0xa:0x0] (8/32) registered
00100000:10000000:6.0:1621283834.081315:0:10966:0:(osd_scrub.c:1855:osd_ios_index_register()) lustre-MDT0000: index object [0x200000005:0x1:0x0] (8/8) registered
00100000:10000000:6.0:1621283834.081477:0:10966:0:(osd_scrub.c:1855:osd_ios_index_register()) lustre-MDT0000: index object [0x200000003:0xe:0x0] (8/32) registered
00100000:10000000:6.0:1621283834.081686:0:10966:0:(osd_scrub.c:1855:osd_ios_index_register()) lustre-MDT0000: index object [0x200000003:0x9:0x0] (8/32) registered
00100000:10000000:6.0:1621283834.083024:0:10966:0:(osd_scrub.c:1855:osd_ios_index_register()) lustre-MDT0000: index object [0x200000006:0x2010000:0x0] (8/32) registered
00100000:10000000:6.0:1621283834.083192:0:10966:0:(osd_scrub.c:1855:osd_ios_index_register()) lustre-MDT0000: index object [0x200000006:0x1010000:0x0] (8/32) registered
00100000:10000000:6.0:1621283834.083402:0:10966:0:(osd_scrub.c:1855:osd_ios_index_register()) lustre-MDT0000: index object [0x200000006:0x10000:0x0] (8/32) registered
00100000:10000000:6.0:1621283834.083531:0:10966:0:(osd_scrub.c:1855:osd_ios_index_register()) lustre-MDT0000: index object [0x200000006:0x1020000:0x0] (8/32) registered
00100000:10000000:6.0:1621283834.083597:0:10966:0:(osd_scrub.c:1855:osd_ios_index_register()) lustre-MDT0000: index object [0x200000006:0x20000:0x0] (8/32) registered
00100000:10000000:6.0:1621283834.083663:0:10966:0:(osd_scrub.c:1855:osd_ios_index_register()) lustre-MDT0000: index object [0x200000006:0x2020000:0x0] (8/32) registered
00000020:00000080:6.0:1621283834.083763:0:10966:0:(obd_config.c:538:class_setup()) finished setup of obd lustre-MDT0000-osd (uuid lustre-MDT0000-osd_UUID)
00080000:01000000:6.0:1621283834.083768:0:10966:0:(osd_handler.c:8039:osd_obd_connect()) connect #0
00000020:00000080:6.0:1621283834.083771:0:10966:0:(genops.c:1417:class_connect()) connect: client lustre-MDT0000-osd_UUID, cookie 0xc56ba8a9d15727d2
00000020:01000004:6.0:1621283834.083774:0:10966:0:(obd_mount_server.c:1964:server_fill_super()) Found service lustre-MDT0000 on device /dev/sda
00000020:01000000:6.0:1621283834.083807:0:10966:0:(obd_mount_server.c:235:server_start_mgs()) Start MGS service MGS
00000020:01000004:6.0:1621283834.083809:0:10966:0:(obd_mount_server.c:119:server_register_mount()) register mount ffff8958fc45e800 from MGS
00000020:01000004:6.0:1621283834.083811:0:10966:0:(obd_mount.c:193:lustre_start_simple()) Starting obd MGS (typ=mgs)
00000020:00000080:6.0:1621283834.083812:0:10966:0:(obd_config.c:1128:class_process_config()) processing cmd: cf001
00000020:00000080:6.0:1621283834.083816:0:10966:0:(genops.c:451:class_newdev()) Allocate new device MGS (ffff8954fc42b180)
00000020:00000080:6.0:1621283834.083845:0:10966:0:(obd_config.c:431:class_attach()) OBD: dev 1 attached type mgs with refcount 1
00000020:00000080:6.0:1621283834.083847:0:10966:0:(obd_config.c:1128:class_process_config()) processing cmd: cf003
00000020:01000004:6.0:1621283834.083878:0:10966:0:(obd_mount_server.c:169:server_get_mount()) get mount ffff8958fc45e800 from MGS, refs=2
00080000:01000000:6.0:1621283834.083881:0:10966:0:(osd_handler.c:8039:osd_obd_connect()) connect #1
00000020:00000080:6.0:1621283834.083882:0:10966:0:(genops.c:1417:class_connect()) connect: client lustre-MDT0000-osd_UUID, cookie 0xc56ba8a9d15727e0
00000040:01000000:6.0:1621283834.087260:0:10966:0:(llog_obd.c:214:llog_setup()) obd MGS ctxt 0 is initialized
10000000:01000000:43.0:1621283834.087390:0:10996:0:(mgc_request.c:90:mgc_name2resid()) log params to resid 0x736d61726170/0x2 (params)
20000000:01000000:6.0:1621283834.087392:0:10966:0:(mgs_nids.c:393:mgs_nidtbl_init_fs()) IR: current version is 2
20000000:01000000:6.0:1621283834.087398:0:10966:0:(mgs_llog.c:560:mgs_find_or_make_fsdb_nolock()) Created new db: rc = 0
00000040:00080000:6.0:1621283834.087407:0:10966:0:(llog_osd.c:232:llog_osd_read_header()) not reading header from 0-byte log
20000000:01000000:6.0:1621283834.087412:0:10966:0:(mgs_llog.c:560:mgs_find_or_make_fsdb_nolock()) Created new db: rc = 0
00000100:00080000:43.0:1621283834.087956:0:11001:0:(pinger.c:588:ping_evictor_main()) Starting Ping Evictor
00000020:00000080:6.0:1621283834.088014:0:10966:0:(obd_config.c:538:class_setup()) finished setup of obd MGS (uuid MGS)
00000020:01000004:6.0:1621283834.088048:0:10966:0:(obd_mount.c:339:lustre_start_mgc()) Start MGC 'MGC10.0.1.70 at tcp'
00000020:01000004:6.0:1621283834.088050:0:10966:0:(obd_mount.c:346:lustre_start_mgc()) mgs nids (null).
00000020:00000080:6.0:1621283834.088051:0:10966:0:(obd_config.c:1128:class_process_config()) processing cmd: cf005
00000020:00000080:6.0:1621283834.088053:0:10966:0:(obd_config.c:1139:class_process_config()) adding mapping from uuid MGC10.0.1.70 at tcp_0 to nid 0x9000000000000 (0 at lo)
00000020:00000080:6.0:1621283834.088055:0:10966:0:(obd_config.c:1128:class_process_config()) processing cmd: cf005
00000020:00000080:6.0:1621283834.088057:0:10966:0:(obd_config.c:1139:class_process_config()) adding mapping from uuid MGC10.0.1.70 at tcp_0 to nid 0x200000a000146 (10.0.1.70 at tcp)
00000020:01000004:6.0:1621283834.088064:0:10966:0:(obd_mount.c:193:lustre_start_simple()) Starting obd MGC10.0.1.70 at tcp (typ=mgc)
00000020:00000080:6.0:1621283834.088065:0:10966:0:(obd_config.c:1128:class_process_config()) processing cmd: cf001
00000020:00000080:6.0:1621283834.088069:0:10966:0:(genops.c:451:class_newdev()) Allocate new device MGC10.0.1.70 at tcp (ffff8954fc42a100)
00000020:00000080:6.0:1621283834.088100:0:10966:0:(obd_config.c:431:class_attach()) OBD: dev 2 attached type mgc with refcount 1
00000020:00000080:6.0:1621283834.088102:0:10966:0:(obd_config.c:1128:class_process_config()) processing cmd: cf003
00010000:00080000:6.0:1621283834.088133:0:10966:0:(ldlm_lib.c:114:import_set_conn()) imp ffff8954ff8d9800 at MGC10.0.1.70@tcp: add connection MGC10.0.1.70 at tcp_0 at head
00000040:01000000:6.0:1621283834.088169:0:10966:0:(llog_obd.c:214:llog_setup()) obd MGC10.0.1.70 at tcp ctxt 1 is initialized
10000000:01000000:43.0:1621283834.088243:0:11002:0:(mgc_request.c:622:mgc_requeue_thread()) Starting requeue thread
00000020:00000080:6.0:1621283834.088250:0:10966:0:(obd_config.c:538:class_setup()) finished setup of obd MGC10.0.1.70 at tcp (uuid 3867756d-71df-4dd5-e513-873bf69178ac)
00000020:00000080:6.0:1621283834.088255:0:10966:0:(genops.c:1417:class_connect()) connect: client 3867756d-71df-4dd5-e513-873bf69178ac, cookie 0xc56ba8a9d15727ee
00000100:00080000:6.0:1621283834.088258:0:10966:0:(import.c:543:import_select_connection()) MGC10.0.1.70 at tcp: connect to NID 0 at lo last attempt 0
00000100:00080000:6.0:1621283834.088260:0:10966:0:(import.c:619:import_select_connection()) MGC10.0.1.70 at tcp: import ffff8954ff8d9800 using connection MGC10.0.1.70 at tcp_0/0 at lo
00000100:00080000:6.0:1621283834.088275:0:10966:0:(pinger.c:387:ptlrpc_pinger_add_import()) adding pingable import 3867756d-71df-4dd5-e513-873bf69178ac->MGS
00000020:01000004:6.0:1621283834.088280:0:10966:0:(obd_mount_server.c:1330:server_start_targets()) starting target lustre-MDT0000
00000020:01000004:6.0:1621283834.088325:0:10966:0:(obd_mount.c:193:lustre_start_simple()) Starting obd MDS (typ=mds)
00000020:00000080:6.0:1621283834.088326:0:10966:0:(obd_config.c:1128:class_process_config()) processing cmd: cf001
00000020:00000080:6.0:1621283834.088328:0:10966:0:(genops.c:451:class_newdev()) Allocate new device MDS (ffff8954fc429080)
00000020:00000080:6.0:1621283834.088359:0:10966:0:(obd_config.c:431:class_attach()) OBD: dev 3 attached type mds with refcount 1
00000020:00000080:6.0:1621283834.088360:0:10966:0:(obd_config.c:1128:class_process_config()) processing cmd: cf003
00010000:00080000:43.0:1621283834.088431:0:11000:0:(ldlm_lib.c:1227:target_handle_connect()) MGS: connection from 3867756d-71df-4dd5-e513-873bf69178ac at 0@lo t0 exp           (null) cur 4567 last 0
00000020:00000080:43.0:1621283834.088458:0:11000:0:(genops.c:1417:class_connect()) connect: client 3867756d-71df-4dd5-e513-873bf69178ac, cookie 0xc56ba8a9d15727fc
00000020:01000000:43.0:1621283834.088464:0:11000:0:(lprocfs_status_server.c:491:lprocfs_exp_setup()) using hash ffff895905c25e00
00000100:00080000:43.0:1621283834.088502:0:11000:0:(import.c:86:import_set_state_nolock()) ffff895501f52800 : changing import state from RECOVER to FULL
00000100:02000000:43.0:1621283834.088507:0:11000:0:(import.c:1597:ptlrpc_import_recovery_state_machine()) MGS: Connection restored to MGC10.0.1.70 at tcp_0 (at 0 at lo)
00000100:00080000:11.0:1621283834.088562:0:6279:0:(import.c:1078:ptlrpc_connect_interpret()) MGC10.0.1.70 at tcp: connect to target with instance 0
00000100:00080000:11.0:1621283834.088572:0:6279:0:(import.c:939:ptlrpc_connect_set_flags()) MGC10.0.1.70 at tcp: Resetting ns_connect_flags to server flags: 0x2000011005002020
10000000:01000000:11.0:1621283834.088576:0:6279:0:(mgc_request.c:1363:mgc_import_event()) import event 0x808005
00000100:00080000:11.0:1621283834.088579:0:6279:0:(import.c:86:import_set_state_nolock()) ffff8954ff8d9800 MGS: changing import state from CONNECTING to FULL
10000000:01000000:11.0:1621283834.088584:0:6279:0:(mgc_request.c:1363:mgc_import_event()) import event 0x808004
00000100:00080000:11.0:1621283834.088589:0:6279:0:(pinger.c:203:ptlrpc_pinger_ir_up()) IR up
00000020:00000080:6.0:1621283834.159132:0:10966:0:(obd_config.c:538:class_setup()) finished setup of obd MDS (uuid MDS_uuid)
00000020:01000004:6.0:1621283834.159141:0:10966:0:(obd_mount_server.c:296:server_mgc_set_fs()) Set mgc disk for /dev/sda
00000040:01000000:6.0:1621283834.159178:0:10966:0:(llog_obd.c:214:llog_setup()) obd MGC10.0.1.70 at tcp ctxt 0 is initialized
00000020:01000004:6.0:1621283834.159188:0:10966:0:(obd_mount_server.c:1223:server_register_target()) Registration lustre-MDT0000, fs=lustre, 10.0.1.70 at tcp, index=0000, flags=0x5
10000000:01000000:6.0:1621283834.159190:0:10966:0:(mgc_request.c:1263:mgc_set_info_async()) register_target lustre-MDT0000 0x10000005
10000000:01000000:6.0:1621283834.159199:0:10966:0:(mgc_request.c:1202:mgc_target_register()) register lustre-MDT0000
20000000:01000000:43.0:1621283834.159274:0:11000:0:(mgs_llog.c:560:mgs_find_or_make_fsdb_nolock()) Created new db: rc = 0
20000000:01000000:43.0:1621283834.159278:0:11000:0:(mgs_handler.c:414:mgs_target_reg()) fs: lustre index: 0 is registered to MGS.
20000000:01000000:43.0:1621283834.159283:0:11000:0:(mgs_handler.c:444:mgs_target_reg()) Server lustre-MDT0000 is running on 0 at lo
10000000:01000000:40.0:1621283834.159369:0:11035:0:(mgc_request.c:90:mgc_name2resid()) log lustre to resid 0x65727473756c/0x2 (lustre)
20000000:01000000:43.0:1621283834.159615:0:11000:0:(mgs_nids.c:393:mgs_nidtbl_init_fs()) IR: current version is 4
20000000:01000000:43.0:1621283834.164929:0:11036:0:(mgs_llog.c:238:mgs_fsdb_handler()) MDT index is 0
00000040:00080000:43.0:1621283834.164936:0:11036:0:(llog.c:705:llog_process_thread()) stop processing plain 0x8:10:0 index 18 count 18
20000000:01000000:5.0:1621283834.165009:0:11000:0:(mgs_llog.c:560:mgs_find_or_make_fsdb_nolock()) Created new db: rc = 0
20000000:01000000:5.0:1621283834.165016:0:11000:0:(mgs_handler.c:558:mgs_target_reg()) replying with lustre-MDT0000, index=0, rc=0
10000000:01000000:6.0:1621283834.168136:0:10966:0:(mgc_request.c:1219:mgc_target_register()) register lustre-MDT0000 got index = 0
00000020:01000004:6.0:1621283834.168140:0:10966:0:(obd_mount_server.c:119:server_register_mount()) register mount ffff8958fc45e800 from lustre-MDT0000
10000000:01000000:6.0:1621283834.168143:0:10966:0:(mgc_request.c:2205:mgc_process_config()) parse_log lustre-MDT0000 from 0
10000000:01000000:6.0:1621283834.168144:0:10966:0:(mgc_request.c:329:config_log_add()) add config log lustre-MDT0000-0000000000000000
10000000:01000000:6.0:1621283834.168146:0:10966:0:(mgc_request.c:211:do_config_log_add()) do adding config log lustre-sptlrpc-ffff8954fc42a100
10000000:01000000:6.0:1621283834.168147:0:10966:0:(mgc_request.c:90:mgc_name2resid()) log lustre-sptlrpc to resid 0x65727473756c/0x0 (lustre)
10000000:01000000:6.0:1621283834.168150:0:10966:0:(mgc_request.c:2059:mgc_process_log()) Process log lustre-sptlrpc-ffff8954fc42a100 from 1
10000000:01000000:6.0:1621283834.168151:0:10966:0:(mgc_request.c:1129:mgc_enqueue()) Enqueue for lustre-sptlrpc (res 0x65727473756c)
10000000:01000000:6.0:1621283834.168354:0:10966:0:(mgc_request.c:2139:mgc_process_log()) MGC10.0.1.70 at tcp: configuration from log 'lustre-sptlrpc' failed (-2).
10000000:01000000:6.0:1621283834.168357:0:10966:0:(mgc_request.c:211:do_config_log_add()) do adding config log params-ffff8958fc45e800
10000000:01000000:6.0:1621283834.168358:0:10966:0:(mgc_request.c:90:mgc_name2resid()) log params to resid 0x736d61726170/0x3 (params)
10000000:01000000:6.0:1621283834.168359:0:10966:0:(mgc_request.c:211:do_config_log_add()) do adding config log lustre-barrier-ffff8958fc45e800
10000000:01000000:6.0:1621283834.168360:0:10966:0:(mgc_request.c:90:mgc_name2resid()) log lustre-barrier to resid 0x65727473756c/0x5 (lustre)
10000000:01000000:6.0:1621283834.168361:0:10966:0:(mgc_request.c:2059:mgc_process_log()) Process log lustre-barrier-ffff8958fc45e800 from 1
10000000:01000000:6.0:1621283834.168362:0:10966:0:(mgc_request.c:1129:mgc_enqueue()) Enqueue for lustre-barrier (res 0x65727473756c)
10000000:01000000:6.0:1621283834.168422:0:10966:0:(mgc_request.c:2139:mgc_process_log()) MGC10.0.1.70 at tcp: configuration from log 'lustre-barrier' succeeded (0).
10000000:01000000:6.0:1621283834.168423:0:10966:0:(mgc_request.c:211:do_config_log_add()) do adding config log lustre-MDT0000-0000000000000000
10000000:01000000:6.0:1621283834.168424:0:10966:0:(mgc_request.c:90:mgc_name2resid()) log lustre-MDT0000 to resid 0x65727473756c/0x0 (lustre)
10000000:01000000:6.0:1621283834.168425:0:10966:0:(mgc_request.c:211:do_config_log_add()) do adding config log lustre-mdtir-ffff8958fc45e800
10000000:01000000:6.0:1621283834.168426:0:10966:0:(mgc_request.c:90:mgc_name2resid()) log lustre-mdtir to resid 0x65727473756c/0x2 (lustre)
10000000:01000000:6.0:1621283834.168427:0:10966:0:(mgc_request.c:2059:mgc_process_log()) Process log lustre-MDT0000-0000000000000000 from 1
10000000:01000000:6.0:1621283834.168428:0:10966:0:(mgc_request.c:1129:mgc_enqueue()) Enqueue for lustre-MDT0000 (res 0x65727473756c)
00000020:01000000:43.0:1621283834.169832:0:11037:0:(obd_config.c:1612:class_config_llog_handler()) Marker, inst_flg=0x0 mark_flg=0x1
00000020:00000080:43.0:1621283834.169838:0:11037:0:(obd_config.c:1128:class_process_config()) processing cmd: cf010
00000020:00000080:43.0:1621283834.169840:0:11037:0:(obd_config.c:1198:class_process_config()) marker 2 (0x1) lustre-MDT0000-m lov setup
00000020:01000000:43.0:1621283834.169845:0:11037:0:(obd_config.c:1686:class_config_llog_handler()) For 2.x interoperability, rename obd type from lov to lod (lustre-MDT0000)
00000020:00000080:43.0:1621283834.169848:0:11037:0:(obd_config.c:1128:class_process_config()) processing cmd: cf001
00000020:00000080:43.0:1621283834.169855:0:11037:0:(genops.c:451:class_newdev()) Allocate new device lustre-MDT0000-mdtlov (ffff8958f9bf4200)
00000020:00000080:43.0:1621283834.169932:0:11037:0:(obd_config.c:431:class_attach()) OBD: dev 4 attached type lod with refcount 1
00000020:00000080:43.0:1621283834.169936:0:11037:0:(obd_config.c:1128:class_process_config()) processing cmd: cf003
00080000:01000000:43.0:1621283834.170017:0:11037:0:(osd_handler.c:8039:osd_obd_connect()) connect #2
00000020:00000080:43.0:1621283834.170021:0:11037:0:(genops.c:1417:class_connect()) connect: client lustre-MDT0000-osd_UUID, cookie 0xc56ba8a9d1572834
00000020:00000080:43.0:1621283834.170095:0:11037:0:(obd_config.c:538:class_setup()) finished setup of obd lustre-MDT0000-mdtlov (uuid lustre-MDT0000-mdtlov_UUID)
00000020:01000000:43.0:1621283834.170098:0:11037:0:(obd_config.c:1612:class_config_llog_handler()) Marker, inst_flg=0x2 mark_flg=0x2
00000020:00000080:43.0:1621283834.170100:0:11037:0:(obd_config.c:1128:class_process_config()) processing cmd: cf010
00000020:00000080:43.0:1621283834.170102:0:11037:0:(obd_config.c:1198:class_process_config()) marker 2 (0x2) lustre-MDT0000-m lov setup
00000020:01000000:43.0:1621283834.170105:0:11037:0:(obd_config.c:1612:class_config_llog_handler()) Marker, inst_flg=0x0 mark_flg=0x1
00000020:00000080:43.0:1621283834.170108:0:11037:0:(obd_config.c:1128:class_process_config()) processing cmd: cf010
00000020:00000080:43.0:1621283834.170110:0:11037:0:(obd_config.c:1198:class_process_config()) marker 3 (0x1) lustre-MDT0000 add mdt
00000020:00000080:43.0:1621283834.170113:0:11037:0:(obd_config.c:1128:class_process_config()) processing cmd: cf001
00000020:00000080:43.0:1621283834.170118:0:11037:0:(genops.c:451:class_newdev()) Allocate new device lustre-MDT0000 (ffff8958f9bf2100)
00000020:00000080:43.0:1621283834.170187:0:11037:0:(obd_config.c:431:class_attach()) OBD: dev 5 attached type mdt with refcount 1
00000020:00000080:43.0:1621283834.170191:0:11037:0:(obd_config.c:1128:class_process_config()) processing cmd: cf007
00000020:00000080:43.0:1621283834.170192:0:11037:0:(obd_config.c:1156:class_process_config()) mountopt: profile lustre-MDT0000 osc lustre-MDT0000-mdtlov mdc (null)
00000020:01000000:43.0:1621283834.170195:0:11037:0:(obd_config.c:838:class_add_profile()) Add profile lustre-MDT0000
00000020:00000080:43.0:1621283834.170199:0:11037:0:(obd_config.c:1128:class_process_config()) processing cmd: cf003
00000020:01000004:43.0:1621283834.170261:0:11037:0:(obd_mount_server.c:169:server_get_mount()) get mount ffff8958fc45e800 from lustre-MDT0000, refs=3
00000020:00000080:43.0:1621283834.170271:0:11037:0:(genops.c:451:class_newdev()) Allocate new device lustre-MDD0000 (ffff8958f9bf6300)
00000020:00000080:43.0:1621283834.170345:0:11037:0:(obd_config.c:431:class_attach()) OBD: dev 6 attached type mdd with refcount 1
00000004:01000000:43.0:1621283834.170410:0:11037:0:(lod_dev.c:1951:lod_obd_connect()) connect #0
00000020:00000080:43.0:1621283834.170413:0:11037:0:(genops.c:1417:class_connect()) connect: client lustre-MDT0000-mdtlov_UUID, cookie 0xc56ba8a9d1572849
00000020:00000080:43.0:1621283834.170460:0:11037:0:(obd_config.c:538:class_setup()) finished setup of obd lustre-MDD0000 (uuid lustre-MDD0000_UUID)
00000004:01000000:43.0:1621283834.170464:0:11037:0:(mdd_device.c:1421:mdd_obd_connect()) connect #0
00000020:00000080:43.0:1621283834.170466:0:11037:0:(genops.c:1417:class_connect()) connect: client lustre-MDD0000_UUID, cookie 0xc56ba8a9d1572850
00080000:01000000:43.0:1621283834.170470:0:11037:0:(osd_handler.c:8039:osd_obd_connect()) connect #3
00000020:00000080:43.0:1621283834.170472:0:11037:0:(genops.c:1417:class_connect()) connect: client lustre-MDT0000-osd_UUID, cookie 0xc56ba8a9d1572857
00000001:00080000:43.0:1621283834.285393:0:11037:0:(tgt_lastrcvd.c:205:tgt_reply_header_read()) lustre-MDT0000: read reply_data header. magic=0xbdabda01 header_size=32 reply_size=32
00000020:00000080:43.0:1621283834.285504:0:11037:0:(genops.c:451:class_newdev()) Allocate new device lustre-QMT0000 (ffff8958f9bf0000)
00000020:00000080:43.0:1621283834.285541:0:11037:0:(obd_config.c:431:class_attach()) OBD: dev 7 attached type qmt with refcount 1
00080000:01000000:43.0:1621283834.285579:0:11037:0:(osd_handler.c:8039:osd_obd_connect()) connect #4
00000020:00000080:43.0:1621283834.285583:0:11037:0:(genops.c:1417:class_connect()) connect: client lustre-MDT0000-osd_UUID, cookie 0xc56ba8a9d1572865
00000020:00000080:5.0:1621283834.285750:0:11037:0:(obd_config.c:538:class_setup()) finished setup of obd lustre-QMT0000 (uuid lustre-QMT0000_UUID)
00000020:00000080:5.0:1621283834.287108:0:11037:0:(genops.c:1417:class_connect()) connect: client lustre-QMT0000_UUID, cookie 0xc56ba8a9d157286c
00000020:00000080:5.0:1621283834.287134:0:11037:0:(obd_config.c:538:class_setup()) finished setup of obd lustre-MDT0000 (uuid lustre-MDT0000_UUID)
00000020:01000000:5.0:1621283834.287385:0:11037:0:(obd_config.c:1612:class_config_llog_handler()) Marker, inst_flg=0x2 mark_flg=0x2
00000020:00000080:5.0:1621283834.287391:0:11037:0:(obd_config.c:1128:class_process_config()) processing cmd: cf010
00000020:00000080:5.0:1621283834.287393:0:11037:0:(obd_config.c:1198:class_process_config()) marker 3 (0x2) lustre-MDT0000 add mdt
00000040:00080000:5.0:1621283834.287399:0:11037:0:(llog.c:705:llog_process_thread()) stop processing plain 0x9:10:0 index 10 count 10
00000020:01000000:6.0:1621283834.287427:0:10966:0:(obd_config.c:1876:class_config_parse_llog()) Processed log lustre-MDT0000 gen 1-9 (rc=0)
10000000:01000000:6.0:1621283834.287439:0:10966:0:(mgc_request.c:2139:mgc_process_log()) MGC10.0.1.70 at tcp: configuration from log 'lustre-MDT0000' succeeded (0).
10000000:01000000:6.0:1621283834.287455:0:10966:0:(mgc_request.c:2059:mgc_process_log()) Process log lustre-mdtir-ffff8958fc45e800 from 1
10000000:01000000:6.0:1621283834.287455:0:10966:0:(mgc_request.c:1129:mgc_enqueue()) Enqueue for lustre-mdtir (res 0x65727473756c)
20000000:01000000:7.0:1621283834.287627:0:11000:0:(mgs_nids.c:632:mgs_get_ir_logs()) Reading IR log lustre-mdtir bufsize 1048576.
20000000:01000000:7.0:1621283834.287634:0:11000:0:(mgs_nids.c:204:mgs_nidtbl_read()) Read IR logs lustre return with 0, version 1
10000000:01000000:6.0:1621283834.287706:0:10966:0:(mgc_request.c:2139:mgc_process_log()) MGC10.0.1.70 at tcp: configuration from log 'lustre-mdtir' succeeded (0).
10000000:01000000:6.0:1621283834.287708:0:10966:0:(mgc_request.c:2059:mgc_process_log()) Process log params-ffff8958fc45e800 from 1
10000000:01000000:6.0:1621283834.287709:0:10966:0:(mgc_request.c:1129:mgc_enqueue()) Enqueue for params (res 0x736d61726170)
00000040:00080000:7.0:1621283834.287880:0:11000:0:(llog_osd.c:232:llog_osd_read_header()) not reading header from 0-byte log
00000040:00080000:43.0:1621283834.288324:0:11039:0:(llog.c:705:llog_process_thread()) stop processing plain 0x2:10:0 index 64768 count 1
00000020:01000000:6.0:1621283834.288339:0:10966:0:(obd_config.c:1876:class_config_parse_llog()) Processed log params gen 1-0 (rc=0)
10000000:01000000:6.0:1621283834.288350:0:10966:0:(mgc_request.c:2139:mgc_process_log()) MGC10.0.1.70 at tcp: configuration from log 'params' succeeded (0).
10000000:01000000:6.0:1621283834.288360:0:10966:0:(mgc_request.c:2205:mgc_process_config()) parse_log lustre-client from 0
10000000:01000000:6.0:1621283834.288362:0:10966:0:(mgc_request.c:329:config_log_add()) add config log lustre-client-ffff8958fc45e800
10000000:01000000:6.0:1621283834.288365:0:10966:0:(mgc_request.c:211:do_config_log_add()) do adding config log lustre-client-ffff8958fc45e800
10000000:01000000:6.0:1621283834.288368:0:10966:0:(mgc_request.c:90:mgc_name2resid()) log lustre-client to resid 0x65727473756c/0x0 (lustre)
10000000:01000000:6.0:1621283834.288371:0:10966:0:(mgc_request.c:2059:mgc_process_log()) Process log lustre-client-ffff8958fc45e800 from 1
10000000:01000000:6.0:1621283834.288374:0:10966:0:(mgc_request.c:1129:mgc_enqueue()) Enqueue for lustre-client (res 0x65727473756c)
00000020:01000004:43.0:1621283834.288773:0:11040:0:(obd_mount.c:193:lustre_start_simple()) Starting obd lustre-MDT0000-lwp-MDT0000 (typ=lwp)
00000020:00000080:43.0:1621283834.288775:0:11040:0:(obd_config.c:1128:class_process_config()) processing cmd: cf001
00000020:00000080:43.0:1621283834.288778:0:11040:0:(genops.c:451:class_newdev()) Allocate new device lustre-MDT0000-lwp-MDT0000 (ffff8958f9bf3180)
00000020:00000080:43.0:1621283834.288813:0:11040:0:(obd_config.c:431:class_attach()) OBD: dev 8 attached type lwp with refcount 1
00000020:00000080:43.0:1621283834.288814:0:11040:0:(obd_config.c:1128:class_process_config()) processing cmd: cf003
00010000:00080000:43.0:1621283834.288857:0:11040:0:(ldlm_lib.c:114:import_set_conn()) imp ffff8958f2eac000 at lustre-MDT0000-lwp-MDT0000: add connection 10.0.1.70 at tcp at head
00000020:00000080:43.0:1621283834.291524:0:11040:0:(obd_config.c:538:class_setup()) finished setup of obd lustre-MDT0000-lwp-MDT0000 (uuid lustre-MDT0000-lwp-MDT0000_UUID)
00000010:01000000:43.0:1621283834.291532:0:11040:0:(lwp_dev.c:503:lwp_obd_connect()) connect #0
00000020:00000080:43.0:1621283834.291535:0:11040:0:(genops.c:1417:class_connect()) connect: client lustre-MDT0000-lwp-MDT0000, cookie 0xc56ba8a9d15728a4
00000100:00080000:43.0:1621283834.291539:0:11040:0:(import.c:543:import_select_connection()) lustre-MDT0000-lwp-MDT0000: connect to NID 0 at lo last attempt 0
00000100:00080000:43.0:1621283834.291542:0:11040:0:(import.c:619:import_select_connection()) lustre-MDT0000-lwp-MDT0000: import ffff8958f2eac000 using connection 10.0.1.70 at tcp/0 at lo
00000100:00080000:43.0:1621283834.291552:0:11040:0:(pinger.c:387:ptlrpc_pinger_add_import()) adding pingable import lustre-MDT0000-lwp-MDT0000_UUID->lustre-MDT0000_UUID
00000100:00080000:11.0:1621283834.291707:0:6279:0:(import.c:86:import_set_state_nolock()) ffff8958f2eac000 lustre-MDT0000_UUID: changing import state from CONNECTING to DISCONN
00000100:00080000:11.0:1621283834.291713:0:6279:0:(import.c:1382:ptlrpc_connect_interpret()) recovery of lustre-MDT0000_UUID on 10.0.1.70 at tcp failed (-11)
00000040:00080000:5.0:1621283834.291818:0:11040:0:(llog.c:705:llog_process_thread()) stop processing plain 0x8:10:0 index 18 count 18
00000020:01000000:6.0:1621283834.291841:0:10966:0:(obd_config.c:1876:class_config_parse_llog()) Processed log lustre-client gen 1-17 (rc=0)
10000000:01000000:6.0:1621283834.291851:0:10966:0:(mgc_request.c:2139:mgc_process_log()) MGC10.0.1.70 at tcp: configuration from log 'lustre-client' succeeded (0).
10000000:01000000:6.0:1621283834.291860:0:10966:0:(mgc_request.c:1263:mgc_set_info_async()) register_target lustre-MDT0000 0x40000005
10000000:01000000:6.0:1621283834.291866:0:10966:0:(mgc_request.c:1202:mgc_target_register()) register lustre-MDT0000
20000000:01000000:7.0:1621283834.291906:0:11000:0:(mgs_handler.c:388:mgs_target_reg()) fs: lustre index: 0 is ready to reconnect.
20000000:01000000:7.0:1621283834.292060:0:11000:0:(mgs_nids.c:543:mgs_ir_update()) Try to revoke recover lock of lustre
20000000:01000000:7.0:1621283834.292064:0:11000:0:(mgs_handler.c:558:mgs_target_reg()) replying with lustre-MDT0000, index=0, rc=0
20000000:01000000:40.0:1621283834.292078:0:11035:0:(mgs_nids.c:441:mgs_ir_notify()) mgs_lustre_notify woken up, phase is 1
10000000:01000000:40.0:1621283834.292083:0:11035:0:(mgc_request.c:90:mgc_name2resid()) log lustre to resid 0x65727473756c/0x2 (lustre)
10000000:01000000:6.0:1621283834.292104:0:10966:0:(mgc_request.c:1219:mgc_target_register()) register lustre-MDT0000 got index = 0
00000020:02000000:6.0:1621283834.292109:0:10966:0:(obd_mount_server.c:2068:server_calc_timeout()) lustre-MDT0000: Imperative Recovery not enabled, recovery window 300-900
10000000:01000000:5.0:1621283834.292261:0:10993:0:(mgc_request.c:1021:mgc_blocking_ast()) Lock res [0x65727473756c:0x2:0x0].0x0 (lustre)
20000000:01000000:43.0:1621283834.292358:0:10992:0:(mgs_nids.c:415:mgs_ir_notify_complete()) Revoke recover lock of lustre completed after 0.000275894s
00000020:00080000:40.0:1621283834.293788:0:11044:0:(update_trans.c:1579:distribute_txn_commit_thread()) lustre-MDT0000: start commit thread committed batchid 0
00000020:00080000:40.0:1621283834.293793:0:11044:0:(update_trans.c:1628:distribute_txn_commit_thread()) lustre-MDT0000: batchid: 0 committed batchid 0
00000040:01000000:8.0:1621283834.293800:0:10966:0:(llog_obd.c:214:llog_setup()) obd lustre-MDT0000-osd ctxt 16 is initialized
00000040:01000000:43.0:1621283834.293870:0:11045:0:(llog_osd.c:2056:llog_osd_get_cat_list()) cat list: disk size=32, read=32
00000040:00080000:43.0:1621283834.294042:0:11045:0:(llog_osd.c:232:llog_osd_read_header()) not reading header from 0-byte log
00000040:01000000:8.0:1621283834.294085:0:10966:0:(llog_obd.c:214:llog_setup()) obd lustre-MDD0000 ctxt 12 is initialized
00000040:00080000:8.0:1621283834.294108:0:10966:0:(llog_osd.c:232:llog_osd_read_header()) not reading header from 0-byte log
00000004:00000080:8.0:1621283834.294682:0:10966:0:(mdd_device.c:474:mdd_changelog_llog_init()) changelog starting index=0
00000040:01000000:8.0:1621283834.294686:0:10966:0:(llog_obd.c:214:llog_setup()) obd lustre-MDD0000 ctxt 14 is initialized
00000040:00080000:8.0:1621283834.294702:0:10966:0:(llog_osd.c:232:llog_osd_read_header()) not reading header from 0-byte log
00000040:00080000:43.0:1621283834.295107:0:11045:0:(llog.c:705:llog_process_thread()) stop processing catalog 0x1:1024:0 index 261376 count 1
00000004:00080000:43.0:1621283834.295112:0:11045:0:(lod_dev.c:449:lod_sub_recovery_thread()) lustre-MDT0000-osd retrieved update log, duration 0, retries 0
00000004:00080000:43.0:1621283834.295114:0:11045:0:(lod_dev.c:473:lod_sub_recovery_thread()) lustre-MDT0000 got update logs from all MDTs.
00000040:00080000:8.0:1621283834.295934:0:10966:0:(llog.c:705:llog_process_thread()) stop processing catalog 0x5:10:0 index 64768 count 1
00000040:00080000:8.0:1621283834.296595:0:10966:0:(llog.c:705:llog_process_thread()) stop processing catalog 0x6:10:0 index 64768 count 1
00000040:01000000:8.0:1621283834.296601:0:10966:0:(llog_obd.c:214:llog_setup()) obd lustre-MDD0000 ctxt 15 is initialized
00000040:00080000:8.0:1621283834.296626:0:10966:0:(llog_osd.c:232:llog_osd_read_header()) not reading header from 0-byte log
00080000:10000000:8.0:1621283834.301909:0:10966:0:(osd_handler.c:4945:osd_index_try()) lustre-MDT0000: index object [0x200000003:0x36:0x0] (8/32) registered
00000020:01000004:8.0:1621283834.303304:0:10966:0:(obd_mount_server.c:314:server_mgc_clear_fs()) Unassign mgc disk
00000020:01000004:8.0:1621283834.303321:0:10966:0:(obd_mount_server.c:1840:server_fill_super_common()) Server sb, dev=41
00000020:01000004:43.0:1621283834.323105:0:11049:0:(obd_mount_server.c:1554:server_put_super()) server put_super lustre-MDT0000
10000000:01000000:43.0:1621283834.323113:0:11049:0:(mgc_request.c:535:config_log_end()) end config log lustre-client (0)
00000100:00080000:43.0:1621283834.323120:0:11049:0:(import.c:157:ptlrpc_deactivate_import_nolock()) setting import lustre-MDT0000_UUID INVALID
00000100:00080000:43.0:1621283834.323124:0:11049:0:(pinger.c:412:ptlrpc_pinger_del_import()) removing pingable import lustre-MDT0000-lwp-MDT0000_UUID->lustre-MDT0000_UUID
00000100:00080000:43.0:1621283834.323127:0:11049:0:(import.c:157:ptlrpc_deactivate_import_nolock()) setting import lustre-MDT0000_UUID INVALID
10000000:01000000:43.0:1621283834.326891:0:11049:0:(mgc_request.c:151:config_log_put()) dropping config log lustre-mdtir
10000000:01000000:43.0:1621283834.326893:0:11049:0:(mgc_request.c:535:config_log_end()) end config log lustre-MDT0000 (0)
00000020:01000004:43.0:1621283834.326896:0:11049:0:(obd_mount_server.c:1592:server_put_super()) stopping lustre-MDT0000
00000020:01000000:43.0:1621283834.326897:0:11049:0:(obd_config.c:2100:class_manual_cleanup()) Manual cleanup of lustre-MDT0000 (flags='FA')
00000020:00000080:43.0:1621283834.326898:0:11049:0:(obd_config.c:1128:class_process_config()) processing cmd: cf004
00000020:02000400:43.0:1621283834.326899:0:11049:0:(obd_config.c:640:class_cleanup()) Failing over lustre-MDT0000
00000004:00000080:43.0:1621283834.326905:0:11049:0:(mdt_handler.c:6681:mdt_iocontrol()) handling ioctl cmd 0x40086672
00000020:00000080:43.0:1621283834.326921:0:11049:0:(obd_config.c:659:class_cleanup()) lustre-MDT0000: forcing exports to disconnect: 0/0
00000020:00080000:43.0:1621283834.326923:0:11049:0:(genops.c:1590:class_disconnect_exports()) OBD device 5 (ffff8958f9bf2100) has no exports
00000020:00080000:43.0:1621283834.326941:0:11049:0:(genops.c:1590:class_disconnect_exports()) OBD device 5 (ffff8958f9bf2100) has no exports
00000020:00000080:5.0:1621283834.499199:0:11049:0:(genops.c:1501:class_disconnect()) disconnect: cookie 0xc56ba8a9d157286c
00000020:00000080:5.0:1621283834.499201:0:11049:0:(genops.c:1024:class_export_put()) final put ffff8959034d8c00/lustre-QMT0000_UUID
00000020:01000000:5.0:1621283834.499205:0:11049:0:(obd_config.c:2100:class_manual_cleanup()) Manual cleanup of lustre-QMT0000 (flags='')
00000020:00000080:5.0:1621283834.499210:0:465:0:(genops.c:974:class_export_destroy()) destroying export ffff8959034d8c00/lustre-QMT0000_UUID for lustre-QMT0000
00000020:00000080:5.0:1621283834.499214:0:11049:0:(obd_config.c:1128:class_process_config()) processing cmd: cf004
00000020:00000080:5.0:1621283834.499216:0:11049:0:(obd_config.c:659:class_cleanup()) lustre-QMT0000: forcing exports to disconnect: 0/0
00000020:00080000:5.0:1621283834.499217:0:11049:0:(genops.c:1590:class_disconnect_exports()) OBD device 7 (ffff8958f9bf0000) has no exports
00000020:00000080:5.0:1621283834.499317:0:11049:0:(genops.c:1501:class_disconnect()) disconnect: cookie 0xc56ba8a9d1572865
00000020:00000080:5.0:1621283834.499319:0:11049:0:(genops.c:1024:class_export_put()) final put ffff8958f2ea0000/lustre-MDT0000-osd_UUID
00000020:00000080:5.0:1621283834.499322:0:465:0:(genops.c:974:class_export_destroy()) destroying export ffff8958f2ea0000/lustre-MDT0000-osd_UUID for lustre-MDT0000-osd
00000020:00000080:5.0:1621283834.499342:0:11049:0:(obd_config.c:1128:class_process_config()) processing cmd: cf002
00000020:00000080:5.0:1621283834.499343:0:11049:0:(obd_config.c:589:class_detach()) detach on obd lustre-QMT0000 (uuid lustre-QMT0000_UUID)
00000020:00000080:5.0:1621283834.499344:0:11049:0:(genops.c:1024:class_export_put()) final put ffff8958fdf6f400/lustre-QMT0000_UUID
00000020:00000080:5.0:1621283834.499345:0:11049:0:(genops.c:974:class_export_destroy()) destroying export ffff8958fdf6f400/lustre-QMT0000_UUID for lustre-QMT0000
00000020:01000000:5.0:1621283834.499346:0:11049:0:(genops.c:481:class_free_dev()) finishing cleanup of obd lustre-QMT0000 (lustre-QMT0000_UUID)
00000020:00080000:5.0:1621283834.499458:0:11049:0:(genops.c:1590:class_disconnect_exports()) OBD device 5 (ffff8958f9bf2100) has no exports
00000020:00000080:5.0:1621283834.556016:0:11049:0:(genops.c:1501:class_disconnect()) disconnect: cookie 0xc56ba8a9d1572834
00000020:00000080:5.0:1621283834.556018:0:11049:0:(genops.c:1024:class_export_put()) final put ffff8959034e7000/lustre-MDT0000-osd_UUID
00000020:00000080:5.0:1621283834.556053:0:465:0:(genops.c:974:class_export_destroy()) destroying export ffff8959034e7000/lustre-MDT0000-osd_UUID for lustre-MDT0000-osd
00000020:00000080:5.0:1621283834.556242:0:11049:0:(genops.c:1501:class_disconnect()) disconnect: cookie 0xc56ba8a9d1572849
00000020:00000080:5.0:1621283834.556243:0:11049:0:(genops.c:1024:class_export_put()) final put ffff89592a956800/lustre-MDT0000-mdtlov_UUID
00000020:01000000:5.0:1621283834.556245:0:11049:0:(obd_config.c:2100:class_manual_cleanup()) Manual cleanup of lustre-MDT0000-mdtlov (flags='')
00000020:00000080:5.0:1621283834.556247:0:465:0:(genops.c:974:class_export_destroy()) destroying export ffff89592a956800/lustre-MDT0000-mdtlov_UUID for lustre-MDT0000-mdtlov
00000020:00000080:5.0:1621283834.556248:0:11049:0:(obd_config.c:1128:class_process_config()) processing cmd: cf004
00000020:00000080:5.0:1621283834.556249:0:11049:0:(obd_config.c:659:class_cleanup()) lustre-MDT0000-mdtlov: forcing exports to disconnect: 0/0
00000020:00080000:5.0:1621283834.556250:0:11049:0:(genops.c:1590:class_disconnect_exports()) OBD device 4 (ffff8958f9bf4200) has no exports
00000020:00000080:5.0:1621283834.556291:0:11049:0:(obd_config.c:1128:class_process_config()) processing cmd: cf002
00000020:00000080:5.0:1621283834.556292:0:11049:0:(obd_config.c:589:class_detach()) detach on obd lustre-MDT0000-mdtlov (uuid lustre-MDT0000-mdtlov_UUID)
00000020:00000080:5.0:1621283834.556294:0:11049:0:(genops.c:1024:class_export_put()) final put ffff8959034e6800/lustre-MDT0000-mdtlov_UUID
00000020:00000080:5.0:1621283834.556295:0:11049:0:(genops.c:974:class_export_destroy()) destroying export ffff8959034e6800/lustre-MDT0000-mdtlov_UUID for lustre-MDT0000-mdtlov
00000020:01000000:5.0:1621283834.556296:0:11049:0:(genops.c:481:class_free_dev()) finishing cleanup of obd lustre-MDT0000-mdtlov (lustre-MDT0000-mdtlov_UUID)
00000020:00000080:5.0:1621283834.556372:0:11049:0:(genops.c:1501:class_disconnect()) disconnect: cookie 0xc56ba8a9d1572850
00000020:00000080:5.0:1621283834.556373:0:11049:0:(genops.c:1024:class_export_put()) final put ffff89592a956c00/lustre-MDD0000_UUID
00000020:01000000:5.0:1621283834.556374:0:11049:0:(obd_config.c:2100:class_manual_cleanup()) Manual cleanup of lustre-MDD0000 (flags='')
00000020:00000080:5.0:1621283834.556375:0:465:0:(genops.c:974:class_export_destroy()) destroying export ffff89592a956c00/lustre-MDD0000_UUID for lustre-MDD0000
00000020:00000080:5.0:1621283834.556377:0:11049:0:(obd_config.c:1128:class_process_config()) processing cmd: cf004
00000020:00000080:5.0:1621283834.556377:0:11049:0:(obd_config.c:659:class_cleanup()) lustre-MDD0000: forcing exports to disconnect: 0/0
00000020:00080000:5.0:1621283834.556378:0:11049:0:(genops.c:1590:class_disconnect_exports()) OBD device 6 (ffff8958f9bf6300) has no exports
00000020:00000080:5.0:1621283834.556443:0:11049:0:(obd_config.c:1128:class_process_config()) processing cmd: cf002
00000020:00000080:5.0:1621283834.556443:0:11049:0:(obd_config.c:589:class_detach()) detach on obd lustre-MDD0000 (uuid lustre-MDD0000_UUID)
00000020:00000080:5.0:1621283834.556444:0:11049:0:(genops.c:1024:class_export_put()) final put ffff89592a950c00/lustre-MDD0000_UUID
00000020:00000080:5.0:1621283834.556445:0:11049:0:(genops.c:974:class_export_destroy()) destroying export ffff89592a950c00/lustre-MDD0000_UUID for lustre-MDD0000
00000020:01000000:5.0:1621283834.556445:0:11049:0:(genops.c:481:class_free_dev()) finishing cleanup of obd lustre-MDD0000 (lustre-MDD0000_UUID)
00000020:00000080:5.0:1621283834.556464:0:11049:0:(genops.c:1501:class_disconnect()) disconnect: cookie 0xc56ba8a9d1572857
00000020:00000080:5.0:1621283834.556465:0:11049:0:(genops.c:1024:class_export_put()) final put ffff89592a954c00/lustre-MDT0000-osd_UUID
00000020:00000080:5.0:1621283834.556467:0:465:0:(genops.c:974:class_export_destroy()) destroying export ffff89592a954c00/lustre-MDT0000-osd_UUID for lustre-MDT0000-osd
00000020:01000004:5.0:1621283834.556469:0:11049:0:(obd_mount_server.c:203:server_put_mount()) put mount ffff8958fc45e800 from lustre-MDT0000, refs=3
00000020:01000004:5.0:1621283834.556470:0:11049:0:(obd_mount.c:661:lustre_put_lsi()) put ffff8958fc45e800 3
00000020:01000004:5.0:1621283834.556472:0:11049:0:(obd_mount_server.c:138:server_deregister_mount()) deregister mount ffff8958fc45e800 from lustre-MDT0000
00000020:00000080:5.0:1621283834.556503:0:11049:0:(obd_config.c:1128:class_process_config()) processing cmd: cf002
00000020:00000080:5.0:1621283834.556504:0:11049:0:(obd_config.c:589:class_detach()) detach on obd lustre-MDT0000 (uuid lustre-MDT0000_UUID)
00000020:00000080:5.0:1621283834.556504:0:11049:0:(genops.c:1024:class_export_put()) final put ffff8959034e7800/lustre-MDT0000_UUID
00000020:00000080:5.0:1621283834.556505:0:11049:0:(genops.c:974:class_export_destroy()) destroying export ffff8959034e7800/lustre-MDT0000_UUID for lustre-MDT0000
00000020:01000000:5.0:1621283834.556506:0:11049:0:(genops.c:481:class_free_dev()) finishing cleanup of obd lustre-MDT0000 (lustre-MDT0000_UUID)
00000020:01000004:5.0:1621283834.556512:0:11049:0:(obd_mount_server.c:272:server_stop_mgs()) Stop MGS service MGS
00000020:01000000:5.0:1621283834.556513:0:11049:0:(obd_config.c:2100:class_manual_cleanup()) Manual cleanup of MGS (flags='F')
00000020:00000080:5.0:1621283834.556514:0:11049:0:(obd_config.c:1128:class_process_config()) processing cmd: cf004
00000020:00000080:5.0:1621283834.556514:0:11049:0:(obd_config.c:659:class_cleanup()) MGS: forcing exports to disconnect: 1/2
00000020:00080000:5.0:1621283834.556516:0:11049:0:(genops.c:1802:print_export_data()) MGS: ACTIVE ffff8959034e3c00 3867756d-71df-4dd5-e513-873bf69178ac 0 at lo 9 (0 0 5) 0 0 0 0:           (null)  0 stale:0
00000020:00080000:5.0:1621283834.556519:0:11049:0:(genops.c:1585:class_disconnect_exports()) OBD device 1 (ffff8954fc42b180) has exports, disconnecting them
00000020:00080000:5.0:1621283834.556521:0:11049:0:(genops.c:1560:class_disconnect_export_list()) MGS: disconnecting export at 0 at lo (ffff8959034e3c00), last request at 1621283834
00000020:00000080:5.0:1621283834.556524:0:11049:0:(genops.c:1501:class_disconnect()) disconnect: cookie 0xc56ba8a9d15727fc
00000020:00080000:5.0:1621283834.556529:0:11049:0:(genops.c:1565:class_disconnect_export_list()) disconnected export at 0 at lo (ffff8959034e3c00): rc 0
00000020:00080000:5.0:1621283834.556532:0:11049:0:(genops.c:1590:class_disconnect_exports()) OBD device 1 (ffff8954fc42b180) has no exports
00000100:00080000:7.0:1621283834.556545:0:11001:0:(pinger.c:654:ping_evictor_main()) Exiting Ping Evictor
00000020:00000080:43.0:1621283834.556636:0:10994:0:(genops.c:1024:class_export_put()) final put ffff8959034e3c00/3867756d-71df-4dd5-e513-873bf69178ac
00000020:00000080:43.0:1621283834.556649:0:467:0:(genops.c:974:class_export_destroy()) destroying export ffff8959034e3c00/3867756d-71df-4dd5-e513-873bf69178ac for MGS
00000020:00000080:43.0:1621283834.556672:0:467:0:(genops.c:1224:obd_zombie_import_free()) destroying import ffff895501f52800 for MGS
00000020:01000000:5.0:1621283834.557122:0:11049:0:(lprocfs_status_server.c:245:lprocfs_free_client_stats()) stat ffff895905ca3960 - data ffff895929952f00/ffff895903eea2e0
00000020:00000080:5.0:1621283834.722153:0:11049:0:(genops.c:1501:class_disconnect()) disconnect: cookie 0xc56ba8a9d15727e0
00000020:00000080:5.0:1621283834.722154:0:11049:0:(genops.c:1024:class_export_put()) final put ffff895502089800/lustre-MDT0000-osd_UUID
00000020:00000080:5.0:1621283834.722164:0:465:0:(genops.c:974:class_export_destroy()) destroying export ffff895502089800/lustre-MDT0000-osd_UUID for lustre-MDT0000-osd
00000020:01000004:0.0F:1621283834.722177:0:11049:0:(obd_mount_server.c:203:server_put_mount()) put mount ffff8958fc45e800 from MGS, refs=2
00000020:01000004:0.0:1621283834.722185:0:11049:0:(obd_mount.c:661:lustre_put_lsi()) put ffff8958fc45e800 2
00000020:01000004:0.0:1621283834.722189:0:11049:0:(obd_mount_server.c:138:server_deregister_mount()) deregister mount ffff8958fc45e800 from MGS
00000020:00000080:0.0:1621283834.722274:0:11049:0:(obd_config.c:1128:class_process_config()) processing cmd: cf002
00000020:00000080:0.0:1621283834.722278:0:11049:0:(obd_config.c:589:class_detach()) detach on obd MGS (uuid MGS)
00000020:00000080:0.0:1621283834.722281:0:11049:0:(genops.c:1024:class_export_put()) final put ffff895502089400/MGS
00000020:00000080:0.0:1621283834.722284:0:11049:0:(genops.c:974:class_export_destroy()) destroying export ffff895502089400/MGS for MGS
00000020:01000000:0.0:1621283834.722289:0:11049:0:(genops.c:481:class_free_dev()) finishing cleanup of obd MGS (MGS)
00000020:01000000:0.0:1621283834.722375:0:11049:0:(obd_config.c:2100:class_manual_cleanup()) Manual cleanup of lustre-MDT0000-lwp-MDT0000 (flags='F')
00000020:00000080:0.0:1621283834.722379:0:11049:0:(obd_config.c:1128:class_process_config()) processing cmd: cf004
00000020:00000080:0.0:1621283834.722382:0:11049:0:(obd_config.c:659:class_cleanup()) lustre-MDT0000-lwp-MDT0000: forcing exports to disconnect: 1/2
00000020:00080000:0.0:1621283834.722387:0:11049:0:(genops.c:1802:print_export_data()) lustre-MDT0000-lwp-MDT0000: ACTIVE ffff8958f2ea1400 lustre-MDT0000-lwp-MDT0000 0 at lo 3 (0 0 0) 0 0 0 0:           (null)  0 stale:0
00000020:00080000:0.0:1621283834.722397:0:11049:0:(genops.c:1585:class_disconnect_exports()) OBD device 8 (ffff8958f9bf3180) has exports, disconnecting them
00000020:00080000:0.0:1621283834.722401:0:11049:0:(genops.c:1560:class_disconnect_export_list()) lustre-MDT0000-lwp-MDT0000: disconnecting export at 0 at lo (ffff8958f2ea1400), last request at 1621283834
00000020:00000080:0.0:1621283834.722406:0:11049:0:(genops.c:1501:class_disconnect()) disconnect: cookie 0xc56ba8a9d15728a4
00000020:00080000:0.0:1621283834.722412:0:11049:0:(genops.c:1565:class_disconnect_export_list()) disconnected export at 0 at lo (ffff8958f2ea1400): rc 0
00000020:00000080:0.0:1621283834.722426:0:11049:0:(genops.c:1024:class_export_put()) final put ffff8958f2ea1400/lustre-MDT0000-lwp-MDT0000
00000020:00000080:0.0:1621283834.722441:0:464:0:(genops.c:974:class_export_destroy()) destroying export ffff8958f2ea1400/lustre-MDT0000-lwp-MDT0000 for lustre-MDT0000-lwp-MDT0000
00010000:01000000:0.0:1621283834.728548:0:11049:0:(obd_class.h:635:obd_cleanup_client_import()) lustre-MDT0000-lwp-MDT0000: client import never connected
00000100:00080000:0.0:1621283834.728552:0:11049:0:(import.c:157:ptlrpc_deactivate_import_nolock()) setting import lustre-MDT0000_UUID INVALID
00000020:00000080:0.0:1621283834.728567:0:464:0:(genops.c:1224:obd_zombie_import_free()) destroying import ffff8958f2eac000 for lustre-MDT0000-lwp-MDT0000
00000020:00000080:0.0:1621283834.728650:0:11049:0:(obd_config.c:1128:class_process_config()) processing cmd: cf002
00000020:00000080:0.0:1621283834.728654:0:11049:0:(obd_config.c:589:class_detach()) detach on obd lustre-MDT0000-lwp-MDT0000 (uuid lustre-MDT0000-lwp-MDT0000_UUID)
00000020:00000080:0.0:1621283834.728657:0:11049:0:(genops.c:1024:class_export_put()) final put ffff8958f2ea0800/lustre-MDT0000-lwp-MDT0000_UUID
00000020:00000080:0.0:1621283834.728659:0:11049:0:(genops.c:974:class_export_destroy()) destroying export ffff8958f2ea0800/lustre-MDT0000-lwp-MDT0000_UUID for lustre-MDT0000-lwp-MDT0000
00000020:01000000:0.0:1621283834.728662:0:11049:0:(genops.c:481:class_free_dev()) finishing cleanup of obd lustre-MDT0000-lwp-MDT0000 (lustre-MDT0000-lwp-MDT0000_UUID)
00000020:01000004:0.0:1621283834.728682:0:11049:0:(obd_mount.c:906:lustre_common_put_super()) dropping sb ffff8958fc45e800
00000100:00080000:0.0:1621283834.728686:0:11049:0:(pinger.c:412:ptlrpc_pinger_del_import()) removing pingable import 3867756d-71df-4dd5-e513-873bf69178ac->MGS
10000000:01000000:0.0:1621283834.728696:0:11049:0:(mgc_request.c:1021:mgc_blocking_ast()) Lock res [0x65727473756c:0x5:0x0].0x0 (lustre)
10000000:01000000:0.0:1621283834.728701:0:11049:0:(mgc_request.c:1035:mgc_blocking_ast()) log lustre-barrier: stopping, won't requeue
10000000:01000000:0.0:1621283834.728704:0:11049:0:(mgc_request.c:151:config_log_put()) dropping config log lustre-barrier
10000000:01000000:0.0:1621283834.728739:0:11049:0:(mgc_request.c:1021:mgc_blocking_ast()) Lock res [0x736d61726170:0x3:0x0].0x0 (params)
10000000:01000000:0.0:1621283834.728743:0:11049:0:(mgc_request.c:1035:mgc_blocking_ast()) log params: stopping, won't requeue
10000000:01000000:0.0:1621283834.728745:0:11049:0:(mgc_request.c:151:config_log_put()) dropping config log params
10000000:01000000:0.0:1621283834.728756:0:11049:0:(mgc_request.c:1021:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre)
10000000:01000000:0.0:1621283834.728760:0:11049:0:(mgc_request.c:1035:mgc_blocking_ast()) log lustre-client: stopping, won't requeue
10000000:01000000:0.0:1621283834.728761:0:11049:0:(mgc_request.c:151:config_log_put()) dropping config log lustre-client
10000000:01000000:0.0:1621283834.728764:0:11049:0:(mgc_request.c:1021:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre)
10000000:01000000:0.0:1621283834.728767:0:11049:0:(mgc_request.c:1035:mgc_blocking_ast()) log lustre-MDT0000: stopping, won't requeue
10000000:01000000:0.0:1621283834.728769:0:11049:0:(mgc_request.c:151:config_log_put()) dropping config log lustre-MDT0000
10000000:01000000:0.0:1621283834.728772:0:11049:0:(mgc_request.c:1021:mgc_blocking_ast()) Lock res [0x65727473756c:0x0:0x0].0x0 (lustre)
10000000:01000000:0.0:1621283834.728775:0:11049:0:(mgc_request.c:1044:mgc_blocking_ast()) log lustre-s: disconnecting, won't requeue
10000000:01000000:0.0:1621283834.728777:0:11049:0:(mgc_request.c:151:config_log_put()) dropping config log lustre-sptlrpc
00000100:00080000:0.0:1621283834.728796:0:11049:0:(import.c:86:import_set_state_nolock()) ffff8954ff8d9800 MGS: changing import state from FULL to CONNECTING
00000400:00000100:0.0:1621283834.728822:0:11049:0:(lib-move.c:3829:lnet_parse_put()) Dropping PUT from 12345-0 at lo portal 26 match 1700036019769216 offset 0 length 224: 4
00000100:00000400:0.0:1621283840.729582:0:11049:0:(client.c:2133:ptlrpc_expire_one_request()) @@@ Request sent has timed out for slow reply: [sent 1621283834/real 1621283834]  req at ffff8954ff8e8000 x1700036019769216/t0(0) o251->MGC10.0.1.70 at tcp@0 at lo:26/25 lens 224/224 e 0 to 1 dl 1621283840 ref 2 fl Rpc:XN/0/ffffffff rc 0/-1
00000100:00080000:0.0:1621283840.729630:0:11049:0:(import.c:157:ptlrpc_deactivate_import_nolock()) setting import MGS INVALID
10000000:01000000:0.0:1621283840.729634:0:11049:0:(mgc_request.c:1363:mgc_import_event()) import event 0x808002
10000000:01000000:0.0:1621283840.729638:0:11049:0:(mgc_request.c:1363:mgc_import_event()) import event 0x808003
00000020:00000080:0.0:1621283840.729646:0:11049:0:(genops.c:1501:class_disconnect()) disconnect: cookie 0xc56ba8a9d15727ee
00000020:00000080:0.0:1621283840.729649:0:11049:0:(genops.c:1024:class_export_put()) final put ffff8958fbddc400/3867756d-71df-4dd5-e513-873bf69178ac
00000020:00000080:0.0:1621283840.729664:0:464:0:(genops.c:974:class_export_destroy()) destroying export ffff8958fbddc400/3867756d-71df-4dd5-e513-873bf69178ac for MGC10.0.1.70 at tcp
00000020:01000000:0.0:1621283840.729673:0:11049:0:(obd_config.c:2100:class_manual_cleanup()) Manual cleanup of MGC10.0.1.70 at tcp (flags='')
00000020:00000080:0.0:1621283840.729677:0:11049:0:(obd_config.c:1128:class_process_config()) processing cmd: cf004
00000020:00000080:0.0:1621283840.729680:0:11049:0:(obd_config.c:659:class_cleanup()) MGC10.0.1.70 at tcp: forcing exports to disconnect: 0/1
00000020:00080000:0.0:1621283840.729684:0:11049:0:(genops.c:1590:class_disconnect_exports()) OBD device 2 (ffff8954fc42a100) has no exports
10000000:01000000:43.0:1621283840.729758:0:11002:0:(mgc_request.c:702:mgc_requeue_thread()) Ending requeue thread
10000000:01000000:0.0:1621283840.729765:0:11049:0:(obd_class.h:635:obd_cleanup_client_import()) MGC10.0.1.70 at tcp: client import never connected
00000100:00080000:0.0:1621283840.729767:0:11049:0:(import.c:157:ptlrpc_deactivate_import_nolock()) setting import MGS INVALID
00000020:00000080:0.0:1621283840.729782:0:464:0:(genops.c:1224:obd_zombie_import_free()) destroying import ffff8954ff8d9800 for MGC10.0.1.70 at tcp
00000020:00000080:0.0:1621283840.729839:0:11049:0:(obd_config.c:1128:class_process_config()) processing cmd: cf002
00000020:00000080:0.0:1621283840.729842:0:11049:0:(obd_config.c:589:class_detach()) detach on obd MGC10.0.1.70 at tcp (uuid 3867756d-71df-4dd5-e513-873bf69178ac)
00000020:00000080:0.0:1621283840.729845:0:11049:0:(genops.c:1024:class_export_put()) final put ffff8958fbddd400/3867756d-71df-4dd5-e513-873bf69178ac
00000020:00000080:0.0:1621283840.729850:0:11049:0:(genops.c:974:class_export_destroy()) destroying export ffff8958fbddd400/3867756d-71df-4dd5-e513-873bf69178ac for MGC10.0.1.70 at tcp
00000020:01000000:0.0:1621283840.729853:0:11049:0:(genops.c:481:class_free_dev()) finishing cleanup of obd MGC10.0.1.70 at tcp (3867756d-71df-4dd5-e513-873bf69178ac)
00000020:00000080:0.0:1621283840.731759:0:11049:0:(obd_config.c:1128:class_process_config()) processing cmd: cf006
00000020:00000080:0.0:1621283840.731761:0:11049:0:(obd_config.c:1147:class_process_config()) removing mappings for uuid MGC10.0.1.70 at tcp_0
00000020:01000004:0.0:1621283840.731766:0:11049:0:(obd_mount.c:661:lustre_put_lsi()) put ffff8958fc45e800 1
00000020:00000080:0.0:1621283840.731770:0:11049:0:(genops.c:1501:class_disconnect()) disconnect: cookie 0xc56ba8a9d15727d2
00000020:00000080:0.0:1621283840.731773:0:11049:0:(genops.c:1024:class_export_put()) final put ffff89550208a000/lustre-MDT0000-osd_UUID
00000020:01000000:0.0:1621283840.731778:0:11049:0:(obd_config.c:2100:class_manual_cleanup()) Manual cleanup of lustre-MDT0000-osd (flags='')
00000020:00000080:0.0:1621283840.731783:0:464:0:(genops.c:974:class_export_destroy()) destroying export ffff89550208a000/lustre-MDT0000-osd_UUID for lustre-MDT0000-osd
00000020:00000080:0.0:1621283840.731789:0:11049:0:(obd_config.c:1128:class_process_config()) processing cmd: cf004
00000020:00000080:0.0:1621283840.731792:0:11049:0:(obd_config.c:659:class_cleanup()) lustre-MDT0000-osd: forcing exports to disconnect: 0/0
00000020:00080000:0.0:1621283840.731794:0:11049:0:(genops.c:1590:class_disconnect_exports()) OBD device 0 (ffff8954fe90b180) has no exports
00000020:00000080:0.0:1621283840.732185:0:11049:0:(obd_config.c:1128:class_process_config()) processing cmd: cf002
00000020:00000080:0.0:1621283840.732187:0:11049:0:(obd_config.c:589:class_detach()) detach on obd lustre-MDT0000-osd (uuid lustre-MDT0000-osd_UUID)
00000020:00000080:0.0:1621283840.732190:0:11049:0:(genops.c:1024:class_export_put()) final put ffff895446fc5c00/lustre-MDT0000-osd_UUID
00000020:00000080:0.0:1621283840.732192:0:11049:0:(genops.c:974:class_export_destroy()) destroying export ffff895446fc5c00/lustre-MDT0000-osd_UUID for lustre-MDT0000-osd
00000020:01000000:0.0:1621283840.732195:0:11049:0:(genops.c:481:class_free_dev()) finishing cleanup of obd lustre-MDT0000-osd (lustre-MDT0000-osd_UUID)
00000020:01000004:0.0:1621283841.177873:0:11049:0:(obd_mount.c:605:lustre_free_lsi()) Freeing lsi ffff8958fc45e000
00000020:01000000:0.0:1621283841.177916:0:11049:0:(obd_config.c:2100:class_manual_cleanup()) Manual cleanup of MDS (flags='F')
00000020:00000080:0.0:1621283841.177916:0:11049:0:(obd_config.c:1128:class_process_config()) processing cmd: cf004
00000020:00000080:0.0:1621283841.177917:0:11049:0:(obd_config.c:659:class_cleanup()) MDS: forcing exports to disconnect: 0/0
00000020:00080000:0.0:1621283841.177918:0:11049:0:(genops.c:1590:class_disconnect_exports()) OBD device 3 (ffff8954fc429080) has no exports
00000020:00000080:0.0:1621283841.219275:0:11049:0:(obd_config.c:1128:class_process_config()) processing cmd: cf002
00000020:00000080:0.0:1621283841.219278:0:11049:0:(obd_config.c:589:class_detach()) detach on obd MDS (uuid MDS_uuid)
00000020:00000080:0.0:1621283841.219280:0:11049:0:(genops.c:1024:class_export_put()) final put ffff8958fbddd000/MDS_uuid
00000020:00000080:0.0:1621283841.219281:0:11049:0:(genops.c:974:class_export_destroy()) destroying export ffff8958fbddd000/MDS_uuid for MDS
00000020:01000000:0.0:1621283841.219282:0:11049:0:(genops.c:481:class_free_dev()) finishing cleanup of obd MDS (MDS_uuid)
00000020:02000400:0.0:1621283841.219363:0:11049:0:(obd_mount_server.c:1642:server_put_super()) server umount lustre-MDT0000 complete


More information about the lustre-discuss mailing list