[lustre-discuss] luster client mount issues

Thu Jul 21 02:33:02 PDT 2016

Hi,

I think your client doesn't have the o2ib lnet (it should appear in the
output of the lctl ping, even if you ping on the tcp lnet).
In your /etc/modprobe.d/lustre.conf o2ib is associated with the ib0
interface, but your /var/log/messages talks about ib1.
If it is a dual port card where just one port is used, the easiest would
be to plug the cable to the other interface. (If there are two ib
connections, things might become a bit more complicated. There are
examples for multi rail configurations using several lnets in the lustre
manual, but maybe this goes too far.)

With the attempt to mount via tcp (or tcp0, which is the same) I think
the problem is that the file system config on the mgs doesn't contain
the tcp-NIDs and/or the routes are not configured correctly. It seems
the attempt to mount via tcp causes the client to use o2ib for the
connections to the MDS and OSSes. So, I would recommend to get that
working first and then look at tcp0 at a later stage (if you need it at
all - native o2ib is more performant).

Last but not least I have noticed a typo in your client mount command:
mount -t lustre 192.168.200.52 at ob2:/mylustre /lustre
this should be "o2ib" here, too.

best regards,
Martin

On 07/20/2016 08:09 PM, sohamm wrote:
> Hi
>
> Any guidance/help on this is greatly appreciated.
>
> Thanks
>
> On Mon, Jul 18, 2016 at 7:25 PM, sohamm <sohamm at gmail.com> wrote:
>
>> Hi Ben
>> Both the networks have netmasks of value 255.255.255.0
>>
>> Thanks
>>
>> On Mon, Jul 18, 2016 at 10:08 AM, Ben Evans <bevans at cray.com> wrote:
>>
>>> What do your netmasks look like on each network?
>>>
>>> From: lustre-discuss <lustre-discuss-bounces at lists.lustre.org> on behalf
>>> of sohamm <sohamm at gmail.com>
>>> Date: Monday, July 18, 2016 at 1:56 AM
>>> To: "lustre-discuss at lists.lustre.org" <lustre-discuss at lists.lustre.org>
>>> Subject: Re: [lustre-discuss] lustre-discuss Digest, Vol 124, Issue 17
>>>
>>> Hi Thomas
>>> Below are the results of the commands you suggested.
>>>
>>> *From Client*
>>> [root at dev1 ~]# lctl ping 192.168.200.52 at o2ib
>>> failed to ping 192.168.200.52 at o2ib: Input/output error
>>> [root at dev1 ~]# lctl ping 192.168.111.52 at tcp
>>> 12345-0 at lo
>>> 12345-192.168.200.52 at o2ib
>>> 12345-192.168.111.52 at tcp
>>> [root at dev1 ~]# mount -t lustre 192.168.111.52 at tcp:/mylustre /lustre
>>> mount.lustre: mount 192.168.111.52 at tcp:/mylustre at /lustre failed:
>>> Input/output error
>>> Is the MGS running?
>>> mount: mounting 192.168.111.52 at tcp:/mylustre on /lustre failed: Invalid
>>> argument
>>>
>>> cat /var/log/messages | tail
>>> Jul 18 01:37:04 dev1 user.warn kernel: [2250504.401397] ib1: multicast
>>> join failed for ff12:401b:ffff:0000:0000:0000:ffff:ffff, status -22
>>> Jul 18 01:37:26 dev1 user.warn kernel: [2250526.257309] LNet: No route to
>>> 12345-192.168.200.52 at o2ib via <?> (all routers down)
>>> Jul 18 01:37:36 dev1 user.warn kernel: [2250536.481862] ib1: multicast
>>> join failed for ff12:401b:ffff:0000:0000:0000:ffff:ffff, status -22
>>> Jul 18 01:41:53 dev1 user.warn kernel: [2250792.947299] LNet: No route to
>>> 12345-192.168.200.52 at o2ib via <?> (all routers down)
>>>
>>>
>>> *From MGS*
>>> [root at lustre_mgs01_vm03 ~]# lctl ping 192.168.111.102 at tcp
>>> 12345-0 at lo
>>> 12345-192.168.111.102 at tcp
>>>
>>> Please let me know what else i can try. Looks like i am missing something
>>> with the ib config? Do i need router setup as part of lnet ?
>>> if i am able to ping mgs from client on the tcp network, it should still
>>> work ?
>>>
>>> Thanks
>>>
>>>
>>> On Sun, Jul 17, 2016 at 1:07 PM, <lustre-discuss-request at lists.lustre.org
>>>> wrote:
>>>> Send lustre-discuss mailing list submissions to
>>>>         lustre-discuss at lists.lustre.org
>>>>
>>>> To subscribe or unsubscribe via the World Wide Web, visit
>>>>         http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>>>> or, via email, send a message with subject or body 'help' to
>>>>         lustre-discuss-request at lists.lustre.org
>>>>
>>>> You can reach the person managing the list at
>>>>         lustre-discuss-owner at lists.lustre.org
>>>>
>>>> When replying, please edit your Subject line so it is more specific
>>>> than "Re: Contents of lustre-discuss digest..."
>>>>
>>>>
>>>> Today's Topics:
>>>>
>>>>    1. llapi_file_get_stripe() and /proc/fs/lustre/osc/  entries
>>>>       (John Bauer)
>>>>    2. luster client mount issues (sohamm)
>>>>    3. Re:
>>>> 
>>>> luster client mount issues (Thomas Roth)
>>>>
>>>>
>>>> ----------------------------------------------------------------------
>>>>
>>>> Message: 1
>>>> Date: Sat, 16 Jul 2016 15:11:22 -0500
>>>> From: John Bauer <bauerj at iodoctors.com>
>>>> To: "lustre-discuss at lists.lustre.org"
>>>>         <lustre-discuss at lists.lustre.org>
>>>> Subject: [lustre-discuss] llapi_file_get_stripe() and
>>>>         /proc/fs/lustre/osc/    entries
>>>> Message-ID: <03ceaaa0-b004-ae43-eaa1-437da2a5b717 at iodoctors.com>
>>>> Content-Type: text/plain; charset="utf-8"; Format="flowed"
>>>>
>>>> I am using *llapi_file_get_stripe()* to get the ost indexes that a file
>>>> is striped on.  That part is working fine. But there are multiple Lustre
>>>> file systems on the node resulting in multiple **OST0000* *in the
>>>> directory /proc/fs/lustre/osc.  Is there something in the *struct
>>>> lov_user_ost_data* or *struct lov_user_md* that would indicate which of
>>>> the following directories pertains to the file's OST ?
>>>>
>>>> dr-xr-xr-x 2 root root 0 Jul 16 12:31 nbp1-OST0000-osc-ffff880287ae4c00
>>>> dr-xr-xr-x 2 root root 0 Jul 16 12:31 nbp2-OST0000-osc-ffff881034d99000
>>>> dr-xr-xr-x 2 root root 0 Jul 16 12:31 nbp6-OST0000-osc-ffff881003cd7800
>>>> dr-xr-xr-x 2 root root 0 Jul 16 12:31 nbp7-OST0000-osc-ffff880ffe051c00
>>>> dr-xr-xr-x 2 root root 0 Jul 16 12:31 nbp8-OST0000-osc-ffff880ffe054c00
>>>> dr-xr-xr-x 2 root root 0 Jul 16 12:31 nbp9-OST0000-osc-ffff880fcf179400
>>>>
>>>> Thanks
>>>>
>>>> --
>>>> I/O Doctors, LLC
>>>> 507-766-0378
>>>> bauerj at iodoctors.com
>>>>
>>>> -------------- next part --------------
>>>> An HTML attachment was scrubbed...
>>>> URL: <
>>>> http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20160716/95176929/attachment.html
>>>> ------------------------------
>>>>
>>>> Message: 2
>>>> Date: Sat, 16 Jul 2016 14:34:35 -0700
>>>> From: sohamm <sohamm at gmail.com>
>>>> To: lustre-discuss at lists.lustre.org
>>>> Subject: [lustre-discuss] luster client mount issues
>>>> Message-ID:
>>>>         <
>>>> CAKGc+eBQ+MCdbSrc7Ft4gd+zmZ6FbAZHaVhSQtpgOshYrJqhhw at mail.gmail.com>
>>>> Content-Type: text/plain; charset="utf-8"
>>>>
>>>> Hi
>>>>
>>>> I am trying to mount lustre client. Below are steps and necessary
>>>> information surrounding the issue. Please let me know if i am missing
>>>> something
>>>>
>>>> Thanks
>>>> Div
>>>>
>>>> *Mgs:*
>>>>
>>>> [root at lustre_mgs01_vm03 ~]# cat /etc/modprobe.d/lustre.conf
>>>>
>>>> options lnet networks=o2ib(ib0),tcp0(eth0)
>>>>
>>>>
>>>>
>>>> [root at lustre_mgs01_vm03 ~]# modprobe lnet
>>>>
>>>> [root at lustre_mgs01_vm03 ~]# lsmod | grep lnet
>>>>
>>>> lnet                  449065  0
>>>>
>>>> libcfs                405839  1 lnet
>>>>
>>>> [root at lustre_mgs01_vm03 ~]# lctl network up
>>>>
>>>> LNET configured
>>>>
>>>> [root at lustre_mgs01_vm03 ~]# lctl list_nids
>>>>
>>>> 192.168.200.52 at o2ib
>>>>
>>>> 192.168.111.52 at tcp
>>>>
>>>> *On Client:*
>>>> I am able to ping MGS on both tcp and ib network
>>>>
>>>> [root at dev1~]# ping 192.168.111.52
>>>>
>>>> PING 192.168.111.52 (192.168.111.52) 56(84) bytes of data.
>>>>
>>>> 64 bytes from 192.168.111.52: icmp_req=1 ttl=64 time=5.81 ms
>>>>
>>>> 64 bytes from 192.168.111.52: icmp_req=2 ttl=64 time=0.802 ms
>>>>
>>>> 64 bytes from 192.168.111.52: icmp_req=3 ttl=64 time=0.780 ms
>>>>
>>>> ^C
>>>>
>>>> --- 192.168.111.52 ping statistics ---
>>>>
>>>> 3 packets transmitted, 3 received, 0% packet loss, time 2000ms
>>>>
>>>> rtt min/avg/max/mdev = 0.780/2.464/5.811/2.366 ms
>>>>
>>>> [root at dev1 ~]# ping 192.168.200.52
>>>>
>>>> PING 192.168.200.52 (192.168.200.52) 56(84) bytes of data.
>>>>
>>>> 64 bytes from 192.168.200.52: icmp_req=1 ttl=64 time=24.4 ms
>>>>
>>>> 64 bytes from 192.168.200.52: icmp_req=2 ttl=64 time=2.14 ms
>>>>
>>>> 64 bytes from 192.168.200.52: icmp_req=3 ttl=64 time=0.782 ms
>>>>
>>>> 64 bytes from 192.168.200.52: icmp_req=4 ttl=64 time=9.30 ms
>>>>
>>>> ^C
>>>>
>>>> --- 192.168.200.52 ping statistics ---
>>>>
>>>> 4 packets transmitted, 4 received, 0% packet loss, time 3005ms
>>>>
>>>>
>>>> *client mount commands*
>>>>
>>>> mount -t lustre 192.168.111.52 at tcp:/mylustre /lustre ( or)
>>>>
>>>> mount -t lustre 192.168.111.52 at tcp0:/mylustre /lustre ( or)
>>>>
>>>> mount -t lustre 192.168.200.52 at ob2:/mylustre /lustre
>>>>
>>>>
>>>> *cat /var/log/messages | tail -40*
>>>>
>>>> Jul 16 17:03:17 dev1 user.err kernel: [2133277.466013] LustreError:
>>>> 162-5:
>>>> Missing mount data: check that /sbin/mount.lustre is installed.
>>>>
>>>> Jul 16 17:03:17 dev1 user.err kernel: [2133277.466064] LustreError:
>>>> 13627:0:(obd_mount.c:1325:lustre_fill_super()) Unable to mount  (-22)
>>>>
>>>> Jul 16 17:03:23 dev1 user.warn kernel: [2133282.680519] Lustre:
>>>> 12364:0:(client.c:1918:ptlrpc_expire_one_request()) @@@ Request sent has
>>>> timed out for slow reply: [sent 1468702998/real 1468702998]
>>>> req at ffff8801e0bc3c00 x1539427524411444/t0(0) o250->MGC192.168.111.52
>>>>
>>>> Jul 16 17:03:24 dev1 user.err kernel: [2133283.680193] LustreError:
>>>> 13628:0:(client.c:1083:ptlrpc_import_delay_req()) @@@ send limit expired
>>>> req at ffff8801e0bc7000 x1539427524411448/t0(0) o101->MGC192.168.111.52 at tcp
>>>> @192.168.111.52 at tcp:26/25 lens 328/344 e 0 to 0 dl
>>>>
>>>> Jul 16 17:03:31 dev1 user.err kernel: [2133290.760978] LustreError:
>>>> 13657:0:(client.c:1083:ptlrpc_import_delay_req()) @@@ send limit expired
>>>> req at ffff8801b7159800 x1539427524411456/t0(0) o101->MGC192.168.111.52 at tcp
>>>> @192.168.111.52 at tcp:26/25 lens 328/344 e 0 to 0 dl
>>>>
>>>> Jul 16 17:03:43 dev1 user.warn kernel: [2133302.681412] Lustre:
>>>> 12364:0:(client.c:1918:ptlrpc_expire_one_request()) @@@ Request sent has
>>>> failed due to network error: [sent 1468703023/real 1468703023]
>>>> req at ffff8801d6bfc800 x1539427524411460/t0(0) o250->MGC192.168.111
>>>>
>>>> Jul 16 17:04:08 dev1 user.warn kernel: [2133327.681402] Lustre:
>>>> 12364:0:(client.c:1918:ptlrpc_expire_one_request()) @@@ Request sent has
>>>> failed due to network error: [sent 1468703048/real 1468703048]
>>>> req at ffff8801d6bfec00 x1539427524411464/t0(0) o250->MGC192.168.111
>>>>
>>>> Jul 16 17:04:15 dev1 user.err kernel: [2133334.680175] LustreError:
>>>> 13628:0:(client.c:1083:ptlrpc_import_delay_req()) @@@ send limit expired
>>>> req at ffff8801e0bc7000 x1539427524411452/t0(0) o101->MGC192.168.111.52 at tcp
>>>> @192.168.111.52 at tcp:26/25 lens 328/344 e 0 to 0 dl
>>>>
>>>> Jul 16 17:04:15 dev1 user.err kernel: [2133334.680316] LustreError:
>>>> 15c-8:
>>>> MGC192.168.111.52 at tcp: The configuration from log 'mylustre-client'
>>>> failed
>>>> (-5). This may be the result of communication errors between this node
>>>> and
>>>> the MGS, a bad configuration, or other e
>>>>
>>>> Jul 16 17:04:15 dev1 user.err kernel: [2133334.680357] LustreError:
>>>> 13628:0:(llite_lib.c:1046:ll_fill_super()) Unable to process log: -5
>>>>
>>>> Jul 16 17:04:15 dev1 user.warn kernel: [2133334.680881] Lustre: Unmounted
>>>> mylustre-client
>>>>
>>>> Jul 16 17:04:15 dev1 user.err kernel: [2133334.731730] LustreError:
>>>> 13628:0:(obd_mount.c:1325:lustre_fill_super()) Unable to mount  (-5)
>>>> -------------- next part --------------
>>>> An HTML attachment was scrubbed...
>>>> URL: <
>>>> http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20160716/28fb2cad/attachment-0001.htm
>>>> ------------------------------
>>>>
>>>> Message: 3
>>>> Date: Sun, 17 Jul 2016 10:19:18 +0200
>>>> From: Thomas Roth <t.roth at gsi.de>
>>>> To: <lustre-discuss at lists.lustre.org>
>>>> Subject: Re: [lustre-discuss] luster client mount issues
>>>> Message-ID: <578B3F86.1060203 at gsi.de>
>>>> Content-Type: text/plain; charset="windows-1252"; format=flowed
>>>>
>>>> Hi,
>>>>
>>>> try 'lctl ping' from your clients to the MDS to check if you get through
>>>> on lnet, e.g.
>>>>
>>>> lctl ping ping 192.168.200.52 at o2ib
>>>>
>>>> or
>>>>
>>>> lctl ping 192.168.111.52 at tcp
>>>>
>>>>
>>>> and vice versa from the MDS to the clients' nids.
>>>>
>>>> Regards,
>>>> Thomas
>>>>
>>>> On 07/16/2016 11:34 PM, sohamm wrote:
>>>>> Hi
>>>>>
>>>>> I am trying to mount lustre client. Below are steps and necessary
>>>>> information surrounding the issue. Please let me know if i am missing
>>>>> something
>>>>>
>>>>> Thanks
>>>>> Div
>>>>>
>>>>> *Mgs:*
>>>>>
>>>>> [root at lustre_mgs01_vm03 ~]# cat /etc/modprobe.d/lustre.conf
>>>>>
>>>>> options lnet networks=o2ib(ib0),tcp0(eth0)
>>>>>
>>>>>
>>>>>
>>>>> [root at lustre_mgs01_vm03 ~]# modprobe lnet
>>>>>
>>>>> [root at lustre_mgs01_vm03 ~]# lsmod | grep lnet
>>>>>
>>>>> lnet                  449065  0
>>>>>
>>>>> libcfs                405839  1 lnet
>>>>>
>>>>> [root at lustre_mgs01_vm03 ~]# lctl network up
>>>>>
>>>>> LNET configured
>>>>>
>>>>> [root at lustre_mgs01_vm03 ~]# lctl list_nids
>>>>>
>>>>> 192.168.200.52 at o2ib
>>>>>
>>>>> 192.168.111.52 at tcp
>>>>>
>>>>> *On Client:*
>>>>> I am able to ping MGS on both tcp and ib network
>>>>>
>>>>> [root at dev1~]# ping 192.168.111.52
>>>>>
>>>>> PING 192.168.111.52 (192.168.111.52) 56(84) bytes of data.
>>>>>
>>>>> 64 bytes from 192.168.111.52: icmp_req=1 ttl=64 time=5.81 ms
>>>>>
>>>>> 64 bytes from 192.168.111.52: icmp_req=2 ttl=64 time=0.802 ms
>>>>>
>>>>> 64 bytes from 192.168.111.52: icmp_req=3 ttl=64 time=0.780 ms
>>>>>
>>>>> ^C
>>>>>
>>>>> --- 192.168.111.52 ping statistics ---
>>>>>
>>>>> 3 packets transmitted, 3 received, 0% packet loss, time 2000ms
>>>>>
>>>>> rtt min/avg/max/mdev = 0.780/2.464/5.811/2.366 ms
>>>>>
>>>>> [root at dev1 ~]# ping 192.168.200.52
>>>>>
>>>>> PING 192.168.200.52 (192.168.200.52) 56(84) bytes of data.
>>>>>
>>>>> 64 bytes from 192.168.200.52: icmp_req=1 ttl=64 time=24.4 ms
>>>>>
>>>>> 64 bytes from 192.168.200.52: icmp_req=2 ttl=64 time=2.14 ms
>>>>>
>>>>> 64 bytes from 192.168.200.52: icmp_req=3 ttl=64 time=0.782 ms
>>>>>
>>>>> 64 bytes from 192.168.200.52: icmp_req=4 ttl=64 time=9.30 ms
>>>>>
>>>>> ^C
>>>>>
>>>>> --- 192.168.200.52 ping statistics ---
>>>>>
>>>>> 4 packets transmitted, 4 received, 0% packet loss, time 3005ms
>>>>>
>>>>>
>>>>> *client mount commands*
>>>>>
>>>>> mount -t lustre 192.168.111.52 at tcp:/mylustre /lustre ( or)
>>>>>
>>>>> mount -t lustre 192.168.111.52 at tcp0:/mylustre /lustre ( or)
>>>>>
>>>>> mount -t lustre 192.168.200.52 at ob2:/mylustre /lustre
>>>>>
>>>>>
>>>>> *cat /var/log/messages | tail -40*
>>>>>
>>>>> Jul 16 17:03:17 dev1 user.err kernel: [2133277.466013] LustreError:
>>>> 162-5:
>>>>> Missing mount data: check that /sbin/mount.lustre is installed.
>>>>>
>>>>> Jul 16 17:03:17 dev1 user.err kernel: [2133277.466064] LustreError:
>>>>> 13627:0:(obd_mount.c:1325:lustre_fill_super()) Unable to mount  (-22)
>>>>>
>>>>> Jul 16 17:03:23 dev1 user.warn kernel: [2133282.680519] Lustre:
>>>>> 12364:0:(client.c:1918:ptlrpc_expire_one_request()) @@@ Request sent
>>>> has
>>>>> timed out for slow reply: [sent 1468702998/real 1468702998]
>>>>> req at ffff8801e0bc3c00 x1539427524411444/t0(0) o250->MGC192.168.111.52
>>>>>
>>>>> Jul 16 17:03:24 dev1 user.err kernel: [2133283.680193] LustreError:
>>>>> 13628:0:(client.c:1083:ptlrpc_import_delay_req()) @@@ send limit
>>>> expired
>>>>> req at ffff8801e0bc7000 x1539427524411448/t0(0)
>>>> o101->MGC192.168.111.52 at tcp
>>>>> @192.168.111.52 at tcp:26/25 lens 328/344 e 0 to 0 dl
>>>>>
>>>>> Jul 16 17:03:31 dev1 user.err kernel: [2133290.760978] LustreError:
>>>>> 13657:0:(client.c:1083:ptlrpc_import_delay_req()) @@@ send limit
>>>> expired
>>>>> req at ffff8801b7159800 x1539427524411456/t0(0)
>>>> o101->MGC192.168.111.52 at tcp
>>>>> @192.168.111.52 at tcp:26/25 lens 328/344 e 0 to 0 dl
>>>>>
>>>>> Jul 16 17:03:43 dev1 user.warn kernel: [2133302.681412] Lustre:
>>>>> 12364:0:(client.c:1918:ptlrpc_expire_one_request()) @@@ Request sent
>>>> has
>>>>> failed due to network error: [sent 1468703023/real 1468703023]
>>>>> req at ffff8801d6bfc800 x1539427524411460/t0(0) o250->MGC192.168.111
>>>>>
>>>>> Jul 16 17:04:08 dev1 user.warn kernel: [2133327.681402] Lustre:
>>>>> 12364:0:(client.c:1918:ptlrpc_expire_one_request()) @@@ Request sent
>>>> has
>>>>> failed due to network error: [sent 1468703048/real 1468703048]
>>>>> req at ffff8801d6bfec00 x1539427524411464/t0(0) o250->MGC192.168.111
>>>>>
>>>>> Jul 16 17:04:15 dev1 user.err kernel: [2133334.680175] LustreError:
>>>>> 13628:0:(client.c:1083:ptlrpc_import_delay_req()) @@@ send limit
>>>> expired
>>>>> req at ffff8801e0bc7000 x1539427524411452/t0(0)
>>>> o101->MGC192.168.111.52 at tcp
>>>>> @192.168.111.52 at tcp:26/25 lens 328/344 e 0 to 0 dl
>>>>>
>>>>> Jul 16 17:04:15 dev1 user.err kernel: [2133334.680316] LustreError:
>>>> 15c-8:
>>>>> MGC192.168.111.52 at tcp: The configuration from log 'mylustre-client'
>>>> failed
>>>>> (-5). This may be the result of communication errors between this node
>>>> and
>>>>> the MGS, a bad configuration, or other e
>>>>>
>>>>> Jul 16 17:04:15 dev1 user.err kernel: [2133334.680357] LustreError:
>>>>> 13628:0:(llite_lib.c:1046:ll_fill_super()) Unable to process log: -5
>>>>>
>>>>> Jul 16 17:04:15 dev1 user.warn kernel: [2133334.680881] Lustre:
>>>> Unmounted
>>>>> mylustre-client
>>>>>
>>>>> Jul 16 17:04:15 dev1 user.err kernel: [2133334.731730] LustreError:
>>>>> 13628:0:(obd_mount.c:1325:lustre_fill_super()) Unable to mount  (-5)
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> lustre-discuss mailing list
>>>>> lustre-discuss at lists.lustre.org
>>>>> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>>>>>
>>>> --
>>>> --------------------------------------------------------------------
>>>> Thomas Roth
>>>> Department: HPC
>>>> Location: SB3 1.262
>>>> Phone: +49-6159-71 1453  Fax: +49-6159-71 2986
>>>>
>>>> GSI Helmholtzzentrum f?r Schwerionenforschung GmbH
>>>> Planckstra?e 1
>>>> 64291 Darmstadt
>>>> www.gsi.de
>>>>
>>>> Gesellschaft mit beschr?nkter Haftung
>>>> Sitz der Gesellschaft: Darmstadt
>>>> Handelsregister: Amtsgericht Darmstadt, HRB 1528
>>>>
>>>> Gesch?ftsf?hrung: Professor Dr. Karlheinz Langanke
>>>> Ursula Weyrich
>>>> J?rg Blaurock
>>>>
>>>> Vorsitzender des Aufsichtsrates: St Dr. Georg Sch?tte
>>>> Stellvertreter: Ministerialdirigent Dr. Rolf Bernhardt
>>>>
>>>>
>>>> ------------------------------
>>>>
>>>> Subject: Digest Footer
>>>>
>>>> _______________________________________________
>>>> lustre-discuss mailing list
>>>> lustre-discuss at lists.lustre.org
>>>> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
>>>>
>>>>
>>>> ------------------------------
>>>>
>>>> End of lustre-discuss Digest, Vol 124, Issue 17
>>>> ***********************************************
>>>>
>>>
>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2252 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20160721/e8d22bb6/attachment-0001.bin>

[lustre-discuss] ​luster client mount issues

[lustre-discuss] luster client mount issues