<div dir="ltr">Thank you very much for your quick help !<br>I reformatted and remounted everything from scratch and can confirm that mounting works, and that the client can communicate with the MDS (210, OSS is 211 and client 212):<br><br>[root@x8602 tests]# lnetctl net show<br>net:<br> - net type: lo<br> local NI(s):<br> - nid: 0@lo<br> status: up<br> - net type: tcp<br> local NI(s):<br> - nid: 10.40.24.212@tcp<br> status: up<br> interfaces:<br> 0: eno1<br>[root@x8602 tests]# lnetctl peer show -v<br>peer:<br> - primary nid: 10.40.24.210@tcp<br> Multi-Rail: True<br> peer ni:<br> - nid: 10.40.24.210@tcp<br> state: NA<br> max_ni_tx_credits: 8<br> available_tx_credits: 8<br> min_tx_credits: 6<br> tx_q_num_of_buf: 0<br> available_rtr_credits: 8<br> min_rtr_credits: 8<br> refcount: 1<br> statistics:<br> send_count: 137546<br> recv_count: 137545<br> drop_count: 0<br> - primary nid: 10.40.24.212@tcp<br> Multi-Rail: True<br> peer ni:<br> - nid: 10.40.24.212@tcp<br> state: NA<br> max_ni_tx_credits: 8<br> available_tx_credits: 8<br> min_tx_credits: -84<br> tx_q_num_of_buf: 0<br> available_rtr_credits: 8<br> min_rtr_credits: 8<br> refcount: 1<br> statistics:<br> send_count: 291726<br> recv_count: 291726<br> drop_count: 0<br> - primary nid: 10.40.24.211@tcp<br> Multi-Rail: True<br> peer ni:<br> - nid: 10.40.24.211@tcp<br> state: NA<br> max_ni_tx_credits: 8<br> available_tx_credits: 8<br> min_tx_credits: 7<br> tx_q_num_of_buf: 0<br> available_rtr_credits: 8<br> min_rtr_credits: 8<br> refcount: 1<br> statistics:<br> send_count: 56<br> recv_count: 56<br> drop_count: 0<br>[root@x8602 tests]# lctl which_nid 10.40.24.210@tcp<br>10.40.24.210@tcp<br>[root@x8602 tests]# lfs df -ih<br>UUID Inodes IUsed IFree IUse% Mounted on<br>test-MDT0000_UUID 4.0M 272 4.0M 1% /lustre[MDT:0]<br>test-OST0000_UUID 640.0K 267 639.7K 0% /lustre[OST:0]<br><br>filesystem_summary: 640.0K 272 639.7K 0% /lustre<br><br>[root@x8602 tests]# ls -lsah /lustre/<br>total 12K<br>4.0K drwxr-xr-x 3 root root 4.0K Jul 18 11:03 .<br>4.0K dr-xr-xr-x. 19 root root 4.0K Jun 28 11:43 ..<br>4.0K -rw-r--r-- 1 root root 14 Jul 18 11:03 test.txt<div><br></div><div>I get the same output from auster though:<br>Client: Lustre version: 2.12.0<br>MDS: No host defined for facet /usr/sbin/lctl<br>OSS: Lustre version: 2.12.0<br><br>From the client I can ssh into the other nodes (and from each node I can ssh into the others).<br>I had tried to debug the scripts behind the above auster output but was unable to track down where it failed...</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, 16 Jul 2019 at 23:09, Andreas Dilger <<a href="mailto:adilger@whamcloud.com">adilger@whamcloud.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On Jul 16, 2019, at 06:11, Baptiste Gerondeau <<a href="mailto:baptiste.gerondeau@linaro.org" target="_blank">baptiste.gerondeau@linaro.org</a>> wrote:<br>
> <br>
> Hi,<br>
> <br>
> I'm currently in the process of bringing up the "3 node" x86 cluster and running "verbose=true ./auster -f multinode -rsv runtests" (on CentOS 7.6 x86 client & server, installed from repos), I keep getting "MDS: No host defined for facet /usr/sbin/lctl".<br>
> <br>
> Auster then prints out some pdsh stuff, "Failures : 0" and exits after 16s obviously without running any tests.<br>
> <br>
> Any suggestions?<br>
> Thanks a lot,<br>
> <br>
> <br>
> PS : My multinode config is attached<br>
> PPS: I posted to the devel list because it concerned auster, if I need to post it elsewhere please let me know<br>
<br>
Before running auster, which tries to launch a lot of tests, start with just a plain mount to see if that is working:<br>
<br>
master.sh:<br>
> MOUNT=/mnt/lustre<br>
> MOUNT2=/mnt/master2<br>
<br>
This is a bit odd for tests, which normally have e.g. /mnt/master and /mnt/master2, but I'm<br>
not sure i there will be a problem or not.<br>
<br>
### assume modules/utils are built<br>
### modules/utils are installed or you are running out of the build directory<br>
### ssh to the MDS and OSS nodes works without a password<br>
### if you are not using @tcp0 for LNet, /etc/modprobe.d/lnet.conf is correct<br>
<br>
all# modprobe ptlrpc ### on client and OSS and MDS to start LNet<br>
x8602# lctl ping x86ohpc ### should print NID(s) of x860hpc<br>
x8602# lctl ping x8601 ### should print NID(s) of x8601<br>
x8602# export NAME=master ### get config from lustre/tests/cfg/master.sh<br>
x8602# sh llmount.sh ### should format x86ohpc:/dev/sda2 and x8601:/dev/sda2<br>
x8602# lfs df ### should show master-MDT0000 and master-OST0000<br>
<br>
Cheers, Andreas<br>
--<br>
Andreas Dilger<br>
Principal Lustre Architect<br>
Whamcloud<br>
<br>
<br>
<br>
<br>
<br>
<br>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><font face="monospace, monospace">Baptiste Gerondeau<br>Engineer - HPC SIG - LDCG - Linaro<br>#irc : BaptisteGer</font><br></div></div>