[Lustre-discuss] Problems getting Lustre started with ZFS

[Thomas Roth]

Hi Andrew,

if you keep having trouble getting the MGS/MDT up, why don't you just mount it and skip the ldev.conf and init script part.
We did
mkfs.lustre  --backfstype=zfs --mgs mgs/mgs /dev/sdX
mkfs.lustre --fsname=zlust --mgsnode=10.16.0.104 at o2ib0 --mdt --index=0 --backfstype=zfs mdt0/mdt0 /dev/sdY

mount -t lustre mgs/mgs /srv/mgs
mount -t lustre mdt0/mdt0 /srv/mdt

That would show you if the formatting options were correct - even if your ldev.conf wasn't set up correctly, or the init script not working for
whatever reason.


As Scott wrote, you have to do the indexing of the OSTs by hand now.
On the OSS we did:
mkfs.lustre  --ost --index=0 --backfstype=zfs --fsname=zlust --mgsnode=10.16.0.104 at o2ib0 ost0/ost0 raidz2 /dev/sdc /dev/sdd ...
...
mkfs.lustre  --ost --index=6 --backfstype=zfs --fsname=zlust --mgsnode=10.16.0.104 at o2ib0 ost6/ost6 raidz2 /dev/sdcs /dev/sdct

mount -t lustre ost0/ost0 /srv/ost0
...
mount -t lustre ost6/ost6 /srv/ost6

Cheers,
Thomas



On 10/23/2013 06:46 PM, Andrew Holway wrote:
> Hello,
> 
> I have hit a wall trying to get lustre started. I have followed this
> to some extent:
> 
> http://zfsonlinux.org/lustre-configure-single.html
> 
> If someone could give me some guidance how to get these services
> started it would be much appreciated.
> 
> I running on Centos 6.4 and am getting my packages from:
> http://archive.zfsonlinux.org/epel/6/SRPMS/
> 
> Thanks,
> 
> Andrew
> 
> 
> [root at lustre1 ~]# zfs get lustre:svname
> NAME              PROPERTY       VALUE           SOURCE
> lustre-mdt0       lustre:svname  -               -
> lustre-mdt0/mdt0  lustre:svname  lustre:MDT0000  local
> lustre-mgs        lustre:svname  -               -
> lustre-mgs/mgs    lustre:svname  MGS             local
> [root at lustre1 ~]# zfs get lustre:svname
> NAME              PROPERTY       VALUE           SOURCE
> lustre-mdt0       lustre:svname  -               -
> lustre-mdt0/mdt0  lustre:svname  lustre:MDT0000  local
> lustre-mgs        lustre:svname  -               -
> lustre-mgs/mgs    lustre:svname  MGS             local
> [root at lustre1 ~]# /etc/init.d/lustre
> anaconda-ks.cfg       .bash_profile         .cshrc
> install.log.syslog    .ssh/                 .viminfo
> .bash_logout          .bashrc               install.log
> ks-post-anaconda.log  .tcshrc
> [root at lustre1 ~]# /etc/init.d/lustre start
> [root at lustre1 ~]# /etc/init.d/lustre start lustre-MDT0000
> lustre-MDT0000 is not a valid lustre label on this node
> [root at lustre1 ~]# /etc/init.d/lustre start MGS
> MGS is not a valid lustre label on this node
> 
> I have configured three OSS's with a single OST:
> 
> Andrews-MacBook-Air:~ andrew$ for i in {201..204}; do ssh
> root at 192.168.0.$i "hostname; zfs get lustre:svname"; done
> lustre1.calthrop.com
> NAME              PROPERTY       VALUE           SOURCE
> lustre-mdt0       lustre:svname  -               -
> lustre-mdt0/mdt0  lustre:svname  lustre:MDT0000  local
> lustre-mgs        lustre:svname  -               -
> lustre-mgs/mgs    lustre:svname  MGS             local
> lustre2.calthrop.com
> NAME              PROPERTY       VALUE           SOURCE
> lustre-ost0       lustre:svname  -               -
> lustre-ost0/ost0  lustre:svname  lustre:OST0000  local
> lustre3.calthrop.com
> NAME              PROPERTY       VALUE           SOURCE
> lustre-ost0       lustre:svname  -               -
> lustre-ost0/ost0  lustre:svname  lustre:OST0000  local
> lustre4.calthrop.com
> NAME              PROPERTY       VALUE           SOURCE
> lustre-ost0       lustre:svname  -               -
> lustre-ost0/ost0  lustre:svname  lustre:OST0000  local
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss
> 

-- 
--------------------------------------------------------------------
Thomas Roth
Department: Informationstechnologie
Location: SB3 1.262
Phone: +49-6159-71 1453  Fax: +49-6159-71 2986

GSI Helmholtzzentrum für Schwerionenforschung GmbH
Planckstraße 1
64291 Darmstadt
www.gsi.de

Gesellschaft mit beschränkter Haftung
Sitz der Gesellschaft: Darmstadt
Handelsregister: Amtsgericht Darmstadt, HRB 1528

Geschäftsführung: Professor Dr. Dr. h.c. Horst Stöcker,
Dr. Hartmut Eickhoff

Vorsitzende des Aufsichtsrates: Dr. Beatrix Vierkorn-Rudolph
Stellvertreter: Ministerialdirigent Dr. Rolf Bernhardt