[lustre-discuss] How to configure OST and OSS on separate nodes

[Patrick Farrell]

One difference: I believe metadata performance is still notably lower with a ZFS based MDT.  I know the gap is closing with changes LLNL and Intel have made, but last I heard it was still significant.  Someone else may be able to give current numbers.

Of course, you can do an ldiskfs MDT and ZFS OSTs.  And since the MDT is much smaller than OSTs, one of the central advantages of ZFS - it allows use of an efficient software RAID, so you can use cheaper hardware options - is less relevant there.  (Because the disk hardware is a smaller % of the cost of a good MDS/MDT.)

- Patrick
________________________________________
From: lustre-discuss [lustre-discuss-bounces at lists.lustre.org] on behalf of Stearman, Marc [stearman2 at llnl.gov]
Sent: Wednesday, January 27, 2016 10:46 AM
To: jeevan.patnaik at wipro.com
Cc: Lustre discussion
Subject: Re: [lustre-discuss] How to configure OST and OSS on separate nodes

Yes, you can run ZFS instead of ldiskfs.  A number of sites, LLNL and SDSC included, have moved over to ZFS only Lustre file systems.  There have been a number of presentations given at the Lustre User Group (LUG) about the advantages of ZFS.  Please look here http://opensfs.org/resources/presentations/ and look at the past few years of talks with ZFS in the title.

I will be giving a tutorial on how to Install and Configure a ZFS file system at the Lustre Ecosystem Workshop in March, but that may be a bit too late depending on your needs.

-Marc

----
D. Marc Stearman
Lustre Operations Lead
stearman2 at llnl.gov
Office:  925-423-9670
Mobile:  925-216-7516




> On Jan 27, 2016, at 2:00 AM, jeevan.patnaik at wipro.com wrote:
>
> Hi,
>
> Now, it cleared my confusion. So, we really should go for having storage as separate, we will fix with this option.
>
> And besides, I have completely ignored ZFS till now. Can I go for ZFS instead of ldiskfs? What’s the main difference I can find?
>
> --
> Regards,
> Jeevan.
>
> From: lustre-discuss [mailto:lustre-discuss-bounces at lists.lustre.org] On Behalf Of Oliver Mangold
> Sent: Wednesday, January 27, 2016 3:25 PM
> To: lustre-discuss at lists.lustre.org
> Subject: Re: [lustre-discuss] How to configure OST and OSS on separate nodes
>
> On 27.01.2016 10:36, jeevan.patnaik at wipro.com wrote:
> But my question is: Can I use OST on client and OSS on separate server (if it is possible to run OST and OSS on separate servers)
>
> I don't understand what you mean. OSS is just a name for the machine which runs one or multiple OSTs. So by definition OSTs always run on OSSes.
>
> The reason for this is we have tens of servers with internal hard disk of 4T each which we want to use it for OSTs, but all these servers are also should be clients (they are the grid execution hosts).
> Sounds like a bad idea to me to do that. Beside the mentioned memory pressure problems, you also don't have a failover configuration for the OSTs, which is strongly recommended. Otherwise one downed server will stall all or most of your active jobs.
>
> So, I am thinking of having separate OSSs and use the servers itself as OSTs. Also, do we need to install lustre kernel in all those OSTs? Because, all these servers are running with kernel 2.6.32-573.12.1.el6.x86_64, which the standard lustre packages won’t be supporting. So, we need to roll back to 32-504_lustre kernel and again we need to work on setting yum repos, installing packages, resolving dependencies required for our tools..it’s really a lot of work.
>
> The last I know is, you need a patched kernel when using ldiskfs as the backing filesystem, but you can use a patchless kernel with ZFS.
>
>
> The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately and destroy all copies of this message and any attachments. WARNING: Computer viruses can be transmitted via email. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email. www.wipro.com_______________________________________________
> lustre-discuss mailing list
> lustre-discuss at lists.lustre.org
> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

_______________________________________________
lustre-discuss mailing list
lustre-discuss at lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org