[Lustre-discuss] Lustre - usage / best practices

[Andreas Dilger]

On 2012-11-04, at 8:54 AM, Jon Yeargers wrote:
> I’ll soon be retiring a number of old file servers and have the opportunity to move to a DFS based system. Im wondering about using Lustre for ‘mixed purposes’ and what the recommended setup is.
>  
> More specifically: lets say there are three groups that want to use the space. One group is primarily storing Tb of data in large numbers of flat files.

So, perfect use case for Lustre.

> A second group is using a DB like MySQL that has fewer but larger files.

This winds up being small random IO to the file.  Not a common workload for Lustre, so I would really recommend testing it out on a small system first.

> A third may have some other process. Would I divide up my DFS space somehow or is it ok for all three groups to comingle? Would I do something on the Lustre clients to force them apart or just rely on Linux security to restrict access to the client folders (on the lustre client)?

The Lustre servers have essentially the same security model as NFS today, so they depend on the clients to enforce user identification.  If malicious users can configure their own client nodes and connect them to the filesystem, then they could access any files in the filesystem.  If the client nodes are secure, then Lustre will properly enforce user access permissions/ACLs to the files.

> I can imagine creating a single Lustre client and then NIS mounting the various volumes on separate machines for each group. This seems like adding an extra step but perhaps it’s the normal pattern?

Do you mean "NFS" when you wrote "NIS"?  I'm not sure why this would be better, but I can definitely think of reasons why it would be worse.

Cheers, Andreas
--
Andreas Dilger                       Whamcloud, Inc.
Principal Lustre Engineer            http://www.whamcloud.com/