[lustre-discuss] Query regarding MDS and OSS functions

Dilger, Andreas andreas.dilger at intel.com
Tue Jun 28 18:39:07 PDT 2016


The MDS is in charge of the storage that is attached to it, so for a particular file or directory the client will always communicate with the same MDS. This is different from e.g. Ceph where the MDS is a "service" running on some node that doesn't have any of its own storage, so the Ceph MDS serving a particular file or directory can change over time.

Cheers, Andreas

On Jun 28, 2016, at 19:24, Sangeetha Banavathi Srinivasa <bsangee at vt.edu<mailto:bsangee at vt.edu>> wrote:

Needed a clarification on the second question.
Whenever a client has to communicate with an MDS is it always the same MDS or can it be any MDS.

On Jun 28, 2016, at 9:19 PM, Patrick Farrell <paf at cray.com<mailto:paf at cray.com>> wrote:


Replies inline.

Hi,

I had the following doubts about the functioning of lustre.

1. Whenever a file creation request is sent from the client, the request is sent to MDS from where a list of OSTs, file metadata etc is received.
        After this point does the data flow through the OSS to the OST or does the client communicate with the OST directly?

That question doesn't quite make sense.  The OST is just a disk volume.  The clients send data to the OSS (a server) that the OST in question is connected to, then the server writes that data to the OST.  This is done without involving the metadata server (MDS) except when opening and closing the file.

2. If a lustre system has more than one MDS will the client always send its requests to the same MDS or how is this decided?
An individual file is always on a particular single MDS.  So for a given file, the requests always go to the same MDS.  Deciding which files are put on which MDS is more complicated.  (This feature is known as distributed namespace or DNE, more info can be found online)

3. How does the MDS decide which OSTs have to be allocated whenever a file request has been made?
There are a number of different policies available.  I believe the default is a mix of round robin and space usage optimization to keep particular OSTs from filling up.

4. How many OSTs can an OSS have? How is this upper limit decided if at all?
I don't know what the upper limit is (it is at least 16), but the software limit is not the main concern.  The OSS (the server) has to be able to move enough data to keep the OSTs fed.  So while you can put many, many OSTs on one server, in practice, you would not.

5. How many clients can an application have?
There is no limit.  Real systems exist with tens of thousands of client nodes connected to one Lustre file system.  The practical limit is not client count, it is performance of the servers, disks, and network.
________________________________
From: lustre-discuss <lustre-discuss-bounces at lists.lustre.org<mailto:lustre-discuss-bounces at lists.lustre.org>> on behalf of Sangeetha Banavathi Srinivasa <bsangee at vt.edu<mailto:bsangee at vt.edu>>
Sent: Tuesday, June 28, 2016 7:58:03 PM
To: lustre-discuss at lists.lustre.org<mailto:lustre-discuss at lists.lustre.org>
Subject: [lustre-discuss] Query regarding MDS and OSS functions

Hi,

I had the following doubts about the functioning of lustre.

1. Whenever a file creation request is sent from the client, the request is sent to MDS from where a list of OSTs, file metadata etc is received.
        After this point does the data flow through the OSS to the OST or does the client communicate with the OST directly?

2. If a lustre system has more than one MDS will the client always send its requests to the same MDS or how is this decided?

3. How does the MDS decide which OSTs have to be allocated whenever a file request has been made?

4. How many OSTs can an OSS have? How is this upper limit decided if at all?

5. How many clients can an application have?


Thanks,
Sangeetha
_______________________________________________
lustre-discuss mailing list
lustre-discuss at lists.lustre.org<mailto:lustre-discuss at lists.lustre.org>
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

_______________________________________________
lustre-discuss mailing list
lustre-discuss at lists.lustre.org<mailto:lustre-discuss at lists.lustre.org>
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20160629/444fac4d/attachment.htm>


More information about the lustre-discuss mailing list