[Lustre-devel] Moving forward on Quotas

Andreas Dilger adilger at sun.com
Mon Jun 2 16:24:33 PDT 2008

On Jun 01, 2008  10:32 +0800, Peter J. Braam wrote:
> I am quite worried about the dynamic qunit patch.
> I am not convinced I want smaller qunits to stick around.
> Please PROVE RIGOROUSLY that qunits are grow large quickly again, otherwise
> they create too much server - server overhead.  The cost of 100MB of disk
> space is barely more than a cent now; what are we trying to address withtiny
> qunits?
> Plan for 5000 OSS servers at the minimum and 1,000,000 clients, and up to
> 100TB/sec in I/O.  Calculate quota RPC traffic from that.  A server cannot
> handle more than 15,000 RPC's / sec.
> No arguing, or opinions here, numbers please.  The original design I did 4
> years ago limited quota calls from one OSS to the master to one per second.
> Qunits were made adaptive without solid reasoning or design.

Just a note - it isn't only shrinking of qunits that is possible, but also
growth of qunits.  I think there was also work done to allow recall of
qunits from the servers, but I'm not sure if it was landed into CVS.

If we are significantly re-architecting quotas, I'd suggest that we also
re-implement grants at the same time and use the DLM to do both of them.
This way we can have grant + quota on a per-file basis (quota + grant are
given to clients as part of extent lock LVB), and are also able to
recall quota + grant.  We may not even want to have separate quota+grant,
since we track the ownership of files on the OSTs and space allocation
is done on a per-file basis.

It would be possible, for example, to take a user's whole quota from
the master, split it evenly into "num_osts * 2" chunks at mount time
to pass to the OSTs, they further grant it to clients when they request
extent locks, and then avoid ALL master->OST quota RPCs unless that user
actually got close to exceeding their quota, either granting out some
of the remaining "num_osts" qunits or recalling some of the outstanding
quota (possibly via lock "conversion" to avoid revoking the quota lock).

Cheers, Andreas
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.

More information about the lustre-devel mailing list