[Lustre-devel] LustreFS performance
Mark.Seger at hp.com
Tue Mar 10 08:07:36 PDT 2009
>>> **** Statistics ****
>>> During all the tests the following is supposed to be running on all
>>> the servers:
>>> 1) vmstat
>>> 2) iostat, if there is some disk activity.
>>> smth else?
>> I would propose either LLNL's LMT or HP's collectl, which both also
>> collect Lustre stats. Those both provide more information than the
>> above, and having the IO/CPU load correlated to Lustre RPC counts is
>> very useful.
> It would be great if we could standardize on a set of tools for performance
> issues. I've got to think a set of tools like this would make it easier for
> customer & partners to gather the correct data the first time.
> Cray has been using lstats, a package of scripts we got from Sun a while back.
> We've added things like AT timeout and sar per-cpu usage to it (see bug 18574
> att 22140 for complete set of scripts).
> I'm all for using collectl, but I think the requirements and setup for LMT makes
> it a tough sell. Does Sun have a set of customizations for collectl or does
> the standard collectl collect enough information?
My goal when I wrote collectl was to provide one-stop shopping for as
much system performance data as seemed relevant and view lustre as only
one of many data sources. To that end, if you do a merge of all the
data collected by the *stat utilities, sar, perfquery (for IB), many of
the lustre stats (but not all) and maybe a few others you'll get closer
to understanding what collectl can collect. On the output side you can
pick and choose what to display - when used interactively only those
data elements are collected but when run as a daemon you can collect
them all and replay the data as ofter as you like looking at different
As for LMT I haven't played with it as my interests are in dealing with
all data. However, as an exercise left to the reader, there are a
number of switches for changing collectl's display as well as --home
which moves the cursor the terminal's home position before displaying
the output, giving a display similar to the feel of top. If you want to
display what's happening to lustre and your disks, cpu, etc all at the
same time on a refreshing display, --top is definitely the way to go.
And finally, if you want something totally different and are feeling
creative, just write your own print routines in perl and tell collectl
to use them with the --export switch.
More information about the lustre-devel