[Lustre-discuss] Lustre SNMP module

[Kilian CAVALOTTI]

On Thursday 20 March 2008 01:15:04 pm Mark Seger wrote:
> not sure if you're talking about collectl 

Not I wasn't, I was referring to the Lustre Monitoring Tool (LMT) from 
LLNL.

> Be careful here.  You can certain stick some data into an rrd but
> certainly not all of it, especially if you want to collect a lot of
> it at a reasonable frequency.  If you want accurate detail plots,
> you've gotta go to the data stored on each separate system.  I just
> don't see any way around this, at least not yet...

Yes, you're absolutely right. Given its intrinsic multi-scale nature, a 
RRD is well suited for keeping historical data on large time scales. 
This could allow a very convenient graphical overview of the different 
system metrics, but would be pointless for debugging purposes, where 
you do need fine-grained data. That's where collectl is the most useful 
for me. 

But what about both? I don't see any reason why collectl couldn't 
provide high-frequency accurate data to diagnose problems locally, and 
at the same time allow to aggregate less precise values in RRD for 
global visualization of multi-hosts systems.

> As a final note, I've put together a tutorial on using collectl in a
> lustre environment and have upload a preliminary copy at
> http://collectl.sourceforge.net/Tutorial-Lustre.html in case anyone
> wants to preview it before I link it into the documentation.  
> If nothing else, look at my very last example where I show what you 
> can see by monitoring lustre at the same time as your network
> interface.  

Very good, thanks for this. The readahead experiment is insightful.

> Did I also mention that collectl is probably one of the few tools
> that can monitor your Infiniband traffic as well?

That's why it rocks. :)

Now the only thing which still make me want to use other monitoring 
software is the ability to get a global view. Centralized data 
collection and easy graphing (RRD feeding) are still what I need most 
of the time.

Cheers,
-- 
Kilian