[Lustre-discuss] LMT 2.6.0

Wed Apr 9 11:32:30 PDT 2008

Hi Klaus,

Nobody's done that yet that I know of, but it would seem doable.

Cerebro is similar to ganglia in that it uses multicast with 
listeners and talkers.  You would need machinery to get lustre proc
values into ganglia on the lustre servers (the talkers), and then from 
ganglia to the mysql database (the listener).

Jim

On Wed, Apr 09, 2008 at 11:02:49AM -0700, Klaus Steden wrote:
> 
> Hi Jim,
> 
> Out of curiosity (and bear with me if this is a stupid question), do you
> know if anyone's integrated LMT with Ganglia? The clusters we have that use
> Lustre are all ROCKS-based, which uses Ganglia for monitoring ... I'm not
> familiar with Cerebro, so I'm not sure how you're using it, but it would be
> nice to be able to integrate LMT's statistics into an existing monitoring
> solution rather than implementing a second one.
> 
> Other than that, I'm looking forward to screenshots -- this sounds like a
> nice add-on.
> 
> cheers,
> Klaus
> 
> On 4/9/08 8:59 AM, "Jim Garlick" <garlick at llnl.gov>did etch on stone
> tablets:
> 
> > Hi Chris,
> > 
> > I think Herb is reporting for jury duty today so I'll speak up and let him
> > follow up later if I missed anything.
> > 
> > There are text clients (ltop and lstat).  I haven't tried running
> > the GUI remotely and tunneling the X session, or running the client
> > locally and tunneling mysql protocol, but I don't know of anything
> > architectural that would prevent either from working.
> > 
> > We'll get some screenshots out there.  Herb was reticent to publish
> > LMT without good documentation, but I encouraged him because there were
> > a couple of people anxious to try it out.  So the lack of screenshots/
> > documentation is my bad - we'll try to fix that as time permits.
> > 
> > Regarding the dependencies: it's always a tradeoff between reinventing the
> > wheel and adding dependencies.  Sigh.  In this case, we leveraged MySQL for
> > the database of historical data; we managed to get a graphics group here
> > which is a Java shop to code the clients, so they interface directly to
> > MySQL using the Java MySQL bindings; and we leveraged the Cerebro multicast
> > based monitoring tool for the data collection because it introduced no
> > new dependencies on lustre servers at our site (since we already use it
> > to monitor other things), and is very lightweight.
> > 
> > I think this is a good architecture: the clients and data collection
> > parts are independent with interfaces defined by the database schema,
> > so they could be independently replaced, for example if someone wanted to
> > write text clients in C/curses/MySQL, or replace the backend with some
> > other data collection infrastructure that they are already using or feel
> > is superior.
> > 
> > Jim
> > 
> > On Wed, Apr 09, 2008 at 08:47:39AM -0600, Chris Worley wrote:
> >> Are there any screenshots/FAQs?
> >> 
> >> I looked at the package, and the depandancy list is long.
> >> 
> >> As most of my work is remote, does the GUI run remotely over an SSH tunnel?
> >> 
> >> Are there any similar console/text-based utilities w/ a shorter
> >> dependency list and lighter weight for remote access?
> >> 
> >> Thanks,
> >> 
> >> Chris
> >> On Mon, Apr 7, 2008 at 5:54 PM, Herb Wartens <wartens2 at llnl.gov> wrote:
> >>> -----BEGIN PGP SIGNED MESSAGE-----
> >>>  Hash: SHA512
> >>> 
> >>> 
> >>>  Hi All,
> >>>  We have a new and improved version of LMT released
> >>>  on Sourceforge.  For those who have never used it, LMT is the Lustre
> >>>  Monitoring Tool developed at LLNL.  It provides realtime monitoring of a
> >>>  Lustre filesystem (or multiple filesystems).  It also graphs data over time
> >>>  for a set of attributes specified by the user.  Please feel free to try it
> >>>  out.
> >>> 
> >>>  Below are the release notes.
> >>>  (please forgive this if it is a repost as it looks like lustre-discuss
> >>>  may have finally come back up)
> >>> 
> >>> 
> >>>  http://sourceforge.net/projects/lmt
> >>> 
> >>>  ========================================================================
> >>>  Release Notes for LMT 2.6.0                                  07 Apr 2008
> >>>  ========================================================================
> >>> 
> >>>  * This version of LMT 2 has been tested to work properly with Lustre 1.6.x
> >>>   It is no longer necessary to keep the old lustre *.xml configuration and
> >>>   there are no more xwatch-lustre.conf or lmt.conf files to set up.
> >>> 
> >>>  * LMT now uses Cerebro to transport the data collected from the Lustre
> >>>   filesystem being monitored. For more info see:
> >>>   http://sourceforge.net/projects/cerebro
> >>> 
> >>>  * Building the rpms requires ibm-java since this is what we use in Chaos 4.
> >>>   This can be changed in the specfile to require Sun Java if that is
> >>>   preferred (the main issue is that we need a full fledged JVM that
> >>>   supports swing).
> >>> 
> >>>  * There is now a dependency on mysql to store the collected data. Since the
> >>>   resolution of the collection is ~5 seconds this can quickly consume a lot
> >>>   of space.  The clients and servers are completely decoupled.
> >>> 
> >>>  * Please refer to the documentation for instructions on how to install and
> >>>   configure LMT 2.
> >>>  -----BEGIN PGP SIGNATURE-----
> >>>  Version: GnuPG v1.4.7 (GNU/Linux)
> >>>  Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org
> >>> 
> >>>  iD8DBQFH+rQhP/62XqEEbMYRCqY6AJ9woGAmDo+lqEoOrr8fhSqGGL1aXwCg1Ml5
> >>>  NwuPz3GIhOU0zAeZDrueQhA=
> >>>  =QWFx
> >>>  -----END PGP SIGNATURE-----
> >>> 
> >>> 
> >>> _______________________________________________
> >>>  Lustre-discuss mailing list
> >>>  Lustre-discuss at lists.lustre.org
> >>>  http://lists.lustre.org/mailman/listinfo/lustre-discuss
> >>> 
> >> _______________________________________________
> >> Lustre-discuss mailing list
> >> Lustre-discuss at lists.lustre.org
> >> http://lists.lustre.org/mailman/listinfo/lustre-discuss
> > _______________________________________________
> > Lustre-discuss mailing list
> > Lustre-discuss at lists.lustre.org
> > http://lists.lustre.org/mailman/listinfo/lustre-discuss