[Lustre-discuss] "ls -l" command in the Lustre Filesystem

Johnlya johnlya at gmail.com
Wed Jul 9 20:00:07 PDT 2008


Thank you!

On Tue 2008-07-08, at 08:16 PM, "Brian J. Murrell"
<Brian.Murr... at Sun.COM> wrote:
> On Mon, 2008-07-07 at 03:29 -0700, Johnlya wrote:
> > I found that executing "ls" command would need more time
> > in the Lustre filesystem than in the ext3 filesystem.
>
> This is not terribly surprising if you think about what Lustre is doing.
>
> An ls on ext3 involves lots of lookup(s) of (meta)data to a very local
> device, namely a disk on a SATA/SCSI/IDE bus local in the machine.  The
> latency of this lookup is very low.  Those same ls lookups on Lustre has
> to go out on a network to another machine to do the same metadata
> lookups plus it has to go to the OST(s) that hold the file to get the
> size and has to return that data all on the network again, with higher
> latencies.
>
> Indeed, there are features in Lustre, such as "statahead" to try to
> streamline the data lookups and data movement and reduce the round-trip
> latencies, but you are still not working with "like" technologies.
>
> Instead you are paying a small price for huge gains.  For people who
> need Lustre, using a local ext3 filesystem simply won't scale for them.
> Lustre will.  Their workload is not usually "ls" either though and they
> appreciate the scalability they get for the small cost of a slower "ls".
>
> That said, the future will bring clustered metadata (CMD) storage, where
> an ls of a large directory could actually be faster than an ls to the
> same sized directory on a local ext3 filesystem due to parallelizing
> metadata the way we can with object data over many OSTs today.
>
> b.
>
>  signature.asc
> 1Kdownload
>
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-disc... at lists.lustre.orghttp://lists.lustre.org/mailman/listinfo/lustre-discuss



More information about the lustre-discuss mailing list