[Lustre-discuss] Benchmarking small file performance

[Ragnar Kjørstad]

Hi Rick,


On Thu, May 15, 2008 at 04:21:56PM -0400, Rick Friedman wrote:
> We build a Lustre appliance that comes fully configured and tuned. A
> number of customers have asked us about small file performance. We are
> looking for feedback on how people "quantify" this type of performance
> with their Lustre based systems. In most cases we are looking at
> applications that are reading and writing files in the 1K to 20M size
> range. 

One tool commonly used to compare file system operation performance for
local filesystems and NFS clients is bonnie++
(http://www.coker.com.au/bonnie++/). It's not really designed for
paralell filesystems but at least it will quantify single-client lustre
performance in a way that your customers can relate to. 

Note that since bonnie++ only runs on a single machine at the time you
need some kind of wrapper to run it on a multi-client lustre cluster.
There is a very primitive one available at
http://ragnark.vestdata.no/download/dbonnie++.txt, but what you should
do is make a version of bonnie++ that use MPI for syncronization instead
of semaphores, so that it can easily be run on clusters.



-- 
Ragnar Kjørstad