[Lustre-discuss] Hardware Question

[Balagopal Pillai]

Hi,

           I have 8 MD1000's (~90 TB of raw disk space) connected to 2 
Dell servers ( 2 quad core cpu's and 4 gigabit interfaces bonded)
on an hpc cluster with 6 lustre volumes and parallel i/o. For bigger 
files, performance is more than adequate and with large number of small
files, performance is close to terrible. Does the job, fast enough and 
is stable. If your storage requirement is for big files, one or two more
pe1950 for parallel i/o helps. I have another Lustre installation with 
one OSS. Again the throughput is better than nfs when mounted on 21 nodes.
The best performance i have seen for large number of small files is with 
gfs and ocfs2. (May be you could have a hybrid, with a few volumes with 
lustre
and few volumes with ocfs2?? never tried that!)

            The onboard broadcoms on pe1950 has the tendency to drop 
frames, just need to bump up the receive buffers with ethtool. I had a 
few occasions
where the two lustre oss had kernel panic on heavy i/o and large number 
of dropped frames. Once the dropping the problem is fixed, never had
that problem so far.

Regards
Balagopal




Aaron Knister wrote:
> I'm planning to hang 58 terabytes off of a PowerEdge 1950 with 4 CPUS  
> and 8 gigs of memory. My MDS is a dual core Opteron with a 250gig  
> raid1 metadata volume and 2GB of ram. Do you think this hardware  
> configuration is sane?
>
> -Aaron
>
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at clusterfs.com
> https://mail.clusterfs.com/mailman/listinfo/lustre-discuss
>