[Lustre-discuss] Multiple IB ports

[Sebastien Piechurski]

Hi Brian,
 
>From my understanding, but confirmation from more skilled people on the
list would be welcomed, using multiple IB ports with a lustre client
will be difficult to manage, and will probably not bring any performance
improvements.
I was told by a colleague that there were currently too many internal
locks in the clients to sustain a big throughput. Lustre is designed for
global throughput on many clients, but not on individual clients.
I can observe this on my site, where I have enough storage and servers
to reach 21GB/s globally, but am unable to get more than 300MB/s on a
single client even though the DDR IB network would sustain +800MB/s ...


________________________________

	From: lustre-discuss-bounces at lists.lustre.org
[mailto:lustre-discuss-bounces at lists.lustre.org] On Behalf Of Brian
O'Connor
	Sent: lundi 21 mars 2011 04:53
	To: lustre-discuss at lists.lustre.org
	Subject: [Lustre-discuss] Multiple IB ports
	
	

	Hi,

	    Any body actually using multiple IB ports on a client for an
aggregated connection?

	 

	Ie. Many oss with one qdr IB each. Clients with 4 qdr IB ports.
Assuming the normal

	issues with bus bandwidth etc, what sort of perf can I expect 

	 

	qdr ~ 3-4Gbytes/Sec

	 

	I'm trying to size a cluster and clients to get ~10GBytes/Sec on
*one* 

	client node. 

	 

	If I can aggregate IB linearly the next step will be to try and
figure out

	How to get 10Gigabytes/s to local storage :-(

	 

	 

	Some times customers are crazy.......

	 

	 

	 

	Brian O'Connor

	 

	-------------------------------------------------

	 

	SGI Consulting

	 

	Email: briano at sgi.com <mailto:briano at sgi.com> , Mobile +61 417
746 452

	 

	Phone: +61 3 9963 1900, Fax: +61 3 9963 1902

	 

	357 Camberwell Road, Camberwell, Victoria, 3124 

	 

	AUSTRALIA http://www.sgi.com/support/services
<http://www.sgi.com/support/services> 

	 

	-------------------------------------------------

	 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20110321/65afa9ba/attachment.htm>