[Lustre-discuss] Lustre 1.8.1 QDR Support
Jagga Soorma
jagga13 at gmail.com
Thu Feb 11 13:44:24 PST 2010
Yet more information. Looks like the switch thinks that this could be set
to 10Gbps (QDR):
hpc116:/mnt/SLES11x86_64 # iblinkinfo.pl -R | grep -i reshpc116
1 34[ ] ==( 4X 5.0 Gbps Active / LinkUp)==> 20 1[ ]
"hpc116 HCA-1" ( Could be 10.0 Gbps)
-J
On Thu, Feb 11, 2010 at 1:26 PM, Jagga Soorma <jagga13 at gmail.com> wrote:
> More information:
>
> hpc116:/mnt/SLES11x86_64 # lspci | grep -i mellanox
> 10:00.0 InfiniBand: Mellanox Technologies MT26428 [ConnectX IB QDR, PCIe
> 2.0 5GT/s] (rev a0)
>
> hpc116:/mnt/SLES11x86_64 # ibstatus
> Infiniband device 'mlx4_0' port 1 status:
> default gid: fe80:0000:0000:0000:0002:c903:0006:9109
> base lid: 0x14
> sm lid: 0x1
> state: 4: ACTIVE
> phys state: 5: LinkUp
> rate: 20 Gb/sec (4X DDR)
>
> Infiniband device 'mlx4_0' port 2 status:
> default gid: fe80:0000:0000:0000:0002:c903:0006:910a
> base lid: 0x0
> sm lid: 0x0
> state: 1: DOWN
> phys state: 2: Polling
> rate: 10 Gb/sec (4X)
>
> hpc116:/mnt/SLES11x86_64 # ibstat
> CA 'mlx4_0'
> CA type: MT26428
> Number of ports: 2
> Firmware version: 2.6.100
> Hardware version: a0
> Node GUID: 0x0002c90300069108
> System image GUID: 0x0002c9030006910b
> Port 1:
> State: Active
> Physical state: LinkUp
> Rate: 20
> Base lid: 20
> LMC: 0
> SM lid: 1
> Capability mask: 0x02510868
> Port GUID: 0x0002c90300069109
> Port 2:
> State: Down
> Physical state: Polling
> Rate: 10
> Base lid: 0
> LMC: 0
> SM lid: 0
> Capability mask: 0x02510868
> Port GUID: 0x0002c9030006910a
>
>
>
> On Thu, Feb 11, 2010 at 1:21 PM, Jagga Soorma <jagga13 at gmail.com> wrote:
>
>> Hi Guys,
>>
>> Wanted to give a bit more information. So for some reason the transfer
>> rates on my ib interfaces are autonegotiating at 20Gb/s (4X DDR). However,
>> these are QDR HCA's.
>>
>> Here is the hardware that I have:
>>
>> HP IB 4X QDR PCI-e G2 Dual Port HCA
>> HP 3M 4X DDR/QDR QSFP IB Cu Cables
>> Qlogic 12200 QDR switch
>>
>> I am using all the lustre provided rpms on my servers (RHEL 5.3) and
>> clients (SLES 11). All my servers in this cluster are auto negotiating to
>> 20Gb/s (4X DDR) which should be 40Gb/s (4X QDR).
>>
>> Are any others out there using QDR? If so, did you run into anything
>> similar to this? Is there any specific configuration that is needed for the
>> servers to detect the higher rates.
>>
>> Thanks in advance for your assistance.
>>
>> -J
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20100211/87edceee/attachment.htm>
More information about the lustre-discuss
mailing list