[Lustre-discuss] Lustre 1.8.1 QDR Support

[Jagga Soorma]

More information:

hpc116:/mnt/SLES11x86_64 # lspci | grep -i mellanox
10:00.0 InfiniBand: Mellanox Technologies MT26428 [ConnectX IB QDR, PCIe 2.0
5GT/s] (rev a0)

hpc116:/mnt/SLES11x86_64 # ibstatus
Infiniband device 'mlx4_0' port 1 status:
    default gid:     fe80:0000:0000:0000:0002:c903:0006:9109
    base lid:     0x14
    sm lid:         0x1
    state:         4: ACTIVE
    phys state:     5: LinkUp
    rate:         20 Gb/sec (4X DDR)

Infiniband device 'mlx4_0' port 2 status:
    default gid:     fe80:0000:0000:0000:0002:c903:0006:910a
    base lid:     0x0
    sm lid:         0x0
    state:         1: DOWN
    phys state:     2: Polling
    rate:         10 Gb/sec (4X)

hpc116:/mnt/SLES11x86_64 # ibstat
CA 'mlx4_0'
    CA type: MT26428
    Number of ports: 2
    Firmware version: 2.6.100
    Hardware version: a0
    Node GUID: 0x0002c90300069108
    System image GUID: 0x0002c9030006910b
    Port 1:
        State: Active
        Physical state: LinkUp
        Rate: 20
        Base lid: 20
        LMC: 0
        SM lid: 1
        Capability mask: 0x02510868
        Port GUID: 0x0002c90300069109
    Port 2:
        State: Down
        Physical state: Polling
        Rate: 10
        Base lid: 0
        LMC: 0
        SM lid: 0
        Capability mask: 0x02510868
        Port GUID: 0x0002c9030006910a


On Thu, Feb 11, 2010 at 1:21 PM, Jagga Soorma <jagga13 at gmail.com> wrote:

> Hi Guys,
>
> Wanted to give a bit more information.  So for some reason the transfer
> rates on my ib interfaces are autonegotiating at 20Gb/s (4X DDR).  However,
> these are QDR HCA's.
>
> Here is the hardware that I have:
>
> HP IB 4X QDR PCI-e G2 Dual Port HCA
> HP 3M 4X DDR/QDR QSFP IB Cu Cables
> Qlogic 12200 QDR switch
>
> I am using all the lustre provided rpms on my servers (RHEL 5.3) and
> clients (SLES 11).  All my servers in this cluster are auto negotiating to
> 20Gb/s (4X DDR) which should be 40Gb/s (4X QDR).
>
> Are any others out there using QDR?  If so, did you run into anything
> similar to this?  Is there any specific configuration that is needed for the
> servers to detect the higher rates.
>
> Thanks in advance for your assistance.
>
> -J
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20100211/3bf97432/attachment.htm>