Qlogic QL45212 Bedienungsanleitung

BC0154503-00 C Third party information brought to

you courtesy of Dell EMC.

User’s Guide

25Gb Intelligent Ethernet Adapter

QL45212

ii BC0154503-00 C

User’s Guide—25Gb Intelligent Ethernet Adapter

QL45212

This document is provided for informational purposes only and may contain errors. QLogic reserves the right, without

notice, to make changes to this document or in product design or specifications. QLogic disclaims any warranty of any

kind, expressed or implied, and does not guarantee that any results or performance described in the document will be

achieved by you. All statements regarding QLogic's future direction and intent are subject to change or withdrawal

without notice and represent goals and objectives only.

Document Revision History

Revision A, January 11, 2015

Revision B, April 19, 2016

Revision C, January 27, 2017

Changes Sections Affected

Removed support for out-of-box OFED Throughout

Updated Dell log Cover page

Removed “Technical Support” topic on preface. “Preface” on page xiii

Added RoCE v1 and v2

Under Performance Features, TCP segmentation

offload: added virtual machine multiqueues

(VMMQ)

“Features” on page 2

Added Applicable IEEE802.3 Ethernet “Standards Specifications” on page 4

Added a reference to driverdownloads.qlogic.com

for a complete list of supported operating systems.

Removed reference to “Full dual-port 40Gb and

single-port 100Gb bandwidth is supported on PCIe

Gen3 x16 or faster slots.

“System Requirements” on page 6

Removed reference to “processor” in the Architec-

ture row, Requirement column.

Updated content on the PCIe row for both the

Hardware and Requirement columns.

Table 2-1 on page 6

Added reference to CentOS.

Updated Windows Server support with Windows

Server 2016 Nano, 2008 SP2 and x64 (12G Only),

2008 R2 with SP1, 2012, 2012 R2, Windows PE

5.0 64bit, Windows PE 10.0 64bit

Updated RHEL versions for Linux to RHEL 7.3,

7.2, 6.8, 6.7 12G and 13G.

Updated SLES versions to SLES 12 SP2, and 11

SP4.

Updated VMware to ESXi 6.0 U2, and 6.5 U1

Added XenServer 7.0 and 6.5

UEFI 2.3, 2.3.1, and 2.5

Table 2-2 on page 6

User’s Guide—25Gb Intelligent Ethernet Adapter

QL45212

iii BC0154503-00 C

Removed “Running the Dell Update Package from

the Command Line” topic.

“Installing Drivers” on page 9

Updated the list of RHEL source and kmod RPM

package names. Added SLES source and kmod

RPM packages.

“Linux Driver Software” on page 10

Updated Step 1, first bullet command

rpm -e kmod-qlgc-fastlinq-<version>.

<arch> to

rpm -e qlgc-fast-

linq-kmp-default-<version>.<arch>

“Removing Linux Drivers in a non-RoCE Environ-

ment” on page 12

Added new Step 1 “Type the following command to

get the path to the currently installed drivers: mod-

info <drivername>.

Replaced references to rmod qede and rmod

qed with modprobe -r qede.

Added new note “If the qedr is present, use mod-

probe -r qedr instead.”

“To remove Linux drivers in a non-RoCE environ-

ment:” on page 12

Added new Step 1 “Type the following command to

get the path to the currently installed drivers: mod-

info <drivername>.

Deleted reference to “modprobe -r qede,

modprobe -r qed, and depmod -a.”

“To remove Linux drivers in a RoCE environment:”

on page 13

Updated Step 2, first bullet command

rpm -e kmod-qlgc-fastlinq-<version>.

<arch> to

rpm -e qlgc-fast-

linq-kmp-default-<version>.<arch>

“Removing Linux Drivers in a RoCE Environment”

on page 13

Added reference to CentOS on Step 2, second bul-

let, under “For RHEL”

“Removing Linux Drivers in a RoCE Environment”

on page 13

Updated Step 1 command

rpm -ivh RPMS/<arch>/kmod-qlgc-fast-

linq-<version>.<arch>.rpm to

rpm -ivh RPMS/<arch>/qlgc-fast-

linq-<version>.<arch>.rpm

Updated Step 2 rmpbuild commands for RHEL

and SLES to use -bb instead of -dd.

Updated Step 3 command

rpm -ivh RPMS/<arch>/kmod-qlgc-fast-

linq-<version>.<arch>.rpm to

rpm -ivh RPMS/<arch>/qlgc-fast-

linq-<version>.<arch>.rpm

“Installing Linux Drivers Using the src RPM Pack-

age” on page 14

User’s Guide—25Gb Intelligent Ethernet Adapter

QL45212

iv BC0154503-00 C

Added reference on CentOS on Step 2 and Step 3,

under “For RHEL”

“Installing Linux Drivers Using the src RPM Pack-

age” on page 14

Updated Step 1 command

rpm -ivh kmod-qlgc-fastlinq-<ver-

sion>.<arch>.rpm to

rpm -ivh qlgc-fastlinq-<ver-

sion>.<arch>.rpm

“Installing Linux Drivers Using the kmp/kmod RPM

Package” on page 15

Added reference to CentOS on Step 2, under “For

RHEL”

“Installing Linux Drivers Using the TAR File” on

page 15

Added note on CentOS. “Installing the Linux Drivers without RoCE” on

page 11

“Installing the Linux Drivers with RoCE” on

page 16

Deleted reference to “modprobe -r qed and

modprobe -qede.”

Added make install_libeqdr.

“To install Linux drivers in an in-box OFED environ-

ment:” on page 16

Updated parameter content and description. Table 3-2 on page 17

Added “See Step 7 on Verifying the RoCE Configu-

ration on Linux on page 59 for more information.”

“Statistics” on page 18

Updated reference from “click View Log” to “click

View Installation Log” on Step 9, first bullet. Also

updated second bullet from “click OK”, to “Click

CLOSE”.

“Running the Dell Update Package in the GUI” on

page 19

Updated screenshot for Step 8, Step 9. Figure 3-7 on page 23

Figure 3-8 on page 24

Removed redundant content from “Installation

Options” topic.

“Installation Options” on page 24

Created new topic “Updating Dell DUP Using .BIN

File”.

“Upgrading Firmware” on page 31

Updated note on Step 3. “To enable NIC partitioning (NPAR), single root

input/output virtualization (SR-IOV), or both:” on

page 42

Added new paragraph and table on FEC Mode

options.

“NIC Configuration” on page 44

Added screenshot of partition configuration details

on Step 4.

“To configure the maximum and minimum band-

width allocations for each partition:” on page 46

Added RoCE v1 and v2 “Configuring RoCE” on page 48

User’s Guide—25Gb Intelligent Ethernet Adapter

QL45212

v BC0154503-00 C

Added new bullet “RoCE is enabled by default for

both V1 and V2.”

“Planning” on page 49

Deleted the following:

qedr cannot operate with RDMA connection man-

ager (CM) when I/O memory management unit

(IOMMU) is enabled. Applications that do not sup-

port RDMA work with IOMMU.

For any RDMA application problem that you

encounter, compare your results with those

reported for other brands of adapters to determine

whether your problem is common. Refer to the

OFED ReadMe and Release Notes for information

about known issues and limitations.

“Planning” on page 49

Added reference to Windows 2016. Updated refer-

ence to RHEL to 6.7, 6.8, 7.1, 7.2, and 7.3. Added

reference to CentOS. Added reference to SLES

12, and 12 SP2 OS.

Table 6-1 on page 49

Updated reference from “type 4” to “type 5” on

Step 3.

“Preparing the Adapter” on page 50

Updated RoCE priority value from “4” to “5” on

Step 2.

“Preparing the Ethernet Switch” on page 51

Updated adapter priority from “4” to “5” on Step 3.

Also updated code reference from “00001000” to

“00000100” on Step 3.

Updated reference to Priorities from “0 1 2 3 5 6 7”

to “0 1 2 3 4 6 7” on Step 4. Also updated Priorities

reference from “4” to “5” on Step 4.

Updated reference to Priorities from “0 1 2 3 5 6 7”

to “0 1 2 3 4 6 7” on Step 6. Also updated Priorities

reference from “4” to “5” on Step 6.

“Dell Z9100 Ethernet Switch” on page 52

In Step 2, updated the screen capture.

Added a new Step 3 to configure RoCE v1 or v2

“Configuring RoCE on the Adapter for Windows

Server” on page 54

Updated second sentence to “The network direct

MTU size must be less than the Ethernet jumbo

packet size” on Step 2.

Updated second sentence to “To optimize perfor-

mance, you can change the number of RDMA con-

nections per RDMA interface to four (or more)” on

Step 7. Also updated sentence to “To increase the

number of RDMA connections to four (or more),

type the...” on Step 7.

“Configuring RoCE on the Adapter for Windows

Server” on page 54

User’s Guide—25Gb Intelligent Ethernet Adapter

QL45212

vi BC0154503-00 C

Added new topic and subtopics on “RoCE v2 Con-

figuring for Linux.”

“RoCE v2 Configuring for Linux” on page 59

Added reference to CentOS.

Added reference to SLES 12.

“iSCSI Extensions for RDMA” on page 69

Deleted reference to “modprobe -r qed and

modprobe -qede.”

“To configure iSER for RHEL:” on page 70

Create new topic “Collecting Debug Data”. “Collecting Debug Data” on page 81

vii BC0154503-00 C

Table of Contents

Preface

Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii

Intended Audience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii

What Is in This Guide. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii

Documentation Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiv

License Agreements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv

Legal Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvi

Warranty . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvi

Laser Safety . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvi

FDA Notice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvi

Agency Certification. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii

EMI and EMC Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii

KCC: Class A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xviii

Product Safety Compliance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xviii

1 Product Overview

Functional Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

Adaptive Interrupt Frequency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

ASIC with Embedded RISC Processor . . . . . . . . . . . . . . . . . . . . . . . . 3

Adapter Specifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

Physical Characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

Standards Specifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2 Installing the Hardware

System Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

Safety Precautions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

Preinstallation Checklist . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

Installing the Adapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

3 Installing Drivers

Linux Driver Software. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

User’s Guide—25Gb Intelligent Ethernet Adapter

QL45212

viii BC0154503-00 C

Installing the Linux Drivers without RoCE . . . . . . . . . . . . . . . . . . . . . . 11

Removing the Linux Drivers . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

Installing Linux Drivers Using the src RPM Package . . . . . . . . . 14

Installing Linux Drivers Using the kmp/kmod RPM Package . . . 15

Installing Linux Drivers Using the TAR File. . . . . . . . . . . . . . . . . 15

Installing the Linux Drivers with RoCE . . . . . . . . . . . . . . . . . . . . . . . . 16

Linux Driver Optional Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

Linux Driver Parameter Defaults . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

Linux Driver Messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

Windows Driver Software. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

Installing the Windows Drivers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

Running the Dell Update Package in the GUI . . . . . . . . . . . . . . 19

Installation Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

Removing the Windows Drivers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

Managing Adapter Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

Setting Power Management Options. . . . . . . . . . . . . . . . . . . . . . . . . . 26

VMware Driver Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

Installing VMware Drivers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

VMware Driver Optional Parameters. . . . . . . . . . . . . . . . . . . . . . . . . . 28

VMware Driver Parameter Defaults. . . . . . . . . . . . . . . . . . . . . . . . . . . 30

Removing the VMware Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

4 Upgrading Firmware

Run by Double-Clicking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

Run from the Command Line . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

Updating Dell DUP Using .BIN File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

5 Preboot Adapter Configuration

Getting Started . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

Displaying Firmware Image Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

Device Level Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

NIC Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

Data Center Bridging Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

NIC Partitioning Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

6 Configuring RoCE

Supported Operating Systems and OFED . . . . . . . . . . . . . . . . . . . . . . . . . . 49

Planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

Preparing the Adapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

User’s Guide—25Gb Intelligent Ethernet Adapter

QL45212

ix BC0154503-00 C

Preparing the Ethernet Switch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

Cisco Nexus 6000 Ethernet Switch . . . . . . . . . . . . . . . . . . . . . . 51

Dell Z9100 Ethernet Switch . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

Configuring RoCE on the Adapter for Windows Server . . . . . . . . . . . . . . . . 54

Configuring RoCE on the Adapter for Linux. . . . . . . . . . . . . . . . . . . . . . . . . 59

RoCE v2 Configuring for Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

Identifying RoCE v2 GID Index or Address . . . . . . . . . . . . . . . . 59

Verifying RoCE v1 or v2 GID Index, and Address from

sys and class Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

Verifying RoCE v1 or v2 Functionality Through

perftest Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

RoCE Configuration for RHEL. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

RoCE Configuration for SLES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

Verifying the RoCE Configuration on Linux . . . . . . . . . . . . . . . . . . . . . 65

VLAN Interfaces and GID Index Values . . . . . . . . . . . . . . . . . . . . . . . 68

7 iSCSI Extensions for RDMA

Configuring iSER for RHEL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

Configuring iSER for SLES12 Linux. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

Optimizing Linux Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

Set CPUs to Maximum Performance Mode. . . . . . . . . . . . . . . . . . . . . 74

Set Kernel sysctl Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

IRQ Affinity Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75

Block Device Staging. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75

8 Troubleshooting

Troubleshooting Checklist . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

Verifying that Current Drivers are Loaded . . . . . . . . . . . . . . . . . . . . . . . . . . 78

Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78

Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78

Testing Network Connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

Network Connectivity Testing for Windows . . . . . . . . . . . . . . . . . . . . . 79

Network Connectivity Testing for Linux . . . . . . . . . . . . . . . . . . . . . . . . 80

Microsoft Virtualization with Hyper-V . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

Linux. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

Miscellaneous. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

Collecting Debug Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

User’s Guide—25Gb Intelligent Ethernet Adapter

QL45212

x BC0154503-00 C

A Adapter LEDS

B Dell Z9100 Switch Configuration

Glossary

Index

Inhaltsverzeichnis

Andere Qlogic Adapter Handbücher

Qlogic

Qlogic SANblade QLA2342 Bedienungsanleitung

Qlogic

Qlogic SANsurfer FC/CNA HBA CLI Bedienungsanleitung

Qlogic

Qlogic QLE3044 Bedienungsanleitung

Qlogic

Qlogic SANblade QEM2462 Bedienungsanleitung

Qlogic

Qlogic 8400 Series Bedienungsanleitung

Qlogic

Qlogic 3400 Series Bedienungsanleitung

Qlogic

Qlogic BR-815 Bedienungsanleitung

Qlogic

Qlogic SANblade 2300 Series Bedienungsanleitung

Qlogic

Qlogic QLE7240 Bedienungsanleitung

Qlogic

Qlogic QME8262-k Bedienungsanleitung

Qlogic

Qlogic InfiniPath QLE7140 Bedienungsanleitung

Qlogic

Qlogic QLE2560 Handbuch

Qlogic

Qlogic 7104-HCA-128LP Bedienungsanleitung

Qlogic

Qlogic SANblade QLA4050 Bedienungsanleitung

Qlogic

Qlogic SANblade QLA4050 Bedienungsanleitung

Qlogic

Qlogic INFINIBAND Bedienungsanleitung

Qlogic

Qlogic QLE2462 Bedienungsanleitung

Qlogic

Qlogic 8200 Series Bedienungsanleitung

Qlogic

Qlogic QLE2562 Bedienungsanleitung

Qlogic

Qlogic QLE7280 Bedienungsanleitung

Qlogic

Qlogic 7104-HCA-LPX1P Bedienungsanleitung

Qlogic

Qlogic QME2462 Kurzanleitung

Qlogic

Qlogic QLogic Fibre Channel Switch Bedienungsanleitung

Qlogic

Qlogic QLE8042 Bedienungsanleitung

Beliebte Adapter Handbücher anderer Marken

PCB Piezotronics

PCB Piezotronics 090B274 Bedienungsanleitung

Auvidea

Auvidea J20 Produkthandbuch

Gossen MetraWatt

Gossen MetraWatt AT3-III E Bedienungsanleitung

Akribis

Akribis ITF-21-BISS-MIT Bedienungsanleitung

Tenda

Tenda W54U Bedienungsanleitung

ROHO

ROHO AGILITY Bedienungsanleitung

PROLiNK

PROLiNK WN2001 Bedienungsanleitung

Hawking

Hawking HUF2 Bedienungsanleitung

Airlink101

Airlink101 AWLC4030 Bedienungsanleitung

NETGEAR

NETGEAR XAV101v1 - Powerline AV Ethernet Adapter Bedienungsanleitung

Philips

Philips SWA4554S/10 Bedienungsanleitung

JVC

JVC TS-C9D2B Gebrauchsanweisung

NETGEAR

NETGEAR A6100 Bedienungsanleitung

D-Link

D-Link DWA-121 Bedienungsanleitung

Philips

Philips SWA2037 Bedienungsanleitung

Basalte

Basalte 0610-20 Bedienungsanleitung

Yealink

Yealink EHS40 Bedienungsanleitung

Balluff

Balluff BIC 1B0-IT1A7-Q40KFU-SM4A4A Bedienungsanleitung