Mellanox Technologies MSX6518-6R Handbuch

www.mellanox.com

324 Port InfiniBand FDR Switch Platform

Hardware Installation Guide

PN: MSX6518-6R, MSX6518-NR

Rev 1.0

Mellanox Technologies

350 Oakmead Parkway Suite 100

Sunnyvale, CA 94085

U.S.A.

www.mellanox.com

Tel: (408) 970-3400

Fax: (408) 970-3403

Mellanox Technologies, Ltd.

Beit Mellanox

PO Box 586 Yokneam 20692

Israel

Tel: +972 (0)4 909 7200 ; 

+972 (0)74 723 7200

Fax: +972 (0)4 959 3245

Mellanox, Mellanox logo, BridgeX, ConnectX, CORE-Direct, InfiniBridge, InfiniHost, InfiniScale, PhyX, SwitchX, Virtual Protocol

Interconnect and Voltaire are registered trademarks of Mellanox Technologies, Ltd. FabricIT, MLNX-OS, Unbreakable-Link, UFM and

Unified Fabric Manager are trademarks of Mellanox Technologies, Ltd.

All other trademarks are property of their respective owners.

324 Port InfiniBand FDR Switch Platform Hardware Installation Guide

Document Number: 3805

Rev 1.0

Mellanox Technologies2

NOTE:

THIS HARDWARE, SOFTWARE OR TEST SUITE PRODUCT (“PRODUCT(S)”) AND ITS RELATED DOCUMENTATION ARE

PROVIDED BY MELLANOX TECHNOLOGIES “AS-IS” WITH ALL FAULTS OF ANY KIND AND SOLELY FOR THE PURPOSE

OF AIDING THE CUSTOMER IN TESTING APPLICATIONS THAT USE THE PRODUCTS IN DESIGNATED SOLUTIONS. THE

CUSTOMER'S MANUFACTURING TEST ENVIRONMENT HAS NOT MET THE STANDARDS SET BY MELLANOX

TECHNOLOGIES TO FULLY QUALIFY THE PRODUCTO(S) AND/OR THE SYSTEM USING IT. THEREFORE, MELLANOX

TECHNOLOGIES CANNOT AND DOES NOT GUARANTEE OR WARRANT THAT THE PRODUCTS WILL OPERATE WITH THE

HIGHEST QUALITY. ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED

WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT ARE

DISCLAIMED. IN NO EVENT SHALL MELLANOX BE LIABLE TO CUSTOMER OR ANY THIRD PARTIES FOR ANY DIRECT,

INDIRECT, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES OF ANY KIND (INCLUDING, BUT NOT LIMITED TO,

PAYMENT FOR PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS

INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,

OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY FROM THE USE OF THE PRODUCT(S) AND

RELATED DOCUMENTATION EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

324 Port InfiniBand FDR Switch Platform Hardware Installation Guide Rev 1.0

Mellanox Technologies 3

Table of Contents

Chapter 1 Chassis Installation 10

1.1 Environmental and Safety Recommendations 10

1.2 ChassisPackageContents 11

1.3 LeafPackageContents 12

1.4 SpinePackageContents 12

1.5 ManagementPackageContents 12

1.6 PhysicalInstallation 12

1.6.1 Starting with the Rack 12

1.6.2 ESD (ElectroStatic Discharge) Connection 13

1.6.3 Installation Procedure 13

1.7 Installing the Leafs 30

1.8 Inserting a Management Module 31

1.9 Inserting a Spine Board 31

Chapter 2 Installing the Cable Holder 33

Chapter 3 Ground Connections 34

Chapter 4 Power Connections 35

4.1 Powering Up the Switch Platform 35

Chapter 5 InfiniBand QSFP Cable Installation 37

5.1 PortConnectorInterfaces 37

5.2 SupportedApprovedCables 38

5.3 CablePower Classes 38

Chapter 6 Chassis Power Up 39

6.1 Power Supply, Management Board, and Spine Board Indicator Status at Power ON 40

Chapter 7 InfiniBand Fabric Initialization and Management 41

7.1 Configuring the Switch Management Modules for the First Time 41

7.2 RerunningtheWizard 46

Chapter 8 MLNX-OS Management Software 47

8.1 Starting an SSH Connection to the Switch 47

8.2 Starting a WebUI Connection to the Switch 47

Chapter 9 Resetting the Switch – RST 49

Chapter 10 TroubleShooting 50

10.1 PowerSupplyUnit 50

10.2 LeafBoard 50

10.3 ManagementModule 51

10.3.1 Yellow Status LED (for the Chassis) on the Management Module is Lit 51

10.3.2 Yellow LED for the Leaf Fan on the Management Module is Lit 51

10.3.3 Yellow LED for the Spine Fan on the Management Module is Lit 52

10.4 SpineBoard 52

10.5 MLNX-OSSoftware 53

Mellanox Technologies4

Rev 1.0

List of Figures

Figure 1: Distance Between the Vertical Supports 13

Figure 2: Installation Kit Parts 15

Figure 3: Shock and Tilt Stickers 16

Figure 4: Container Screws 17

Figure 5: Vertical Rack Supports Dimension 18

Figure 6: Adjustable Shelf Brackets 19

Figure 7: Top of Shelf Bracket 19

Figure 8: Placing the Shelf Bracket 20

Figure 9: Caged Nut Locations for the Shelf Brackets 20

Figure 10: Shelf Brackets 21

Figure 11: U Numbering Above Shelf 21

Figure 12: Putting the Chassis in the Rack 22

Figure 13: Place the Upper Bracket Offset 23

Figure 14: Attach the Shelf Extension 23

Figure 15: Location for the Lock-down Bar 24

Figure 16: Caged Nut Placement by U Numbers 25

Figure 17: Fork Lift 26

Figure 18: Face Plate Mounting Bolt Locations 26

Figure 19: Upper Bracket Installation 27

Figure 20: Upper Brackets and Offset Location 28

Figure 21: Offset to Upper bracket Bolts 28

Figure 22: Location for the Lock-down Bar 29

Figure 23: Lock-Down Bar 29

Figure 24: Bolt and Washer Order 30

Figure 25: Ejector Latch 30

Figure 26: Ejector Latch 31

Figure 27: Spine Board Insertion 31

Figure 28: Ejector Latch 32

Figure 29: Cable Holder 33

Figure 30: Ground Connection 34

Figure 31: Multiple Power Inlets - Electric Caution Notification 36

Figure 32: Spine Module 36

Figure 33: Port Numbering 38

Figure 34: Top and Bottom Ports 38

Figure 35: Spine Side Panel Display Status Indications 40

Figure 36: Management Module Status Indications for Normal Operation 40

Figure 37: Connecting the Harness to the Console 41

324 Port InfiniBand FDR Switch Platform Hardware Installation Guide Rev 1.0

Mellanox Technologies 5

Figure 38: Web UI Login Page 48

Figure 39: Reset Button 49

Mellanox Technologies6

Rev 1.0

List of Tables

Table 1: Revision History of this Installation Guide 7

Table 2: Reference Documents and Web Sites 8

Table 3: Serial Terminal Program Configuration 42

Table 4: Configuration Wizard Session - IP Configuration by DHCP 42

Table 5: Configuration Wizard Session - IP Zeroconf Configuration 44

Table 6: Configuration Wizard Session - Static IP Configuration 45

324 Port InfiniBand FDR Switch Platform Hardware Installation Guide Rev 1.0

Mellanox Technologies 7

Revision History

Table 1 - Revision History of this Installation Guide

Revision Date Details

1.0 March 2012 Initial release

Mellanox Technologies8

Rev 1.0

About this Manual

This manual provides installation and set-up instructions for the SX6518 QSFP Chassis InfiniBand

Switch Platform.

Intended Audience

This manual is intended for users and system administrators responsible for installing and setting

up the switch platform.

The manual assumes familiarity with the InfiniBand®architecture specification.

Related Documentation

The documentation set accompanying the QSFP Chassis InfiniBand Switch platform includes the

following:

Table 2 - Reference Documents and Web Sites

Document Conventions

Throughout this manual, the name SX6518 and the terms chassis and switch are used to describe

the switch, unless explicitly indicated otherwise.

The following pictures are used throughout this document to indicate information that is important

to the user.

InfiniBand Architecture Specification, Vol. 1,

Release 1.2.1 The InfiniBand Architecture Specification that is provided by

IBTA

Switch Hardware Errata For any possible errata due to hardware issues see the switch

support product page. This requires a customer support login.

Mellanox MLNX-OS™ SwitchX® Software WebUI

User’s Manual WebUI Overview for MLNX-OS software.

Mellanox MLNX-OS™ SwitchX® Software User

Manual This document contains information regarding configuring and

managing Mellanox Technologies' SwitchX Switch Platforms.

MLNX-OS™ Software Command Reference Guide Command Reference Guide for MLNX-OS listing all of the

commands available through MLNX-OS with explanations and

examples.

MLNX-OS™ Software Configuration Guide Configuration Guide for MLNX-OS displaying different config-

uration scenarios.

This symbol makes recommendations to the user.

324 Port InfiniBand FDR Switch Platform Hardware Installation Guide Rev 1.0

Mellanox Technologies 9

This symbol indicates information that is helpful to the user.

This symbol indicates a situation that can potentially cause damage to hardware or

software.

WARNING! This symbol indicates a situation that can potentially cause personal

injury and / or damage to hardware or software.

Chassis Installation

Mellanox Technologies10

Rev 1.0

1 Chassis Installation

Installation and initialization of the chassis is a simple process requiring attention to the normal

mechanical, power, and thermal precautions for rack-mounted equipment. Your chassis comes

only with the power supplies and fans pre-installed. The rest of the openings are populated with

blanks. All of the leafs, spines, and management modules come shipped in separate packages.

The chassis requires initial configuration to get the chassis and Fabric management up and running

through remote management. see “Configuring the Switch Management Modules for the First

Time” on page 41.

1.1 Environmental and Safety Recommendations

The following are Mellanox recommendations.

This chassis can be installed in standard 19” racks that have depths between 65cm

(25.6in.) and 80cm (31.5in.) between the vertical supports of the rack.

This unit is intended for installation in a Restricted Access Location. A restricted

access area can be accessed only through the use of a special tool, lock and key, or

other means of security.

Recommended ambient temperature in the System room is 20o± 5oC.

Recommended humidity ranges is 40% ± 15% without condensing.

It is highly recommended that the installation sites be as isolated as possible from all

sources of radio transmissions and electrical interference.

It is highly recommended that the installation site building be equipped with a light-

ning rod.

It is highly recommended that the installation site be equipped with smoke detectors

and a fire alarm warning system.

Dieses Handbuch passt für folgende Modelle

Inhaltsverzeichnis

Andere Mellanox Technologies Schalten Handbücher

Mellanox Technologies

Mellanox Technologies 1G Switch Air Duct Rail Kit Bedienungsanleitung

Mellanox Technologies

Mellanox Technologies Mellanox SX1018 Bedienungsanleitung

Mellanox Technologies

Mellanox Technologies IS5025D-1BRC Bedienungsanleitung

Mellanox Technologies

Mellanox Technologies MTS3600 Bedienungsanleitung

Mellanox Technologies

Mellanox Technologies Spectrum SN2740 Gebrauchsanweisung

Mellanox Technologies

Mellanox Technologies Switch-IB Bedienungsanleitung

Mellanox Technologies

Mellanox Technologies MIS5300 Handbuch

Mellanox Technologies

Mellanox Technologies SB7700 Bedienungsanleitung

Mellanox Technologies

Mellanox Technologies Spectrum SN2100 Gebrauchsanweisung

Mellanox Technologies

Mellanox Technologies MSX6536-10R Handbuch

Mellanox Technologies

Mellanox Technologies MIS5025 Bedienungsanleitung

Mellanox Technologies

Mellanox Technologies SwitchX-2 MSX6005F-1BFS Gebrauchsanweisung

Mellanox Technologies

Mellanox Technologies Mellanox SX1018 Bedienungsanleitung

Mellanox Technologies

Mellanox Technologies Spectrum SN2100 Bedienungsanleitung

Mellanox Technologies

Mellanox Technologies SX1710 Bedienungsanleitung

Mellanox Technologies

Mellanox Technologies SX6710 Gebrauchsanweisung

Mellanox Technologies

Mellanox Technologies SX6720 Gebrauchsanweisung

Mellanox Technologies

Mellanox Technologies TX6100 MetroX Series Gebrauchsanweisung

Mellanox Technologies

Mellanox Technologies SX67X0 Gebrauchsanweisung

Mellanox Technologies

Mellanox Technologies MSX6506-3R Handbuch

Mellanox Technologies

Mellanox Technologies SX1012 Bedienungsanleitung

Mellanox Technologies

Mellanox Technologies Grid Director 4700 Bedienungsanleitung

Mellanox Technologies

Mellanox Technologies IS5025D-1BRC Bedienungsanleitung

Mellanox Technologies

Mellanox Technologies Mellanox SX1018 Betriebsanleitung

Beliebte Schalten Handbücher anderer Marken

URC

URC Total Control MFS-8 Bedienungsanleitung

Azbil

Azbil HPQ-D11 Bedienungsanleitung

Sony

Sony DFS-900M Bedienungsanleitung

SICK

SICK i17S Bedienungsanleitung

F&F

F&F SZR-277 Bedienungsanleitung

Shure

Shure UAMS/BK Bedienungsanleitung

Dräger

Dräger X-zone Switch Off Bedienungsanleitung

Asco

Asco Joucomatic 349 Series Bedienungsanleitung

BT Mini Hub Bedienungsanleitung

D-Link

D-Link DES-1226G Bedienungsanleitung

Johnson Controls

Johnson Controls FS80-C Bedienungsanleitung

Alcatel-Lucent

Alcatel-Lucent 7450 ESS-12 Bedienungsanleitung

MicroNet

MicroNet SP1200A Bedienungsanleitung

Ruijie

Ruijie RG-PF2920 Series Installations- und Betriebsanleitung

Altinex

Altinex DA1914SX Bedienungsanleitung

MiLAN

MiLAN MIL-SM2401M Bedienungsanleitung

RJM

RJM MASTERMIND PBC/6X Bedienungsanleitung

Atlas

Atlas MMK-KVM8 Bedienungsanleitung