Supermicro SuperServer 2028GR-TRT Bedienungsanleitung

USER’S MANUAL

Revision 1.0a

SUPERSERVER®

2028GR-TR

2028GR-TRT

2028GR-TRH

2028GR-TRHT

The information in this User’s Manual has been carefully reviewed and is believed to be accurate.

The vendor assumes no responsibility for any inaccuracies that may be contained in this document,

makes no commitment to update or to keep current the information in this manual, or to notify any

person or organization of the updates. Please Note: For the most up-to-date version of this

manual, please see our web site at www.supermicro.com.

Super Micro Computer, Inc. ("Supermicro") reserves the right to make changes to the product

described in this manual at any time and without notice. This product, including software and

documentation, is the property of Supermicro and/or its licensors, and is supplied only under a

license. Any use or reproduction of this product is not allowed, except as expressly permitted by

the terms of said license.

IN NO EVENT WILL SUPERMICRO BE LIABLE FOR DIRECT, INDIRECT, SPECIAL, INCIDENTAL,

SPECULATIVE OR CONSEQUENTIAL DAMAGES ARISING FROM THE USE OR INABILITY TO

USE THIS PRODUCT OR DOCUMENTATION, EVEN IF ADVISED OF THE POSSIBILITY OF

SUCH DAMAGES. IN PARTICULAR, SUPERMICRO SHALL NOT HAVE LIABILITY FOR ANY

HARDWARE, SOFTWARE, OR DATA STORED OR USED WITH THE PRODUCT, INCLUDING THE

COSTS OF REPAIRING, REPLACING, INTEGRATING, INSTALLING OR RECOVERING SUCH

HARDWARE, SOFTWARE, OR DATA.

Any disputes arising between manufacturer and customer shall be governed by the laws of Santa

Clara County in the State of California, USA. The State of California, County of Santa Clara shall

be the exclusive venue for the resolution of any such disputes. Super Micro's total liability for all

claims will not exceed the price paid for the hardware product.

FCC Statement: This equipment has been tested and found to comply with the limits for a Class

A digital device pursuant to Part 15 of the FCC Rules. These limits are designed to provide

reasonable protection against harmful interference when the equipment is operated in a commercial

environment. This equipment generates, uses, and can radiate radio frequency energy and, if not

installed and used in accordance with the manufacturer’s instruction manual, may cause harmful

interference with radio communications. Operation of this equipment in a residential area is likely

to cause harmful interference, in which case you will be required to correct the interference at your

own expense.

California Best Management Practices Regulations for Perchlorate Materials: This Perchlorate

warning applies only to products containing CR (Manganese Dioxide) Lithium coin cells. “Perchlorate

Material-special handling may apply. See www.dtsc.ca.gov/hazardouswaste/perchlorate”

WARNING: Handling of lead solder materials used in this

product may expose you to lead, a chemical known to

the State of California to cause birth defects and other

reproductive harm.

Manual Revision 1.0a

Release Date: April 27, 2016

Unless you request and receive written permission from Super Micro Computer, Inc., you may not

copy any part of this document.

Information in this document is subject to change without notice. Other products and companies

referred to herein are trademarks or registered trademarks of their respective companies or mark

holders.

Printed in the United States of America

iii

Preface

About this Manual

This manual is written for professional system integrators and PC technicians.

It provides information for the installation and use of the SuperServer

2028GR-TR/TRT/TRH/TRHT. Installation and maintainance should be performed

by experienced technicians only.

Please refer to the server specications page on our Web site for updates on

supported memory, processors and operating systems (http://www.supermicro.

com).

Notes

For your system to work properly, please follow the links below to download all

necessary drivers/utilities and the user’s manual for your server.

• Supermicro product manuals: http://www.supermicro.com/support/manuals/

• Product drivers and utilities: ftp://ftp.supermicro.com

• Product safety info: http://super-dev/about/policies/safety_information.cfm

If you have any questions, please contact our support team at:

[email protected]

This manual may be periodically updated without notice. Please check the

Supermicro Web site for possible updates to the manual revision level.

Warnings

Special attention should be given to the following symbols used in this manual.

Warning! Indicates high voltage may be encountered when performing

a procedure.

Warning! Indicates important information given to prevent equipment/

property damage or personal injury.

SUPERSERVER 2028GR-TR/TRT/TRH/TRHT Manual

Contents

Chapter 1 Introduction..............................................................................1-1

1-1 Overview ......................................................................................................... 1-1

1-2 Serverboard Features ..................................................................................... 1-2

Processors ...................................................................................................... 1-2

Memory ........................................................................................................... 1-2

Serial ATA ....................................................................................................... 1-2

PCI Expansion Slots ....................................................................................... 1-2

Input/Output Ports........................................................................................... 1-2

IPMI................................................................................................................. 1-2

1-3 Server Chassis Features ................................................................................ 1-3

System Power................................................................................................. 1-3

SATA Subsystem............................................................................................. 1-3

Front Control Panel......................................................................................... 1-3

Cooling System............................................................................................... 1-3

1-4 GPU Subsystem.............................................................................................. 1-3

1-5 Contacting Supermicro.................................................................................... 1-5

Chapter 2 Server Installation

2-1 Overview ......................................................................................................... 2-1

2-2 Unpacking the System .................................................................................... 2-1

2-3 Preparing for Setup......................................................................................... 2-1

Choosing a Setup Location............................................................................. 2-1

2-4 Warnings and Precautions.............................................................................. 2-2

Rack Precautions............................................................................................ 2-2

Server Precautions.......................................................................................... 2-2

Rack Mounting Considerations ....................................................................... 2-3

Ambient Operating Temperature ................................................................ 2-3

Reduced Airow ......................................................................................... 2-3

Mechanical Loading ................................................................................... 2-3

Circuit Overloading..................................................................................... 2-3

Reliable Ground ......................................................................................... 2-3

2-5 Installing the System into a Rack ................................................................... 2-4

Identifying the Sections of the Rack Rails...................................................... 2-4

Releasing the Inner Rail ................................................................................. 2-5

Installing the Inner Rails on the Chassis ........................................................ 2-6

Installing the Outer Rails onto the Rack......................................................... 2-7

Sliding the Chassis onto the Rack Rails......................................................... 2-8

Preface

2-6 Checking the Serverboard Setup.................................................................... 2-9

2-7 Checking the Chassis Setup......................................................................... 2-10

Chapter 3 System Interface......................................................................3-1

3-1 Overview ......................................................................................................... 3-1

3-2 Control Panel Buttons ..................................................................................... 3-2

3-3 Control Panel LEDs ........................................................................................ 3-2

Overheating..................................................................................................... 3-3

3-4 Drive Carrier LEDs.......................................................................................... 3-4

3-5 Power Supply LEDs ........................................................................................ 3-4

Chapter 4 Standardized Warning Statements for AC Systems ............ 4-1

Warning Denition ........................................................................................... 4-1

Installation Instructions.................................................................................... 4-4

Circuit Breaker ................................................................................................ 4-5

Power Disconnection Warning ........................................................................ 4-6

Equipment Installation..................................................................................... 4-8

Restricted Area................................................................................................ 4-9

Battery Handling............................................................................................ 4-10

Redundant Power Supplies .......................................................................... 4-12

Backplane Voltage ........................................................................................ 4-13

Comply with Local and National Electrical Codes........................................ 4-14

Product Disposal........................................................................................... 4-15

Hot Swap Fan Warning................................................................................. 4-16

Power Cable and AC Adapter ...................................................................... 4-18

Chapter 5 Advanced Serverboard Setup

5-1 Handling the Serverboard ............................................................................... 5-1

Precautions ..................................................................................................... 5-1

Unpacking ....................................................................................................... 5-1

5-2 Connecting Cables.......................................................................................... 5-2

Connecting Data Cables ................................................................................. 5-2

Connecting Power Cables .............................................................................. 5-2

Connecting the Control Panel......................................................................... 5-2

5-3 I/O Ports.......................................................................................................... 5-3

5-4 Installing the Processor and Heatsink ............................................................ 5-4

Installing an LGA 2011 Processor................................................................... 5-4

Installing a CPU Heatsink ............................................................................... 5-7

Removing the Heatsink .................................................................................. 5-7

SUPERSERVER 2028GR-TR/TRT/TRH/TRHT Manual

5-5 Installing Memory............................................................................................ 5-8

Memory Support.............................................................................................. 5-8

Processor/DIMM Population Congurations .............................................. 5-8

5-6 Serverboard Details ...................................................................................... 5-10

5-7 Connector Denitions ................................................................................... 5-13

I/O Connections ............................................................................................ 5-17

5-8 Jumper Settings ............................................................................................ 5-20

5-9 Onboard Indicators........................................................................................ 5-22

5-10 SATA Ports .................................................................................................... 5-23

5-11 Installing Software......................................................................................... 5-24

SuperDoctor®5 ............................................................................................. 5-25

5-12 Onboard Battery............................................................................................ 5-26

Chapter 6 Advanced Chassis Setup........................................................6-1

6-1 Static-Sensitive Devices.................................................................................. 6-2

Precautions ..................................................................................................... 6-2

6-2 Control Panel .................................................................................................. 6-2

6-3 Installing Drives............................................................................................... 6-3

6-4 Installing Graphics (GPU) Cards..................................................................... 6-6

6-5 Expansion Card Setup .................................................................................. 6-8

6-6 System Cooling..............................................................................................6-11

System Fan Failure........................................................................................6-11

Installing the Air Shroud................................................................................ 6-14

6-7 Power Supply................................................................................................ 6-15

Chapter 7 BIOS .........................................................................................7-1

7-1 Introduction...................................................................................................... 7-1

Starting BIOS Setup Utility.............................................................................. 7-1

How To Change the Conguration Data ......................................................... 7-1

How to Start the Setup Utility ......................................................................... 7-2

7-2 Main Setup...................................................................................................... 7-2

7-3 Advanced Setup Congurations...................................................................... 7-4

7-4 Event Logs .................................................................................................... 7-30

7-5 IPMI............................................................................................................... 7-32

7-6 Security Settings ........................................................................................... 7-34

7-7 Boot Settings................................................................................................. 7-35

7-8 Save & Exit ................................................................................................... 7-38

Appendix A BIOS Error Beep Codes ..................................................... A-1

Appendix B System Specications ........................................................ B-1

Chapter 1

Introduction

1-1 Overview

The SuperServer 2028GR-TR/TRT/TRH/TRHT is a GPU/Xeon Phi optimized server

comprised of two main subsystems: the SC218GHTS-R2K03BP 2U server chassis

and the X10DRG-H/HT serverboard. The models and features are:

• 2028GR-TR has 1Gb Ethernet and up to four GPUs

• 2028GR-TRT has 10GBase-T and up to four GPUs

• 2028GR-TRH has 1Gb Ethernet and up to six GPUs

• 2028GR-TRHT has 10GBase-T and up to six GPUs

Other components are:

• Five 8-cm fans (FAN-0136L4)

• One air shroud (MCP-310-21803-0N)

• Two passive CPU heatsinks (SNK-P0048PS:CPU1 and SNK-P0047PS:CPU2)

• SATA Accessories

One SAS backplane (BPN-SAS-218A)

Ten hot-swap drive carriers (MCP-220-00047-0B)

• Three riser cards (RSC-G2F-A66, RSC-G2F2-A66, RSC-R2UG-E16R-X9; the

2028GR-TRH/TRHT models also have one additional riser card, RSC-G2B-

A66-O-P)

• One rail set (MCP-290-00057-0N)

Operating systems certied for use with the system are listied on the web site.

Note: For your system to work properly, please follow the links below to download

all necessary drivers/utilities and the user’s manual for your server.

• Supermicro product manuals: http://www.supermicro.com/support/manuals/

• Product drivers and utilities: ftp://ftp.supermicro.com

• Product safety info:

http://www.supermicro.com/about/policies/safety_information.cfm

If you have any questions, please contact our support team at:

[email protected]

Chapter 1: Introduction

1-1

1-2

SUPERSERVER 2028GR-TR/TRT/TRH/TRHT User's Manual

1-2 Serverboard Features

At the heart of the server is the X10DRG-H/HT, a dual processor serverboard based

on the Intel PCH C612 chipset. Below are the main features. (See Figure 1-1 for

a block diagram)

Processors

The X10DRG-H/HT supports two Intel E5-2600 v3/v4 series processors in LGA

2011 sockets (Socket R3). Refer to the serverboard description pages on our web

site for a complete listing of supported processors (www.supermicro.com).

Memory

The serverboard has 16 DIMM slots that can support up to 2 TB of Registered Load

Reduced (LRDIMM), or 1 TB of RDIMM, DDR4 ECC 2400/2133/1866/1600 MHz

RDIMMs. See Chapter 5 for details.

Serial ATA

The serverboard supports a 10-port SATA 3.0 subsystem. RAID 0, 1, 5 and 10 can

be enabled. The SATA drives are hot-swappable.

PCI Expansion Slots

The serverboard has four PCI-Express 3.0 x16 slots to support up to six double-

width GPU cards using riser cards. The 2028GR-TR/THT supports up to four GPUs

and the 2028GR-TRH/TRHT supports up to six GPUs.

An additional slot supports one low prole PCI-Express 3.0 x8 (in x16 slot) card.

Note that both CPUs must be installed for full use of the PCIe slots.

Input/Output Ports

The rear I/O ports include a VGA (monitor) port, two USB 3.0 ports, two Ethernet

LAN ports (1Gb for X10DRG-H, 10Gb for X10DRG-HT) and one dedicated IPMI

LAN port.

IPMI

IPMI (Intelligent Platform Management Interface) is a hardware-level interface

specication that provides remote access, monitoring and administration for

Supermicro server platforms. IPMI allows server administrators to view a server’s

hardware status remotely, receive an alarm automatically if a failure occurs, and

power cycle a system that is non-responsive.

1-3

Chapter 1: Introduction

1-3 Server Chassis Features

System Power

The SC218GHTS-R2K03BP features a high-efciency 2000W 80-Plus Platinum

level redundant power supply system composed of two hot-plug power modules.

One power module may be removed without shutting down the system. See Chapter

6 for details.

SATA Subsystem

The chassis includes ten 2.5" drive bays, which house hot-swappable SATA drives.

RAID 0, 1, 5 and 10 are supported (RAID 5 is not supported with Linux OS).

Front Control Panel

The control panel provides a system monitoring and control interface. LEDs indicate

system power, HDD activity, network activity, and a system overheat/fan fail/ UID

LED. A main power button and a system reset button are also included.

Cooling System

The chassis has an innovative cooling design that includes ve 8-cm heavy-duty

fans. The power supply modules also include a cooling fan. All chassis and power

supply fans operate continuously.

1-4 GPU Subsystem

The 2028GR-TR/TRT/TRH/TRHT server represents one of Supermicro's massively

parallel processing multiple GPU/Xeon Phi servers, with support for up to six GPUs/

Xeon Phis. NVIDIA Kepler GPUs place this system at the forefront of today's GPU

computing solutions.

Please refer to the NVIDIA web site (www.nvidia.com) for details on Kepler GPUs.

The GPUs process complex image calculations and then route the data out through

the VGA port on the serverboard.

NVIDIA Kepler K10/K20/K40 GPUs have been tested for use with this system. Any

combination of these cards (up to a total of four/six) may have come bundled with

the system. They come with a passive heatsink attached.

Power Notes

To provide power for the GPU cards, connect the GPU power cable from each of

the GPUs to JPW3, 4, 5, 6, 7 and 8 on the serverboard (one cable for each card).

1-4

SUPERSERVER 2028GR-TR/TRT/TRH/TRHT User's Manual

SPI

LAN3

RGRMII

Debug Card

FRONT PANEL

SYSTEM POWER

CTRL

FAN SPEED

PCI-E X1 G2

USB 2.0

#12 USB2.0

#6/7/8

PCH

6.0 Gb/S

LPC

SATA

RTL8211E-VB-CG

RJ45

BIOS

SPI

Temp Sensor

EMC1402-1 *2 at diff SMBUS

TPM HEADER

USB 3.0

BIOS

HEADER

SPI

AST2400

BMC

RMII/NCSI

COM

Header

VGA CONN

BMC Boot Flash

DDR3

SLOT 1

5 PHASE

145W

VR12.5

QPI

9.6G

SLOT 3

PCI-E X16 G3

DMI2

PCI-E X16

PCI-E X16 G3 (LAN REVERSE)

DDR4

DMI2

QPI

9.6G

4GB/s

5 PHASE

145W

VR12.5

PCI-E X16

I350/X540

LAN

PCI-E X8

SLOT 4

PCI-E X16

SLOT2

PCI-E X8

PCI-EX8 G3(w/ Re-driver)

PCI-EX16 G3(LAN REVERSE)

PCI-EX16 G3

#1 #2 #3

#2 #1

<=1.758W (average)

2.3W (Peak)

MAX:12.5W

STBY:2.5W

TDP:6.5W (WORKSTATION)

5W (SERVER)

USB & SATA useage different

Idle:0.45W

5V:1.2A

3.3V:0.1A

3.3 STBY:0.2A

1.05 PCH

1.05 ASW

1.5 PCH

PVCCIO 1.0/0.95

3.3STBY:0.5A

VCCP1 12v

VCCP0 12v

PVCCIO

(1.05/0.95)

from 3.3v

P5V_AUX

PX2V5_I3V3

PX1V2_I1V8

PX0V8_I1V0

PX0V67_I1V0

P3V3_STBY

P1V5_AUX_BMC

P1V2_AUX_BMC

P3V3_PCH

P1V5_PCH 3.3v

P1V05_PCH 5v

P1V05_STBY 3.3v STBY

Sagevill: 9W

Processor 1 Processor 2

SLOT5

DMI2

DDR4

up to 2133

DDR4

#1-4

#1-3

#1-2

#1-1

#1-5

#1-6

#1-7

#1-8

#2-1

#2-2

#2-3

#2-4

#2-5

#2-6

#2-7

#2-8

up to 2133

DDR4

SATA

USBSATA

Figure 1-1. Intel PCH C612 Chipset System Block Diagram

Note: This is a general block diagram. Please see Chapter 5 for details.

Dieses Handbuch passt für folgende Modelle

Inhaltsverzeichnis

Andere Supermicro Server Handbücher

Supermicro

Supermicro SuperServer 6026TT-HTRF Bedienungsanleitung

Supermicro

Supermicro SuperServer E300-12C Bedienungsanleitung

Supermicro

Supermicro SUPERSERVER 5018A-TN7B Bedienungsanleitung

Supermicro

Supermicro AS-1041M-82 Bedienungsanleitung

Supermicro

Supermicro superserver 5019S-ML Bedienungsanleitung

Supermicro

Supermicro A+ Server AS -1115CS-TNR Bedienungsanleitung

Supermicro

Supermicro SuperServer SYS-220ME-FN6R Bedienungsanleitung

Supermicro

Supermicro SuperBlade SBI-7125W-S6 Bedienungsanleitung

Supermicro

Supermicro SuperWorkstation 540A-TR Bedienungsanleitung

Supermicro

Supermicro SuperServer SYS-1029UX-LL1-S16 Bedienungsanleitung

Supermicro

Supermicro SuperServer SYS-110P-FRN2T Bedienungsanleitung

Supermicro

Supermicro SUPERSERVER 1018R-WC0R Bedienungsanleitung

Supermicro

Supermicro SUPERSERVER 1028UX-LL1-B8 Bedienungsanleitung

Supermicro

Supermicro SSG-937R-E2CJB Bedienungsanleitung

Supermicro

Supermicro SuperServer 7089P-TR4T Bedienungsanleitung

Supermicro

Supermicro SUPERSERVER 1018R-WR Bedienungsanleitung

Supermicro

Supermicro SuperStorage SSG-640SP-DE2CR60 Bedienungsanleitung

Supermicro

Supermicro X5SS8-GM Bedienungsanleitung

Supermicro

Supermicro SUPERSERVER 6011H Bedienungsanleitung

Supermicro

Supermicro SuperServer SYS-110D-4C-FRAN8TP Bedienungsanleitung

Supermicro

Supermicro SUPERSERVER 6016XT-TF Bedienungsanleitung

Supermicro

Supermicro SUPERSERVER 8016B-TF Bedienungsanleitung

Supermicro

Supermicro SuperServer 5019S-W4TR Bedienungsanleitung

Supermicro

Supermicro SBI-7428R-C3 Bedienungsanleitung

Beliebte Server Handbücher anderer Marken

HP AlphaServer ES47 Bedienungsanleitung

Dell

Dell PowerEdge R410 Bedienungsanleitung

PowerNAS

PowerNAS CMA Bedienungsanleitung

Supero

Supero FatTwin F617R3-FT Bedienungsanleitung

MSI

MSI P1-109N-L70 Bedienungsanleitung

Dell

Dell PowerEdge XE9640 Handbuch

Dell

Dell PS4100 Montageanleitung

EtroVISION

EtroVISION EV3151 Bedienungsanleitung

NEC

NEC Express5800/T110j Bedienungsanleitung

HP ProLiant Gen9 Bedienungsanleitung

Compaq

Compaq ProLiant 3000 Handbuch

Dell

Dell PowerEdge 3YPMN Bedienungsanleitung

iRobo

iRobo IPC2U Bedienungsanleitung

Nortel

Nortel 1000 Con?guration guide Bedienungsanleitung

Asus

Asus AP7500 Bedienungs- und Wartungshandbuch

Avid Technology

Avid Technology AirSpeed 5000 Bedienungsanleitung

HP Integrity rx2600 Installationsanleitung

Milestone

Milestone Husky IVO 350T Bedienungsanleitung