Intel Xeon Processor E5-2600 Bedienungsanleitung

Reference Number: 327043-001

Intel® Xeon® Processor E5-2600

Product Family Uncore Performance

Monitoring Guide

March 2012

2Reference Number: 327043-001

INFORMATION IN THIS DOCUMENT IS PROVIDED IN CONNECTION WITH INTEL®PRODUCTS. NO LICENSE, EXPRESS OR IMPLIED,

BY ESTOPPEL OR OTHERWISE, TO ANY INTELLECTUAL PROPERTY RIGHTS IS GRANTED BY THIS DOCUMENT. EXCEPT AS

PROVIDED IN INTEL'S TERMS AND CONDITIONS OF SALE FOR SUCH PRODUCTS, INTEL ASSUMES NO LIABILITY WHATSOEVER,

AND INTEL DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY, RELATING TO SALE AND/OR USE OF INTEL PRODUCTS INCLUDING

LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY, OR INFRINGEMENT OF ANY

PATENT, COPYRIGHT OR OTHER INTELLECTUAL PROPERTY RIGHT. Intel products are not intended for use in medical, life saving,

life sustaining applications.

A "Mission Critical Application" is any application in which failure of the Intel Product could result, directly or indirectly, in personal

injury or death. SHOULD YOU PURCHASE OR USE INTEL'S PRODUCTS FOR ANY SUCH MISSION CRITICAL APPLICATION, YOU

SHALL INDEMNIFY AND HOLD INTEL AND ITS SUBSIDIARIES, SUBCONTRACTORS AND AFFILIATES, AND THE DIRECTORS,

OFFICERS, AND EMPLOYEES OF EACH, HARMLESS AGAINST ALL CLAIMS COSTS, DAMAGES, AND EXPENSES AND REASONABLE

ATTORNEYS' FEES ARISING OUT OF, DIRECTLY OR INDIRECTLY, ANY CLAIM OF PRODUCT LIABILITY, PERSONAL INJURY, OR DEATH

ARISING IN ANY WAY OUT OF SUCH MISSION CRITICAL APPLICATION, WHETHER OR NOT INTEL OR ITS SUBCONTRACTOR WAS

NEGLIGENT IN THE DESIGN, MANUFACTURE, OR WARNING OF THE INTEL PRODUCT OR ANY OF ITS PARTS.

Intel may make changes to specifications and product descriptions at any time, without notice.

Designers must not rely on the absence or characteristics of any features or instructions marked “reserved” or “undefined.” Intel

reserves these for future definition and shall have no responsibility whatsoever for conflicts or incompatibilities arising from future

changes to them.

Code names represented in this document are only for use by Intel to identify a product, technology, or service in development,

that has not been made commercially available to the public, i.e., announced, launched or shipped. It is not a “commercial” name

for products or services and is not intended to function as a trademark.

The Intel®64 architecture processors may contain design defects or errors known as errata, which may cause the product to deviate

from published specifications. Current characterized errata are available on request.

Hyper-Threading Technology requires a computer system with an Intel®processor supporting Hyper-Threading Technology and

an HT Technology enabled chipset, BIOS and operating system. Performance will vary depending on the specific hardware and

software you use. For more information, see http://www.intel.com/technology/hyperthread/index.htm; including details on which processors

support HT Technology.

Intel®Virtualization Technology requires a computer system with an enabled Intel®processor, BIOS, virtual machine monitor

(VMM) and for some uses, certain platform software enabled for it. Functionality, performance or other benefits will vary depending

on hardware and software configurations. Intel®Virtualization Technology-enabled BIOS and VMM applications are currently in

development.

64-bit computing on Intel architecture requires a computer system with a processor, chipset, BIOS, operating system, device driv-

ers and applications enabled for Intel®64 architecture. Processors will not operate (including 32-bit operation) without an Intel®

64 architecture-enabled BIOS. Performance will vary depending on your hardware and software configurations. Consult with your

system vendor for more information.

Contact your local Intel sales office or your distributor to obtain the latest specifications and before placing your product order.

Copies of documents which have an order number and are referenced in this document, or other Intel literature, may be obtained

by calling 1-800-548-4725, or by visiting Intel's Web Site.

Intel, Pentium, Intel Xeon, Intel NetBurst, Intel Core Solo, Intel Core Duo, Intel Core 2 Duo, Intel Core 2 Extreme, Intel Pentium

D, Itanium, Intel SpeedStep, MMX, and VTune are trademarks or registered trademarks of Intel Corporation or its subsidiaries in

the United States and other countries.

*Other names and brands may be claimed as the property of others.

Reference Number: 327043-001 3

Contents

1Introduction..............................................................................................................9

1.1 Introduction .......................................................................................................9

1.2 Uncore PMON Overview........................................................................................9

1.3 Section References............................................................................................10

1.4 Uncore PMON - Typical Control/Counter Logic .......................................................11

1.5 Uncore PMU Summary Tables .............................................................................12

1.6 On Parsing and Using Derived Events...................................................................14

1.6.1 On Common Terms found in Derived Events ..............................................15

2 Intel® Xeon® Processor E5-2600 Product Family Uncore Performance Monitoring.17

2.1 Uncore Per-Socket Performance Monitoring Control................................................17

2.1.1 Setting up a Monitoring Session ...............................................................17

2.1.2 Reading the Sample Interval....................................................................18

2.2 UBox Performance Monitoring .............................................................................19

2.2.1 Overview of the UBox .............................................................................19

2.2.2 UBox Performance Monitoring Overview ....................................................19

2.2.3 UBox Performance Monitors.....................................................................19

2.2.3.1 UBox Box Level PMON State.......................................................19

2.2.3.2 UBox PMON state - Counter/Control Pairs.....................................20

2.2.4 UBox Performance Monitoring Events........................................................21

2.2.5 UBOX Box Events Ordered By Code ..........................................................21

2.2.6 UBOX Box Performance Monitor Event List.................................................21

2.3 Caching Agent (Cbo) Performance Monitoring........................................................22

2.3.1 Overview of the CBo...............................................................................22

2.3.2 CBo Performance Monitoring Overview ......................................................23

2.3.2.1 Special Note on CBo Occupancy Events........................................23

2.3.3 CBo Performance Monitors.......................................................................24

2.3.3.1 CBo Box Level PMON State.........................................................27

2.3.3.2 CBo PMON state - Counter/Control Pairs ......................................27

2.3.3.3 CBo Filter Register (Cn_MSR_PMON_BOX_FILTER) ........................28

2.3.4 CBo Performance Monitoring Events..........................................................30

2.3.4.1 An Overview: ...........................................................................30

2.3.4.2 Acronyms frequently used in CBo Events: ....................................30

2.3.4.3 The Queues: ............................................................................31

2.3.5 CBo Events Ordered By Code...................................................................31

2.3.6 CBO Box Common Metrics (Derived Events)...............................................32

2.3.7 CBo Performance Monitor Event List..........................................................34

2.4 Home Agent (HA) Performance Monitoring............................................................45

2.4.1 Overview of the Home Agent ...................................................................45

2.4.2 HA Performance Monitoring Overview........................................................46

2.4.3 HA Performance Monitors ........................................................................46

2.4.3.1 HA Box Level PMON State ..........................................................46

2.4.3.2 HA PMON state - Counter/Control Pairs........................................47

2.4.4 HA Performance Monitoring Events ...........................................................49

2.4.4.1 On the Major HA Structures: ......................................................49

2.4.5 HA Box Events Ordered By Code ..............................................................50

2.4.6 HA Box Common Metrics (Derived Events).................................................50

2.4.7 HA Box Performance Monitor Event List.....................................................51

2.5 Memory Controller (iMC) Performance Monitoring ..................................................59

2.5.1 Overview of the iMC ...............................................................................59

2.5.2 Functional Overview ...............................................................................59

2.5.3 iMC Performance Monitoring Overview.......................................................59

2.5.4 iMC Performance Monitors .......................................................................60

4Reference Number: 327043-001

2.5.4.1 MC Box Level PMON State ..........................................................60

2.5.4.2 MC PMON state - Counter/Control Pairs........................................61

2.5.5 iMC Performance Monitoring Events...........................................................62

2.5.5.1 An Overview:............................................................................62

2.5.6 iMC Box Events Ordered By Code..............................................................63

2.5.7 iMC Box Common Metrics (Derived Events) ................................................63

2.5.8 iMC Box Performance Monitor Event List ....................................................64

2.6 Power Control (PCU) Performance Monitoring ........................................................72

2.6.1 Overview of the PCU ...............................................................................72

2.6.2 PCU Performance Monitoring Overview ......................................................72

2.6.3 PCU Performance Monitors.......................................................................72

2.6.3.1 PCU Box Level PMON State.........................................................73

2.6.3.2 PCU PMON state - Counter/Control Pairs.......................................74

2.6.3.3 Intel® PCU Extra Registers - Companions to PMON HW..................76

2.6.4 PCU Performance Monitoring Events..........................................................76

2.6.4.1 An Overview:............................................................................76

2.6.5 PCU Box Events Ordered By Code .............................................................78

2.6.6 PCU Box Common Metrics (Derived Events)................................................79

2.6.7 PCU Box Performance Monitor Event List....................................................79

2.7 Intel® QPI Link Layer Performance Monitoring.......................................................87

2.7.1 Overview of the Intel® QPI Box................................................................87

2.7.2 Intel® QPI Performance Monitoring Overview.............................................87

2.7.3 Intel® QPI Performance Monitors..............................................................88

2.7.3.1 Intel® QPI Box Level PMON State ...............................................88

2.7.3.2 Intel® QPI PMON state - Counter/Control Pairs .............................89

2.7.3.3 Intel® QPI Registers for Packet Mask/Match Facility.......................90

2.7.3.4 Intel® QPI Extra Registers - Companions to PMON HW...................94

2.7.4 Intel® QPI LL Performance Monitoring Events.............................................94

2.7.4.1 An Overview.............................................................................94

2.7.4.2 Acronyms frequently used in Intel® QPI Events: ...........................95

2.7.5 Intel® QPI LL Box Events Ordered By Code................................................95

2.7.6 Intel QPI LL Box Common Metrics (Derived Events).....................................96

2.7.7 Intel® QPI LL Box Performance Monitor Event List ......................................98

2.8 R2PCIe Performance Monitoring.........................................................................111

2.8.1 Overview of the R2PCIe Box...................................................................111

2.8.2 R2PCIe Performance Monitoring Overview................................................111

2.8.3 R2PCIe Performance Monitors.................................................................112

2.8.3.1 R2PCIe Box Level PMON State...................................................112

2.8.3.2 R2PCIe PMON state - Counter/Control Pairs ................................113

2.8.4 R2PCIe Performance Monitoring Events....................................................114

2.8.4.1 An Overview...........................................................................114

2.8.5 R2PCIe Box Events Ordered By Code.......................................................114

2.8.6 R2PCIe Box Common Metrics (Derived Events) .........................................114

2.8.7 R2PCIe Box Performance Monitor Event List .............................................115

2.9 R3QPI Performance Monitoring ..........................................................................119

2.9.1 Overview of the R3QPI Box ....................................................................119

2.9.2 R3QPI Performance Monitoring Overview .................................................119

2.9.3 R3QPI Performance Monitors..................................................................120

2.9.3.1 R3QPI Box Level PMON State....................................................120

2.9.3.2 R3QPI PMON state - Counter/Control Pairs..................................121

2.9.4 R3QPI Performance Monitoring Events.....................................................122

2.9.4.1 An Overview...........................................................................122

2.9.5 R3QPI Box Events Ordered By Code ........................................................122

2.9.6 R3QPI Box Common Metrics (Derived Events)...........................................123

2.9.7 R3QPI Box Performance Monitor Event List...............................................123

2.10 Packet Matching Reference................................................................................131

Reference Number: 327043-001 5

Figures

1-1 Uncore Sub-system Block Diagram of Intel Xeon Processor E5-2600 Family ................9

1-2 Perfmon Control/Counter Block Diagram...............................................................11

Tables

1-1 Per-Box Performance Monitoring Capabilities.........................................................10

1-2 MSR Space Uncore Performance Monitoring Registers.............................................12

1-3 PCICFG Space Uncore Performance Monitoring Registers ........................................13

2-1 UBox Performance Monitoring MSRs.....................................................................19

2-2 U_MSR_PMON_CTL{1-0} Register – Field Definitions .............................................20

2-3 U_MSR_PMON_CTR{1-0} Register – Field Definitions.............................................20

2-4 U_MSR_PMON_FIXED_CTL Register – Field Definitions ...........................................21

2-5 U_MSR_PMON_FIXED_CTR Register – Field Definitions...........................................21

2-8 CBo Performance Monitoring MSRs ......................................................................24

2-9 Cn_MSR_PMON_BOX_CTL Register – Field Definitions ............................................27

2-10 Cn_MSR_PMON_CTL{3-0} Register – Field Definitions............................................27

2-11 Cn_MSR_PMON_CTR{3-0} Register – Field Definitions ...........................................28

2-12 Cn_MSR_PMON_BOX_FILTER Register – Field Definitions........................................29

2-13 Opcode Match by IDI Packet Type for Cn_MSR_PMON_BOX_FILTER.opc ...................29

2-33 HA Performance Monitoring MSRs........................................................................46

2-34 HA_PCI_PMON_BOX_CTL Register – Field Definitions .............................................47

2-35 HA_PCI_PMON_CTL{3-0} Register – Field Definitions.............................................47

2-36 HA_PCI_PMON_CTR{3-0} Register – Field Definitions ............................................48

2-37 HA_PCI_PMON_BOX_OPCODEMATCH Register – Field Definitions............................48

2-38 HA_PCI_PMON_BOX_ADDRMATCH1 Register – Field Definitions...............................48

2-39 HA_PCI_PMON_BOX_ADDRMATCH0 Register – Field Definitions...............................49

2-59 iMC Performance Monitoring MSRs.......................................................................60

2-60 MC_CHy_PCI_PMON_BOX_CTL Register – Field Definitions......................................60

2-61 MC_CHy_PCI_PMON_CTL{3-0} Register – Field Definitions .....................................61

2-62 MC_CHy_PCI_PMON_FIXED_CTL Register – Field Definitions ................................... 62

2-63 MC_CHy_PCI_PMON_CTR{FIXED,3-0} Register – Field Definitions ...........................62

2-73 PCU Performance Monitoring MSRs ......................................................................72

2-74 PCU_MSR_PMON_BOX_CTL Register – Field Definitions ..........................................73

2-75 PCU_MSR_PMON_CTL{3-0} Register – Field Definitions..........................................74

2-76 PCU_MSR_PMON_CTR{3-0} Register – Field Definitions .........................................75

2-77 PCU_MSR_PMON_BOX_FILTER Register – Field Definitions......................................76

2-78 PCU_MSR_CORE_C6_CTR Register – Field Definitions.............................................76

2-79 PCU_MSR_CORE_C3_CTR Register – Field Definitions.............................................76

2-80 PCU Configuration Examples...............................................................................77

2-84 Intel® QPI Performance Monitoring Registers........................................................88

2-85 Q_Py_PCI_PMON_BOX_CTL Register – Field Definitions ..........................................89

2-86 Q_Py_PCI_PMON_CTL{3-0} Register – Field Definitions..........................................89

2-87 Q_Py_PCI_PMON_CTR{3-0} Register – Field Definitions .........................................90

2-88 Q_Py_PCI_PMON_PKT_MATCH1 Registers.............................................................91

2-89 Q_Py_PCI_PMON_PKT_MATCH0 Registers.............................................................91

2-90 Q_Py_PCI_PMON_PKT_MASK1 Registers...............................................................92

2-91 Q_Py_PCI_PMON_PKT_MASK0 Registers...............................................................92

2-92 Message Events Derived from the Match/Mask filters..............................................93

6Reference Number: 327043-001

2-93 QPI_RATE_STATUS Register – Field Definitions......................................................94

2-104 R2PCIe Performance Monitoring Registers...........................................................112

2-105 R2_PCI_PMON_BOX_CTL Register – Field Definitions ............................................112

2-106 R2_PCI_PMON_CTL{3-0} Register – Field Definitions............................................113

2-107 R2_PCI_PMON_CTR{3-0} Register – Field Definitions ...........................................113

2-118 R3QPI Performance Monitoring Registers ............................................................120

2-119 R3_Ly_PCI_PMON_BOX_CTL Register – Field Definitions .......................................120

2-120 R3_Ly_PCI_PMON_CTL{2-0} Register – Field Definitions.......................................121

2-121 R3_Ly_PCI_PMON_CTR{2-0} Register – Field Definitions ......................................121

2-142 Intel® QuickPath Interconnect Packet Message Classes ........................................131

2-143 Opcode Match by Message Class........................................................................131

2-144 Opcodes (Alphabetical Listing)...........................................................................132

Reference Number: 327043-001 7

Revision History

Revision Description Date

327043-001 Initial release. March 2012

8Reference Number: 327043-001

Reference Number: 327043-001 9

Introduction

1Introduction

1.1 Introduction

The uncore subsystem of the Intel® Xeon® processor E5-2600 product family is shown in Figure 1-1.

The uncore subsystem also applies to the Intel® Xeon® processor E5-1600 product family in a

single-socket platform1. The uncore sub-system consists of a variety of components, ranging from the

CBox caching agent to the power controller unit (PCU), integrated memory controller (iMC) and home

agent (HA), to name a few. Most of these components provide similar performance monitoring

capabilities.

1.2 Uncore PMON Overview

The uncore performance monitoring facilities are organized into per-component performance

monitoring (or ‘PMON’) units. A PMON unit within an uncore component may contain one of more

sets of counter registers. With the exception of the UBox, each PMON unit provides a unit-level

control register to synchronize actions across the counters within the box (e.g., to start/stop

counting).

1. The uncore sub-system in Intel® CoreTM i7-3930K and i7-3820 processors are derived from

above, hence most of the descriptions of this document also apply.

Figure 1-1. Uncore Sub-system Block Diagram of Intel Xeon Processor E5-2600 Family

Introduction

10 Reference Number: 327043-001

Events can be collected by reading a set of local counter registers. Each counter register is paired with

a dedicated control register used to specify what to count (i.e. through the event select/umask fields)

and how to count it. Some units provide the ability to specify additional information that can be used

to ‘filter’ the monitored events (e.g., C-box; see Section 2.3.3.3, “CBo Filter Register

(Cn_MSR_PMON_BOX_FILTER)”).

Uncore performance monitors represent a per-socket resource that is not meant to be affected by

context switches and thread migration performed by the OS, it is recommended that the monitoring

software agent establish a fixed affinity binding to prevent cross-talk of event counts from different

uncore PMU.

The programming interface of the counter registers and control registers fall into two address spaces:

• Accessed by MSR are PMON registers within the Cbo units, PCU, and U-Box, see Table 1-2.

• Access by PCI device configuration space are PMON registers within the HA, iMC, Intel® QPI,

R2PCIe and R3QPI units, see Table 1-3.

Irrespective of the address-space difference and with only minor exceptions, the bit-granular layout of

the control registers to program event code, unit mask, start/stop, and signal filtering via threshold/

edge detect are the same.

The general performance monitoring capabilities of each box are outlined in the following table.

1.3 Section References

The following sections provide a breakdown of the performance monitoring capabilities for each box.

• Section 2.1, “Uncore Per-Socket Performance Monitoring Control”

• Section 2.2, “UBox Performance Monitoring”

• Section 2.3, “Caching Agent (Cbo) Performance Monitoring”

• Section 2.6, “Power Control (PCU) Performance Monitoring”

• Section 2.4, “Home Agent (HA) Performance Monitoring”

• Section 2.5, “Memory Controller (iMC) Performance Monitoring”

• Section 2.7, “Intel® QPI Link Layer Performance Monitoring”

• Section 2.9, “R3QPI Performance Monitoring”

• Section 2.8, “R2PCIe Performance Monitoring”

• Section 2.10, “Packet Matching Reference”

Table 1-1. Per-Box Performance Monitoring Capabilities

Box # Boxes # Counters/

Box #Queue

Enabled Bus

Lock? Packet Match/

Mask Filters? Bit Width

C-Box 8 4 1 N Y 44

HA 1 4 4 Y Y 48

iMC 1

(4 channels) 4 (+1)

(per channel) 4N N 48

PCU 1 4 (+2) 4 N N 48

QPI 1

(2 ports) 4

(per port) 4N Y? 48

R2PCIe 1 4 1 N N 44

R3QPI 1

(2 links) 31N N44

U-Box 1 2 (+1) 0 N/A N 44

Andere Handbücher für Xeon Processor E5-2600

Inhaltsverzeichnis

Andere Intel Computerhardware Handbücher

Intel

Intel X5550 - Quad Core Xeon Leitfaden

Intel

Intel LF80537GF0484M - Cpu Core 2 Duo T7400 2.16Ghz Fsb667Mhz 4Mb Fcpga6... Bedienungsanleitung

Intel

Intel T8300 - Core 2 Duo 2.4GHz 800MHz 3MB Socket P Mobile... Bedienerhandbuch

Intel

Intel 80286 Bedienungsanleitung

Intel

Intel E5345 - Xeon 2.33 GHz 8M L2 Cache 1333MHz FSB LGA771 Active Quad-Core... Bedienungsanleitung

Intel

Intel ETHEREXPRESS PRO/100 Bedienungsanleitung

Intel

Intel Pentium OverDrive Bedienungsanleitung

Intel

Intel BX80605X3440 - Quad Core Xeon X3440 Service-Handbuch

Intel

Intel RMS25CB080 Einrichtungshandbuch

Intel

Intel RS25NB008 Bedienungsanleitung

Intel

Intel 80960HA Bedienungsanleitung

Intel

Intel D2550DC2 Bedienungsanleitung

Intel

Intel E5420 - CPU XEON QUAD CORE 2.50GHZ FSB1333MHZ 12M LGA771 HALOGEN FREE... Bedienungsanleitung

Intel

Intel MFS5000SI - Multi-Flex Server Compute Module Bedienungsanleitung

Intel

Intel DP45SG - CARACTERISTIQUES TECHNIQUES Bedienungsanleitung

Intel

Intel SL6NQ - Xeon 2.4 GHz/533MHz/512 KB CPU Processor... Bedienungsanleitung

Intel

Intel P8700 - Core 2 Duo Processor Bedienungsanleitung

Intel

Intel ETHEREXPRESS PRO/100 Bedienungsanleitung

Intel

Intel CORE 2 DUO E7000 - UPDATE 7-2010 Bedienungsanleitung

Intel

Intel Atom C2000 Bedienungsanleitung

Intel

Intel 520J - Pentium 4 2.80GHz 800MHz 1MB Socket 775... Bedienungsanleitung

Intel

Intel SRCSASRB - RAID Controller Installationshandbuch

Intel

Intel D815EEA2 Bedienungsanleitung

Intel

Intel CORE 2 DUO E4000 - 3-2008 Bedienungsanleitung

Beliebte Computerhardware Handbücher anderer Marken

EMC2

EMC2 VNX Series Betriebsanleitung

Panasonic

Panasonic DV0PM20105 Bedienungsanleitung

Mitsubishi Electric

Mitsubishi Electric Q81BD-J61BT11 Bedienungsanleitung

Gigabyte

Gigabyte B660M DS3H AX DDR4 Bedienungsanleitung

Raidon

Raidon iT2300 Bedienungsanleitung

National Instruments

National Instruments PXI-8186 Bedienungsanleitung

Hauppauge

Hauppauge WinTV-HVR-2250 Bedienungsanleitung

Kontron

Kontron CP6002 Bedienungsanleitung

Manhattan

Manhattan 702843 Bedienungsanleitung

Aphex

Aphex 230 Bedienungsanleitung

AMD

AMD Phenom II Betriebshandbuch

dbx

dbx Zone Pro 1260 Bedienungsanleitung

Galaxy

Galaxy GHDX2-2430S-24F4D Bedienungsanleitung

Kontron

Kontron DIMM-PC/MD Bedienungsanleitung

ST X-NUCLEO-SAFEA1B Betriebsanleitung

STEINWAY LYNGDORF

STEINWAY LYNGDORF SP-1 Bedienungsanleitung

Advantech

Advantech ASMB-935 Series Bedienungsanleitung

Jupiter

Jupiter RAM PACK Bedienungsanleitung