Advantech MIC-5332 User manual

User Manual
MIC-5332
AdvancedTCA® 10GbE Dual Socket CPU Blade
with Intel® Xeon® E5-2600 series EP Processors

Revision History
Revision
Index
Brief Description of Changes Date of Issue
0.1 Initial Draft November 15th,
2011
0.2 Modification April 11th, 2012
0.3 Modification June 15th, 2012
0.4 Modification July 16th, 2012

Copyright
The documentation and the software included with this product are copyrighted 2012 by
Advantech Co., Ltd. All rights are reserved. Advantech Co., Ltd. reserves the right to make
improvements in the products described in this manual at any time without notice. No part of
this manual may be reproduced, copied, translated or transmitted in any form or by any means
without the prior written permission of Advantech Co., Ltd. Information provided in this manual
is intended to be accurate and reliable. However, Advantech Co., Ltd. assumes no
responsibility for its use, nor for any infringements of the rights of third parties, which may
from its use.
Acknowledgements
ATCA and AMC are trademarked by PCI Industrial Computer Manufacturers Group whilst
QPI and C600-B are trademarked by the Intel Corp. All other product names or trademarks are
properties of their respective owners.
Product Warranty (2 years)
Advantech warrants to you, the original purchaser, that each of its products will be free from
defects in materials and workmanship for two years from the date of purchase.
This warranty does not apply to any products which have been repaired or altered by persons
other than repair personnel authorized by Advantech, or which have been subject to misuse,
abuse, accident or improper installation. Advantech assumes no liability under the terms of this
warranty as a consequence of such events.
Because of Advantech’s high quality-control standards and rigorous testing, most of our
customers never need to use our repair service. If an Advantech product is defective, it will be
repaired or replaced at no charge during the warranty period. For out-of-warranty repairs, you
will be billed according to the cost of replacement materials, service time and freight. Please
consult your dealer for more details.
If you think you have a defective product, follow these steps:
1. Collect all the information about the problem encountered, for example, Advantech
products used, other hardware and software used, etc. Note anything abnormal and list
any onscreen messages you get when the problem occurs.
2. Call your dealer and describe the problem. Please have your manual, product, and any
helpful information readily available.
3. If your product is diagnosed as defective, obtain an RMA (return merchandise
number from your dealer. This allows us to process your return more quickly.
4. Carefully pack the defective product, a fully-completed Repair and Replacement Order
Card and a photocopy proof of purchase date (such as your sales receipt) in a shippable
container. A product returned without proof of the purchase date is not eligible for warranty
service.
5. Write the RMA number visibly on the outside of the package and ship it prepaid to your
dealer.

Declaration of Conformity
CE
This product has passed the CE test for environmental specifications when shielded cables
are used for external wiring. We recommend the use of shielded cables.
FCC Class A
Note: This equipment has been tested and found to comply with the limits for a Class A digital
device, pursuant to part 15 of the FCC Rules. These limits are designed to provide reasonable
protection against harmful interference when the equipment is operated in a commercial
environment. This equipment generates, uses, and can radiate radio frequency energy and, if
not installed and used in accordance with the instruction manual, may cause harmful
interference to radio communications. Operation of this equipment in a residential area is likely
to cause harmful interference in which case the user will be required to correct the interference
at his or her own expense.
Technical Support and Assistance
1. Visit the Advantech web site at www.advantech.com/support where you can find the
latest information about the product.
2. Contact your distributor, sales representative, or Advantech’s customer service center for
technical support if you need additional assistance. Please have the following information
ready before you call:
Product name and serial number
Description of your peripheral attachments
Description of your firmware version
A complete description of the problem
The exact wording of any error messages

Warnings, Cautions and Notes
Warning! Warnings indicate conditions, which if not observed, can cause personal injury.
Caution! Cautions are included to help you avoid damaging hardware or losing data, for
example, there is a danger of a new battery exploding if it is incorrectly installed.
Do not attempt to recharge, force open, or heat the battery. Replace the battery only with the
same or equivalent type recommended by the manufacturer.
Discard used batteries according to the manufacturer’s instructions.
Note! Notes provide optional additional information.
Document Feedback
To assist us in making improvements to this manual, we would welcome comments and
constructive criticism. Please send all such - in writing to: [email protected]
Packing List
RJ45 to DB9 Console Cable x1, p/n: 1700002270
Mini-USB to USB Console Cable x1, p/n: 1700018550
Warranty certificate document x, p/n: 2190000902
If any of these items are missing or damaged, contact your distributor or sales representative
immediately.

Safety Instructions
1. Read these safety instructions carefully.
2. Keep this User Manual for later reference.
3. Keep this equipment away from humidity.
4. Put this equipment on a reliable surface during installation. Dropping it or letting it fall
may cause damage.
5. All cautions and warnings on the equipment should be noted.
6. Always use caution when handling/operating the computer. Only qualified, experienced,
authorized electronics service personnel should access the interior of the computer. Te
power supplies produce high voltages and energy hazards, which can cause bodily
harm.
7. If the equipment is not used for a long time, disconnect it from the power source to avoid
damage by transient over-voltage.
8. Never pour any liquid into an opening. This may cause fire or electrical shock.
9. If one of the following situations arises, get the equipment checked by service personnel:
The power cord or plug is damaged.
Liquid has penetrated into the equipment.
The equipment has been exposed to moisture.
The equipment does not work well, or you cannot get it to work according
The equipment has been dropped and damaged.
The equipment has obvious signs of breakage.
10. The sound pressure level at the operator’s position according to IEC 704-1:1982 is no
more than 70 dB (A).
DISCLAIMER: This set of instructions is given according to IEC 704-1. Advantech disclaims all
responsibility for the accuracy of any statements contained herein.
Safety Precaution - Static Electricity
Follow these simple precautions to protect yourself from harm and the products from damage.
1. To avoid electrical shock, always disconnect the power from your system chassis before
you work on it. Don’t touch any components on the CPU card or other cards while the
system is on.
2. Disconnect power before making any configuration changes. The sudden rush of power
as you connect a jumper or install a card may damage sensitive electronic components.
3. When
unpacking a static-sensitive component from its shipping carton, do not
remove the component's antistatic packing material until you are ready to install
the omponent in a computer. Just before unwrapping the antistatic packaging, be
sure you are at an ESD workstation or grounded. This will discharge any static
electricity that may have built up in your body.
4. When
transporting a sensitive component, first place it in an antistatic container
or packaging.
We Appreciate Your Input
Please let us know of any aspect of this product, including the manual, which could use
improvement or correction. We appreciate your valuable input in helping make our products
better.

This page is left blank intentionally.

Glossary
ACPI Advanced Configuration and Power Interface
AHCI Advanced Host Controller Interface
AMC Advanced Mezzanine Card
APIC Advanced Programmable Interrupt Controller
ATCA Advanced Telecommunications Computing Architecture
BI Base Interface
BMC Baseboard Management Controller
CMC Carrier Management Controller
EHCI Enhanced Host Controller Interface
FI Fabric Interface
FMM FabricMezzanineModule
FRU Field Replaceable Unit
FW Firmware
GbE Gigabit Ethernet
HPM Hardware Platform Management
IOH I/O Controller Hub
IPMC Intelligent Platform Management Controller
IPMI Intelligent Platform Management Interface
MCH MemoryControllerHub
NVRAM Non-volatile Random Access Memory
OOS Out Of Service
PCH PlatformControllersHub
PCIe PCI Express
PECI Platform Environment Control Interface
PICMG PCI Industrial Computer Manufacturers Group
PXE Pre-boot Execution Environment
QPI QuickPath Interconnection
RDIMM RegisteredDIMMs
RMCP Remote Management Control Protocol
RTM Rear Transition Module
RX Receive
SAS Serial Attached SCSI
SATA Serial Advanced Technology Attachment
SCSI Small Computer System Interface
SDR Sensor Data Record
SerDes Serializer/Deserializer

ShMC Shelf Manager Controller
SOL Serial Over LAN
TCLK Telecom Clock
TPM Trusted Platform Module
TX Transmit
UDIMM UnbufferedDIMMs
UHCI Universal Host Controller Interface
VLP Very Low Profile
XAUI X (means ten) Attachment Unit Interface

Chapter 1
Product Overview
This chapter briefly describes the MIC-5332.

1.1 MIC-5332 Overview
The MIC-5332 is a dual socket AdvancedTCA blade based on the Intel® Xeon
E5-2600 series EP processors and C600 PCH (codename Patsburg). The MIC-5332
enables the highest performance available in an ATCA form factor with up to 16-cores
and 32-threads of processing power, fast PCI Express gen 3 lanes running at up to
8Gbps, and best in class virtualization support. Two QPI interfaces between the CPUs
improve memory and I/O access throughput and latencies when one processor needs
to access resources hosted by the other socket. With four DDR3 DIMMs per socket in
a quad channel design running up to 1600MT/s, the MIC-5332 not only offers superior
memory bandwidth over 3-channel designs, but can also support memory densities
up 256GB using latest LR DIMM technology. It outperforms previous generation dual
socket designs while keeping similar thermal characteristics with balanced airflow
resistance.
Using Intel’s latest PCH (C604) with its integrated 4-port SAS controller, the need for
an external storage controller is eliminated, making the MIC-5332 an ideal choice for
cost sensitive control plane applications. While supporting two 10GBaseKX4
interfaces in the base model, support for dual-dual star fabric implementations can be
added by installing the FMM-5001B Fabric Mezzanine Module (FMM). Beyond that,
the FMM type II socket with PCIe x16 connectivity provides extension possibilities for
additional front port I/O, offload and acceleration controllers such as Intel
QuickAssist™ accelerators, IPSec offload engines or customer specific logic. FMMs
not only have higher PCI Express bandwidth than AMCs, but also integrate well in
terms of thermal design and board real estate when compared to Advanced
Mezzanine Cards. Moreover, FMMs can be reused on RTMs and across different
blade designs. This unmatched flexibility combined with the highest performance
Intel® Xeon® processors available make the MIC-5332 equally well suited for
application and data plane workloads.
The onboard IPMI firmware was developed entirely by Advantech to offer greater
modularity and flexibility for the customization of system management features,
especially when it comes to tailoring a system design to meet target cost points
without sacrificing features and time to market. HPM.1 based updates are available
for all programmable components (BIOS, IPMC firmware, FPGA) including rollback
support. Advantech’s IPMI solution, combined with an optimized AMI UEFI BIOS
continues to offer advanced features used on previous generation MIC-532x blades,
such as BIOS redundancy, Real Time Clock Synchronization, and MAC

Mirroring.(please refer to chapter 4) Advantech IPMI firmware has been tested for
CP-TA compliance using the Polaris Networks ATCA Test Suite.
The MIC-5332 supports hot-swappable RTMs such as the RTM-5104 for High
Availability (HA) needs, rear I/O and dual SAS storage with RAID as well as an
optional FMM. Please contact Advantech for more information on available RTMs.
An on-board FPGA design facilitates customer-specific modifications, and the core
board design can be modified or adapted to other form factors through Advantech’s
DMS customization services.
Figure 1.1 MIC-5332 Overview (Top Side)

1.2 Block Diagram
The hardware implementation is shown in the following block diagram. Refer to Table
1.1 (next page) for the detailed product technical specification.
:Option
Figure 1.2 MIC-5332 Block Diagram

1.3 Product Configurations
Model Name Configurations
MIC-5332SA1-P1E MIC-5332 RJ45 version with dual Intel® Xeon® E5-2648L CPU
MIC-5332SA1-P2E MIC-5332 RJ45 version with dual Intel® Xeon® E5-2658 CPU
MIC-5332SB1-P1E MIC-5332 SFP version with dual Intel® Xeon® E5-2648L CPU
MIC-5332SB1-P2E MIC-5332 SFP version with dual Intel® Xeon® E5-2658 CPU
Table 1.1 MIC-5332 Configurations
Note:
Support max 256GB using 8 pieces of 32GB DDR3-1600 VLP DIMM modules.
1.4 Related Products
Model Name Configurations
RTM-5104 I/O extension ATCA RTM for MIC-5332
FMM-5001BE Dual 10GE Module with 2x fabric ports for dual dual star support
based on i82599EB
FMM-5001FE Dual 10GE Module with 2x SFP+ front IO based on i82599ES
FMM-5002E Server Graphics Module with external VGA Port
FMM-5006E MIC-5332 QuickAssist Accelerator FMM
Table 1.2 MIC-5332 Related Products
Note:
Contact Advantech for information on available and future RTMs and FMMs.

Chapter 2
Board Features
This chapter describes the MIC-5332 hardware features.

2.1 Technical Data
Processor
System
CPU Dual Intel® Xeon® E5-2648L/E5-2658 8-core processors
(1)
Max. Speed 2.1GHz
Chipset Intel® C604
BIOS
Dual 64-Mbit BIOS firmware flashes with AMI UEFI based BIOS
QPI 8.0 GT/s
Memory
T echnology
Four channel DDR3 1066/1333/1600MHz SDRAM (72-bit ECC Un-/
Registered), LR DIMM support
Max. Capacity Configurable up to 256 GB
Socket 8 VLP DIMMs
Zone 2
Fabric interface
2 x Intel® 82599 Dual 10GE MAC/PHY supporting four 10GBase
ports (XAUI) (one by default and the second one is optional, via a
FMM-5001BE)
Base interface i350 GbE MAC/PHY supporting two 10/100/1000Mbps ports
Front I/O
Interface
Serial (COM)
2 x 16C550 compatible S erial Ports (1 RJ-45 connector, 1
connector)
Ethernet
2 x 10/100/1000BASE-T or SFP through i350 MAC/PHY, 1x
10/100/1000 BASE-T Chipset LAN
USB 2.0 2 x Type A ports
Operating
System
Compatibility WindRiver PNE/LE 4.2, RedHat Enterprise 5.7 & 6.2, CentOS 6.1,
Windows Server 2008
IPMC
BMC Controller NXP LPC1768 (Cortex M)
IPMI Compliant with IPMI 1.5 using Advantech IPMI code base
Watchdog Timer
Supervision 1 for x86 BIOS POST, OS Boot, Application
Interval IPMI compliant
FMM
Site 1 FMM type II socket
Interface 1 x PCIe x16 or 2 x PCIe x8
Miscellaneous
Storage 2 x CFast / 1 x 2.5” SSD*, , 4-port SAS controller integrated in PCH
to zone 3
Real Time Clock Built-in
Power
Requirement
Configuration 2 x E5-2648L70W, 32GB memory, no FMM, no RTM
Consumption 230W (estimated)
Zone 3 (RTM)
RTM Advantech common RTM interface Type 2
Interface 4 x SAS/SATA, 1 x PCIex16, 4 x USB, 2 x UART
Physical
Characteristics
PCB Dimensions 6HP, 280.00 x 322.25 mm (11.02" x 12.69") (W x D)
Weight 3.275kg

Environment
Operating Non-operating
Temperature 0 ~ 55° C (32 ~ 131° F) - 40 ~ 70° C (-40 ~ 158° F)
Humidity 5 to 93%@40°C (non
condensing)
95% @ 40° C (non-condensing)
Shock 4 G each axis 20 G each axis
Vibration (5~500Hz) 0.5 Grms 2.16 Grms, 30 mins each axis
Compliance
Environment ETSI EN300019-2-1 Class1.2, EN300019-2-2 Class 2.3, ETSI
EN300019-2-3 Class 3.1E, Designed to meet GR63-CORE
PICMG 3.0 R3.0, 3.1 R1.0, HPM.1
Safety CE mark (EN60950-2001), UL60950-1/CSAC22.2
EMC FCC47 CFR Part15, Class A, CE Mark (EN55022 / EN55024 /
EN300386), Designed to meet GR1089-CORE
Table 2.1 MIC-5332 Technical Data
Note:MIC-5332 supports 2 x 95W CPUs. Special system airflow requirements
apply.
CFast and 2.5” SSD are mutually exclusive.
2.2 Product Features
2.2.1 Processors
The MIC-5332 supports dual Intel® Xeon® E5-2600 series processors, using latest
32nm silicon architectures with built-in memory controller. It is a two-chip platform
(CPU and PCH) as opposed to traditional three-chip platforms (CPU, MCH and IOH).
The Intel® Xeon® E5-2600 series feature per socket, two Intel® QuickPath
Interconnect point-to-point links capable of up to 8.0 GT/s, up to 40 lanes of Gen 3
PCI Express links capable of 8.0 GT/s, and 4 lanes of DMI2/PCI Express Gen 2
interface with a peak transfer rate of 5.0 GT/s. The processor supports up to 46 bits of
physical address space and 48-bit of virtual address space. It is also capable to
support up to 4 channels DDR3 DIMM, supports both to UDIMMs and RDIMMs (more
details about DDR3 DIMM supported specification, please refer to section 2.3). The
MIC-5332 currently supported processors are listed as table 2.2:
Model Cores/Threads Frequency L3 cache DDR3 support TDP Socket
Xeon E5-2620 6C/12T 2.00 GHz 15 MB DDR3-1333 95 Watt LGA2011
Xeon E5-2648L 8C/16T 1.80 GHz 20 MB DDR3-1600 70 Watt LGA2011
Xeon E5-2658 8C/16T 2.10 GHz 20 MB DDR3-1600 95 Watt LGA2011
Table 2.2 MIC-5332 Supported Processors List

The E5 series Xeon processors support cache memory as listed below:
A 32-KB instruction and 32-KB data first-level cache (L1) for each core.
A 256-KB shared instruction/data mid-level (L2) cache for each core.
Up to 20 MB last level cache (LLC): up to 2.5 MB per core instruction/data last level
cache (LLC), shared among all cores.
2.2.2 Platform Controller Hub (PCH)
An Intel® C604 provides the peripheral connection in the Intel® Sandy Bridge
platform. It contains DMI Gen2, 8x PCIe Gen2 root ports, 2x SATA 6Gb/s, 4x SATA
3Gb/s, 4x SAS 3Gb/s, 14x USB 2.0 and supports one internal GbE MAC. For more
details, please refer to section 2.3.
2.2.3 DMI Gen2
Direct Media Interface (DMI) is the chip-to-chip connection between the processor
and PCH. This high-speed interface integrates advanced priority-based servicing
allowing for concurrent traffic and true isochronous transfer capabilities. Base
functionality is completely software transparent permitting current and legacy software
to operate normally. New for C604 chipset, the DMI interface operates at 5.0 GT/s.
2.2.4 PCI Express Port Configuration
Intel® Xeon® E5-2600 series processors support 40 PCI Express Gen3 ports. They
are configured to two x16 ports, one x8 port and five x4 ports in the MIC-5332. The
PCI Express interface is connected to RTM (Port1 and 2) and update channel (Port 3,
optional).
CPU Port No. Width Description
0
0 x4 Connection for DMI between the CPU and PCH
1 x4 Base Interface and I/O Control, connected to Intel® i350-AM4
2 x8 Fabric Interface, connected to Intel® i82599 10GbE controller
3 x4 For update channel, connected to Zone 2 (optional)
1
0 x4 Not connected
1 x4 For RTM use, connected to Zone 3
2 x16 For RTM use, connected to Zone 3
3 x16
connected to fabric mezzanine (FMM); configurable as 1x16 or
2x8
Table 2.3 PCI Express Port Configuration on the MIC-5332
1. Note:

PCIe hot swap is not supported for graphic controllers (e.g. FMM-5002E) installed on a
RTM.
2.2.5 Redundant BIOS Flash
The MIC-5332 has two SPI flash devices for storing redundant x86 firmware (BIOS,
BIOS configuration, etc.). The integrated management controller (IPMC) controls
which flash device is active. Failover and rollback operations between the flash
devices are compliant to HPM.1. By default, the MIC-5332 starts to boot from the
active BIOS chip and will switch to the backup BIOS chip if it detects a problem, i.e.
the BIOS is stuck in POST or fails to boot. For more details, please refer to Chapter 5,
about AMI BIOS setup. From OS side of view, there is only the active BIOS SPI flash
visible at any time.
2.2.6 Serial ATA and Serial Attached SCSI Controller
The MIC-5332 supports a total of 6 SATA and 4 SAS lanes.
The built-in Serial ATA (SATA) and Serial Attached SCSI (SAS) controller in the
Intel® C600 PCH has three modes of operation to support different operating system
conditions. In the case of a Native IDE enabled operating system, the C600 PCH
utilizes two controllers to enable all six ports of the bus. The first controller (Device 31:
Function 2) supports ports 0-3 and the second controller (Device 31: Function 5)
supports ports 4 and 5. When using a legacy operating system, only one controller
(Device 31: Function 2) is available which supports ports 0-3. In AHCI or RAID mode,
only one controller (Device 31: Function 2) is utilized, enabling all six ports and the
second controller (Device 31: Function 5) will be disabled. Please contact your
Advantech technical support team for more detail information.
In the MIC-5332, SATA 6 Gbps support is available on PCH Ports 0 and 1 only, and
reserved for onboard storage modules like CFast or 2.5” SSD. For detailed
configurations, please refer to table 2.4:
Port No. Speed Description
0 6Gbps
Onboard Storage Module, 1x 2.5” SSD (default) or 2x CFast (option)
1 6Gbps
2 3Gbps SATA devices on the RTM (connected to Zone 3)
3 3Gbps
4 3Gbps on board SATA NAND Flash (optional)
5 3Gbps SATA devices on FMM (optional)

Table 2.4 SATA Port Configuration on the MIC-5332
The MIC-5332 is also able to support SAS devices. 4 SAS 2.0 channels are reserved
for SAS or SATA devices on RTM boards. For details, please see table 2.5.
Port No. Speed Description
0
3Gbps Supports extended SAS/SATA devices on RTM (connected to Zone
3)
1
2
3
Table 2.5 SAS Port Configuration on the MIC-5332
2.2.7 USB Controller
In the Intel® C600 PCH there are 14x USB ports, controlled either through the
standard Universal Host Controller Interface (UHCI) for full/low speed support, or
through Enhanced Host Controller Interface (EHCI) for USB 2.0 high speed support.
The USB port connection in the MIC-5332 is listed in table 2.6.
Port No. Description
0-1 Front Panel Ports
2-3 USB devices on a FMM
4-7 Not used
8-11 USB devices on an RTM (connect to Zone 3)
12-13 Not used
Table 2.6 USB Ports C on the MIC-5332
2.2.8 Real-time Clock (RTC)
Because there is no battery assembled on the MIC-5332, the integrated real-time
clock is fed by IPMC management power. Due to the on-board super cap, the date
and time can be kept for up to 2 hour periods of power loss
2.3 DDR3 DIMMs
2.3.1 Memory Characteristics
The MIC-5332 uses DDR3 VLP SDRAM. As shown in Figure 2.2, each CPU uses 4
channels on the MIC-5332, and each channel supports one un-buffered/registered
ECC VLP DIMM, for a total of 8 sockets. Supported memory characteristics on the