Cisco NCS 1002 User manual

Troubleshooting Guide for Cisco NCS 1002, IOS XR Release 6.3.x
Americas Headquarters
Cisco Systems, Inc.
170 West Tasman Drive
San Jose, CA 95134-1706
USA
http://www.cisco.com
Tel: [phone]
800 553-NETS (6387)
Fax: [phone]

THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS IN THIS MANUAL ARE SUBJECT TO CHANGE WITHOUT NOTICE. ALL STATEMENTS,
INFORMATION, AND RECOMMENDATIONS IN THIS MANUAL ARE BELIEVED TO BE ACCURATE BUT ARE PRESENTED WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED. USERS MUST TAKE FULL RESPONSIBILITY FOR THEIR APPLICATION OF ANY PRODUCTS.
THE SOFTWARE LICENSE AND LIMITED WARRANTY FOR THE ACCOMPANYING PRODUCT ARE SET FORTH IN THE INFORMATION PACKET THAT SHIPPED WITH
THE PRODUCT AND ARE INCORPORATED HEREIN BY THIS REFERENCE. IF YOU ARE UNABLE TO LOCATE THE SOFTWARE LICENSE OR LIMITED WARRANTY,
CONTACT YOUR CISCO REPRESENTATIVE FOR A COPY.
The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of California, Berkeley (UCB) as part of UCB's public domain version of
the UNIX operating system. All rights reserved. Copyright ©1981, Regents of the University of California.
NOTWITHSTANDING ANY OTHER WARRANTY HEREIN, ALL DOCUMENT FILES AND SOFTWARE OF THESE SUPPLIERS ARE PROVIDED “AS IS" WITH ALL FAULTS.
CISCO AND THE ABOVE-NAMED SUPPLIERS DISCLAIM ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING, WITHOUT LIMITATION, THOSE OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE.
IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING, WITHOUT
LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THIS MANUAL, EVEN IF CISCO OR ITS SUPPLIERS
HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
Any Internet Protocol (IP) addresses and phone numbers used in this document are not intended to be actual addresses and phone numbers. Any examples, command display output, network
topology diagrams, and other figures included in the document are shown for illustrative purposes only. Any use of actual IP addresses or phone numbers in illustrative content is unintentional
and coincidental.
Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this URL:
https://www.cisco.com/go/trademarks. Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership relationship
between Cisco and any other company. (1721R)
©2018 Cisco Systems, Inc. All rights reserved.

For information on obtaining documentation, using the Cisco Bug Search Tool (BST), submitting a service request, and gathering additional information, see What's New in Cisco Product
Documentation.
To receive new and revised Cisco technical content directly to your desktop, you can subscribe to the . RSS feeds are a free service.
©2018 Cisco Systems, Inc. All rights reserved.

CONTENTS
General Troubleshooting 1
CHAPTER 1
Validating and Troubleshooting Installation of the Software Packages 1
Troubleshooting Problems with Node 3
Node is Unreachable 3
Console and Node are Not Responsive 4
Verifying the Status of Hardware Modules 4
Verifying the Software Version 8
Troubleshooting the Management Interface 8
Troubleshooting Slice Provisioning 10
Troubleshooting Environmental Parameters 10
Troubleshooting Firmware Upgrade Failure 14
Verifying the Firmware Version 16
Troubleshooting Optical Connectivity 18
Using Loopbacks 18
Using Link Layer Discovery Protocol Snooping 20
Using Trail Trace Identifier 21
Troubleshooting the Trunk Port 23
Troubleshooting Breakout Ports 25
Troubleshooting Breakout Patch Panel 26
Troubleshooting a Failed Commit Configuration 26
Removing and Re-inserting DIMMs on the Controller Card 27
Verifying Wavelength and Channel Mapping for Optics Controllers 28
Verifying the Performance Monitoring Parameters of Controllers 29
Verifying and Troubleshooting Headless State Settings 31
Monitoring Headless Statistics 33
Using SNMP for Troubleshooting 34
Troubleshooting Guide for Cisco NCS 1002, IOS XR Release 6.3.x
v

Using Netconf for Troubleshooting 35
Verifying Alarms 38
Using Onboard Failure Logging 38
Capturing Logs 39
Verifying Process Details and Crash Dump 41
Alarm Troubleshooting 45
CHAPTER 2
Alarm Logical Objects 45
Alarm Severity 46
Alarms 46
ALL-FAN-TRAY-REMOVAL Alarm 46
Clear the ALL-FAN-TRAY-REMOVAL Alarm 46
CD Alarm 46
Clear the CD Alarm 47
DAISYDUKE-CPU-FPGA-PCIE-ERROR Alarm 47
Clear the DAISYDUKE-CPU-FPGA-PCIE-ERROR Alarm 47
DAISYDUKE-CPU-PROCESSOR-HOT Alarm 47
Clear the DAISYDUKE-CPU-PROCESSOR-HOT Alarm 47
DGD Alarm 48
Clear the DGD Alarm 48
EQPT-FAIL-SLICE Alarm 48
Clear the EQPT-FAIL-SLICE Alarm 48
FAM-FAULT-TAG-UNIT-UNSUPPORTED ALARM 49
Clear the FAM-FAULT-TAG-UNIT-UNSUPPORTED Alarm 49
FAM-FAULT-TAG-PMO-UNIT-UNSUPPORTED ALARM 49
Clear the FAM-FAULT-TAG-PMO-UNIT-UNSUPPORTED Alarm 49
FAN FAIL Alarm 49
Clear the FAN FAIL Alarm 50
FAN MISSING Alarm 50
Clear the FAN MISSING Alarm 50
FAN-TRAY-INSERT Alarm 50
Clear the FAN-TRAY-INSERT Alarm 50
FAN-TRAY-REMOVAL 51
Clear the FAN-TRAY-REMOVAL Alarm 51
Troubleshooting Guide for Cisco NCS 1002, IOS XR Release 6.3.x
vi
Contents

HI-LASERBIAS Alarm 51
Clear the HI-LASERBIAS Alarm 51
HI-RXPOWER Alarm 52
Clear the HI-RXPOWER Alarm 52
HI-TXPOWER Alarm 52
Clear the HI-TXPOWER Alarm 52
HIBER Alarm 53
Clear the HIBER Alarm 53
IMPROPRMVL Alarm 53
Clear the IMPROPRMVL Alarm 53
LO-RXPOWER Alarm 54
Clear the LO-RXPOWER Alarm 54
LO-TXPOWER Alarm 55
Clear the LO-TXPOWER Alarm 55
LOCAL-FAULT Alarm 55
Clear the LOCAL-FAULT Alarm 55
LOM Alarm 56
Clear the LOM Alarm 56
LOS-P Alarm 56
Clear the LOS-P Alarm 56
LOWER-CTRL-FPGA-PCIE-ERROR 57
Clear the LOWER-CTRL-FPGA-PCIE-ERROR Alarm 57
MEA Alarm 57
Clear the MEA Alarm 57
MIXED-AC-DC-PTS Alarm 58
Clear the MIXED-AC-DC-PTS Alarm 58
MKA-LOCALLY-SECURED Alarm 58
Clear the MKA-LOCALLY-SECURED Alarm 58
MKA-START-FAIL-MISSING-PSK Alarm 58
Clear the MKA-START-FAIL-MISSING-PSK Alarm 59
MKA-UNSECURE-MISSING-PSK Alarm 59
Clear the MKA-UNSECURE-MISSING-PSK Alarm 59
MORE-THAN-ONE-FAN-TRAY-REMOVED Alarm 59
Clear the MORE-THAN-ONE-FAN-TRAY-REMOVED Alarm 59
Troubleshooting Guide for Cisco NCS 1002, IOS XR Release 6.3.x
vii
Contents

NM-I2C-ACCESS-ERROR Alarm 59
Clear the NM-I2C-ACCESS-ERROR Alarm 60
NM-SHUTDOWN-CARD Alarm 60
Clear the NM-SHUTDOWN-CARD Alarm 60
NOT ABLE TO COMMUNICATE WITH PATCH-PANEL Alarm 60
Clear the NOT ABLE TO COMMUNICATE WITH PATCH-PANEL Alarm 60
OSNR Alarm 60
Clear the OSNR Alarm 61
OTUK-AIS Alarm 61
Clear the OTUK-AIS Alarm 61
OTUK-BDI Alarm 61
Clear the OTUK-BDI Alarm 62
OTUK-LOF Alarm 62
Clear the OTUK-LOF Alarm 62
OTUK-SD Alarm 62
Clear the OTUK SD Alarm 63
OTUK-SF Alarm 63
Clear the OTUK SF Alarm 63
OTUK-TIM Alarm 63
Clear the OTUK-TIM Alarm 63
PATCH-PANEL POWER REDUNDANCY LOST ALARM 63
Clear the PATCH-PANEL POWER REDUNDANCY LOST Alarm 64
PEM-MODULE-INSERT Alarm 64
Clear the PEM-MODULE-INSERT Alarm 64
PEM-MODULE-REMOVAL Alarm 64
Clear the PEM-MODULE-REMOVAL Alarm 64
POWER MODULE OUTPUT DISABLED Alarm 64
Clear the POWER MODULE OUTPUT DISABLED Alarm 65
PEM-PWR-TRAY-LVL-RED-LOST 65
Clear the PEM-PWR-TRAY-LVL-RED-LOST Alarm 65
PLL-FAIL-SLICE Alarm 65
Clear the PLL-FAIL-SLICE Alarm 65
PPM FAIL Alarm 66
Clear the PPM FAIL Alarm 66
Troubleshooting Guide for Cisco NCS 1002, IOS XR Release 6.3.x
viii
Contents

PKG-K9SEC-REQ Alarm 66
Clear the PKG-K9SEC-REQ Alarm 66
PSU FAIL Alarm 66
Clear the PSU FAIL Alarm 66
PSU INSERTED Alarm 67
Clear the PSU INSERTED Alarm 67
PSU REDUNDANCY LOST Alarm 67
Clear the PSU REDUNDANCY LOST Alarm 67
REMOTE-FAULT Alarm 67
Clear the REMOTE-FAULT Alarm 67
RM-DEVICE-OUT-OF-TOL Alarm 68
Clear the RM-DEVICE-OUT-OF-TOL Alarm 68
SIGLOSS Alarm 68
Clear the SIGLOSS Alarm 68
SYNCLOSS Alarm 68
Clear the SYNCLOSS Alarm 69
SQUELCHED Alarm 69
Clear the SQUELCHED Alarm 70
TEMPERATURE Alarm 70
Clear the TEMPERATURE Alarm 71
TX-POWER-PROV-MISMATCH Alarm 71
Clear the TX-POWER-PROV-MISMATCH Alarm 71
UNC-WORD Alarm 72
Clear the UNC-WORD Alarm 72
UPPER-CTRL-FPGA-PCIE-ERROR Alarm 72
Clear the UPPER-CTRL-FPGA-PCIE-ERROR Alarm 72
VOLTAGE Alarms 73
Clear the VOLTAGE Alarms 73
WVL-OUT-OF-LOCK Alarm 73
Clear the WVL-OUT-OF-LOCK Alarm 74
Troubleshooting Guide for Cisco NCS 1002, IOS XR Release 6.3.x
ix
Contents

Troubleshooting Guide for Cisco NCS 1002, IOS XR Release 6.3.x
x
Contents