Cisco NCS 1002 User manual

Troubleshooting Guide for Cisco NCS 1002, IOS XR Release 6.3.x
Americas Headquarters
Cisco Systems, Inc.
170 West Tasman Drive
San Jose, CA 95134-1706
USA
http://www.cisco.com
Tel: [phone]
800 553-NETS (6387)
Fax: [phone]

THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS IN THIS MANUAL ARE SUBJECT TO CHANGE WITHOUT NOTICE. ALL STATEMENTS,
INFORMATION, AND RECOMMENDATIONS IN THIS MANUAL ARE BELIEVED TO BE ACCURATE BUT ARE PRESENTED WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED. USERS MUST TAKE FULL RESPONSIBILITY FOR THEIR APPLICATION OF ANY PRODUCTS.
THE SOFTWARE LICENSE AND LIMITED WARRANTY FOR THE ACCOMPANYING PRODUCT ARE SET FORTH IN THE INFORMATION PACKET THAT SHIPPED WITH
THE PRODUCT AND ARE INCORPORATED HEREIN BY THIS REFERENCE. IF YOU ARE UNABLE TO LOCATE THE SOFTWARE LICENSE OR LIMITED WARRANTY,
CONTACT YOUR CISCO REPRESENTATIVE FOR A COPY.
The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of California, Berkeley (UCB) as part of UCB's public domain version of
the UNIX operating system. All rights reserved. Copyright ©1981, Regents of the University of California.
NOTWITHSTANDING ANY OTHER WARRANTY HEREIN, ALL DOCUMENT FILES AND SOFTWARE OF THESE SUPPLIERS ARE PROVIDED “AS IS" WITH ALL FAULTS.
CISCO AND THE ABOVE-NAMED SUPPLIERS DISCLAIM ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING, WITHOUT LIMITATION, THOSE OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE.
IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING, WITHOUT
LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THIS MANUAL, EVEN IF CISCO OR ITS SUPPLIERS
HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
Any Internet Protocol (IP) addresses and phone numbers used in this document are not intended to be actual addresses and phone numbers. Any examples, command display output, network
topology diagrams, and other figures included in the document are shown for illustrative purposes only. Any use of actual IP addresses or phone numbers in illustrative content is unintentional
and coincidental.
Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this URL:
https://www.cisco.com/go/trademarks. Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership relationship
between Cisco and any other company. (1721R)
©2018 Cisco Systems, Inc. All rights reserved.

For information on obtaining documentation, using the Cisco Bug Search Tool (BST), submitting a service request, and gathering additional information, see What's New in Cisco Product
Documentation.
To receive new and revised Cisco technical content directly to your desktop, you can subscribe to the . RSS feeds are a free service.
©2018 Cisco Systems, Inc. All rights reserved.

CONTENTS
General Troubleshooting 1
CHAPTER 1
Validating and Troubleshooting Installation of the Software Packages 1
Troubleshooting Problems with Node 3
Node is Unreachable 3
Console and Node are Not Responsive 4
Verifying the Status of Hardware Modules 4
Verifying the Software Version 8
Troubleshooting the Management Interface 8
Troubleshooting Slice Provisioning 10
Troubleshooting Environmental Parameters 10
Troubleshooting Firmware Upgrade Failure 14
Verifying the Firmware Version 16
Troubleshooting Optical Connectivity 18
Using Loopbacks 18
Using Link Layer Discovery Protocol Snooping 20
Using Trail Trace Identifier 21
Troubleshooting the Trunk Port 23
Troubleshooting Breakout Ports 25
Troubleshooting Breakout Patch Panel 26
Troubleshooting a Failed Commit Configuration 26
Removing and Re-inserting DIMMs on the Controller Card 27
Verifying Wavelength and Channel Mapping for Optics Controllers 28
Verifying the Performance Monitoring Parameters of Controllers 29
Verifying and Troubleshooting Headless State Settings 31
Monitoring Headless Statistics 33
Using SNMP for Troubleshooting 34
Troubleshooting Guide for Cisco NCS 1002, IOS XR Release 6.3.x
v

Using Netconf for Troubleshooting 35
Verifying Alarms 38
Using Onboard Failure Logging 38
Capturing Logs 39
Verifying Process Details and Crash Dump 41
Alarm Troubleshooting 45
CHAPTER 2
Alarm Logical Objects 45
Alarm Severity 46
Alarms 46
ALL-FAN-TRAY-REMOVAL Alarm 46
Clear the ALL-FAN-TRAY-REMOVAL Alarm 46
CD Alarm 46
Clear the CD Alarm 47
DAISYDUKE-CPU-FPGA-PCIE-ERROR Alarm 47
Clear the DAISYDUKE-CPU-FPGA-PCIE-ERROR Alarm 47
DAISYDUKE-CPU-PROCESSOR-HOT Alarm 47
Clear the DAISYDUKE-CPU-PROCESSOR-HOT Alarm 47
DGD Alarm 48
Clear the DGD Alarm 48
EQPT-FAIL-SLICE Alarm 48
Clear the EQPT-FAIL-SLICE Alarm 48
FAM-FAULT-TAG-UNIT-UNSUPPORTED ALARM 49
Clear the FAM-FAULT-TAG-UNIT-UNSUPPORTED Alarm 49
FAM-FAULT-TAG-PMO-UNIT-UNSUPPORTED ALARM 49
Clear the FAM-FAULT-TAG-PMO-UNIT-UNSUPPORTED Alarm 49
FAN FAIL Alarm 49
Clear the FAN FAIL Alarm 50
FAN MISSING Alarm 50
Clear the FAN MISSING Alarm 50
FAN-TRAY-INSERT Alarm 50
Clear the FAN-TRAY-INSERT Alarm 50
FAN-TRAY-REMOVAL 51
Clear the FAN-TRAY-REMOVAL Alarm 51
Troubleshooting Guide for Cisco NCS 1002, IOS XR Release 6.3.x
vi
Contents

HI-LASERBIAS Alarm 51
Clear the HI-LASERBIAS Alarm 51
HI-RXPOWER Alarm 52
Clear the HI-RXPOWER Alarm 52
HI-TXPOWER Alarm 52
Clear the HI-TXPOWER Alarm 52
HIBER Alarm 53
Clear the HIBER Alarm 53
IMPROPRMVL Alarm 53
Clear the IMPROPRMVL Alarm 53
LO-RXPOWER Alarm 54
Clear the LO-RXPOWER Alarm 54
LO-TXPOWER Alarm 55
Clear the LO-TXPOWER Alarm 55
LOCAL-FAULT Alarm 55
Clear the LOCAL-FAULT Alarm 55
LOM Alarm 56
Clear the LOM Alarm 56
LOS-P Alarm 56
Clear the LOS-P Alarm 56
LOWER-CTRL-FPGA-PCIE-ERROR 57
Clear the LOWER-CTRL-FPGA-PCIE-ERROR Alarm 57
MEA Alarm 57
Clear the MEA Alarm 57
MIXED-AC-DC-PTS Alarm 58
Clear the MIXED-AC-DC-PTS Alarm 58
MKA-LOCALLY-SECURED Alarm 58
Clear the MKA-LOCALLY-SECURED Alarm 58
MKA-START-FAIL-MISSING-PSK Alarm 58
Clear the MKA-START-FAIL-MISSING-PSK Alarm 59
MKA-UNSECURE-MISSING-PSK Alarm 59
Clear the MKA-UNSECURE-MISSING-PSK Alarm 59
MORE-THAN-ONE-FAN-TRAY-REMOVED Alarm 59
Clear the MORE-THAN-ONE-FAN-TRAY-REMOVED Alarm 59
Troubleshooting Guide for Cisco NCS 1002, IOS XR Release 6.3.x
vii
Contents

NM-I2C-ACCESS-ERROR Alarm 59
Clear the NM-I2C-ACCESS-ERROR Alarm 60
NM-SHUTDOWN-CARD Alarm 60
Clear the NM-SHUTDOWN-CARD Alarm 60
NOT ABLE TO COMMUNICATE WITH PATCH-PANEL Alarm 60
Clear the NOT ABLE TO COMMUNICATE WITH PATCH-PANEL Alarm 60
OSNR Alarm 60
Clear the OSNR Alarm 61
OTUK-AIS Alarm 61
Clear the OTUK-AIS Alarm 61
OTUK-BDI Alarm 61
Clear the OTUK-BDI Alarm 62
OTUK-LOF Alarm 62
Clear the OTUK-LOF Alarm 62
OTUK-SD Alarm 62
Clear the OTUK SD Alarm 63
OTUK-SF Alarm 63
Clear the OTUK SF Alarm 63
OTUK-TIM Alarm 63
Clear the OTUK-TIM Alarm 63
PATCH-PANEL POWER REDUNDANCY LOST ALARM 63
Clear the PATCH-PANEL POWER REDUNDANCY LOST Alarm 64
PEM-MODULE-INSERT Alarm 64
Clear the PEM-MODULE-INSERT Alarm 64
PEM-MODULE-REMOVAL Alarm 64
Clear the PEM-MODULE-REMOVAL Alarm 64
POWER MODULE OUTPUT DISABLED Alarm 64
Clear the POWER MODULE OUTPUT DISABLED Alarm 65
PEM-PWR-TRAY-LVL-RED-LOST 65
Clear the PEM-PWR-TRAY-LVL-RED-LOST Alarm 65
PLL-FAIL-SLICE Alarm 65
Clear the PLL-FAIL-SLICE Alarm 65
PPM FAIL Alarm 66
Clear the PPM FAIL Alarm 66
Troubleshooting Guide for Cisco NCS 1002, IOS XR Release 6.3.x
viii
Contents

PKG-K9SEC-REQ Alarm 66
Clear the PKG-K9SEC-REQ Alarm 66
PSU FAIL Alarm 66
Clear the PSU FAIL Alarm 66
PSU INSERTED Alarm 67
Clear the PSU INSERTED Alarm 67
PSU REDUNDANCY LOST Alarm 67
Clear the PSU REDUNDANCY LOST Alarm 67
REMOTE-FAULT Alarm 67
Clear the REMOTE-FAULT Alarm 67
RM-DEVICE-OUT-OF-TOL Alarm 68
Clear the RM-DEVICE-OUT-OF-TOL Alarm 68
SIGLOSS Alarm 68
Clear the SIGLOSS Alarm 68
SYNCLOSS Alarm 68
Clear the SYNCLOSS Alarm 69
SQUELCHED Alarm 69
Clear the SQUELCHED Alarm 70
TEMPERATURE Alarm 70
Clear the TEMPERATURE Alarm 71
TX-POWER-PROV-MISMATCH Alarm 71
Clear the TX-POWER-PROV-MISMATCH Alarm 71
UNC-WORD Alarm 72
Clear the UNC-WORD Alarm 72
UPPER-CTRL-FPGA-PCIE-ERROR Alarm 72
Clear the UPPER-CTRL-FPGA-PCIE-ERROR Alarm 72
VOLTAGE Alarms 73
Clear the VOLTAGE Alarms 73
WVL-OUT-OF-LOCK Alarm 73
Clear the WVL-OUT-OF-LOCK Alarm 74
Troubleshooting Guide for Cisco NCS 1002, IOS XR Release 6.3.x
ix
Contents

Troubleshooting Guide for Cisco NCS 1002, IOS XR Release 6.3.x
x
Contents

CHAPTER 1
General Troubleshooting
This chapter provides procedures for troubleshooting the most common problems encountered when operating
an NCS 1002. To troubleshoot specific alarms, see Alarm Troubleshooting, on page 45. If you cannot find
what you are looking for, contact Cisco Technical Support (1 800 553-2447).
This chapter includes the following sections:
•Validating and Troubleshooting Installation of the Software Packages, on page 1
•Troubleshooting Problems with Node, on page 3
•Troubleshooting the Management Interface, on page 8
•Troubleshooting Slice Provisioning, on page 10
•Troubleshooting Environmental Parameters, on page 10
•Troubleshooting Firmware Upgrade Failure, on page 14
•Troubleshooting Optical Connectivity, on page 18
•Troubleshooting the Trunk Port, on page 23
•Troubleshooting Breakout Ports, on page 25
•Troubleshooting Breakout Patch Panel, on page 26
•Troubleshooting a Failed Commit Configuration, on page 26
•Removing and Re-inserting DIMMs on the Controller Card, on page 27
•Verifying Wavelength and Channel Mapping for Optics Controllers, on page 28
•Verifying the Performance Monitoring Parameters of Controllers, on page 29
•Verifying and Troubleshooting Headless State Settings, on page 31
•Using SNMP for Troubleshooting, on page 34
•Using Netconf for Troubleshooting, on page 35
•Verifying Alarms, on page 38
•Using Onboard Failure Logging, on page 38
•Capturing Logs, on page 39
•Verifying Process Details and Crash Dump, on page 41
Validating and Troubleshooting Installation of the Software
Packages
Step 1 show version
Troubleshooting Guide for Cisco NCS 1002, IOS XR Release 6.3.x
1

Displays the software version and details such as system uptime.
Example:
RP/0/RP0/CPU0:ios# show version
Wed Nov 11 06:08:46.785 UTC
Cisco IOS XR Software, Version 6.0.0.22I
Copyright (c) 2013-2015 by Cisco Systems, Inc.
Build Information:
Built By : xxxxx
Built On : Fri Nov 13 17:08:39 IST 2015
Build Host : agl-ads-111
Workspace : /nobackup/xxxxx/idprom
Version : 6.0.0.22I
Location : /opt/cisco/XR/packages/
cisco NCS1K () processor
System uptime is 3 hours, 3 minutes
Step 2 show install repository
Displays a list of all the installed software packages on the NCS 1002.
Example:
RP/0/RP0/CPU0:ios# show install repository
Wed Nov 11 06:05:33.699 UTC
1 package(s) in XR repository:
ncs1k-xr-6.0.0.22I
Step 3 show install active
Displays a list of all the installed and active software packages on the NCS 1002.
The following sample output displays active software packages in the EXEC mode.
Example:
RP/0/RP0/CPU0:ios# show install active
Wed Nov 11 06:06:40.221 UTC
Node 0/RP0/CPU0 [RP]
Boot Partition: xr_lv0
Active Packages: 1
ncs1k-xr-6.0.0.22I version=6.0.0.22I [Boot image]
The following sample output displays active software packages in the system admin EXEC mode.
sysadmin-vm:0_RP0# show install active
Wed Nov 11 06:06:47.804 UTC
Node 0/RP0 [RP]
Active Packages: 1
ncs1k-sysadmin-6.0.0.22I version=6.0.0.22I [Boot image]
Step 4 show install committed
Displays a list of all committed software packages on the NCS 1002.
The committed software packages are the software packages that are booted on an NCS 1002 reload. Committed packages
are the packages that are persistent across reloads. If you install and activate a package, it remains active until the next
reload. If you commit a package set, all packages in that set remain active across reloads until the package set is replaced
with another committed package set.
Troubleshooting Guide for Cisco NCS 1002, IOS XR Release 6.3.x
2
General Troubleshooting
Validating and Troubleshooting Installation of the Software Packages

The following sample output displays the committed software packages in the EXEC mode.
Example:
RP/0/RP0/CPU0:ios# show install committed
Wed Nov 11 06:07:53.181 UTC
Node 0/RP0/CPU0 [RP]
Boot Partition: xr_lv0
Committed Packages: 1
ncs1k-xr-6.0.0.22I version=6.0.0.22I [Boot image]
The following sample output displays the committed software packages in the system admin EXEC mode.
sysadmin-vm:0_RP0# show install committed
Wed Nov 11 06:08:02.409 UTC
Node 0/RP0 [RP]
Committed Packages: 1
ncs1k-sysadmin-6.0.0.22I version=6.0.0.22I [Boot image]
Step 5 show install log
Displays information on the history of the installation operations. This command provides information about both
successful and failed installation operations on the NCS 1002. You can also verify a Service Maintenance Update (SMU)
installation using this command.
Example:
RP/0/RP0/CPU0:ios# show install log 49 detail
Wed Dec 9 01:19:18.680 UTC
Dec 09 01:19:07 Install operation 49 started by root:
install add source tftp://10.105.236.167 ncs1k-k9sec.rpm
Dec 09 01:19:08 Action 1: install add action started
Dec 09 01:19:08 ERROR! Either file is not proper or error in getting rpm metadata from rpm file
Dec 09 01:19:08 ERROR!! failed to complete install add precheck
Dec 09 01:19:09 Install operation 49 aborted
Dec 09 01:19:10 Ending operation 49
In the above example, either a wrong rpm package is used or the rpm package is corrupted.
For failure on install add source, check that the package is correctly named and is available at the location.
What to do next
If the expected active software packages are not displayed, install the packages (if required) and activate the
packages using the install activate package_name command.
Troubleshooting Problems with Node
Node is Unreachable
Step 1 Verify cable connectivity.
Step 2 Verify that the power supply is on.
Troubleshooting Guide for Cisco NCS 1002, IOS XR Release 6.3.x
3
General Troubleshooting
Troubleshooting Problems with Node

Step 3 Reboot the NCS 1002.
Step 4 Verify the hardware module and inventory data. For more information, see Verifying the Status of Hardware Modules,
on page 4.
Console and Node are Not Responsive
Console problems occur when the NCS 1002 becomes unresponsive to an input at the console port. If the
console is not responsive, it means that a high priority process prevents the console driver from responding
to input.
Step 1 Verify cable connectivity.
Step 2 Verify that the power supply is on.
Step 3 Verify the NCS 1002 LED status. If all LEDs are down, it might be an issue with the power supply.
Step 4 Verify that the CPU is inserted properly.
Step 5 Reboot the NCS 1002.
Verifying the Status of Hardware Modules
You can verify the state of the hardware modules in the following scenarios:
• Node is not reachable.
• Node recovers from a problem.
• Node had a power cycle.
• Node reboot.
• Node upgrade.
• Node settles down after the Cisco IOS XR has continuously reloaded.
Step 1 show platform
When you execute this command from the Cisco IOS XR EXEC mode, the status of the Cisco IOS XR is displayed.
Verify that the node state is Operational and the admin state is UP.
Example:
RP/0/RP0/CPU0:ios# show platform
Wed Nov 11 01:22:28.953 UTC
Node name Node type Node state Admin state Config state
-----------------------------------------------------------------------------------
0/RP0 NCS1K-CNTLR-K9 OPERATIONAL UP NSHUT
a) If the Cisco IOS XR is not operational, no output is shown in the result. In this case, verify the state of service domain
router (SDR) on the node using the show sdr command.
Troubleshooting Guide for Cisco NCS 1002, IOS XR Release 6.3.x
4
General Troubleshooting
Console and Node are Not Responsive

The following example shows sample output from the show sdr command in Cisco IOS XR EXEC mode.
RP/0/RP0/CPU0:ios# show sdr
RP/0/RP0/CPU0:ios#sh sdr
Tue Nov 10 22:57:20.921 UTC
Type NodeName NodeState RedState PartnerName
--------------------------------------------------------------------------------
RP 0/RP0/CPU0 IOS XR RUN ACTIVE NONE
NCS1K-CNTLR-K9 0/RP0 OPERATIONAL N/A
The following example shows sample output from the show sdr command in system admin EXEC mode.
sysadmin-vm:0_RP0# show sdr
Tue Nov 10 22:56:41.225 UTC
sdr default-sdr
location 0/RP0/VM1
sdr-id 2
IP Address of VM 192.0.0.4
MAC address of VM E2:3A:DD:0A:8D:03
VM State RUNNING
start-time 2020-11-06T10:41:52.340092+00:00
Last Reload Reason FIRST_BOOT
Reboot Count 1
Step 2 admin
Enters system admin EXEC mode.
Example:
RP/0/RP0/CPU0:ios# admin
Step 3 show platform
Displays information and status for each node in the system.
Example:
sysadmin-vm:0_RP0# show platform
Tue Feb 27 10:26:58.763 UTC
Location Card Type HW State SW State Config State
----------------------------------------------------------------------------
0/0 NCS1002 OPERATIONAL N/A NSHUT
0/RP0 NCS1002--RP OPERATIONAL OPERATIONAL NSHUT
0/FT0 NCS1K-FTA OPERATIONAL N/A NSHUT
0/FT1 NCS1K-FTA OPERATIONAL N/A NSHUT
Verify that all the modules of the NCS 1002 are displayed in the result. The software state and the hardware state must
be OPERATIONAL.
The various hardware and software states are:
Hardware states:
• OPERATIONAL—Node is operating normally and is fully functional
• POWERED_ON—Power is on and the node is booting up
• FAILED—Node is powered on but has experienced some internal failure
• PRESENT—Node is in the shutdown state
Troubleshooting Guide for Cisco NCS 1002, IOS XR Release 6.3.x
5
General Troubleshooting
Verifying the Status of Hardware Modules

• OFFLINE—User has changed the node state to OFFLINE. The node is accessible for diagnostics
Software states:
• OPERATIONAL—Software is operating normally and is fully functional
• SW_INACTIVE—Software is not completely operational
• FAILED—Software is operational but the card has experienced some internal failure
Step 4 show platform detail
Displays the hardware and software states, and other details of the node.
Example:
sysadmin-vm:0_RP0# show platform detail
Wed Aug 5 09:49:06.521 UTC
Platform Information for 0/0
PID : NCS1002
Description : "Network Convergence System 1000 Controller"
VID/SN : V01
HW Oper State : OPERATIONAL
SW Oper State : N/A
Configuration : "NSHUT RST"
HW Version : 0.1
Last Event : HW_EVENT_OK
Last Event Reason : "HW Event OK"
Platform Information for 0/RP0
PID : NCS1002--RP
Description : "Network Convergence System 1000 Controller"
VID/SN : V01
HW Oper State : OPERATIONAL
SW Oper State : OPERATIONAL
Configuration : "NSHUT RST"
HW Version : 0.1
Last Event : UNKNOWN
Last Event Reason : UNKNOWN
Step 5 show inventory
Displays the details of the physical entities of the NCS 1002 along with the details of QSFPs and CFPs when you execute
this command in the Cisco IOS XR EXEC mode.
You can verify if any QSFP or CFP has been removed from the NCS 1002.
Example:
RP/0/RP0/CPU0:ios# show inventory
RP/0/RP0/CPU0:ios#show inventory
Fri May 18 10:46:51.323 UTC
NAME: "0/0", DESCR: "Network Convergence System 1002 20 QSFP28/QSFP+ slots"
PID: NCS1002-K9 , VID: V03, SN: CAT2116B170
NAME: "0/0-Optics0/0/0/1", DESCR: "Non-Cisco QSFP28 100G LR4 Pluggable Optics Module"
PID: SPQCELRCDFB , VID: 01 , SN: G9I2011804
NAME: "0/0-Optics0/0/0/4", DESCR: "Non-Cisco QSFP28 100G LR4 Pluggable Optics Module"
PID: TR-FC13L-N00 , VID: 01 , SN: INGAJ0930306
NAME: "0/0-Optics0/0/0/6", DESCR: "Cisco CFP2 DWDM Pluggable Optics"
PID: ONS-CFP2-WDM , VID: V01 , SN: OUK1936006S
Troubleshooting Guide for Cisco NCS 1002, IOS XR Release 6.3.x
6
General Troubleshooting
Verifying the Status of Hardware Modules

NAME: "0/0-Optics0/0/0/7", DESCR: "Cisco 4x10GE QSFP+ LR-S Pluggable Optics Module"
PID: QSFP-4X10G-LR-S , VID: V02 , SN: INL20410069
NAME: "0/0-Optics0/0/0/8-LANE1", DESCR: "Cisco 10G SFP LR Pluggable Optics Module"
PID: SFP-10G-LR , VID: V01 , SN: SPC1907074N
NAME: "0/0-Optics0/0/0/9", DESCR: "Cisco 40GE QSFP+ SR4 Pluggable Optics Module"
PID: QSFP-40G-SR4 , VID: V03 , SN: JFQ20332088
NAME: "0/0-Optics0/0/0/10", DESCR: "Non-Cisco QSFP28 100G LR4 Pluggable Optics Module"
PID: SPQCELRCDFB , VID: 01 , SN: GAV2008935
NAME: "0/0-Optics0/0/0/11-LANE1", DESCR: "Cisco 10G SFP LR Pluggable Optics Module"
PID: SFP-10G-LR , VID: V01 , SN: SPC190707YP
NAME: "0/0-Optics0/0/0/17-LANE1", DESCR: "Cisco 10G SFP SR Pluggable Optics Module"
PID: SFP-10G-SR , VID: V03 , SN: JUR1904073P
NAME: "0/0-Optics0/0/0/18", DESCR: "Non-Cisco QSFP28 100G LR4 Pluggable Optics Module"
PID: FTLC1151RDPL , VID: A0 , SN: UVE1C6C
NAME: "0/0-Optics0/0/0/19", DESCR: "Cisco CFP2 DWDM Pluggable Optics"
PID: ONS-CFP2-WDM , VID: V05 , SN: OVE204404PA
NAME: "0/0-Optics0/0/0/21", DESCR: "Cisco 4x10GE QSFP+ LR-S Pluggable Optics Module"
PID: QSFP-4x10G-LR-S , VID: V01 , SN: INL20200012
NAME: "0/0-Optics0/0/0/22-LANE1", DESCR: "Cisco 10G SFP LR Pluggable Optics Module"
PID: SFP-10G-LR , VID: V01 , SN: SPC190707YS
NAME: "0/0-Optics0/0/0/23", DESCR: "Cisco 40GE QSFP+ SR4 Pluggable Optics Module"
PID: QSFP-40G-SR4 , VID: V03 , SN: JFQ2033201H
NAME: "0/0-Optics0/0/0/24", DESCR: "Non-Cisco QSFP28 100G LR4 Pluggable Optics Module"
PID: FTLC1151RDPL , VID: A0 , SN: UWD2QMM
NAME: "0/0-Optics0/0/0/25-LANE1", DESCR: "Cisco 10G SFP ER Pluggable Optics Module"
PID: SFP-10G-ER , VID: V02 , SN: ONT213100BW
NAME: "0/RP0", DESCR: "Network Convergence System 1000 Controller"
PID: NCS1K-CNTLR , VID: V04, SN: CAT2052B0FZ
NAME: "Rack 0", DESCR: "Network Convergence System 1002 20 QSFP28/QSFP+ slots"
PID: NCS1002-K9 , VID: V03, SN: CAT2116B170
NAME: "0/FT0", DESCR: "Network Convergence System 1000 Fan"
PID: NCS1K-FTA , VID: V01, SN: N/A
NAME: "0/FT1", DESCR: "Network Convergence System 1000 Fan"
PID: NCS1K-FTA , VID: V01, SN: N/A
NAME: "0/FT2", DESCR: "Network Convergence System 1000 Fan"
PID: NCS1K-FTA , VID: V01, SN: N/A
NAME: "0/PM0", DESCR: "Network Convergence System 1000 2KW AC PSU"
PID: NCS1K-2KW-AC , VID: V01, SN: POG2041J0BW
NAME: "0/PM1", DESCR: "Network Convergence System 1000 2KW AC PSU"
PID: NCS1K-2KW-AC , VID: V01, SN: POG2041J01C
Troubleshooting Guide for Cisco NCS 1002, IOS XR Release 6.3.x
7
General Troubleshooting
Verifying the Status of Hardware Modules

What to do next
Verify the software version of the NCS 1002. For more information, see Verifying the Software Version, on
page 8
Verifying the Software Version
The NCS 1002 is shipped with a pre-installed Cisco IOS XR software. Verify that the latest version of the
software is installed. If a newer version is available, perform a system upgrade. This will install the newer
version of the software and provide the latest feature set on the NCS 1002.
show version
Displays the software version and details such as system uptime in the Cisco IOS XR EXEC mode.
Example:
RP/0/RP0/CPU0:ios# show version
Tue Nov 10 23:02:37.683 UTC
Cisco IOS XR Software, Version 6.0.0.26I
Copyright (c) 2013-2015 by Cisco Systems, Inc.
Build Information:
Built By : xxxx
Built On : Tue Dec 1 17:02:18 PST 2015
Build Host : build-lnx-100
Workspace : /auto/build-lnx-106-san1/r60x-ws6/nightly_r60x/151201B_ncs1k/workspace
Version : 6.0.0.26I
Location : /opt/cisco/XR/packages/
cisco NCS1K () processor
System uptime is 4 days, 12 hours, 20 minutes
What to do next
Verify the result to ascertain whether a system upgrade is required. If the upgrade is required, see the System
Setup and Software Installation Guide for Cisco NCS 1000 Series.
Troubleshooting the Management Interface
Before you begin
Management interface should be configured.
Step 1 show interfaces mgmtEth instance
Displays the management interface configuration.
Example:
Troubleshooting Guide for Cisco NCS 1002, IOS XR Release 6.3.x
8
General Troubleshooting
Verifying the Software Version

RP/0/RP0/CPU0:ios# show interfaces MgmtEth 0/RP0/CPU0/0
Fri Nov 13 19:42:29.716 UTC
MgmtEth0/RP0/CPU0/0 is administratively down, line protocol is administratively down
Interface state transitions: 0
Hardware is Management Ethernet, address is badb.adba.d098 (bia badb.adba.d098)
Internet address is 10.58.227.183/24
MTU 1514 bytes, BW 100000 Kbit (Max: 100000 Kbit)
reliability 255/255, txload 0/255, rxload 0/255
Encapsulation ARPA,
Full-duplex, 100Mb/s, CX, link type is autonegotiation
loopback not set,
ARP type ARPA, ARP timeout 04:00:00
Last input never, output never
Last clearing of "show interface" counters never
5 minute input rate 0 bits/sec, 0 packets/sec
5 minute output rate 0 bits/sec, 0 packets/sec
0 packets input, 0 bytes, 0 total input drops
0 drops for unrecognized upper-level protocol
Received 0 broadcast packets, 0 multicast packets
0 runts, 0 giants, 0 throttles, 0 parity
0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
0 packets output, 0 bytes, 0 total output drops
Output 0 broadcast packets, 0 multicast packets
0 output errors, 0 underruns, 0 applique, 0 resets
0 output buffer failures, 0 output buffers swapped out
0 carrier transitions
a) In the above result, the management interface is administratively down. Use the no shut command to enable the
management interface.
The following example shows sample output from the show running-config interface mgmtEth command when
the management interface is in the no shut state.
RP/0/RP0/CPU0:ios#show running-config interface mgmtEth 0/RP0/CPU0/0
Fri Nov 13 19:42:54.368 UTC
interface MgmtEth0/RP0/CPU0/0
ipv4 address 10.58.227.183 255.255.255.0
!
You can also use the show interfaces summary and show interfaces brief commands in the Cisco IOS XR EXEC mode
to verify the management interface status.
• The following example shows sample output from the show interfaces summary command.
RP/0/RP0/CPU0:ios# show interfaces summary
Sun Nov 15 19:31:46.469 UTC
Interface Type Total UP Down Admin Down
-------------- ----- -- ---- ----------
ALL TYPES 2 2 0 0
--------------
IFT_ETHERNET 1 1 0 0
IFT_NULL 1 1 0 0
• The following example shows sample output from the show interfaces brief command.
RP/0/RP0/CPU0:ios# show interfaces brief
Sun Nov 15 19:31:41.806 UTC
Intf Intf LineP Encap MTU BW
Name State State Type (byte) (Kbps)
--------------------------------------------------------------------------------
Nu0 up up Null 1500 0
Troubleshooting Guide for Cisco NCS 1002, IOS XR Release 6.3.x
9
General Troubleshooting
Troubleshooting the Management Interface

Mg0/RP0/CPU0/0 up up ARPA 1514 100000
Step 2 When the line protocol is down, you must verify the Layer 3 connectivity. You can perform the following steps.
a) Check the Ethernet cable connection and physical connectivity of the NCS 1002 to get the line protocol up.
b) Ensure ARP connectivity.
c) Use the ping command to check reachability and network connectivity on the IP network.
d) Verify the static IP and default gateway configuration.
Troubleshooting Slice Provisioning
Step 1 show hw-module slice slicenumber
Displays details of the slice provisioning.
Example:
RP/0/RP0/CPU0:ios# show hw-module slice 3
Fri Nov 6 10:12:16.684 UTC
Slice ID: 3
Status:Provisioning Failed [ETNA Config Failure]
Client Bitrate: 100
Trunk Bitrate: 100
In the above example, the slice provisioning has failed because of an ETNA configuration failure.
Some of the failure reasons that can appear in the command output are:
• CFG INIT Config Failure—Indicates DP FPGA download issue.
• PPM Config Failure—Indicates PPM Driver failure.
• CDR Config Failure—Indicates CDR failure.
• ETNA Config Failure—Indicates ETNA device or programming failure.
• DP FPGA Config Failure—Indicates DP FPGA programming failure.
Step 2 upgrade hw-module slice slice_number re-provision
Reprovisions the slice.
Step 3 Reload the Cisco IOS-XR if reprovisioning the slice does not work.
Troubleshooting Environmental Parameters
Some of the common environmental problems are listed below.
• Fan failure
Troubleshooting Guide for Cisco NCS 1002, IOS XR Release 6.3.x
10
General Troubleshooting
Troubleshooting Slice Provisioning