Nortel 3050 User manual

Nortel VPN Gateway
Troubleshooting Guide
Release: 7.0
Document Revision: 01.01
www.nortel.com
NN46120-700 324371-A
.

Nortel VPN Gateway
Release: 7.0
Publication: NN46120-700
Document status: Standard
Document release date: 12 October 2007
Copyright © 2007 Nortel Networks
All Rights Reserved.
The information in this document is subject to change without notice. The statements, configurations, technical
data, and recommendations in this document are believed to be accurate and reliable, but are presented without
express or implied warranty. Users must take full responsibility for their applications of any products specified in this
document. The information in this document is proprietary to Nortel Networks.
Export
This product, software and related technology is subject to U.S. export control and may be subject to export or
import regulations in other countries. Purchaser must strictly comply with all such laws and regulations. A license to
export or reexport may be required by the U.S. Department of Commerce.
Licensing
This product includes software developed by the OpenSSL Project for use in the OpenSSL Toolkit
(http://www.openssl.org/).
This product includes cryptographic software written by Eric Young ([email protected]).
This product includes software written by Tim Hudson ([email protected]).
This product includes software developed by the Apache Software Foundation http://www.apache.org/.
This product includes a TAP-Win32 driver derived from the CIPE-Win32 kernel driver, Copyright © Damion K.
Wilson, and is licensed under the GPL.
See Appendix D, "License Information", in the User’s Guide for more information
.

3
.
Contents
New in this release 5
Introduction 7
Prerequisites 7
Navigation 7
Acronyms 8
Troubleshooting Fundamentals 9
Navigation 9
SSL acceleration 9
Log files 10
Traffic Generators 11
NVG hardware 12
Virtual IP addresses 12
Global troubleshooting tasks 13
Navigation 13
Dumping the log files to another server 13
Displaying the audit trials 14
Viewing SSL traffic generators 14
Viewing TCP traffic generators 14
Feature-specific troubleshooting tasks 17
Navigation 17
Troubleshooting SSL acceleration 17
SSL acceleration troubleshooting tools 18
SSL acceleration troubleshooting tools navigation 18
Viewing current configuration 18
Resetting default configuration 18
Troubleshooting Net Direct 18
Troubleshooting TunnelGuard 19
Recovering using boot.img 20
Upgrading code using .pkg 21
Troubleshooting authentication tasks 25
Navigation 25
Nortel VPN Gateway
Troubleshooting Guide
NN46120-700 01.01 Standard
12 October 2007
Copyright © 2007 Nortel Networks
.

4
Troubleshooting RADIUS authentication 25
Troubleshooting RADIUS authentication navigation 25
Troubleshooting LDAP authentication with Active Directory 37
Troubleshooting LDAP authentication with Active Directory navigation 37
Troubleshooting LDAP authentication issues 37
Adding a SSL VPN gateway user into the Active Directory 38
Configuring the LDAP Attributes 40
Configuring LDAPs authentication with Active Directory 41
Importing certificates 44
Troubleshooting NTLM authentication with Primary Domain Controller 46
Troubleshooting NTLM authentication with Primary Domain Controller
navigation 46
Creating the Windows group and add a user into that group 46
Adding users to the new group 47
Emergency Recovery Trees 49
Cannot access NVG for management -- recovery tree 49
Cannot access VPN -- recovery tree 50
Reference to third party Application Guides 53
Contact Nortel technical support 55
Navigation 55
Gathering critical information 55
Getting help from the Nortel Web site 56
Getting help over the phone from a Nortel Solutions Center 56
Getting help from a specialist by using an Express Routing Code 57
Getting help through a Nortel distributor or reseller 57
Glossary 59
Nortel VPN Gateway
Troubleshooting Guide
NN46120-700 01.01 Standard
12 October 2007
Copyright © 2007 Nortel Networks
.

5
.
New in this release
Nortel VPN Gateway Troubleshooting Guide (Part number 324371-A,
NN46120-700) is a new document for Nortel VPN Gateway Release 7.0.
Some of the contents in this document originally appeared in the following
sources:
•Application Guide for SSL Acceleration (Part number 216370-D,
NN46120-100)
•CLI Application Guide (Part number 216371-E, NN46120-101)
•BBI Application Guide for VPN (Part number 217239-E, NN46120-102)
•Command Reference Guide (Part number 216369-E, NN46120-103)
•User Guide (Part number 216368-F, NN46120-104)
•Administrator Guide (Part number 217238-D, NN46120-105)
Nortel VPN Gateway
Troubleshooting Guide
NN46120-700 01.01 Standard
12 October 2007
Copyright © 2007 Nortel Networks
.

6New in this release
Nortel VPN Gateway
Troubleshooting Guide
NN46120-700 01.01 Standard
12 October 2007
Copyright © 2007 Nortel Networks
.

7
.
Introduction
This chapter describes the prerequisites and various tools used to
troubleshoot the Nortel VPN Gateway (NVG). Use the troubleshooting
tools to enhance overall performance, resolve error messages, and
increase response time for a specific feature.
Each tool is described by purpose, usage procedures, and how to interpret
the output.
Prerequisites
Nortel recommends you to use one or more of the following commercially
available troubleshooting tools as well as the tools described in this
document.
•Capture and analyze HTTP and HTTPS with the HTTP Analyzer from
IE Inspector http://www.ieinspector.com/
•Capture and analyze HTTP and HTTPS with Tamper Data, a plug-in
available for Mozilla Firefox https://addons.mozilla.org/en-US/firefox/
addon/966
•Display the time to load Web pages with Faster Fox, a plug-in available
for Mozilla Firefox https://addons.mozilla.org/en-US/firefox/addon/1269
•Capture and analyze packets with either Sniffer or Wireshark from
Network General http://www.wireshark.org/ and http://www.networkge
neral.com/
Navigation •“Troubleshooting Fundamentals” (page 9)
•“Global troubleshooting tasks” (page 13)
•“Feature-specific troubleshooting tasks” (page 17)
•“Troubleshooting authentication tasks” (page 25)
•“Emergency Recovery Trees” (page 49)
Nortel VPN Gateway
Troubleshooting Guide
NN46120-700 01.01 Standard
12 October 2007
Copyright © 2007 Nortel Networks
.

8Introduction
•“Reference to third party Application Guides” (page 53)
•“Contact Nortel technical support” (page 55)
•“Glossary” (page 59)
Acronyms Table 1 "Acronyms" (page 8) lists the acronyms used in this guide.
Table 1
Acronyms
BootP Bootstrap Protocol
CLI Command Line Interface
CPU Central Processing Unit
IP Internet Protocol
I/O Input/Output
MAC Media Access Control
NVG Nortel VPN Gateway
VPN Virtual Private Network
LDAP Lightweight Directory Application Protocol
AD Active Directory
NTLM New Technology LAN Manager
Nortel VPN Gateway
Troubleshooting Guide
NN46120-700 01.01 Standard
12 October 2007
Copyright © 2007 Nortel Networks
.

9
.
Troubleshooting Fundamentals
This section provides conceptual information about the methods and
tools that you can use to troubleshoot and isolate problems in the VPN
Gateway.
Navigation •“SSL acceleration” (page 9)
•“Log files” (page 10)
•“Traffic Generators” (page 11)
•“NVG hardware” (page 12)
•“Virtual IP addresses” (page 12)
SSL acceleration
The Secure Sockets Layer (SSL) protocol runs above the TCP/IP protocol
and below higher-level protocols such as HTTP or IMAP. SSL uses TCP/IP
on behalf of the higher-level protocols and, in the process, allows an
SSL-enabled server to authenticate itself to an SSL-enabled client. The
client then authenticates itself to the server, and both machines establish
an encrypted connection. The current standard is TLS (Transport Layer
Security) but the name SSL is still used.
See “Dumping the log files to another server” (page 13) for instructions on
how to view the ssl.log files.
Table 2 "Interpreting SSL acceleration log files" (page 10) gives the
information about log files messages and their descriptions.
Nortel VPN Gateway
Troubleshooting Guide
NN46120-700 01.01 Standard
12 October 2007
Copyright © 2007 Nortel Networks
.

10 Troubleshooting Fundamentals
Table 2
Interpreting SSL acceleration log files
Message Severity Type Description
Failed to initialize SSL
hardware ERROR Traffic processing The SSL acceleration hardware failed to
initialize and the NVG
runs with degraded
performance.
Set CSWIFT as default INFO Startup The NVG is using
CSWIFT SSL
hardware acceleration.
ssl-hw-fail ALARM (MAJOR) System control The SSL hardware
acceleration card
could not be found or
initiated and the NVG
runs with degraded
performance.
Using <hardware
type> hardware INFO Startup The NVG is using <hardware type>
hardware for SSL
acceleration.
Log files View the log files to see the history of system events.
This troubleshooting guide documents only the most common messages
from the ssl.log.
/maint/debug/proxydebug [on|off|once]
on: enable simpleproxy to print out debug message.
off: disable simpleproxy to print out debug message.
once: enable simpleproxy to print out debug message only once.
ATTENTION
Enabling proxydebug will use more CPU resource. Make sure to disable it after
you finish debugging.
Transmit the event log from the Nortel VPN Gateway to a file on a TFTP,
FTP, or SFTP server. Specify the IP address or host name of the server
as well as the file name. The default value is TFTP.
Table 3 "Log file types in a log dump" (page 11) lists the log file types in
a log dump.
Nortel VPN Gateway
Troubleshooting Guide
NN46120-700 01.01 Standard
12 October 2007
Copyright © 2007 Nortel Networks
.

Traffic Generators 11
Table 3
Log file types in a log dump
Log file
type Description
clierror This log provides information on the CLI engine and is used by
engineering to debug issues while in development.
erlerror This log provides information on the internal Erlang language
engine and SSL acceleration. It is used by Engineers to debug
issues while in development.
erlstart This log provides information on the internal Erstart language
engine and SSL acceleration. It is used by Engineers to debug
issues while in development.
conslog This log contains messages displayed on the console port of the
device. These messages are the one that are generated during
boot sequence. These messages are generated during boot
sequence.
dmesg This log contains messages generated by the kernel.
ssl.log This log contains messages generated by the simpleproxy
features. The user needs to completely logoff and then close the
browser ; login and then active the ssldump.
ikelog IPsec module related messages.
message This log contains standard syslog types of messages and
contains general information such as system-level status and
non-application acceleration errors across the device.
Procomm Kernel panic message with debug information can be captured
from console directly. Use Procomm to capture those messages.
Traffic Generators
Use the traffic dump commands to display either the SSL or TCP traffic
capture tools in the system.
Table 4 "Interpreting an SSL traffic dump" (page 11)provides information
on SSL dump entry and its description.
Table 4
Interpreting an SSL traffic dump
Dump entry Description
New TCP connection #<connection number>:
<sending host IP> (<port number>) <->
<receiving host IP>(<port number>)
A new TCP connection <number> made
between the sending host using port <number>
to the receiving host using port <number>.
<connection number> <start timestamp>
(<previous record timestamp>) S>C <record
type> FIN
The timestamp when the connection occurred
and the elapsed time since the last connection.
The connection is from server to client (S>C) or
client to server (C>S) and is a record type.
Nortel VPN Gateway
Troubleshooting Guide
NN46120-700 01.01 Standard
12 October 2007
Copyright © 2007 Nortel Networks
.

12 Troubleshooting Fundamentals
Table 5 "Interpreting an TCP traffic dump" (page 12) gives the information
about TCP dump entry and its description.
Table 5
Interpreting an TCP traffic dump
Dump entry Description
<ID number> IP <source IP and port>>
<destination IP and port>: <flags><data
sequence number> ack <sequence number
of next data> win <window number> Urg <tcp
options>
It is a TCP connection between a source and
destination. Flag values are S (SYN), F (FIN), P
(PUSH), R (RST), or a single period (no falgs).
The data sequence space covered by the data
in this packet. Ack is the sequence number of
the next data expected in the other direction on
this connection. Window number is the number
of bytes of receive buffer space available in the
other direction on this connection. Urg indicates
there is urgent data in the packet. TCP options
are enclosed in angle brackets.
NVG hardware
This section provides information to troubleshoot problems related to the
Nortel VPN Gateway hardware components.
Table 6 "Front Panel LEDs" (page 12) describes the Front Panel LED
indicators on the VPN Gateway.
Table 6
Front Panel LEDs
LED Indicator (from left to right) Description
Amber system status LED The amber system status LED lights
up when the system needs attention
due to a problem with power supplies,
fans, CPU, or system temperature.
Hard-disk drive activity LED This LED blinks when activity is
detected on the hard-disk drive.
System power LED This LED is green when the power
supply is turned on.
ATTENTION
Call Nortel for RMA if Amber System status LED can not be cleared.
Virtual IP addresses
In instances where virtual IP addresses are used without an external load
balancer, ensure that the effected services are set to standalone mode.
Nortel VPN Gateway
Troubleshooting Guide
NN46120-700 01.01 Standard
12 October 2007
Copyright © 2007 Nortel Networks
.

13
.
Global troubleshooting tasks
This section describes the global troubleshooting tools available at the
operating system level.
Navigation •“Dumping the log files to another server” (page 13)
•“Displaying the audit trials” (page 14)
•“Viewing SSL traffic generators” (page 14)
•“Viewing TCP traffic generators” (page 14)
Dumping the log files to another server
This section explains the steps to dump the log files to another server.
Procedure steps
Step Action
1From the command line, enter this command.
/maint/dumplogs
2Enter the transfer protocol ( tftp, ftp, or sftp).
3Enter the hostname or IP address of the target server.
4Enter the name of the file to be dumped to the target server. The
filename must have an extension of .tgz. The file is an archive
file, bundled with tar, and compressed with gzip.
5Confirm that the dump collects information from all the hosts in
the cluster.
6If using FTP, enter the user name and password.
7Use a newer version of WinZip to unpack the file on the targeted
server.
Nortel VPN Gateway
Troubleshooting Guide
NN46120-700 01.01 Standard
12 October 2007
Copyright © 2007 Nortel Networks
.

14 Global troubleshooting tasks
8Use the following table to identify the types of log files in a dump.
--End--
Displaying the audit trials
Use this tool to view changes in the system. The audit trial must be used
in conjunction with the system.
Procedure steps
Step Action
1From the command line, enter this command to add an audit
server to the device.
/cfg/sys/adm/audit/servers/add <IP address>
<udp-port> <shared server password>.
2From the command line, enter this command to enable audit
logging.
/cfg/sys/adm/audit/ena.
3All configuration changes are sent to the RADIUS.
--End--
Following are the tips to implement SSL acceleration.
•Make sure your system is configured for SSL acceleration.
•Use browsers on the client machines that are capable of doing HTTPS.
Viewing SSL traffic generators
This section explains the step to view SSL traffic generators.
Procedure steps
Action
From the command line, enter this command to view the SSL
traffic dump.
/cfg/vpn # /server/trace/ssldump
Viewing TCP traffic generators
This section explains the step to view SSL traffic generators.
Procedure steps
Nortel VPN Gateway
Troubleshooting Guide
NN46120-700 01.01 Standard
12 October 2007
Copyright © 2007 Nortel Networks
.

Viewing TCP traffic generators 15
Action
From the command line, enter this command to view the TCP
traffic dump.
/cfg/vpn # /server/trace/tcpdump.
Nortel VPN Gateway
Troubleshooting Guide
NN46120-700 01.01 Standard
12 October 2007
Copyright © 2007 Nortel Networks
.

16 Global troubleshooting tasks
Nortel VPN Gateway
Troubleshooting Guide
NN46120-700 01.01 Standard
12 October 2007
Copyright © 2007 Nortel Networks
.

17
.
Feature-specific troubleshooting tasks
This section describes the feature-specific troubleshooting tools available
at the operating system level.
Navigation •“Troubleshooting SSL acceleration” (page 17)
•“SSL acceleration troubleshooting tools” (page 18)
•“Troubleshooting Net Direct” (page 18)
•“Troubleshooting TunnelGuard” (page 19)
•“Recovering using boot.img” (page 20)
•“Upgrading code using .pkg” (page 21)
Troubleshooting SSL acceleration
This section explains the steps to troubleshoot SSL acceleration. These
steps uses the troubleshooting tools described in this section. See
Displaying Traffic Generators VPN Command Line Interface Guide
(NN46120-101) for commands not covered in this document.
Procedure steps
Step Action
1Make sure SSL acceleration is enabled for the service.
2View the output from the ssldump for the service. See for the
ssldump command.
--End--
ATTENTION
NVG 7.0 supports strong CIPHER by default. Some of the old legacy products
can interact with weak cipher, (ex) NAS 2424. Change the CIPHER string to
ALL@STRONG if problem occurred after 7.0 upgrade.
Nortel VPN Gateway
Troubleshooting Guide
NN46120-700 01.01 Standard
12 October 2007
Copyright © 2007 Nortel Networks
.

18 Feature-specific troubleshooting tasks
SSL acceleration troubleshooting tools
Use the traffic load data and SSL acceleration-specific error messages
to troubleshoot SSL acceleration.
SSL acceleration troubleshooting tools navigation
•“Viewing current configuration” (page 18)
•“Resetting default configuration” (page 18)
Viewing current configuration
This section explains the step to view current configuration.
Procedure steps
Action
From the command line, enter this command
/stats/sslstats/server # is the host number associated
with the current SSL acceleration service.
Resetting default configuration
To remove all configuration settings from the Nortel VPN Gateway, use the
/boot/delete command. This command resets the system to the default
settings.
Procedure steps
Action
From the command line, enter this command. /boot/delete
ATTENTION
After you enter this command, you must log in to the NVG and perform the initial
setup.
Troubleshooting Net Direct
For troubleshooting Net Direct, you need to get the following Net Direct
log files from the client machine. These log files are located in the users
temp folder ( %TEMP% ).
•netdirect.log
•netdirectrunner.log
•netdirectocx.log
Nortel VPN Gateway
Troubleshooting Guide
NN46120-700 01.01 Standard
12 October 2007
Copyright © 2007 Nortel Networks
.

Troubleshooting TunnelGuard 19
•tapinstall.log
•ConNDLog.log
Troubleshooting TunnelGuard
For troubleshooting the TunnelGuard Installed Agent issues, follow these
steps:
Step Action
1Enable debug logging on NVG/NVR.
The debugging information is used to track the errors in the
TunnelGuard SRS rules.
2To debug the information from the TunnelGuard applet, enter the
following command cfg/ vpn #/aaa/ tg/ loglevel debug.
3Enter apply.
4Add agent.lcf in the path <TG-Install-Dir>\resources directory.
5Reboot the system.
6Collect TunnelGuard logs from <TG-Install-Dir>log directory.
ATTENTION
The default path to access TG-Install-Dir is C:\Program Files\Nortel
Networks\TunnelGuard\.
7Attach these logs to the CR.
8For all users, attach Profiles.ini from %ALLUSERSPROFILE%\A
pplication Data\Nortel\TunnelGuard directory.
ATTENTION
The default profile path for all users is C:\Documents and Settings\All
Users.
9For logged in users, attach Profiles.ini from %APPDATA%\Nortel
\TunnelGuard directory.
ATTENTION
You can attach Profile.ini for both ’all users’ and ’logged in users’ if it
is available in their respective paths.
Nortel VPN Gateway
Troubleshooting Guide
NN46120-700 01.01 Standard
12 October 2007
Copyright © 2007 Nortel Networks
.

20 Feature-specific troubleshooting tasks
ATTENTION
The default profile path for logged in users is C:\Documents and
Settings\<user>.
--End--
For troubleshooting the TunnelGuard Applet issues, follow these steps:
Step Action
1Enable debug logging on NVG/NVR.
The debugging information is used to track the errors in the
TunnelGuard SRS rules.
2To debug the information from the TunnelGuard applet, enter the
following command cfg/ vpn #/aaa/ tg/ loglevel debug.
3Enter apply.
4Open Java console from IE Tools->Sun Java Console.
5Copy all the logs in a text file.
6Attach these logs to the CR.
--End--
Recovering using boot.img
When you log in as the boot user and perform a reinstallation of the
software, the VPN Gateway is reset to its factory default configuration.
All configuration data and current software is wiped out, including old
software image versions or upgrade packages that may be stored in the
flash memory card or on the hard disk. Also note that a reinstall must be
performed on each VPN Gateway through a console connection.
A reinstall wipes out all configuration data (including network settings).
Therefore you should first save all configuration data to a file on a
TFTP/FTP/SCP/SFTP server. Using the ptcfg command, installed keys
and certificates are included in the configuration data, and can later be
restored by using the gtcfg command.
To reinstall a VPN Gateway, you need the following:
•Access to the VPN Gateway through a console connection.
•An install image, loaded on a FTP/SCP/SFTP server on your network.
•The IP address of the FTP/SCP/SFTP server.
Nortel VPN Gateway
Troubleshooting Guide
NN46120-700 01.01 Standard
12 October 2007
Copyright © 2007 Nortel Networks
.