Alcatel-Lucent Alcatel-Lucent Security Management Server... User manual

Alcatel-Lucent Security Management Server
(SMS)
Release 9.4
Installation Guide
060274-00 REV B
CC109735423
260-100-018R9.4 Issue 2, July 2009
Alcatel-Lucent - Proprietary
This document contains proprietary information of Alcatel-Lucent and
is not to be disclosed or used except in accordance with applicable agreements.
Copyright © 2009 Alcatel-Lucent
Unpublished and Not for Publication
All Rights Reserved
See notice on first page

Alcatel, Lucent, Alcatel-Lucent and the Alcatel-Lucent logo are trademarks of Alcatel-Lucent. All other trademarks
are the property of their respective owners.
The information presented is subject to change without notice. Alcatel-Lucent assumes no responsibility for
inaccuracies contained herein.
Copyright © 2009 Alcatel-Lucent. All Rights Reserved.
Notice
Every effort was made to ensure that this information product was complete and accurate at the time of printing.
However, information is subject to change.
Security statement
In rare instances, unauthorized individuals make connections to the telecommunications network through the use of
remote access features. In such an event, applicable tariffs require the customer to pay all network charges for
traffic. Alcatel-Lucent cannot be responsible for such charges and will not make any allowance or give any credit for
charges that result from unauthorized access.
Limited warranty
For terms and conditions of sale, contact your Alcatel-Lucent Account Team.
Ordering Information
The ordering number for this information product is 260-100-018R9.4.
Technical Support
In the contintental United States, when you need additional technical assistance, the Alcatel-Lucent Global TSS
Contact Center is your first point of contact. Technical assistance is available 24 hours a day, 7 days a week. Contact
the Global TSS Contact Center at 1-866-582-3688.
Outside the United States, contact your Local Customer Support (LCS) or the support organization designated by
your Alcatel-Lucent customer team representative. If you are unsure of who to call, contact the Global TSS Contact
Center at [phone].
Information product support
Use the following email address if you have any errors to report or questions to ask about this information product:
[e-mail] .
See notice on first page

Contents
About this information product ix
Purpose ...................................................................................................................................................................... ixix
How to use this information product ........................................................................................................... ixix
Supported Brick devices .................................................................................................................................... xixi
Related information ............................................................................................................................................. xixi
How to order ......................................................................................................................................................... xiixii
Safety information .............................................................................................................................................. xiixii
How to comment ................................................................................................................................................. xiixii
1Safety Information 1
Structure of hazard statements ......................................................................................................................... 11
2SMS on a Microsoft®Windows®Server Platform 3
Overview ................................................................................................................................................................... 33
Introduction .............................................................................................................................................................. 44
Hardware Requirements ...................................................................................................................................... 66
Software Requirements ........................................................................................................................................ 77
Pre-Installation Requirements (Clean Installations) ................................................................................ 88
To Install the SMS Application (Clean Installation) ............................................................................ 1010
To Manually Un-Install SMS ......................................................................................................................... 1616
To Upgrade the SMS Software ..................................................................................................................... 1717
What To Do Next ................................................................................................................................................ 2323
3SMS on a Microsoft®Vista®Server Platform 25
Overview ................................................................................................................................................................. 2525
Introduction ............................................................................................................................................................ 2626
Hardware Requirements ................................................................................................................................... 2929
Software Requirements ..................................................................................................................................... 3030
Pre-Installation Requirements (Clean Installations) .............................................................................. 3131
...................................................................
iii

To Install the SMS Application (Clean Installations With Server Running Microsoft®
Vista®)................................................................................................................................................................... 3333
To Manually Un-Install SMS ......................................................................................................................... 3939
To Upgrade the SMS Software ..................................................................................................................... 4040
What To Do Next ................................................................................................................................................ 4444
4SMS on a Sun®Solaris®Server Platform 45
Overview ................................................................................................................................................................. 4545
Introduction ............................................................................................................................................................ 4646
Hardware Requirements ................................................................................................................................... 4848
Software Requirements ..................................................................................................................................... 4949
Pre-Installation Requirements (Clean Installations) .............................................................................. 5050
To Install the SMS Application (Clean Installation) ............................................................................ 5353
To Manually Un-Install SMS ......................................................................................................................... 5858
To Upgrade the SMS Software .................................................................................................................... 5959
What To Do Next ................................................................................................................................................ 6464
5SMS on a Linux Server Platform 65
Overview ................................................................................................................................................................. 6565
Introduction ............................................................................................................................................................ 6666
Minimum Hardware/Software Requirements .......................................................................................... 6868
Pre-Installation Requirements (Clean Installations) .............................................................................. 6969
To Install the SMS Application (Clean Installation) ............................................................................ 7272
Post-Installation Steps ....................................................................................................................................... 7878
To Manually Un-Install SMS ......................................................................................................................... 7979
To Upgrade the SMS Software ..................................................................................................................... 8080
What To Do Next ................................................................................................................................................ 8484
ARegistering SMS Software License Keys and Obtaining Installation Keys A-1
Overview ................................................................................................................................................................. 8585
To Register a Software License Key and Obtain an Installation Key .......................................... 8686
BSMS Hardening Guidelines B-1
Overview ............................................................................................................................................................... 111111
SMS and Operating System Hardening Guidelines ........................................................................... 112112
CLicense Terms for Third Party Software C-1
Contents
...................................................................
iv

Overview ............................................................................................................................................................... 117117
License terms for third party software ..................................................................................................... 118118
Index 125
...................................................................
v

List of figures
VPN Firewall Product Registration and Support Login Page .......................................................... 8787
Registration Web Page for New Users (Step 1: Verify Registration Keys) ............................... 8888
Verify Registration Keys Information (Example) .................................................................................. 8989
Registration Web Page for New Users (Step 2: Enter Customer & Contact Info) ................. 9090
Customer Information and Key Contact Information (Example) ................................................... 9292
Registration Web Page for New Users (Step 3: Enter Product Info) ........................................... 9494
Brick Hardware and Vendor Information (Example) ........................................................................... 9696
Registration Web Page for New Users (Step 4: Enter Username & Password) ...................... 9797
Username and Password Information (Example) .................................................................................. 9898
Registration Web Page for New Users (Step 5: Verify & Submit) ............................................... 9999
Registration Completed Page Showing Registered Software License Key and Installation
Key Obtained (Example) ........................................................................................................................... 100100
VPN Firewall Product Registration and Support Login Page ....................................................... 101101
VPN Firewall Registration Web Page ...................................................................................................... 103103
Registered Keys Web Page ........................................................................................................................... 104104
SMS Help > About Window ........................................................................................................................ 105105
Register New Keys Web Page .................................................................................................................... 106106
Register New Keys Upgrade License Key Entry (Example) ......................................................... 107107
Register New Keys Confirmation Window ........................................................................................... 108108
Register New Key Confirmation Window, Selecting Associated License Key From Previous
Release (Example) ........................................................................................................................................ 109109
Registration Completed Page, Successful Registration of Upgrade License Key and
Installation Key Obtained (Example) ................................................................................................... 110110
...................................................................
vii

About this information productAbout this information product
Purpose
This preface provides an overview of this information product, which is the Alcatel-Lucent
Security Management Server (SMS) Installation Guide.
The purpose of the Alcatel-Lucent Security Management Server (SMS) Installation Guide is
to explain how to install the Alcatel-Lucent Security Management Server (SMS)
application.
How to use this information product
This document is divided into chapters that describe how to install the SMS on a
Microsoft®Windows®,Microsoft®Vista®,Sun®Solaris®, and Linux server platform,
respectively.
This Installation Guide is organized as follows:
Chapter Title Description
Chapter 2, “SMS on a Microsoft®Windows®
Server Platform” Describes how to install the SMS application
on a host running Microsoft®Windows®XP
Professional or Windows®Server 2003.
Includes hardware and software
requirements, pre-installation requirements,
and detailed installation procedures for a
new and upgrade installation for both a
Primary SMS and redundant SMS
configurations. Also provides
post-installation configuration requirements
and how to access the SMS application on
your PC desktop. Also provides a procedure
for manually un-installing the SMS from a
de-commissioned server as needed.
...................................................................
ix

Chapter Title Description
Chapter 3, “SMS on a Microsoft®Vista®
Server Platform” Describes how to install the SMS application
on a host running Microsoft®Vista®.
Includes hardware and software
requirements, pre-installation requirements,
and detailed installation procedures for a
new and upgrade installation for both a
Primary SMS and redundant SMS
configurations. Also provides
post-installation configuration requirements
and how to access the SMS application on
your PC desktop. Also provides a procedure
for manually un-installing the SMS from a
de-commissioned server as needed.
Chapter 4, “SMS on a Sun®Solaris®Server
Platform” Describes how to install the SMS application on a host running the Sun®Solaris®Release
9 or 10 operating system. Describes
hardware and software requirements for
installing SMS on a Solaris®server
platform, pre-installation requirements, and
and detailed installation procedures for a
new and upgrade installation for both a
Primary SMS and redundant SMS
configurations. Also provides
post-installation configuration requirements
and how to access the SMS application on
your PC desktop. Also provides a procedure
for manually un-installing the SMS from a
de-commissioned server as needed.
Chapter 5, “SMS on a Linux Server
Platform” Describes how to install the SMS application on a host running Red Hat Enterprise Linux
4 (RHEL4) or Red Hat Enterprise Linux 5
(RHEL5). Includes hardware and software
requirements, pre-installation requirements,
and detailed instructions for a new and
upgrade installations on a Linux platform
server. Also provides post-installation
configuration requirements and how to
access the SMS application on your PC
desktop. Also provides a procedure for
manually un-installing the SMS from a
de-commissioned server as needed.
Appendix A, “Registering SMS Software
License Keys and Obtaining Installation
Keys”
This appendix provides instructions on how
to register the SMS product and software
license key, and obtain an installation key,
which is needed to install a new or upgraded
SMS release or feature option.
Appendix B, “SMS Hardening Guidelines” This appendix provides general security
hardening recommendations for the SMS and
server operating system.
About this information product
...................................................................
x

Chapter Title Description
Appendix C, “License Terms for Third Party
Software” This appendix contains information about licensing terms and agreements for third
party software.
Supported Brick devices
The following available Brick models are supported by the current SMS release:
•Alcatel-Lucent VPN Firewall Brick™Model 20 Security Appliance
•Alcatel-Lucent VPN Firewall Brick™Model 50 Security Appliance
•Alcatel-Lucent VPN Firewall Brick™Model 150 Security Appliance
•Alcatel-Lucent VPN Firewall Brick™Model 350 Security Appliance
•Alcatel-Lucent VPN Firewall Brick™Model 1100/1100A Security Appliance
•Alcatel-Lucent VPN Firewall Brick™Model 700 Security Appliance
•Alcatel-Lucent VPN Firewall Brick™Model 1200 Standard and HS VPN Security
Appliances
Important! Note: only later Model 20 Brick devices are supported with this release.
Model 20 Bricks that have 6-8 MB of RAM and 8 MB of flash are not supported with
this release.
Some of the above Brick device models require a specific patch of the current SMS release in order
to be fully supported. For details about the SMS patch release required for a specific Brick device
model, refer to the User’s Guide for the Brick device model or contact your Alcatel-Lucent
customer support team representative for more information.
Related information
The Alcatel-Lucent Security Management Server (SMS) Installation Guide is part of a set of
documents that support the SMS.
An online version of this document set, in PDF format, is available through the Help
feature of the SMS application software.
Documentation
The document set that supports the SMS application consists of the following documents:
•Alcatel-Lucent Security Management Server (SMS) Policy Guide, which instructs users
how to create security policies and set up VPN tunnels on Alcatel-Lucent VPN
Firewall Brick™Security Appliances.
•Alcatel-Lucent Security Management Server (SMS) Administration Guide, which
instructs users how to administer the SMS application.
•Alcatel-Lucent Security Management Server (SMS) Installation Guide, which instructs
users how to install the SMS application.
...................................................................
xi

•Alcatel-Lucent Security Management Server (SMS) Reports, Alarms, and Logs Guide,
which instructs users how to use log files, configure triggers and actions to generate
alarms, and how to compile and view reports.
•Alcatel-Lucent Security Management Server (SMS) Tools and Troubleshooting Guide,
which instructs users how to use the Command Line Interface (CLI) commands.
•Alcatel-Lucent Security Management Server (SMS) Technical Overview, which provides
a general technical description of the Alcatel-Lucent VPN Firewall solution.
How to order
To order SMS information products, contact your Alcatel-Lucent Technologies customer
team representative or contact Alcatel-Lucent at one of the following telephone numbers:
•From the United States, call 888-582-3688, prompt 1.
•From Canada, call 317-322-6616.
•From Europe, the Middle East, Asia, Africa, the Pacific, China, the Caribbean, and
Latin America, call 317-322-6416.
Safety information
This information product contains hazard statements for your safety. Hazard statements are
given at points where safety consequences to personnel, equipment, and operation may
exist. Failure to follow these statements may result in serious consequences.
How to comment
To comment on this information product, go to the Online Comment Form
(http://www.lucent-info.com/comments/enus/) or e-mail your comments to the Comments
Hotline ([email protected]).
About this information product
...................................................................
xii

11Safety Information
Structure of hazard statements
.........................................................................................................................................................................................
Overview
Hazard statements describe the safety risks relevant while performing tasks on
Alcatel-Lucent products during deployment and/or use. Failure to avoid the hazards may
have serious consequences.
General structure
Hazard statements include the following structural elements:
Item Structure element Purpose
1 Personal-injury symbol Indicates the potential for personal injury
(optional)
2 Hazard-type symbol Indicates hazard type (optional)
3 Signal word Indicates the severity of the hazard
...................................................................
1

Item Structure element Purpose
4 Hazard type Describes the source of the risk of damage or
injury
5 Damage statement Consequences if protective measures fail
6 Avoidance message Protective measures to take to avoid the hazard
7 Identifier The reference ID of the hazard statement
(optional)
Signal words
The signal words identify the hazard severity levels as follows:
Signal word Meaning
DANGER Indicates an imminently hazardous situation (high risk) which, if not
avoided, will result in death or serious injury.
WARNING Indicates a potentially hazardous situation (medium risk) which, if
not avoided, could result in death or serious injury.
CAUTION When used with the personal injury symbol:
Indicates a potentially hazardous situation (low risk) which, if not
avoided, may result in personal injury.
When used without the personal injury symbol:
Indicates a potentially hazardous situation (low risk) which, if not
avoided, may result in property damage, such as service interruption
or damage to equipment or other materials.
Safety Information
...................................................................
2

22SMS on a Microsoft®Windows®Server
Platform
Overview
.........................................................................................................................................................................................
Purpose
This chapter explains how to install or upgrade the SMS application on a
Microsoft®Windows®server platform. It also includes a procedure for manually
un-installing the SMS as needed.
Contents
Introduction 4
Hardware Requirements 6
Software Requirements 7
Pre-Installation Requirements (Clean Installations) 8
To Install the SMS Application (Clean Installation) 10
To Manually Un-Install SMS 16
To Upgrade the SMS Software 17
What To Do Next 23
...................................................................
3

Introduction
.........................................................................................................................................................................................
Overview
The SMS application can be installed on a host running Microsoft®Windows®XP
Professional or Server 2003.
Release 9.4 can be installed as a first time (clean) installation or as an upgrade.
This chapter provides step-by-step instructions for installing the SMS application as:
•a Primary SMS. Use this procedure to install a single SMS or the first SMS in a
multi-site SMS redundancy configuration
•a Secondary SMS associated with a Primary SMS in a redundant pair or as part of a
multi-site SMS redundancy configuration
•a Compute Server associated with a Primary or Secondary SMS
Upgrades
CAUTION
Service-disruption hazard
Security Precaution
It is highly recommended that the SMS server be protected with a Brick device and that as
little as possible be put on the same trusted subnet as the SMS.
SMS configurations
The SMS can be deployed in one of the following configurations:
•A Primary SMS (stand-alone configuration)
•A Primary SMS and Secondary SMS (in a redundant configuration)
•A Primary SMS and up to three Secondary SMSs (Multi-Site configuration)
•A Primary SMS and up to three Secondary SMSs, each with up to five Compute
Servers (Compute Server configuration)
Multi-Site SMS configuration
A multi-site SMS configuration consists of a Primary SMS and up to three Secondary
SMSs. The Primary SMS and Secondary SMS (s) share the same database, which is
updated periodically across the network. The Primary SMS and Secondary SMS(s) are
simultaneously active, synchronizing status and configuration at the same time.
Compute servers
To maximize scalability of the SMS/Brick security solution, the SMS provides the option
of adding a separate set of servers called Compute Servers (CSs), which are associated with
a Primary or Secondary SMS and act as collection points for Brick log traffic. Using a CS
to collect Brick log data frees up computing resources on the SMS itself and extends the
SMS on a Microsoft®Windows®Server Platform
...................................................................
4

number of Brick devices and total log traffic that can be handled. Each Brick device
managed by the SMS can be homed to one of the associated CSs or the managing SMS for
logging purposes.
Up to five CSs can be configured for a Primary or Secondary SMS.
Implementing Primary SMS/Secondary SMS configurations
During the installation of the Secondary SMS, there is a point at which the Secondary SMS
attempts to contact the Primary SMS to replicate the Primary SMS database on the
Secondary SMS. If the Secondary SMS cannot contact the Primary SMS, correct the
problem and retry the operation on the Secondary SMS.
For reasons of security, we strongly recommend that you deploy a Brick device ″in front″
of the Primary and Secondary SMS(s) to protect all servers.
To ensure that the Primary SMS and Secondary SMS(s) can contact each other through
both Brick devices, follow the course of action outlined below when you order the
installation:
1. Install the Primary SMS first. The two installation procedures in this chapter provide
step-by-step instructions for a new installation and an upgrade installation.
2. Once the Primary SMS is operational, use it to configure two Bricks and assign the
pre-configured ruleset administrativezone to the ports that will be connected to the
SMS. Refer to the Configuring Brick Ports section in the SMS Administration Guide
for instructions on how to do this.
3. Connect the Primary SMS to the port on one Brick, and the host that will be the
Secondary SMS to the port on the other. Then, deploy the two Bricks: the primary
SMS and the host that will be the Secondary SMS in the network.
4. To ensure that the Primary SMS and remote host can communicate, add a ping rule
(dir=both, source=*, dest=*, service=ping_request,action=pass) to the
administrativezone ruleset, and then ping the host from the SMS. Once the ping is
successful, remove the rule for security reasons. (Refer to the Brick Zone Rulesets
section in the SMS Policy Guide for instructions on how to create a rule.)
5. When you have established that the two SMS servers can communicate, install the
Secondary SMS.
Introduction
...................................................................
5

Hardware Requirements
.........................................................................................................................................................................................
Minimum hardware requirements
Regardless of which operating system you are running, the host on which you install the
SMS application must meet the following minimum hardware requirements:
•400 MHz Pentium processor
•512 MB of RAM
•Swap space at least as large as the amount of RAM
•1 GB free space on an NTFS partition
•CD-ROM drive
•3.5 inch floppy drive, USB port, or serial port. For information on the hardware
required to boot specific Lucent Brick device models, consult the User’s Guide for the
specific Brick or contact your Alcatel-Lucent customer support team representative.
•Ethernet interface card
•Video card capable of 1024 x 768 resolution (65,535 colors)
Important! Because of the memory requirements of SMS R9.4, older Brick 20 models
with only 8 MB of RAM may not be able to be configured with all features.
To verify the amount of RAM on the Brick device, log into the Brick device via the
remote console and run the display mem command.
If you are managing many Brick devices, supporting many IPSec clients, or generating large
amounts of audit data, you may require additional memory. Refer to the Sizing Guidelines appendix
in the SMS Administration Guide to help you determine how much additional memory you may
need.
One optional piece of equipment you may want to consider is a modem. Having a modem in the
host allows you to set up alarms that page an administrator when a problem is encountered. If you
decide to add a modem, it must be Hayes-compatible and configured so that it cannot accept
incoming calls and cannot be shared by other services, such as a Remote Access Server.
SMS on a Microsoft®Windows®Server Platform
...................................................................
6

Software Requirements
.........................................................................................................................................................................................
Required software components
The following software is required to run the SMS application on a Microsoft Windows®
platform:
•Windows®XP Professional and Service Pack 1 2, or 3, or Windows®Server 2003 and
Service Pack 4 or higher.
•A hard drive with at least one NTFS partition. The drive can have the FAT file system
installed, but it must have at least one NTFS partition to hold the SMS software.
In addition, the following software is not required, but is highly recommended:
•A browser such as Microsoft®Internet Explorer®or Firefox to view SMS reports and
display the on-line help.
•Adobe Acrobat Reader. This application is required to display the on-line manuals.
Installations on a Windows®Server 2003 platform
If you are installing the SMS application on aWindows®Server 2003 platform, do not
install Terminal Server in Windows®. An error message is generated when attempting to
install SMS and it does not work.
Security patches
It is strongly recommended that you install security patches on a regular basis to keep
security on your machine up-to-date. Use caution when installing service packs. It is not
guaranteed that the current SMS software can be installed or run with new service packs.
...................................................................
7

Pre-Installation Requirements (Clean Installations)
.........................................................................................................................................................................................
Overview
Before you proceed with the actual installation of the SMS application for the first time on
a host, some pre-installation steps are required:
1. Install the service pack.
2. Resolve potential web server port conflicts.
3. Install Adobe Acrobat Reader.
4. Obtain the SMS installation keys.
Install the service pack
Check the host that you are using to see if the correct service pack is installed. If it is not,
you must install the appropriate service pack before beginning the installation. Service
packs can be downloaded from the Microsoft®website.
Resolve potential web server port conflicts
The SMS application contains a web server which will conflict with any existing web
server currently active on the host machine. If there is another web server running on the
SMS host (such as Microsoft®Internet Information Server or Apache), you must either:
•Shut the web server down
•Check system services and set the Startup Type to Manual or Disabled rather than
Automatic so that the web server will not start up when the OS is rebooted.
Port 80 is the default SMS web server port. In the event that another web server is already
using port 80 (such as Apache), you must either change the port on that web server or
select a different port when configuring the SMS web server.
Install Adobe Acrobat Reader
Adobe Acrobat Reader is required to view the on-line documentation that is provided with
the application. A copy of this application can be downloaded from the Adobe website,
http://www.adobe.com.
Obtain the SMS installation keys
To install the SMS application, two keys are required:
•Software license key
The software license key is provided with the SMS application. You will need this key
to register the product and to obtain the installation key, which is required to perform
the installation.
•Installation key
The installation key is required to install the product.
SMS on a Microsoft®Windows®Server Platform
...................................................................
8