Kemp High Availability User manual

Copyright © 2002 - 2014 KEMP Technologies, Inc. All Rights Reserved. Page 1 / 37
High Availability (HA)
Feature Description
High Availability
(HA)
Feature Description
VERSION: 1.7
UPDATED: JUNE 2014

Copyright © 2002 - 2014 KEMP Technologies, Inc. All Rights Reserved. Page 2 / 37
High Availability (HA)
Feature Description
Copyright Notices
Copyright © 2002-2014 KEMP Technologies, Inc.. All rights reserved.. KEMP Technologies and the KEMP
Technologies logo are registered trademarks of KEMP Technologies, Inc..
KEMP Technologies, Inc. reserves all ownership rights for the LoadMaster product line including software
and documentation. The use of the LoadMaster Exchange appliance is subject to the license agreement.
Information in this guide may be modified at any time without prior notice.
Microsoft Windows is a registered trademarks of Microsoft Corporation in the United States and other
countries. All other trademarks and service marks are the property of their respective owners.
Limitations: This document and all of its contents are provided as-is. KEMP Technologies has made efforts
to ensure that the information presented herein are correct, but makes no warranty, express or implied,
about the accuracy of this information. If any material errors or inaccuracies should occur in this document,
KEMP Technologies will, if feasible, furnish appropriate correctional notices which Users will accept as the
sole and exclusive remedy at law or in equity. Users of the information in this document acknowledge that
KEMP Technologies cannot be held liable for any loss, injury or damage of any kind, present or prospective,
including without limitation any direct, special, incidental or consequential damages (including without
limitation lost profits and loss of damage to goodwill) whether suffered by recipient or third party or from
any action or inaction whether or not negligent, in the compiling or in delivering or communicating or
publishing this document.
Any Internet Protocol (IP) addresses, phone numbers or other data that may resemble actual contact
information used in this document are not intended to be actual addresses, phone numbers or contact
information. Any examples, command display output, network topology diagrams, and other figures
included in this document are shown for illustrative purposes only. Any use of actual addressing or contact
information in illustrative content is unintentional and coincidental.
Portions of this software are; copyright (c) 2004-2006 Frank Denis. All rights reserved; copyright (c) 2002
Michael Shalayeff. All rights reserved; copyright (c) 2003 Ryan McBride. All rights reserved.
Redistribution and use in source and binary forms, with or without modification, are permitted provided
that the following conditions are met:
1. Redistributions of source code must retain the above copyright notice, this list of conditions and the
following disclaimer
2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the
following disclaimer in the documentation and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE ABOVE COPYRIGHT HOLDERS ''AS IS'' AND ANY EXPRESS OR IMPLIED
WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE ABOVE COPYRIGHT
HOLDERS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
POSSIBILITY OF SUCH DAMAGE.
The views and conclusions contained in the software and documentation are those of the authors and
should not be interpreted as representing official policies, either expressed or implied, of the above
copyright holders..
Portions of the LoadMaster software are copyright (C) 1989, 1991 Free Software Foundation, Inc. -51
Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA- and KEMP Technologies Inc. is in full compliance
of the GNU license requirements, Version 2, June 1991. Everyone is permitted to copy and distribute
verbatim copies of this license document, but changing it is not allowed.
Portions of this software are Copyright (C) 1988, Regents of the University of California. All rights reserved.

Copyright © 2002 - 2014 KEMP Technologies, Inc. All Rights Reserved. Page 3 / 37
High Availability (HA)
Feature Description
Redistribution and use in source and binary forms are permitted provided that the above copyright notice
and this paragraph are duplicated in all such forms and that any documentation, advertising materials, and
other materials related to such distribution and use acknowledge that the software was developed by the
University of California, Berkeley. The name of the University may not be used to endorse or promote
products derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED ``AS IS'' AND WITHOUT ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING,
WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE.
Portions of this software are Copyright (C) 1998, Massachusetts Institute of Technology
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated
documentation files (the "Software"), to deal in the Software without restriction, including without
limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the
Software, and to permit persons to whom the Software is furnished to do so, subject to the following
conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions
of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING
BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
Portions of this software are Copyright (C) 1995-2004, Jean-loup Gailly and Mark Adler
This software is provided 'as-is', without any express or implied warranty. In no event will the authors be
held liable for any damages arising from the use of this software.
Permission is granted to anyone to use this software for any purpose, including commercial applications,
and to alter it and redistribute it freely, subject to the following restrictions:
1. The origin of this software must not be misrepresented; you must not claim that you wrote the original
software. If you use this software in a product, an acknowledgment in the product documentation would
be appreciated but is not required.
2. Altered source versions must be plainly marked as such, and must not be misrepresented as being the
original software.
3. This notice may not be removed or altered from any source distribution.
Portions of this software are Copyright (C) 2003, Internet Systems Consortium
Permission to use, copy, modify, and/or distribute this software for any purpose with or without fee is
hereby granted, provided that the above copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING
ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT,
INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS,
WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE
USE OR PERFORMANCE OF THIS SOFTWARE.
Used, under license, U.S. Patent Nos. 6,473,802, 6,374,300, 8,392,563, 8,103,770, 7,831,712, 7,606,912, 7,346,695, 7,287,084 and
6,970,933

Copyright © 2002 - 2014 KEMP Technologies, Inc. All Rights Reserved. Page 4 / 37
High Availability (HA)
Feature Description
Table of Contents
1 Introduction ....................................................................................................................................6
1.1 Intended Audience.................................................................................................................. 6
2 Advantages of High Availability (HA) ..............................................................................................7
3 Prerequisites ...................................................................................................................................8
4 HA Components .............................................................................................................................. 9
5 Setting Up HA................................................................................................................................10
5.1 Set up the First Unit ..............................................................................................................10
5.2 Set Up the Second Unit.........................................................................................................12
5.3 Enable the “Use for HA Checks” Option ...............................................................................15
5.4 Test Failover..........................................................................................................................15
6 How to Perform a Firmware Update on HA Pairs.........................................................................17
7 HA Disaster Recovery Process (Replacing a Defective Unit).........................................................18
7.1 Prerequisites .........................................................................................................................18
7.2 Installation ............................................................................................................................18
7.2.1 Reboot the Active Unit..................................................................................................19
7.2.2 Change the Switch to Preferred Server Option ............................................................20
8 HA WUI Options............................................................................................................................21
8.1.1 Interfaces ......................................................................................................................21
8.1.2 HA Parameters ..............................................................................................................22
9 Troubleshooting............................................................................................................................ 26
9.1 General Troubleshooting Tips...............................................................................................26
9.2 No HA Status Squares are Visible in the WUI .......................................................................26
9.3 Green/Red HA Status Squares ..............................................................................................26
9.4 Blue HA Status Square ..........................................................................................................27
9.5 Both Units are Active and the WUI is Unresponsive –Blue or Red Status Square...............27
9.6 Grey HA Status Square..........................................................................................................27
9.7 Virtual Services Temporarily Unavailable After Failover ......................................................27
9.8 No Access to WUI on HA1/2 or Shared.................................................................................28
9.9 Nothing Works ...................................................................................................................... 28
9.10 Issue with Hyper-V and HA on a Pair of VLMs ...................................................................... 28

Copyright © 2002 - 2014 KEMP Technologies, Inc. All Rights Reserved. Page 5 / 37
High Availability (HA)
Feature Description
9.11 HA Issue on VMware.............................................................................................................28
9.11.1 Both Units Think That They Are the Master Unit..........................................................28
9.11.2 Two Virtual LoadMasters on Different Hosts................................................................30
10 Replacing HA Units....................................................................................................................31
References ............................................................................................................................................36
Document History .................................................................................................................................37

Copyright © 2002 - 2014 KEMP Technologies, Inc. All Rights Reserved. Page 6 / 37
High Availability (HA)
Feature Description
1Introduction
The LoadMaster system can be deployed as a single unit or in an active/standby dual-unit
configuration (HA). HA allows two physical or virtual machines to become one logical device.
Only one of these units is active and handling traffic at any one time. One unit is active and the
other is a hot standby (passive). This provides redundancy and resiliency, meaning if one
LoadMaster goes down for any reason, the hot standby can become active, therefore avoiding
any downtime.
HA in the LoadMaster for Azure product works a different way to that of a
regular LoadMaster. For more information and instructions on how to
configure HA on the LoadMaster for Azure, refer to the HA for Azure,
Feature Description document.
Three IP addresses are needed per active interface; one for the active unit, one for the passive
unit and one IP address for the shared interface.
1.1 Intended Audience
This document is intended to be read by anyone who is interested in learning about the HA
feature in the KEMP LoadMaster.

Copyright © 2002 - 2014 KEMP Technologies, Inc. All Rights Reserved. Page 7 / 37
High Availability (HA)
Feature Description
2Advantages of High Availability (HA)
Figure 2-1: Typical 1-armed HA-setup
The ultimate goal of redundant LoadMasters is to provide reliable traffic management, even if
one LoadMaster becomes unavailable. The advantages of HA are as follows:
Eliminates a single point of failure
The second (passive) unit monitors the active unit to detect if a failure has occurred
Active connections and sessions are not lost when a fail-over occurs

Copyright © 2002 - 2014 KEMP Technologies, Inc. All Rights Reserved. Page 8 / 37
High Availability (HA)
Feature Description
3Prerequisites
There are some prerequisites to be aware of before setting up HA:
Two LoadMasters must be set up and licensed
LoadMasters must be located on the same subnet in order to be in a HA pair
LoadMasters must be in the same physical location
A layer 2 connection (Ethernet/VLAN) is required
The LoadMasters must not be located further than 100 meters from each other
Ensure that you have more than one interconnection between the two LoadMasters to
avoid data or lack of availability
The two LoadMasters need the same default gateway
Use Network Time Protocol (NTP) to keep times on LoadMasters up-to-date. This
ensures that the times will be correct on any logs and that Common Address
Redundancy Protocol (CARP) message timestamps are in sync.
Ensure that any switches do not prevent MAC spoofing
Latency on the link between the two LoadMasters must be below 100 milliseconds
Multicast traffic flow is required in both directions between the devices
The appropriate LoadMaster license should be in place
Three IP addresses are required for each subnet in which the LoadMaster is configured

Copyright © 2002 - 2014 KEMP Technologies, Inc. All Rights Reserved. Page 9 / 37
High Availability (HA)
Feature Description
4HA Components
The HA components are listed and described below.
CARP:
Keeps updated on the health of the partner
Governs whether the LoadMaster will take the active role
The Use for HA Checks option enables CARP requests to be sent over enabled interfaces.
This can be set for multiple interfaces.
When CARP is being used, packet analysis tools such as Wireshark,
incorrectly displays the protocol used as Virtual Router Redundancy
Protocol (VRRP). Any IP addresses displayed by the packet analysis tools
are fictitious and are not part of the CARP protocol.
Sync:
Sync maintains a “single image view” of the LoadMaster settings. It keeps the
LoadMaster up-to-date with changes made to Virtual Services and all other
configurations.
Notable exceptions that are not synchronized are; time and passwords
Keeps the standby LoadMaster updated on persistence updates

Copyright © 2002 - 2014 KEMP Technologies, Inc. All Rights Reserved. Page 10 / 37
High Availability (HA)
Feature Description
5Setting Up HA
5.1 Set up the First Unit
To build a HA LoadMaster environment there are a number of settings that must be carefully
specified. Follow the steps below to set up HA:
1. Log in to the LoadMaster that is desired to be the active (master) unit.
2. In the main menu, select the System Configuration > Miscellaneous Options >HA
Parameters option.
Figure 5-1: HA Mode option
3. Select HA (First) Mode in the HA Mode drop-down.
Figure 5-2: HA question
4. Click OK on the resulting message.
Do not reboot at this time.
Figure 5-3: Shared IP address reminder
5. Click OK on the resulting message reminding not to forget to set the Shared IP address.
Figure 5-4: HA settings

Copyright © 2002 - 2014 KEMP Technologies, Inc. All Rights Reserved. Page 11 / 37
High Availability (HA)
Feature Description
6. Specify the desired shared IP address in the HA Shared IP address field and click Set
Shared Address.
7. A confirmation message may appear. Click OK.
8. A message may appear asking to reconnect to the shared IP address. Click OK.
Do not reboot or reconnect at this time.
9. Enter the IP address of the passive (slave) unit in the HA Partner IP address and click the
Set Partner address button.
10. A confirmation message will appear. Click OK.
11. Configure any other settings as needed.
12. Click Reboot Now.
Figure 5-5: Rebooting message
13. Click Continue.
14. Refresh the page after the LoadMaster has rebooted (this may take a few minutes).
Figure 5-6: Log in screen
A log in screen appears. After logging in, a different menu will appear than
before. This is the Local Administration menu displayed for HA units.
This menu has far fewer options. Only configuration settings pertaining to
that specific unit are accessible via the Local Administration option. All
management of the HA units should be done via the shared IP address. To
see the full menu and configure the units, access the WUI of the shared IP
address, which was specified above.
15. Log in to the shared IP WUI by entering the shared IP address in the address bar of the
browser and pressing Enter.
In the top-left of the screen there will be 2 indicator squares. These squares indicate the
status of the HA pair. The left square always represents HA1 and the right represents HA2. The A
represents which unit is active. The first or second HA unit can be opened by clicking the
relevant status icon. Green and green status colors indicate a properly paired configuration.

Copyright © 2002 - 2014 KEMP Technologies, Inc. All Rights Reserved. Page 12 / 37
High Availability (HA)
Feature Description
Currently, they will probably be green and red since the HA2 unit has not yet joined the pair. For
an explanation of all icon colors and statuses, refer to Section 8.1.2.
16. Go to HA Parameters in the main menu.
Figure 5-7: HA Parameters
17. Enter a different number (different from the IDs of other HA devices) in the HA Virtual
ID text box and click Set Virtual ID. Using the same ID as other HA devices may cause
problems.
KEMP highly recommends using a higher value than 10, as any other HA
device using the same ID could interfere with HA operations.
5.2 Set Up the Second Unit
Now that HA has been configured on the first unit, the second unit needs to be setup. Follow the
steps below to do this.
1. Enter the IP address of the second unit in the address bar of the browser and press
Enter.
Ensure to enter https:// before the IP address.
2. In the main menu, select the System Configuration > Miscellaneous Options > HA
Parameters option
Figure 5-8: HA Parameters screen
3. Select HA (Second) Mode as the HA Mode.

Copyright © 2002 - 2014 KEMP Technologies, Inc. All Rights Reserved. Page 13 / 37
High Availability (HA)
Feature Description
Figure 5-9: Confirmation message
4. Click OK on the resulting message.
Figure 5-10: Shared IP address reminder
5. Click OK on the resulting message.
Figure 5-11: Network Interface screen
6. Enter the HA Shared IP address and click the Set Shared address button.
The HA Shared IP address must be the same as the HA Shared IP
address which was set when configuring the first unit in Section 5.1.
7. Click OK on the resulting message.
8. Click OK on the message asking to reconnect to the shared IP address.
9. Enter the IP address of the first (master) unit in the HA pair in the HA Partner IP address
field and click Set Partner address.
10. Click OK on the resulting message.
11. Change any other settings as needed.
12. Click the Reboot Now button.

Copyright © 2002 - 2014 KEMP Technologies, Inc. All Rights Reserved. Page 14 / 37
High Availability (HA)
Feature Description
Figure 5-12: Rebooting screen
13. Click the Continue button.
Passwords for the bal account are not synchronized across HA pairs, so
ensure to use the same password on both units. Problems may occur if
different passwords are used.
After rebooting, the HA pair will establish a TCP connection (using port 6973) between the 2
addresses. The synchronization process is started for the configuration.
The indicator squares should now show green and green. The A indicates the active unit of
the pair. If the first synchronization attempt fails (i.e. the icons are not green and green) a second
attempt might be needed.
Figure 5-13: Credentials section
On the home screen, the IP address field has changed. In addition to specifying the shared IP
address of the pair, it also specifies the active unit in the pair. The left IP address is the shared
address. The IP address in brackets is the address of the current, active unit. Next to that, the HA
version (Carp) is displayed.

Copyright © 2002 - 2014 KEMP Technologies, Inc. All Rights Reserved. Page 15 / 37
High Availability (HA)
Feature Description
5.3 Enable the “Use for HA Checks” Option
Some guidelines relating to the Use for HA checks option are below:
If you have a physical LoadMaster, you can connect a direct cable on eth1 between both
boxes. Leave the IP configuration blank. Select the Use for HA checks check box.
In a hardware configuration, if a direct cable has been deployed between both units over
eth1 and there are IP addresses set in the interface management screen (and Use for HA
checks is enabled on that interface) this causes problems because the LoadMaster will
think it is a production link and if one of the LoadMasters reboot the other will fail too.
If the Use for HA checks check box is greyed out it means that this is the only interface
configured to be used for HA checks and cannot be deselected.
To enable the Use for HA checks option, follow the steps below:
1. Go to the WUI of the shared IP address.
2. In the main menu, select System Configuration.
3. Select the relevant interface.
4. Select the Use for HA checks check box.
These steps can be repeated if the Use for HA checks option needs to be enabled on more than
one interface.
5.4 Test Failover
Now that the HA units have been set up, failover can be tested if needed. The easiest way to do
this is to reboot the active unit. To reboot the unit, follow the steps below:
1. Log in to the IP address of the active unit.
2. In the main menu, click Local Administration.
3. Select System Reboot.
Figure 5-14: System Reboot screen
4. Click the Reboot button.

Copyright © 2002 - 2014 KEMP Technologies, Inc. All Rights Reserved. Page 16 / 37
High Availability (HA)
Feature Description
Figure 5-15: Confirmation message
5. A confirmation message may appear. Click OK.
Figure 5-16: Rebooting screen
6. Click Continue.
When HA1 is back online, both HA status icons should be green. The A should have moved into
the right green square. This means that the secondary unit is now the active unit.

Copyright © 2002 - 2014 KEMP Technologies, Inc. All Rights Reserved. Page 17 / 37
High Availability (HA)
Feature Description
6How to Perform a Firmware Update on HA Pairs
KEMP recommends that firmware updates are performed outside of working hours. This is to
ensure there is no interruption to client connectivity. If it has to be done during working hours
we suggest that a window of maintenance is put in place.
Before updating the firmware, ensure that the Switch to Preferred Server
drop-down list is set to No Preferred Host in System Configuration >
Miscellaneous Options > HA Parameters.
To update the firmware on a HA pair; perform the following steps via the shared IP address:
1. Update the currently active LoadMaster (“A”).
2. When the update is complete, reboot A so that the passive unit becomes active.
3. When the first unit is back up, update the other unit.
4. When the update is complete, reboot other unit.
Now A should be active as it originally was.

Copyright © 2002 - 2014 KEMP Technologies, Inc. All Rights Reserved. Page 18 / 37
High Availability (HA)
Feature Description
7HA Disaster Recovery Process (Replacing a Defective
Unit)
In a disaster recovery scenario you may need to replace one of the HA units. Prerequisites and
installation steps for this scenario are provided in the sections below.
7.1 Prerequisites
There are some prerequisites that are needed before replacing a HA unit in a disaster recovery
situation. These are listed below:
The new LoadMaster should be licensed:
Verify that the license is valid for the relevant features
Gather the following data:
IP address of the LoadMaster
IP address of the partner
Shared IP address
7.2 Installation
Installation of a new HA unit should be performed during a maintenance window. Allow
approximately 20 minutes for the installation.
Follow the steps below:
1. Ensure that the HA ports are connected and both machines can reach each other by
using the Ping option in System Configuration > Logging Options > System Log Files >
Debug Options.
2. Connect to the WUI on the local (machine) IP address of the new LoadMaster.
3. In the main menu of the WUI, select System Configuration > Miscellaneous Options >
HA Parameters.
Figure 7-1: HA Mode
4. Select the relevant HA Mode (first or second).
5. Click OK.
6. Click OK again.
Figure 7-2: Network Interface

Copyright © 2002 - 2014 KEMP Technologies, Inc. All Rights Reserved. Page 19 / 37
High Availability (HA)
Feature Description
7. Enter the HA Shared address. Click Set Shared address.
8. Enter the HA Partner IP address. Click Set Partner address.
9. Click Reboot Now.
10. Wait 45 to 60 seconds, then click Continue.
11. Wait until the HA icons at the top of the screen are green.
12. If not yet done, connect the LoadMaster to the Real Server network and the open
network side.
13. In the main menu, select Local Administration > System Reboot.
Figure 7-3: Reboot
14. Click Reboot.
15. Click OK.
16. Click Continue.
It is optional, but KEMP recommends simulating a failover to the newly restored partner to
check that everything is working OK. There are two ways to do this, both are detailed in the
sections below.
7.2.1 Reboot the Active Unit
Rebooting the active unit should cause the new unit to take over. To do this, follow the steps
below in the WUI of the active unit:
1. In the main menu of the WUI, select Local Administration > System Reboot.
Figure 7-4: Reboot
2. Click Reboot.
3. Wait 45 to 60 seconds, then click Continue.

Copyright © 2002 - 2014 KEMP Technologies, Inc. All Rights Reserved. Page 20 / 37
High Availability (HA)
Feature Description
7.2.2 Change the Switch to Preferred Server Option
To change the Switch to Preferred Server option, follow the steps below in the WUI of the
shared IP address:
1. In the main menu, select System Configuration > Miscellaneous Options > HA
Parameters.
Figure 7-5: HA Parameters
2. In the Switch to Preferred Server drop-down list, select the relevant option in order to set
the new LoadMaster as the preferred master.
This manual suits for next models
1
Table of contents