Privileged Access Manager POC Guidelines

Hitachi ID Privileged Access Manager

Product Evaluation Guidelines

© 2011 Hitachi ID Systems, Inc. All rights reserved.

Contents

1 Introduction 1

2 Focusing on Technical Challenges 1

3 Replication, Latency and Bandwidth 2

4 Automatic Discovery and Classiﬁcation 6

5 Firewalls 9

6 Strong Authentication 11

7 Access Controls 12

8 Approval Workﬂow 14

9 Building the Lab Environment 15

APPENDICES 17

A Why Replication Across Cities is Mandatory 18

i

Hitachi ID Privileged Access Manager Product Evaluation Guidelines

1 Introduction

This document outlines best practices for evaluating privileged access management solutions. It offers
guidance regarding how to construct a lab environment, what features to test and how to test them. Prod-
uct evaluators can use this document to guide a robust testing process that highlights architectural and
technological differences between competing products.

This document is intended for a technical audience. In particular, it should be helpful to anyone planning
a lab evaluation of a privileged access management system. It describes the systems that must be made
available in the lab and the key tests which should be performed on candidate products.

2 Focusing on Technical Challenges

This document begins with the assumption that every product being considered can meet the most ba-
sic requirements – randomize passwords for privileged accounts, store passwords in an encrypted vault,
disclose passwords to users after they have been authenticated and subjected to access control policies.
Rather than focusing on these basics, this document will help prospective buyers focus on more advanced
elements – fault tolerance, scalability, workﬂow processes and more.

By focusing on the more technically difﬁcult components of the system, prospective customers will be able
to see clear differences between products.

© 2011 Hitachi ID Systems, Inc.. All rights reserved. 1


3 Replication, Latency and Bandwidth

Once a PAM system is deployed, it will change passwords on privileged accounts to random values on a
scheduled basis. This is good for security – attackers have a shorter time interval to guess passwords and
departed staff automatically lose access.

Randomizing passwords goes hand in hand with storing those password values in a secure vault, so that
authorized users will still be able to sign into these accounts. With passwords changing frequently, IT users
will become dependent on the vault’s availability to do their work.

While this is a secure arrangement, it introduces the potential for disaster: if the password vault is discon-
nected, damaged or destroyed, system administrators will be unable to do their jobs. There is a potential
here to escalate a disaster from a purely local problem – with the system where the password vault is stored
– to a global one, where IT change management is interrupted everywhere.

To avoid this problem, a PAM system must:

1. Replicate the vault between different servers, so that loss of a single server does not cause data loss.
2. Place replicated servers in different data centers, as far apart (physically, geographically) as possible,
so that a regional disaster does not cause data loss at other locations.

For a sobering historical review that explains why geo-diverse replication is important, refer to the notes in
Appendix A on Page 18.

Replication is conceptually simple, but can be complicated in practice:

1. Wide area networks, between data centers in different cities and perhaps on opposite sides of the
world, have the following characteristics:
(a) High network latency – up to 50ms across a continent and 350ms around the planet.
(b) Low bandwidth – typically no more than 10Mbps on a private WAN across a continent and 1Mbps
on a private WAN across an ocean.
(c) More frequent connection outages than LANs (e.g., a few times per year).
(d) Higher packet loss than LANs (e.g., 1 out of 1000 packets may be dropped).
2. Configuring the replication features native to database software can be very complex, requiring database
administration expertise, a deep knowledge of how each application uses the (to be replicated)
database and many days of configuration and testing work.
3. WAN connections are normally carried on infrastructure owned by third parties, so they usually cannot
be trusted to be secure against packet sniffing or injection.

When evaluating a PAM system, organizations should always:

1. Deploy at least two replicated nodes, each with its own vault. This simulates an arrangement where
loss of a single application server or database instance does not cause loss of current password
values.



2. Simulate a WAN between the two nodes – with high latency, low bandwidth and a small amount of
packet loss.

An evaluation using a single node is simply unrealistic and may be dangerous, since the selected product
may demo well in the lab but not be able to support a reliable, replicated architecture in practice.

An evaluation using replication over a high speed, low latency network is also unsafe, since replication that
works in a lab may not work well in a multi-site real-world deployment.

To evaluate products in this way, at least two PAM servers plus a WAN emulator are required. Fortunately,
this is easy to setup using virtual machines and open source software, as illustrated in Figure 1.

Privileged Access
Management System
Node 1

Latency: 100ms
Bandwidth: 1Mbps
Packet loss: 0.1%

WAN Emulator

Password Vault: Privileged Access
Node 1 Management System
Node 2

Application
Level
Replication

Password Vault:
Node 2

Figure 1: Multiple, replicated nodes and a wide area network simulator

A more robust evaluation would use three replicated nodes, where each node replicates through its own
WAN emulator instance to the other two.

The WAN emulator can be downloaded from this project:
http://wanem.sourceforge.net
including using this (free) virtual appliance:
http://wanem.sourceforge.net/download-vma.html

For products where replication is provided by the database server, rather than by the PAM system, the
conﬁguration should be as shown in Figure 2.

To summarize:



Database Server 1

Privileged Access Latency: 100ms
Management System Bandwidth: 1Mbps
Node 1 Packet loss: 0.1%

WAN Emulator

Password Vault:
Node 1

DB Native Database Server 2
Replication

Password Vault:
Node 2

Privileged Access
Management System
Node 2

Figure 2: Using a WAN emulator with native database replication

1. Evaluate with at least two nodes.
2. Replicate data between the nodes.
3. Pass replication traffic through a WAN emulator.

Testing with a single node or without a WAN emulator represents a significant business risk once the system
moves to production.

Once a replicated environment is configured, test with load and failure conditions:

1. Have each password management system change many passwords in a short time interval. Verify
that new password values are available on other nodes after a short time (ideally, just a few seconds).
2. Turn off one of the password management nodes. Verify that other nodes still have a full data set
and that they respond to the failure in a suitable fashion – for example, by stopping the scheduled
password change process so that new passwords are not stored in just one vault.
3. Re-enable the disabled node and verify that queued up updates from other nodes are automatically
written to its database.



4. Turn off and delete one node and walk through the process to rebuild its replacement. Gauge difﬁculty
and estimate the amount and impact of data loss.



4 Automatic Discovery and Classification

It is not practical to manually configure thousands of managed systems where a PAM system will change
passwords and control administrative access. In practice, automatic discovery of systems and accounts,
combined with automatic application of appropriate policies to each discovered system, is essential.

It is not practical to deploy thousands of servers or workstations in a lab environment. On the other hand,
it is essential to test the system’s capability to discover, auto-manage and auto-unmanage systems and
accounts on thousands of systems. The solution to this problem is to simulate thousands of systems,
without actually provisioning them physically or even virtually.

A simulator is a complex piece of software. Organizations considering deployment of a PAM system cannot
seriously be expected to develop one, purely to perform a lab evaluation. Moreover, there are no enterprise-
scale infrastructure simulators available for download (at least none that Hitachi ID Systems is aware of).
The challenge here is to acquire and use an infrastructure simulator in the PAM evaluation despite it being
too costly to develop one and there being none available for purchase or as open source downloads.

PAM software vendors must have such a simulator, or they would not be able to test the scalability and
flexibility of the auto-discovery infrastructure in their solution, especially as they prepare new versions for
release.

This leads to just three possibilities:

1. The vendor does not have an infrastructure simulator. Such vendors should be disqualified immedi-
ately, since they cannot test their own ability to deliver scalable, flexible solutions.
2. The vendor has a proprietary infrastructure simulator but is unable or unwilling to share it with prospec-
tive customers. This is suspicious – either the simulator is needlessly complex, which means that its
use by the vendor may be limited, or the simulator is not very capable, which means that the vendor’s
software is not well tested.
3. The vendor has a proprietary infrastructure simulator and is willing to share it with customers – typically
under terms of a non-disclosure agreement (NDA). This is the best outcome as it shows that the
vendor has robust internal quality assurance processes and is willing to show them to customers.

Organizations are encouraged to only consider PAM systems from vendors who can provide a robust in-
frastructure simulator for use in their prospective customers’ lab environments.

Using an infrastructure simulator, Hitachi ID Systems recommends testing that has the PAM system:

1. Automatically discover at least 1000 managed systems, drawn from computer objects in Active Direc-
tory. Increase this number based on the ultimate scale of the deployment and the capacity of PAM
servers available in the lab.
2. Automatically probe these systems, to extract their local users, groups, services, machine identifiers
and so on.
3. Simulate latency and connectivity problems when probing individual systems – i.e., some are slow to
respond (simulating network distance or busy systems) and others are simply unreachable.
4. Apply rules to decide which systems to manage at all. How flexible are these rules?



5. Apply rules to decide what security policies to apply to each system.
6. Apply rules to decide which accounts on managed systems are considered “privileged” and therefore
should be managed.
7. Apply rules to decide what security policies to apply to each account.
8. Apply rules to decide what systems and accounts to stop managing – for example because systems
have been decommissioned and are no longer responsive, or accounts have been deleted.

Since Windows servers are often both the most numerous and the easiest to discover, in the context of a
proof of concept (POC) it makes sense to focus the auto-discovery testing on them. With this in mind, it is
helpful to construct an Active Directory domain with many computer objects and link this to the simulator,
so that they can each simulated computer has a "state" (login IDs, groups, group memberships, services
and service accounts, password values, etc.). Each computer can then be "probed" by the PAM system, to
find its state. This is illustrated in Figure 3.

As can be seen in Figure 3, a simulator for auto-discovery can be thought of as a test harness for the PAM
system. It should include components to:

1. Create a landscape of computers and load this data into Active Directory computer objects.
2. Track the state of each simulated computer.
3. Simulate daily changes – new computers are provisioned, old ones modified, moved and deactivated.

4. Simulate connectivity problems – slow connections and failures to connect.
5. Allow the PAM system to "discover" both administrator and service accounts. For the latter, it should
be able to not only change passwords on the service accounts, but also notify suitable operating
system components of new password values.

A robust auto-discovery system is crucial. It should be able to execute in a reasonable amount of time (say
no more than a few hours, running daily) and be both flexible and reasonably easy to manage. Anything
else will lead to a system with very high total cost of ownership – which might ultimately be abandoned.

Always keep in mind that just because a PAM system can manage access to accounts on a few hundred
systems, this does not imply that the same system will be an appropriate solution in a larger organization,
with thousands of managed systems and where many of these managed systems are added or retired,
daily.



Generate Computer
List of State
computers
(all days)
Generate Computer
List
Represent the state
of each “computer.”

Add/mod/delete Initial load List of
AD domain
service accts

1 file per PC:
Read previous Accts, Grps, Store new service
state of Services, etc. account password
LDIF: Create computer
LDIF: change
computers,
computers
service accts
Randomly change Randomly “fail to connect”
(day 1,2,3,...) (day 0) some computer state to some systems
Write:
new state
Script file

Change admin,
service account
LDIFDE password
Active Directory
(real, not
simulated)

Simulated connector
(model Windows
List accts, computers Verify pw: chg svc acct pw servers)

Active Directory
Connector
Change
passwords
Auto discovery
process

Password
Vault

Scheduled password Simulated connector
randomization set service account PW
process
Privileged Access
Management System

Figure 3: Components required to simulate managed system auto-discovery



5 Firewalls

In larger organizations, the corporate network may be segmented into different parts, where segments are
isolated using firewalls. If a PAM system is deployed on one side of a firewall but systems with privileged
accounts exist on the other side, then the firewall is likely to prevent connections to target systems, as
shown in Figure 4.

System with
Privileged Accounts

Privileged Access
Management System

Password Vault

Figure 4: The need to traverse firewalls to securing access to privileged accounts

A simple approach is to modify firewall configuration to allow connections from the PAM system, as illus-
trated in Figure 5. This is a reasonable approach if there are very few firewall rules to change, but if the
firewall protects a large network segment with hundreds or thousands of systems, this approach will not
scale.

To address problems of scale and complexity (too many target systems on the other side of the firewall);
security (protocols used to connect to target systems are not intrinsically encrypted and safe) and perfor-
mance (protocols are sensitive to bandwidth or latency and the distance between the PAM system and
target systems is too great), a proxy server is needed, as illustrated in Figure 6.

A proxy server can be fast, secure, manageable and efficient. The drawback is that another system must
be deployed – typically a VM – in each isolated network segment.

If a proxy will be needed in production, the PAM system POC should include one and connect to some
systems through it. Test the installation footprint (should be small), configuration complexity and bandwidth
consumption of the proxy. Verify that communication between the main PAM system and proxy is encrypted.



System with
Privileged Accounts

Privileged Access
Management System

Password Vault

Figure 5: Enabling access by opening ports through the ﬁrewall

System with
Privileged Accounts

PAM
Proxy Server

Privileged Access
Management System

Password Vault

Figure 6: Enabling access by deploying a proxy server



6 Strong Authentication

A PAM system controls access to the most sensitive accounts across an entire organization. It is there-
fore reasonable to apply stronger authentication policies to it as compared to other applications, including
individual systems access to which it controls.

In practical deployments, strong authentication may mean use of smart cards or one time password tokens.

In a lab environment, setting up a PKI and CA for smart cards or a token server may be cost-prohibitive. A
less costly alternative suitable to demonstrating a ﬂexible authentication model is therefore appropriate.

A simple approach to multi-factor authentication is to enroll mobile phone numbers from users and send
them a random PIN - via SMS - as one of the login steps. Another inexpensive authentication factor is
simply asking users to answer personal security questions.

In a POC environment, it is reasonable to use one or both of these – both if the ability to route different
users to different login processes is needed:

1. Authentication process A: user types his own Active Directory password, which the PAM server vali-
dates against an AD DC.

2. Authentication process B: user is sent a random PIN via SMS to his mobile phone. He must enter this
ﬁrst, before being prompted to enter his AD password, as above.
3. Authentication process C: user completes AD/password login as in A above and then is asked to
answer 2 or 3 security questions.

The ability to use more than simply AD ID/password authentication and to dynamically select appropriate
login processes for different users should be tested and the above guidance shows how this can be done
without special hardware or software.



7 Access Controls

Once the PAM system has randomized passwords to accounts with elevated privileges, users will not be
able to sign into those accounts except by using the PAM system. The PAM system must then enforce
access controls: who can connect to what?

Access controls are defined using access control lists, which connect users to rights.

Assigning ACLs directly to users or directly to managed systems is not a good solution, since there may be
hundreds of users and thousands of systems under management. The number of ACLs connecting users
to systems would simply be unmanageably large.

In order to make access control manageable, it should be possible to group users, group accounts and
systems and link the two types of groups with rights. This means that the PAM system must support:

1. Defining sets of users that should be granted some rights. It is important to leverage existing informa-
tion about users when doing this:

(a) Define a set of users based on their membership in one or more Active Directory or LDAP groups.
(b) Define a set of users based on values of their attributes in a trusted system, such as AD or LDAP.
(c) Define a set of users based on the OU that houses their personal (unprivileged) account.

2. Defining sets of managed systems that users should get rights to. It is important to leverage data
about systems when doing this, such as the following

(a) Managed system’s DNS or NetBIOS name.
(b) Where in a directory (OU) the machine’s computer object was found. Active Directory or LDAP
groups.
(c) IP, in particular via subnet matching.
(d) Operating system type and patchlevel.

3. Identifying managed accounts through characteristics of the account within its system:

(a) Account is a member of a specified group (group name, SID).
(b) Account is used to run a service.
(c) Account name matches a pattern.

Access controls can then be defined in a pattern such as the following:

User group Rights System Account
• Member of AD group Launch RDP • IP in subnet 1.2.3.4/24. • Member of group with
Windows Admins on connections • OS type "Windows" SID S-1-5-32-544
domain AD-NA. (Administrators)
• Account in OU IT on
domain AD-NA.

• Member of LDAP group Launch SSH • IP in subnet 5.6.7.8/24. • UID 0 (root)
Linux Admins in connections • OS type "Linux|Unix"
directory LDAP1.


A lab evaluation should verify that:

1. Users can be collected into sets, which can be used to assign rights, based on their group membership
and attributes on multiple directories.

2. Systems can be collected into sets, based on their name, address, type and other attributes.
3. Accounts can be collected into the same sets as systems, based on their name, group memberships
(group ID or name) and whether they are used to run services (Windows-only).
4. Access rights can be assigned using the pattern described above – sets of users linked via rights to
sets of systems and accounts.

Anything less would create an excessive administrative problem.



8 Approval Workflow

In some cases, someone who does not satisfy the criteria described in Section 7 on Page 12 above will need
access to a privileged account. This can be thought of as a one-time or non-pre-authorized scenario. In
such cases, an authorization process is needed to control who can request access and who must approve
it before the user is allowed access to a privileged account.

Authorization means that multiple human beings are involved:

1. Exactly one requester – the person asking for access rights.
2. At least one recipient – the person who will get access rights.
3. Zero or more authorizers – people who will review the request and either approve or reject it.

The PAM must support:

1. Limiting who can act as a requester at all.
2. Limiting who can be a recipient:
• For some requesters, only self-service (requester and recipient are the same person).
• For others, only some recipients, based on their relationship (e.g., requester must be a manager
of the recipient, or in the same location, etc.)
3. Limiting what access can be requested:
• For example, the system being requested must be in the same location as the requester, or of a
specific type for a given recipient.
4. Requiring and validating form inputs, such as start/end date or an incident number.
5. Automatically selecting authorizers:
• Zero authorizers means that the request is automatically approved due to some business rule.
• One or more authorizers means that these people must review and approve the request before
it proceeds.
• Support an unlimited number of authorizers (not just one).
6. Intelligently cope with non-responsive authorizers:
• Check an authorizer’s e-mail out-of-office status, and select an alternate if the authorizer is known
to be unavailable.
• Send reminder e-mails to non-responsive authorizers.
• Be able to invite multiple authorizers at once, and accept approval from just a subset (e.g., any
two of these five...). Invitations sent all at once result in faster approvals as does the ability to
approve a request with just a subset of the invited people.
• Assign alternate authorizers in the event that a request remains open after multiple reminders
have been sent, as the original authorizers are likely unavailable.

The product evaluation should attempt to exercise as many of these options as possible – especially dy-
namic authorizer routing, multiple approvers and N of M approvers.



9 Building the Lab Environment

Pulling together the requirements from the preceding sections, a reasonable lab environment in which to
evaluate a PAM system should include the following:

1. Two or three servers on which to install the PAM software. A reasonable configuration for each of
these in a lab is:
(a) Virtual hardware: 8GB RAM, single core CPU, 250GB disk.
(b) Operating system: Windows 2008 32-bit (in most organizations it is reasonable to avoid 2008R2
as it requires a 64-bit virtual host).
(c) Database server (installed on the same server as the application in the POC): SQL Server 2008
Standard.
2. A virtual machine to host a WAN emulator.
(a) This is essential to test real-world, multi-site data replication.
(b) WANem is a 32-bit Linux-based VM.
3. A virtual Active Directory domain controller (AD DC).
(a) Should have at least 1000 users, several hundred groups, extensive group membership and an
organizational hierarchy. This data will be used to test authentication, authorization policies and
approval workflows.
(b) Hitachi ID Systems can provide sample contents for an arbitrarily-sized AD environment in the
form of LDIF files.
4. A target system simulator, so that auto-discovery, auto-classification and management of administrator
and service account passwords on thousands of systems can be evaluated.
(a) This has to be provided by each PAM vendor, since one or more simulated connectors are used
to accomplish this and since connector architectures are unique to each product.
(b) Hitachi ID Systems provides a simulator to every customer who wishes to use it.
5. A load balancer, capable of making web sessions “sticky” to a single server. A simple and free one,
again for Linux VMs, is Pen: http://siag.nu/pen/ – definitely suitable for a POC, though production
systems will likely demand more.
6. At least one real target system of each major type. Consider deploying the following (VMs are appro-
priate in all cases):
(a) A Windows Server 2003 target system.
(b) A Windows Server 2008 target system.
(c) A Linux server target system (RedHat, SuSe, Debian, Ubuntu, etc.).
(d) A SQL Server where access to SA accounts will be controlled.
(e) An Oracle Database where access to DBA accounts will be controlled.
7. Multiple workstation operating systems, both to test the PAM system’s ability to secure access to
administrator and service accounts on laptops and as user interfaces to the system:
(a) Windows XP.



(b) Windows 7.
(c) Others as required.

In practice, all of the above can be provisioned as virtual machines on two or three large virtual hosts –
quad-core CPUs, 24GB RAM, 1TB RAID disk with ESXi, Xen or similar. It can all be provisioned on a cloud
IaaS vendor such as Amazon EC2 as well.

The lab environment should be able to connect to the public Internet. This will support:

1. Remote assistance by the PAM vendor.
2. Sending e-mails to mobile phones through public e-mail-to-SMS gateways, for two factor authentica-
tion (AD password plus SMS random PIN).

The lab environment may include an e-mail service, to test workflow use of e-mails. A simple mail service
such as Postfix (http://postfix.org) is appropriate as it is much easier to setup than Exchange. Alternately,
the PAM system can simply drop e-mail messages on its filesystem.

The lab environment may include an incident management system (BMC Remedy, HP ServiceDesk, etc.).
This is used to test its ability to create, update and close incidents and also to verify that ticket numbers
entered by IT staff are valid, open, etc.

The lab environment may include a security incident and event management system (SIEM), or just a
Linux server with a networked SYSLOG service, to test log aggregation from the PAM system to a central
monitoring solution.



APPENDICES



A Why Replication Across Cities is Mandatory

If replication across multiple vaults, each in a different city, seems extreme, consider a recent history of
disasters and what would have happened to a global organization that had a data center containing their
sole password vault in an affected area. The human toll of each event is major, but in the following we focus
on the physical effects:

1. 1995: Kobe earthquake – destroys thousands of buildings in Kobe, Japan.
2. 2003: Northeast blackout – power lost in much of Eastern and Midwest US and Canada for up to 24
hours in some areas. Telecommunications and other infrastructure disrupted.
3. 2004: Indonesian tsunami – destroys towns and villages on Indonesian, Thai, Indian and other coast-
lines.
4. 2005: Hurricane Katrina – knocked out much of Louisiana and Mississippi.
5. 2008: Major earthquake in Szechuan – knocked out major infrastructure in Chengdu (a city of tens of
millions).

6. 2010: Haiti earthquake – destroyed much of Port-au-prince and the country’s infrastructure.
7. 2011: Tohoku earthquake and subsequent meltdown at the Fukushima power plant – physically de-
stroyed and made uninhabitable a large area in Northeast Japan.
8. 2011: New Zealand earthquake damages much of Christchurch.

The point here is that major natural and man-made disasters happen somewhere in the world with some
frequency and cannot be safely ignored as “infrequent” or “only happens somewhere else.” A robust system
must be built with the assumption that a single data center may be destroyed or at least disconnected from
power and/or telecommunications for at least a few hours and perhaps permanently.

500, 1401 - 1 Street SE, Calgary AB Canada T2G 2J3 Tel: 1.403.233.0740 Fax: 1.403.233.0725 E-Mail: sales@Hitachi-ID.com

File: /home/dawnm/hipam-poc-guidelines.tex
www.Hitachi-ID.com Date: 2011-09-23

Privileged Access Manager POC Guidelines

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à Privileged Access Manager POC Guidelines

Similaire à Privileged Access Manager POC Guidelines (20)

Plus de Hitachi ID Systems, Inc.

Plus de Hitachi ID Systems, Inc. (20)

Dernier

Dernier (20)

Privileged Access Manager POC Guidelines