Organizations worldwide are continually required to make significant investments in upgrading, re-engineering, and protecting their IT infrastructure. However, unlike application software development, many companies lack a structured quality assurance approach for infrastructure testing. Creating an infrastructure quality practice is an answer, but it's not without its challenges. However, if your company is interested in avoiding headline-grabbing outages, rooted in deployment problems with infrastructure—server, network, storage, middleware, telephony, hardware, IT security, cloud, virtual, and Data Center Ops—then come to this session. Carl Delmolino and Hitesh Patel explain how to identify and address infrastructure testing opportunities, how to build a diversely skilled infrastructure test team, and how to apply familiar SDLC testing process rigor to enterprise-level infrastructure change. When addressed effectively, infrastructure testing is risk mitigation at the far end of “left,” reduces organizational technical risk, and helps ensure higher system availability for employees and customers, alike.

1.
T18
Special
Topics
5/11/17
13:30
Infrastructure
Testing:
The
Ultimate
“Shift
Left”
Presented
by:
Carl
Delmolino,
Travelers
Insurance
Hitesh
Patel,
Infosys
Ltd.
Travelers
Insurance
Brought
to
you
by:
350
Corporate
Way,
Suite
400,
Orange
Park,
FL
32073
888-­‐-­‐-­‐268-­‐-­‐-­‐8770
·∙·∙
904-­‐-­‐-­‐278-­‐-­‐-­‐0524
-­‐
info@techwell.com
-­‐
http://www.starwest.techwell.com/

2.
Carl
Delmolino
Carl
Delmolino
is
the
director
of
quality
assurance
within
the
infrastructure
technology
division
of
Travelers
Insurance.
Before
going
off
the
rails
to
build
an
infrastructure
testing
practice,
Carl
spent
more
than
twenty
years
in
application
software
testing,
leading
BA,
QA,
and
automated
test
teams
in
the
personal
and
commercial
areas
at
Travelers.
As
the
creator
of
a
test
automation
community
of
practice
and
a
past
chairperson
of
Traveler's
Quality
Assurance
Roundtable,
Carl
is
a
proponent
of
enterprise
QA
collaboration.
Hitesh
Patel
Hitesh
Patel
is
a
group
project
manager
at
Infosys
Ltd.
and
co-­‐leads
the
Infrastructure
quality
assurance
team
with
Carl.
Hitesh
has
experience
in
various
QA
roles
in
his
eighteen
years
in
planning,
executing,
and
delivering
on-­‐time,
end-­‐to-­‐
end
testing
solutions.
He
is
passionate
about
quality
and
providing
innovative
QA
solutions
for
the
challenges
faced
by
testing
teams.

3. Infrastructure
Tes-ng
The
Ul-mate
Shi5
Le5
StarEast
2017,
Thursday,
May
11th
1:30pm
–
T18
1

4. Carl
Delmolino
Travelers
Insurance
Hitesh
Patel
Infosys
Presenters:
2

5. Infrastructure
Failures
DALLAS -- Southwest Airlines struggled to recover Thursday from a massive computer outage,
as several hundred more flights were canceled or delayed around the country.
Airline executives said a router breakdown set off a chain of failures in critical technology systems
a day earlier. Backup systems didn't work as expected, they said.
3

6. Infrastructure
Failures
John
HackeN,
HSBC
COO
said:
"We
profoundly
apologize
for
any
inconvenience
this
has
caused.
"There
is
a
complex
technical
issue
with
our
internet
banking
systems,
and
our
IT
team
has
been
working
non-­‐stop
since
yesterday
morning
to
find
a
solu-on.
This
has
involved
many
tests,
diagnos-cs
and
trial
runs.
We
are
geZng
closer
to
solving
the
problem,
but
are
not
there
yet.
We
can,
however,
confirm
this
is
not
a
cyber-­‐aNack
or
any
other
malicious
act."
Karl
Flinders,
Emea
Content
Editor,
Computer
Weekly
05
Jan
2016
15:20
4

7. Infrastructure
Failures
“Airlines
have
spent
heavily
to
introduce
new
features
such
as
automated
check-­‐in
kiosks,
real-­‐-me
luggage
tracking
and
slick
mobile
apps.
But
they
have
avoided
the
steep
cost
of
rebuilding
their
reserva-ons
systems
from
the
ground
up,
former
airline
execu-ves
said.”
A
failure
to
ensure
backup
power
for
some
of
its
computer
servers
led
to
the
meltdown
at
Delta
Air
Lines
this
week
that
forced
more
than
2,000
flight
cancella-ons
through
Wednesday
a5ernoon,
the
airline’s
top
execu-ve
said.
5

8. Infrastructure
Failures
Amazon's
cloud
service
has
outage,
disrup-ng
sites
SAN
FRANCISCO
—
Por-ons
of
Amazon
Web
Services,
the
na-on's
largest
cloud
compu-ng
company,
went
offline
Tuesday
a5ernoon,
affec-ng
millions
of
companies
across
the
United
States.
"This
is
a
preNy
big
outage,"
said
Dave
BartoleZ,
a
cloud
analyst
with
Forrester.
"AWS
had
not
had
a
lot
of
outages
and
when
they
happen,
they're
famous.
People
s-ll
talk
about
the
one
in
September
of
2015
that
lasted
five
hours,"
he
said.
The
outage
appeared
to
have
begun
around
12:35
pm
ET,
according
to
Catchpoint
Systems,
a
digital
experience
monitoring
company.
It
was
centered
in
AWS'
S3
storage
system
on
the
east
coast.
Elizabeth
Weise
,
USATODAY
,
KARE
3:20
PM.
CST
February
28,
2017
6

9. Why
is
Infrastructure
Tes-ng
needed?
Schedule
slippage
due
to
last
minute
issues
Diverse
and
rapidly
changing
landscape
Lack
of
tes-ng
rigor
and
tes-ng
skills
Missing
industry
best
prac-ces
and
standards
High
cost
of
Infrastructure
projects
7

10. How
does
Infrastructure
Tes-ng
shi5
quality
le5?
Infrastructure
Tes-ng
achieves
“Ul-mate
Shi5
Le5”
status
when
it…
• Employs
a
methodical
quality
approach
to
the
design,
evalua-on,
standing
up,
and
maintenance
of
various
infrastructure.
• Takes
place
before
any
related
development
of
applica-ons
on
a
given
infrastructure
begins.
• Enables
subsequent
so5ware
development
on
an
evaluated
founda-on.
• Helps
segregate
issues
found
in
So5ware
development
to
the
so5ware
itself,
rather
than
Infrastructure
issues
being
suspect.
• Exists!!!!
What
is
“Shi5
Le5”?
• An
approach
used
in
so5ware
tes-ng
and
system
tes-ng
in
which
tes-ng
is
performed
earlier
in
the
lifecycle
(i.e.,
moved
le5
on
the
project
-meline).
• An
approach
to
accelerate
so5ware
tes-ng
and
facilitate
development
by
moving
the
tes-ng
process
to
an
earlier
point
in
the
development
cycle.
8
Prod
Live
UAT
Perf
Test
Func-onal
Test
Unit
Test
Req/Design
Valida-on
Infrastructure
Tes-ng

11. What
is
Infrastructure?
• Computer
Hardware
-­‐
IBM,
HP,
Linux,
Apple,
Switches,
Hubs,
Servers,
Virtual,
etc
• End
User
Compu-ng
-­‐
Desktop
and
Mobile
technologies
• Middleware
-­‐
CICS,
IMS,
MQ,
Jboss,
.Net,
• Internet
Plauorms
-­‐
Apache,
MS
IIS,
.Net,
Unix,
Cisco,
Java
• Content
&
Collabora-on
Services
-­‐
Email,
Messaging,
Fax,
KX,
Audio/Visual,
SP,
Filenet
• Opera-ng
Systems
-­‐
MS
Windows,
Unix,
Linux,
Mac
OS
• Enterprise
Applica-ons
-­‐
SAP,
Oracle,
MS
Produc-vity,
Source
Code
Mgmt,
Customer
Rela-onship
Mgmt,
Employee
Mgmt,
etc.
• Database
Administra-on
• Networking/Telecommunica-ons
-­‐
Windows
Server,
Linux,
Cisco,
VOIP,
VPN,
Wireless
• Mainframe
• Voice
&
Service
Center
Technology
• Data
Management
&
Storage
-­‐
IBM
DB2,
SQL
Server,
Oracle
• Data
Centers/Facili-es
-­‐
Print,
Disaster
Recovery
• IT
Security
9

12. When
to
do
Infrastructure
Tes-ng
Infrastructure
Acquisi-on
• New
infrastructure
design
and
build
• New
infrastructure
func-onality
and
integra-on
• New
scalable
infrastructure
is
planned
Infrastructure
Migra-on
• Standardiza-on/
Consolida-on
• Migra-ons
–
Infrastructure,
applica-on
• Data
Center
migra-ons
• Database
migra-on
• Cloud
transi-on
Infrastructure
BAU
changes
• Environment
provisioning
• Patch
upgrades
and
updates
• OS
or
Database
changes
• Environment
standardiza-on
New
Laptops
New
phones
New
3rd
party
tools
Windows
10
W2K3
to
W2K8
W2K8
to
W2K12
New
patches
Security
fixes
Upgrades
Examples:
10

13. What
is
Infrastructure
Tes-ng?
• Environment
Verifica-on
Tes-ng
• Deployment
/
Installa-on
/
Uninstalla-on
/
Fallback
Tes-ng
• Hardware
/
Device
Tes-ng
• Connec-vity
Tes-ng
• Opera-onal
Readiness
Tes-ng
• Disaster
Recovery
Tes-ng
• Image
Tes-ng
(Client
/
Server)
Core
Infrastructure
• Configura-on
Tes-ng
• System
Integra-on
Tes-ng
• Volume
&
Performance
Tes-ng
• Security
Tes-ng
• MDM
Tes-ng
• Component
Tes-ng
3rd
Party
App
Integra-on
• Smoke/Sanity
Tes-ng
• Func-onality
/
System
Tes-ng
• Integra-on
Tes-ng
• Regression
Tes-ng
• Performance
Tes-ng
• User
Acceptance
Tes-ng
• Usability
Tes-ng
Home
Grown
App’s
11

14. Examples
of
Infrastructure
Tes-ng……
Client/Server
• Test
and
Cer-fy
the
quality
of
Applica-ons,
Servers,
Desktops,
Patches
,OS,
Image
and
Hardware.
• Test
the
infrastructure
components
to
reduce
the
possible
down-me
impact
in
produc-on
and
help
to
improve
the
performance
of
the
infrastructure.
Cloud
• Test
and
cer-fy
the
Private
Cloud
-­‐
IaaS
service
offerings
for
Release
and
Itera-ons.
• Self
service
portal
–
Catalog
service
request
form.
• Build
-­‐
VM
lifecycle
management
–
VM
Provisioning/
De-­‐Provisioning
• VM
Templates–
OS
images
and
Packages
• Test
and
Cer-fy
the
Cloud
automa-ons
and
Orchestra-ons.
Data
Center
Migra3on
• Test
and
cer-fy
the
migra-on
of
DR
capabili-es
from
DR
datacenter
to
new
data
center.
• Test
the
different
migra-on
methodologies
such
as
New
server
build,
V2V,
P2P,
Li5
and
shi5.
• Test
the
servers
and
ensure
that
there
are
no
changes
due
to
migra-on
• Test
and
ensure
no
applica-on
latency
or
performance
are
observed.
Mobility
• Verify
corporate
mobile
devices
are
able
to
manage
their
corporate
emails,
calendar
,
contacts,
personal
documents,
intranet
access,
text
and
video
messaging.
• Verify
Mobile
Device
Mgmt
(MDM)
func-onality
a5er
changes
(MDM,
OS
version,
App
version)
FileNet
• Ensure
Filenet
IBM
Content
Navigator(ICN)
func-onali-es
are
not
impacted
a5er
every
bundle
releases
(consists
of
IBM
updates,
Security
updates
mandated
by
IS
security
team)
• Control,
Maintain
and
support
for
future
Filenet
upgrades.
• Ensure
Filenet
web
services
are
not
impacted
a5er
every
bundle
release.
Test
Environment
Mgmt
• So5ware
installa-on
• Applica-on
configura-on
setup
and
changes
• Inventory
management
• Ac-ve
Directory
administra-on
• Currency
maintenance,
Problem
resolu-on
12

15. How
to
build
a
Infrastructure
Tes-ng
prac-ce?
Process
Metrics & Repor/ng
Opera/ng Model
Infrastructure
Organiza/on
Methodology
§ Weekly
/
Monthly
Report
§ Produc-vity
and
quality
§ Basic
Tes-ng
metrics
§ Advanced
Tes-ng
metrics
§ Test
techniques
and
methods
§ Test
Case
management
§ Test
Data
Management
§ Defect
Preven-on
§ Performance
tes-ng
§ Tools
&
Automa-on
§ Test
Environment
§ Technology
stack
§ Sourcing
Model
§ Effort
Es-ma-on
§ Workload
Management
§ Communica-on
§ Knowledge
Management
§ Structure
§ Goals
&
Objec-ves
§ Collabora-on
§ People
capabili-es
§ Strategic
Alignment
§ R&R
§ Quality
plan
§ Opera-ons
guide
§ Test
Requirement
§ Test
Strategy
&
Planning
§ Change
Request
§ Risk
Management
13

16. What
are
the
Benefits
of
Infrastructure
Tes-ng?
Improved
quality
of
service
Higher
availability
of
systems,
stable
environment
Reduced
incidents
caused
by
change
Reduced
opera-onal
risks
&
costs
Improved
-me
to
market
Improved
stability
of
applica-ons
Enhanced
custom
knowledgebase
–
faster
problem
resolu-on
14

17. Summary
1. Catastrophic
Infrastructure
failures
are
rare
but,
when
they
happen,
are
costly
in
many
ways
2. Smaller
infrastructure
issues
are
more
frequent
and
these
result
in
a
loss
of
produc-vity
and
increased
opera-ng
cost,
which
can
be
cumula-vely
significant
3. Infrastructure
change
is
inevitable
4. Infrastructure
tes-ng
helps
you
shi5
le5,
by
finding
issues
long
before
other
types
of
tes-ng
begin
5. An
Infrastructure
Tes-ng
Team’s
job
is
to
iden-fy
what,
when
and
how
to
test
6. SeZng
up
a
Infrastructure
tes-ng
team
will
help
your
organiza-on
in
the
long
term

18. THANKS
16

19. Infrastructure
Tes-ng
–
Appendix
17
1. Server
/
Client
2. Cloud
3. Data
Center
Migra-on
4. Disaster
Recovery
5. Test
Environment
Management
6. Mobility
7. Filenet
8. Sharepoint
9. Documaker

20. Infrastructure
Tes-ng
Procedure
–
Server/Client
Test
Objec3ve
• Test
and
Cer-fy
the
quality
of
Applica-ons,
Servers,
Desktops,
Patches
,OS,
Image
and
Hardware.
• Test
the
infrastructure
components
to
reduce
the
possible
down-me
impact
in
produc-on
and
help
to
improve
the
performance
of
the
infrastructure.
• Infrastructure
tes-ng
helps
to
ensure
an
error
free
environment
in
produc-on
and
keeps
the
environment
healthy.
• Reduce
the
security
vulnerability
risks
earlier
by
tes-ng
in
QA
before
releasing
to
Produc-on.
• Ensure
fallback
plans
are
tested
and
ready
before
the
actual
implementa-on
in
produc-on
• Provides
worry
free
migra-on
of
So5ware/Hardware/OS
upgrades
in
infrastructure.
Test
Procedure
• Stakeholder
submits
the
test
request
with
all
required
informa-on
needed
for
tes-ng
• Tester
collects
the
requirements
from
the
request
and
dra5s
a
Test
plan
with
an
understanding
in
structure
including
in-­‐scope/out
of
scope
considera-ons,
prerequisites,
applicable
OS,
fresh/upgrade
scenarios
and
func-onality
tes-ng.
• Tested
discusses
the
created
Test
plan
with
the
Stakeholder
and
obtains
the
signoff.
• Tester
executes
the
scenarios
outline
in
Test
plan
on
selec-ve
target
machines
and
validates
the
expected
checkout
results.
In
case
of
discrepancies,
a
defect
is
logged
in
the
defect
tracking
tool
and
tracked
un-l
closure.
• Stakeholder
will
respond
to
the
defect(s).
Tester
will
retest
and
verify
that
defect(s)
are
fixed.
• Tester
documents
the
results
obtained
during
execu-on
of
test
plan.
• The
procedure/package
is
cer-fied
and
approved
for
implementa-on
in
produc-on.
18

21. Infrastructure
Tes-ng
Procedure
–
Server/Client
Test
Types
• Stability
Test
• Smoke
tes-ng
• Installa-on
/
Uninstalla-on
tes-ng
• Package
/Bundle
tes-ng
• Security
Patch
tes-ng
• Browser
/extension
tes-ng
• Fallback
Tes-ng
• Integra-on
Tes-ng
• Func-onality
Tes-ng
• Hardware
Cer-fica-on
Benefits
• Package
methods
used
for
so5ware
fresh/upgrade
installa-on
and
configura-on
changes
are
tested
on
QA
servers/clients.
This
validates
the
code
behavior
earlier
and
thereby
reduces
the
possible
impact
in
produc-on.
• Microso5
security
patches
are
released
every
month
and
are
tested
in
QA
on
different
opera-ng
plauorms
such
as
Windows
7,
Windows
10,
Windows
Server
2008,
Windows
Server
2012,
etc.,
before
rollout
to
produc-on.
• All
types
of
browsers
and
extensions
released
by
vendor
are
tested
in
QA
before
rollout
in
produc-on
.
• Monthly
bundles,
consis-ng
of
several
standalone
packages
are
tested
on
servers/clients
in
the
QA
environment,
providing
the
ability
to
iden-fy
poten-al
risks
earlier
.
• Fallback
procedures/packages
are
tested
in
QA
to
ensure
back
out
plans
are
working
and
in
place
before
the
actual
migra-on/deployment
in
produc-on.
19

22. Infrastructure
Tes-ng
Procedure
–
Cloud
Test
Objec3ve
• Test
and
cer-fy
the
Private
Cloud
-­‐
IaaS
service
offerings
for
Release
and
Itera-ons.
• Self
service
portal
–
Catalog
service
request
form.
• Build
-­‐
VM
lifecycle
management
–
VM
Provisioning/
De-­‐Provisioning
• VM
Templates–
OS
images
and
Packages
• Test
and
Cer-fy
the
Cloud
automa-ons
and
Orchestra-ons.
Test
Procedure
• Migrate
the
templates,
Code
and
Blue
prints
from
DEV
to
QA
Environment.
• Configure
and
validate
the
Catalogue
items,
Blue
prints,
Build
Profiles,
Proper-es
and
workflows.
• Set
up
the
QA
environment
to
kick
off
the
builds
based
on
the
requirements.
• Build
-­‐
Provision
the
VM’s
for
OS,
Web
and
DB
service
offerings.
• Test
-­‐
Perform
the
regression
tes-ng
and
ensure
that
required
so5ware's
are
installed.
• Cer-fy
–
Provide
the
QA
sign
off
to
migrate
the
templates,
code
and
Blue
prints
to
the
produc-on
environment.
Test
types
and
results
• Component
Tes-ng
• System
Tes-ng
• Installa-on
Tes-ng
• Uninstalla-on
Tes-ng
• End
to
End
Tes-ng
• Regression
Tes-ng
Benefits
• Error
free
implementa-on
in
produc-on
environment.
• Simplified
request
form
for
beNer
user
experience
• Automated
test
cases
reduces
the
tes-ng
cycle
and
improves
the
go
to
market
-me.
20

23. Infrastructure
Tes-ng
Procedure
–
Data
center
Migra-on
Test
Objec3ve
Ensuring
the
partnership
with
Data
center
team
to
test
&
cer-fy
the
datacenter
migra-on
and
focusing
on
con-nuous
Quality
improvements.
• Test
and
cer-fy
the
migra-on
of
DR
capabili-es
from
DR
datacenter
to
new
data
center.
• Test
the
different
migra-on
methodologies
such
as
New
server
build,
V2V,
P2P,
Li5
and
shi5.
• Test
the
servers
and
ensure
that
there
are
no
changes
due
to
migra-on
• Test
and
ensure
no
applica-on
latency
or
performance
are
observed.
Test
Procedure
–
Pre,
Post
and
Compare
• VM
/
Physical
-­‐
Test
the
servers
before
and
a5er
the
migra-on
and
ensure
that
there
are
no
configura-on
changes.
• New
Build
–
Test
and
ensure
that
newly
build
servers
are
built
based
on
the
exis-ng
server
requirements.
• P2P
/Li5
shi5
–
Ensure
that
checklist
is
followed
to
physically
move
server
from
exis-ng
DC
to
new
Datacenter
and
no
changes
on
configura-on
during
the
physical
move.
• Produc-on
checkouts
are
performed
a5er
the
successful
infrastructure
migra-on.
• Confirmed
that
there
are
no
changes
on
applica-on
performance.
Test
types
and
results
• Installa-on
test
• Configura-on
Test
• Cer-fica-on
Test
• Stability
Test
• Produc-on
checkout
Value
adds
• Iden-fies
the
poten-al
server
configura-on
mismatches
• Iden-fies
the
server
build
issues
in
during
the
build
phase
• Improves
the
Quality
of
deliverable
across
the
project
life
cycle
21

24. Infrastructure
Tes-ng
Procedure
–
Disaster
Recovery
Test
Test
Objec3ve
Disaster
Recovery
Tes-ng
is
to
conduct
disaster
recovery
exercises
in
the
DR
Data
Center
that
test
end-­‐to-­‐end
business
func-ons
and
services
• Reduce
Overall
Risk
during
disaster.
• Maintain
and
Test
Your
Disaster
Recovery
Plan
• Improve
the
confident
of
business
owners
• Restore
cri-cal
business
func-ons
and
mission
cri-cal
applica-ons
• Rapid
Response
during
the
failure
Test
Procedure
• Isolated
network
–
Designed
to
perform
the
DR
test
without
impac-ng
the
produc-on.
• DR
tests
are
designed
to
perform
the
tes-ng
on
business
func-ons
or
mission
cri-cal
applica-ons.
• Core
infrastructure
recovery
–
Bring
up
and
test
the
core
infra
services
like
AD,
DNS
and
DHCP
in
Isolated
network.
• Infrastructure
recovery
–
The
required
infrastructure
will
be
brought
up
on
the
isolated
network
and
infrastructure
checkout
will
be
performed.
• Applica3on
recovery
–
The
applica-ons
are
brought
up
as
required
and
sanity
test
will
be
performed
on
the
recovered
applica-ons.
Test
types
and
results
• Component
Tes-ng
• Tabletop
Test
• Dry
Run
Test
• Structured
Walkthrough
Test
• Parallel
/
Combined
Component
Tes-ng
• Cutover
Test
Value
adds
• Improve
the
recovery
-me
• Con-nuously
improve
the
RPO
and
RTO
• Automate
the
recovery
plans
22

25. Tes3ng
Objec3ves
• So5ware
installa-on
• Applica-on
configura-on
setup
and
changes
• Inventory
management
• Ac-ve
Directory
administra-on
• Currency
maintenance
• Problem
resolu-on
TEMS
Roles
and
Responsibili3es
1. Inventory
(Monthly)
a. Asset
management
b. Scans
&
reports
c. Server,
client,
and
storage
costs
d. LAN
Center
&
domain
breakdown
of
QA
technology
resources
2. Administra3on
(Day-­‐to-­‐day)
a. Manage
system
access
&
permissions
for
QA
technology
resources
b. Domains
including
IP
Address
configura-ons,
Sites
c. Client
lab
d. Environment
incidents
e. Monitors
&
alerts
f. Test
bed
designa-ons
for
servers
3. Provisioning
(On
demand)
a. Submit
server
build
requests
b. Build
client
machines
c. Implementa-on
target
machine
groups
4. Compliance
(Monthly)
a. Schedule
backfills
b. Execute
and
review
compliance
jobs
c. Report
discrepancies
5. Environment
backfills
&
currency
(Ongoing)
a. Execute
IQA
backfill
requests
b. Define
targets
c. Maintain
machine
groups
6. Schedule
development
(Annual)
a. Environment
backfills
including
gold
server
images
b. Develop
backfill
schedules
for
all
QA
technology
resources
c. Schedule
server
build
image
updates
for
monthly
infra
releases
–
Windows
&
Unix
7. Repor3ng
(Monthly)
a. Provisioning
for
all
QA
technology
resources
b. Project
ac-vity
c. QA
alerts
with
a
breakdown
by
classifica-on
(warning,
low,
med,
high)
d. Backfill
volumes
and
work
effort
8. Monitors
&
Alerts
a. QA
environment
alert
implementa-on
b. Produc-on
LFA/ELA/other…
alerts
for
QA
tes-ng
in
QA
environment
9. Ac3ve
Directory
health
checks
(Annual)
a. Par-cipate
in
Microso5
review
sessions
b. Implement
Microso5
produc-on
recommenda-ons
in
QA
c. Resolve
ac-ve
directory
issues
10. Manage
server
and
client
environment
issues
a. Resolve
environmental
incidents
for
all
technology
resources
b. Designate
environment
incidents
to
owning
poruolios
as
necessary
Test
Environment
Management
23

26. Infrastructure
Tes-ng
Procedure
–
Mobility
Test
Objec3ve
• Ensure
employees
having
corporates
iPhone
are
able
to
manage
their
corporate
emails,
Calendar
,
contacts
,
personal
documents
,
Access
intranet
sites
,
manage
Skype
for
business
on
mobile.
• Ensure
MDM
solu-on
is
correctly
managing
corporate
devices.
• Ensure
MDM
func-onality
works
a5er
any
change(
AirWatch
update
,
IOS
version
,
Applica-on
version
etc.)
Test
Procedure
• Receive
Test
request
from
Project
team
• Create
test
strategy
if
required
,
Test
cases
etc.
• Prepare
devices
for
tes-ng
• Verify
func-onality
in
lower
environment
and
provide
sign-­‐off
• PROD
change
and
checkout
• Tes-ng
on
IOS
versions
,
IOS
beta
versions
,
Applica-on
beta
version
in
PROD
region
Test
types
and
results
• Configura3on
tes3ng
–
Test
configura-on
of
servers
,
Enterprise
systems
integra-on
,
Ac-ve
Directory
Sync
• Enrollment
tes3ng
-­‐
Test
Device
enrollment
in
AirWatch
,
test
user
authen-ca-on
and
policy
or
terms
tes-ng
• Security
Tes3ng
–
Test
all
restric-ons
applied
on
managed
devices
as
per
corporate
or
industry
standards
,
Compliance
tes-ng
• Profiles/Policies
tes3ng
–
Test
policies
for
app
user
creden-als
for
accessing
enterprise
resources
,
Test
profiles
applied
to
managed
devices
–
e.g.
–
Email
profile
,
VPN
profile
,
Wi-­‐Fi
profile
etc.
• Distribu3on
tes3ng
–
Test
mul-ple
devices
to
ensure
internal/public/purchased
apps
installed/distributed
through
company
app
catalogue
• Monitor
and
tes3ng
–
Test
console/dashboard
to
monitor
devices
a5er
distribu-on
of
apps
,
Test
push
no-fica-on
,
test
commands
for
managed
devices
• Func3onality
tes3ng
-­‐
Test
Corporate
Emails
,
contacts
,
calendar
,
Content
management
,
browser
to
access
intranet
sites
etc.
on
managed
devices
,
Skype
for
Business
mobile
• Backup
and
restore
tes3ng
–
Test
iTunes
to
ensure
device
backup
and
restore
works
on
latest
IOS
version
• Regression
tes3ng
–
Test
all
exis-ng
test
scenarios
with
every
IOS
release
,
Environment
updates
etc.
,
Execute
few
cri-cal
test
cases
everyday
in
PROD
region
to
iden-fy
outage
if
any
• Beta
tes3ng
–
Test
on
IOS
beta
version
,
AirWatch
beta
applica-ons
to
ensure
func-onality
works
on
beta
versions
24

27. Infrastructure
Tes-ng
Procedure
–
SharePoint
Test
Objec3ve
• Ensure
SharePoint
func-onality
is
not
impacted
a5er
Microso5
patches
are
applied
to
the
environment.
• Control,
Maintain
and
support
for
future
SharePoint
upgrades(e.g
SharePoint
2013
to
SharePoint
2016).
• Ensure
SharePoint
func-onali-es
are
not
impacted
a5er
every
enhancement.
• Ensure
that
Company
branding
(look
and
feel
of
the
SharePoint
portal)
is
not
affected
by
any
of
the
above
men-oned
change.
Test
Procedure
• Receive
Test
request
from
SharePoint
Dev
Ops
team
or
engage
in
the
project
through
the
Demand
Management
process.
• Schedule
a
mee-ng
with
PM,
BA,
Engineers
to
discuss
the
requirements.
• Define
Test
Strategy,
Design
Test
Condi-ons,
Test
Cases.
• Ensure
Test
Environment
readiness
and
prepare
test
data.
• Perform
Test
execu-on
and
publish
the
Test
Execu-on
report,
defect
repor-ng.
Test
types
and
results
• Smoke
tes-ng
• System
Integra-on
tes-ng
• Func-onal
tes-ng
• UAT
tes-ng
• Regression
tes-ng
• Produc-on
checkout
25

28. Infrastructure
Tes-ng
Procedure
–
Filenet
Test
Objec3ve
• Ensure
Filenet
IBM
Content
Navigator(ICN)
func-onali-es
are
not
impacted
a5er
every
bundle
releases
(consists
of
IBM
updates,
Security
updates
mandated
by
IS
security
team)
• Control,
Maintain
and
support
for
future
Filenet
upgrades.
• Ensure
Filenet
web
services
are
not
impacted
a5er
every
bundle
releases.
• Ensure
ICC
Inges-on
is
working
as
expected.
Test
Procedure
• Receive
Test
request
from
Filenet
team
or
engage
in
the
project
through
the
Demand
Management
process.
• Schedule
a
mee-ng
with
PM,
BA,
Engineers
to
discuss
the
requirements.
• Define
Test
Strategy,
Design
Test
Condi-ons,
Test
Cases.
• Ensure
Test
Environment
readiness
and
prepare
test
data.
• Perform
Test
execu-on
and
publish
the
Test
Execu-on
report,
defect
repor-ng.
Test
types
and
results
• Smoke
tes-ng
• System
Integra-on
tes-ng
• Func-onal
tes-ng
• Middleware
tes-ng
• UAT
Tes-ng
• Security
Tes-ng
• Performance
Tes-ng
• Regression
tes-ng
• Produc-on
checkout
26

29. Infrastructure
Tes-ng
Procedure
–
Documaker
Test
Objec3ve
• Ensure
Documaker
forms
are
working
a5er
every
Documaker
upgrade.
• Ensure
each
LOB(Lines
of
Business)
forms
are
working
by
execu-ng
the
XML
script
and
verify
the
file
generated.
• Ensure
Documaker
forms
are
working
a5er
every
patches
applied
to
Documaker
tool.
Test
Procedure
• Receive
Test
request
from
the
project
team
or
engage
in
the
project
through
the
Demand
Management
process.
• Schedule
a
mee-ng
with
PM,
BA,
Engineers
to
discuss
the
requirements.
• Define
Test
Strategy,
Design
Test
Condi-ons,
Test
Cases.
• Ensure
Test
Environment
readiness
and
prepare
test
data.
• Perform
Test
execu-on
and
publish
the
Test
Execu-on
report,
defect
repor-ng.
Test
types
and
results
• Smoke
tes-ng
• System
Integra-on
tes-ng
• Func-onal
tes-ng
• UAT
tes-ng
• Regression
tes-ng
• Produc-on
checkout
27