presentation by Daniel Reimann at Dominopoint Days 2013

1. Back to the future
Understand and Optimize your IBM Notes/Domino infrastructure
Daniel Reimann

2. Introduction
• The goal: Thinking about going to the cloud, implementing Connections,
migrating, consolidating or growing your environment?
– Crucial data lives in various places in your IT infrastructure
– Doing any of the above projects requires you to look at this data in a “big picture”
view
• The challenge: How do you connect the various information silos?
– Knowledge is spread out in the environment in numerous repositories on several
servers
– In many situations, companies aren't even aware of the information available to
them
• How we can help: Going to back to the future to move forward
– Find out how your environment looks today and get surprising results
– Use these gathered results to solve existing issues and be well prepared for
implementing or connecting new services and optimizing or extending your
infrastructure

3. Session content:
(Some) results of a Domino Network Analysis Health Check

4. About DNA®
• Service Offering
• Unique multi dimensional Insight in Utilisation &
Configuration of IBM Notes/Domino
• Overall Health Check / SWOT of Customer Environment
• Data Collection -> Analysis -> Reporting
• IBM & Business Partners reselling DNA World Wide
• Executed remotely by panagenda/Trust Factory
ZEROImpact
Zero Impact: means:
Impact
means:
Nothing to Install
No Installation
Leaves No Footprint
Leave No Footprint
No Dependancy
Deliver in a Few Days
No Footprint

5. DNA® Service Modules
DNA® Health Check
Starting Point for
all DNA Services
Server Consolidation Support
Optional Service
Root Cause & Performance
See slides at the end of
this presentation
Optional Service
See slides at the end of
this presentation
Source Code Analysis
Optional Service
See slides at the end of
this presentation

6. What Results will it Provide?
• Reveales Realistic Ambitions for:
•
•
•
•
Cleanup & Optimization Opportunities
Server Consolidation & Network (Optional Module)
Application Migration (Optional Module)
Performance Optimization (Optional Module)
• Interactive Slide Deck
• Linked to Common Spreadsheets & Check Lists
• Recommendations
• Presented Live
• Along with Explanation & Interpretation of Analysis Results by Subject Matter
Expert

7. What can you do with the results?
• IT Director / CIO:
•
•
•
•
Executive Decision-Making Support
Validation of Business Case
Define Innovation & Cloud Strategy
Current State of Affairs
Each DNA
®
delivers:
Quick Wins
Project Wins
Strategic Wins
• Project Manager & Teams:
• DNA Facts & Findings Help Focus & Prioritize
• Detailed Helicopter View replaces Micro View
• Prepare any Scenario with DNA Data Points
• For Administration & Support:
• Cleanup & Optimize by Means of Actionable Check Lists
• Improve Service Levels

8. Scope
• Activity by Rich Clients
• Web users not included
• 7 days
• 26 Servers in Scope

9. Management Summary
• Today:
26 Domino Servers
4 Different Releases
22544, GB Storage
Databases:
102, Integrity Issues
1020, Open to Anonymous
Directories:
110, Conflicts/Duplicates
18 Weak Passwords
• Tomorrow:
6 Domino Servers *
1 Single 9.0 Release
8316, GB Storage
Issues solved
* Based on observed session concurrency of 2.167
(clustered, excl. special functions )

10. Domino Environment Overview
1 Domino Directory
2,806 Users Registered
2,064 Users Active
3,345 Databases Touched
2,055 Users sending email
46 Servers Registered
149,515 Views Indexed
26 Servers Analyzed
15,572 Databases Deployed
1,177,671 Views Defined
4 Domino Releases
176 View Storage (GB)
4 Operating Systems
144,164 ACL Entries
589 Groups Registered
22,594 Db Storage (GB)
23,605 Group Members

11. DNA Benchmark
Active versus Registered Users
100 %
2,064 active users
80 %
60 %
40 %
20 %
0%
Demo Inc
Lowest Customer
Unused Licenses, Web Users, Regular Absense
DNA Average
Highest Customer

12. DNA Bechmark
Time Online
50
20
40
15
30
10
20
5
10
-
Demo Inc.
Lowest Customer
DNA Average
Highest Customer
Session Duration
3
1
4
28
Online Time
37
5
17
44
-
Online Time
(hours per user)
60
25
Session Duration
(mins per session)
30

13. DNA Bechmark
Document Reads/Writes
750
2.000
500
1.000
250
-
Demo Inc.
Lowest Customer
DNA Average
Highest Customer
Document Reads
1.983
304
2.093
3.652
Document Writes
301
152
385
906
-
Document Writes
1.000
3.000
Document Reads
4.000

14. DNA Bechmark
Network Bandwidth Consumption
8
30
6
20
4
10
2
0
Demo Inc.
Lowest Customer
DNA Average
Highest Customer
server to clients
8,7
1,9
10,3
38,2
clients to server
1,6
0,5
2,1
8,1
0
clients to server
(kilobits per sec)
10
40
server to clients
(kilobits per sec)
50

15. User Demand Profiling
(Demo Company, 2064, active accounts)
20%
Remote Users <
> Of f ice Workers
System Accounts
15%
10%
5%
0%
2
4
6
8
10
12
14
Distinct Hours Online per Day
16
18
20
22
24

16. End User Demand Characteristics
Demo Company
100%
75%
50%
25%
0%
Notes Sessions
Document Reads
Document Writes
Db Transactions
Network Traffic
Session Duration
19%
0%
0%
4%
0%
3%
6%
5%
47%
8%
1%
1%
mail files
52%
69%
47%
62%
87%
89%
directories
17%
22%
2%
23%
6%
3%
applications
6%
4%
4%
3%
5%
4%
check new mail
system dbs

17. KiloBytes Sent to Server
User Demand on 3341, Databases
Demo Company
100.000.000
10.000.000
1.000.000
100.000
10.000
1.000
100
10
1
0
1
10
100
1.000
10.000
100.000
1.000.000
10.000.000
100.000.000
KiloBytes Read from Server
Application
Domino Directory
Mailfile
Mailin database
Server Mail Box
System database

18. End User Demand at Demo Company
Classified by Demand Level
Document
Writes
Document
Reads
Database
Transactions
Network Traffic
(client to server)
Network Traffic
(server to client)
User Sessions
0%
25%
Extreme (,0)
50%
Intensive (2,0)
75%
Moderate (62,0)
100%
Light (2000,0)

19. User Activity by Top 10 Accounts
(Document Reads)
25.000
20.000
user_810
user_864
user_1831
15.000
user_1364
user_894
user_1950
user_2652
10.000
user_996
user_2243
user_657
5.000
18
12
6
DD-Okt-YY 0
18
12
6
DD-Okt-YY 0
18
12
6
DD-Okt-YY 0
18
12
6
DD-Okt-YY 0
18
12
6
DD-Okt-YY 0
18
12
6
DD-Okt-YY 0
18
12
DD-Okt-YY 0
6
0

20. Domino Servers at Demo Company
Classified by Maximum Session Concurrency
20
Redistributing the load can reduce nr. of
servers with up to 18,
15
10
5
0
Level
Servers
Very Low
< 50
Low
50 - 249
Average
250 - 749
Normal
750 - 1499
High
> 1500
18
6
0
2
0

21. 20
16
12
08
04
2010-10-27 00
20
16
12
08
04
2010-10-26 00
20
16
12
08
04
2010-10-25 00
20
16
12
08
04
2010-10-24 00
20
16
12
08
04
2010-10-23 00
20
16
12
08
04
2010-10-22 00
20
16
12
08
04
2010-10-21 00
Concurrent User Session
End User Demand
Session Concurrency
2,500
Max Observed Maxim um : 2,167
2,000
1,500
1,000
500
0

22. 20
16
12
08
04
2010-10-27 00
20
16
12
08
04
2010-10-26 00
20
16
12
08
04
2010-10-25 00
20
16
12
08
04
2010-10-24 00
20
16
12
08
04
2010-10-23 00
20
16
12
08
04
2010-10-22 00
20
16
12
08
04
2010-10-21 00
Avg per Hou
End User Demand
Document Reads
2,500
Max Observed Average: 1,911
2,000
1,500
1,000
500
0

23. 20
16
12
08
04
2010-10-27 00
20
16
12
08
04
2010-10-26 00
20
16
12
08
04
2010-10-25 00
20
16
12
08
04
2010-10-24 00
20
16
12
08
04
2010-10-23 00
20
16
12
08
04
2010-10-22 00
20
16
12
08
04
2010-10-21 00
Avg per Hou
End User Demand
Document Writes
800
Max Observed Average: 678
700
600
500
400
300
200
100
0

24. 20
16
12
08
04
2010-10-27 00
20
16
12
08
04
2010-10-26 00
20
16
12
08
04
2010-10-25 00
20
16
12
08
04
2010-10-24 00
20
16
12
08
04
2010-10-23 00
20
16
12
08
04
2010-10-22 00
20
16
12
08
04
2010-10-21 00
Avg per Hou
End User Demand
Database Transactions
16,000
Max Observed Average: 13,498
14,000
12,000
10,000
8,000
6,000
4,000
2,000
0

25. 20
16
12
08
04
2010-10-27 00
20
16
12
08
04
2010-10-26 00
20
16
12
08
04
2010-10-25 00
20
16
12
08
04
2010-10-24 00
20
16
12
08
04
2010-10-23 00
20
16
12
08
04
2010-10-22 00
20
16
12
08
04
2010-10-21 00
Avg per Hour (kbps, to clien
End User Demand
Network Bandwidth Consumption
25,000
Max Observed Average: 23,616
20,000
15,000
10,000
5,000
0

26. Network Compression
How Much is Notes Network Compression Used?
Includes Traffic
from Users and
Servers
# Users making use of
Notes Network Compression
Disabled
99%
100%
Enabled
Disabled
75%
% Active User
Enabled
1%
50%
25%
0%
Persons
Servers

27. Deployment Integrity
Entries appearing in multiple documents
Integrity check
Duplicate Replica On Same Server
Duplicate Template on same Server
# Databases
80
0
Full Names
Name Variations
Group
0
2
Mailin / Resource
4
6
Document Type
Replicas Acting As Different Template
22
Person
0
95
Same Replica but Different Inheritance
0
Other
3
0
Grand Total
7
103
Grand Total
102
86 Group Cycles Detected

28. DB Storage Profile for Demo Company
Distributed by Size
25%
Dbs > 1 GB: 1,323
Dbs > 10 GB: 431
System database
15%
Mailin database
Mailfile
Domino Directory
Application
10%
5%
100.000
10.000
1.000
100
Size (megabytes)
10
0%
1
% of Total Storage
20%

29. Database Storage Consolidation Potential
Number of
Database Type
Domino Directory
Total
Unique
Consolidated
Storage
Databases
Storage
Replicas
Storage
Savings
100%
79
3
30
7,832
20,723
2,768
7,517
64%
Mailin database
261
107
144
53
50%
Server Mail Box
54
8
54
8
0%
QuickPlace
165
1
147
1
0%
BlackBerry
1,425
11
718
6
45%
Application
2,904
1,691
1,200
731
57%
12,720
22,544
5,061
8,316
63%
Mailfile
Grand Total
-
Top 10 databases ranked by size
Database Title
Db Type
Storage (GB)
061cc52a3244e8bad944519170c3ff06
Mailfile
42.2
96140ee5ec036ec69136a74647729a2a
Mailfile
32.2
cad93d0aa156514e70237f1b370cc9c5
Mailfile
31.9
cfef8f68b2847cdd2eea9dda44d94acc
Mailfile
30.5
861455d1d12b1b2d990fc9e7c41ceea7
Mailfile
28.9
5128de940e6d560d37723b6448d48e00
Mailfile
28.7
d6de2ffe48a037e2e93ff13c688df77c
Mailfile
28.3
490eeb54ce626d24d63d1038be4de073
Mailfile
28.0
928c35784596e9487a0db1304573a310
Mailfile
27.8
1ab2006c3ef8fe4ba6e7daca15ee29d6
Application
27.3

30. View Size Distribution for Demo Company
Views > 100 MB: 234,
Views > 1 GB: 8,
12%
Showing 148355, views
8%
System database
Mailin database
Mailfile
Domino Directory
6%
Application
4%
2%
10 GB
1 GB
100 MB
10 MB
1 MB
100 KB
0%
10 KB
% of Total View Storage
10%

31. View Size Distribution for Demo Company
End User Mail Files
25%
Views > 100 MB: 13,0
Views > 1 GB: ,0
Showing 7104, views
($Meetings)
15%
($ThreadsEmbed)
($All)
($Sent)
($Inbox)
10%
5%
100 MB
10 MB
1 MB
100 KB
0%
10 KB
% of Total View Storage
20%

32. Basic Security Checks
Internet Password Strength
Databases with Anonymous Access
Variations found
Accounts
Company Name
1
First Name
1
Last Name
1
Depositor
Short Name
7
'password'
8
18
Databases
Templates
0
28
Reader
262
80
Author
35
82
Editor
Grand Total
Access Level
2
0
Manager
0
21
Grand Total
299
211
Grand Total
598
422

33. Applications Touched by Users
Application Migration Effort
15,572
Databases on all Servers
5,507
Unique Replica IDs
1,349
Excl. mail and system files
193
Applications Used in past 7 days
Recoding Effort
Optimize Platform

34. Server Health Checks
The following Slides show several higlights of the
platform checks that were performed

35. Namelookup Cache Utilization
on server server_17
Set this cache higher to
prevent 100% utilization
2 other server(s) have sim ilar issues
100
%
75
50
25
18
12
6
27-Oct-10 0
18
12
6
26-Oct-10 0
18
12
6
25-Oct-10 0
18
12
6
24-Oct-10 0
18
12
6
23-Oct-10 0
18
12
6
22-Oct-10 0
18
12
6
21-Oct-10 0
0

36. NSF Events & Monitor Pool Size Utilization
on server server_17
Cache Size is Sufficient
No issues detected on other servers
100
%
75
50
25
18
12
6
27-Oct-10 0
18
12
6
26-Oct-10 0
18
12
6
25-Oct-10 0
18
12
6
24-Oct-10 0
18
12
6
23-Oct-10 0
18
12
6
22-Oct-10 0
18
12
6
21-Oct-10 0
0

37. Miss Rate on Database BufferPool
on server server_24
Longer periods of High
Miss Rate may indicate
Performance Constraint
%
5 other server(s) m ay have sim ilar issues
25
20
15
10
5
18
12
6
27-Oct-10 0
18
12
6
26-Oct-10 0
18
12
6
25-Oct-10 0
18
12
6
24-Oct-10 0
18
12
6
23-Oct-10 0
18
12
6
22-Oct-10 0
18
12
6
21-Oct-10 0
0

38. Database File IO in KBytes per Second
Read activity on server server_16
Show ing busiest server
3,500
3,000
2,500
2,000
1,500
1,000
500
18
12
6
27-Oct-10 0
18
12
6
26-Oct-10 0
18
12
6
25-Oct-10 0
18
12
6
24-Oct-10 0
18
12
6
23-Oct-10 0
18
12
6
22-Oct-10 0
18
12
6
21-Oct-10 0
0

39. Database File IO in KBytes per Second
Write activity on server server_26
Show ing busiest server
1,000
900
800
700
600
500
400
300
200
100
18
12
6
27-Oct-10 0
18
12
6
26-Oct-10 0
18
12
6
25-Oct-10 0
18
12
6
24-Oct-10 0
18
12
6
23-Oct-10 0
18
12
6
22-Oct-10 0
18
12
6
21-Oct-10 0
0

40. Full Text Index Utilization
Search activity on server server_17
Show ing busiest server
0:00:22
0:00:13
0:00:09
0:00:04
18
12
6
27-Oct-10 0
18
12
6
26-Oct-10 0
18
12
6
25-Oct-10 0
18
12
6
24-Oct-10 0
18
12
6
23-Oct-10 0
18
12
6
22-Oct-10 0
18
12
6
0:00:00
21-Oct-10 0
Hours:Minut
0:00:17

41. Mail Delivery Rates
Categorized by Msg Size
Msgs per Hour, Show ing all servers
30,000
25,000
20,000
15,000
10,000
5,000
under_1kb
1kb_to_10kb
10kb_to_100kb
100kb_to_1mb
1mb_to_10mb
10mb_to_100mb
18
12
6
27-Oct-10 0
18
12
6
26-Oct-10 0
18
12
6
25-Oct-10 0
18
12
6
24-Oct-10 0
18
12
6
23-Oct-10 0
18
12
6
22-Oct-10 0
18
12
6
21-Oct-10 0
0
over_100mb

42. Mail Transfer Rates
Categorized by Msg Size
Msgs per Hour, Show ing all servers
7,000
6,000
5,000
4,000
3,000
2,000
1,000
under_1kb
1kb_to_10kb
10kb_to_100kb
100kb_to_1mb
1mb_to_10mb
10mb_to_100mb
18
12
6
27-Oct-10 0
18
12
6
26-Oct-10 0
18
12
6
25-Oct-10 0
18
12
6
24-Oct-10 0
18
12
6
23-Oct-10 0
18
12
6
22-Oct-10 0
18
12
6
21-Oct-10 0
0
over_100mb

43. DNA for Root Cause & Performance:
• Service Offering
• Assist Customers with Root Cause Analysis
» Application & Server Performance
» Server Stability and Complex Support Topics
» Network Latency Impact
• Data Collection -> Analysis -> Reporting
• Executed Remotely by panagenda/Trust Factory
ZEROImpact
Zero Impact: means:
Impact
means:
• Prerequisite
• DNA Health Check Performed
Nothing to Install
No Installation
Leaves No Footprint
Leave No Footprint
No Dependancy
Deliver in a Few Days
No Footprint

44. What Results will it Provide?
• Root Cause Analysis Report:
• Server & OS Configuration and Settings
• Application Performance
• Impact of Network Delay on Applications
• Interactive Slide Deck
• Linked to Common Spreadsheets & Check Lists
• Recommendations
• Presented Live
• Along with Explanation & Interpretation of Analysis Results
by Subject Matter Expert

45. What can you do with the results?
• Application Owners:
• Improve User Satisfaction
• Developers:
• Improve Source Code of Applications
• Adminstrators:
• Improve Server Performance

46. DNA Server Consolidation:
• Service Offering
• Multi Dimensional Insight into Network Demand,
Consolidation Potential & Data Center Scenarios
• Detailed Roadmap to Plan & Execute Server Consolidations
• Data Collection -> Analysis -> Reporting
• Executed Remotely by BP Trust Factory
ZEROImpactmeans:
Zero Impact:
Impact means:
Nothing to Install
No Installation
• Prerequisite
Leaves No Footprint
Leave No Footprint
No Dependancy
• DNA Health Check Performed
Deliver in a Few Days
No Footprint

47. What Results will it Provide?
•
•
•
•
•
Calculates Network Bandwidth Requirements
Consolidation Potential & Placement Scenarios
System & User Traffic Reduction & Optimization
Sizing Parameters for Servers, Storage & Network
Data Points for Cloud / Lotus Live
• Interactive Slide Deck
• Linked to Common Spreadsheets & Check Lists
• Recommendations
• Presented Live
• Along with Explanation & Interpretation of Analysis Results by Subject Matter Expert

48. What can you do with the results?
• IT Director / CIO:
• Executive Decision-Making Support
• Validation of Business Cases & Vendor Proposals
• Effective Realization of Cost Reduction
• Project Manager & Teams:
•
•
•
•
Input for Business Case & Project Proposal
DNA Facts & Findings Help Focus & Prioritize
Define Service Levels
Use Sizing Parameters for RFP’s

49. DNA Source Code Analysis:
• Service Offering
• Multi Dimensional Insight into Entire Application Landscape
» All Design Elements
» Source Code
• Real Impact of Application Landscape on Platform Migration
» Factual Quantification of Migration Effort for Redevelopment
» Identify Applications that Depend on Notes Mail
• Data Collection -> Analysis -> Reporting
• Executed Remotely by Trust Factory
• Prerequisite
• DNA Health Check Performed
ZEROImpactmeans:
Zero Impact:
Impact means:
Nothing to Install
No Installation
Leaves No Footprint
Leave No Footprint
No Dependancy
Deliver in a Few Days
No Footprint

50. What Results will it Provide?
• Complete Inventory of De-Duplicated Design & Source Code
• List of Applications that will Break after Migrating Notes Mail away
from Domino
• True Migration Effort based on Cocomo II
• Interactive Slide Deck
• Linked to Common Spreadsheets & Check Lists
• Recommendations
• Presented Live
• Along with Explanation & Interpretation of Analysis Results by Subject Matter
Expert

51. What can you do with the results?
• IT Director / CIO:
• Executive Decision-Making Support
• Validation of Business Cases & Vendor Proposals
• Develop Innovation Strategy
• Project Manager & Teams:
•
•
•
•
Input for Business Case & Project Proposal
Remediate Apps & Code Interacting with Notes Mail
DNA Facts & Findings Help Focus & Prioritize
Consolidate Source Code

52. Upcoming: panagenda iDNA
The In-house Version of Trust Factory‘s DNA Service
http://www.panagenda.com/en_uk/idna

53. Instant and Ongoing Analytics for Servers, Clients, Apps & More
• Unique Insights and Instant Value
• Executive Decision-Making Support
• Validation of Business Cases & Vendor Proposals
• Develop Innovation Strategy
• Turn Data into Knowledge
•
•
•
•
Hassle-free data collection from many different data sources
Instantly turns your data into meaningful reports
Move from reactive to proactive operations
Helps to fix, foresee and prevent problems with root cause
identification
• Gain answers to questions you never knew you could ask

54. Instant and Ongoing Analytics for Servers, Clients, Apps & More
• Facts and Architecture
• Up and running in half an hour - turnkey virtual softwareappliance
(Linux/VMWare)
• Non-intrusive, agent-less software - no installations required on
analyzed systems
• Data ware house access possible for data use with existing
reporting solutions
• HTML5 and PDF export for offline reports

66. Thank you!!!

67. How to Engage
• Master Agreement is in Place with IBM since 2003:
• Global Master Agreement Number:
• Trust Factory Supplier Number:
• panagenda:
•
•
•
•
4902NL0239
1000216663
Schreyvogelgasse 3/10 :: 1010 Vienna :: Austria
Web: http://www.panagenda.com
Email: office@panagenda.com
Fax: +43 1 89 012 89 – 15
• Our business partners in Italy: