Learn about the critical best practices and considerations for optimizing and growing SharePoint farms, storing user data efficiently and securely, while backing up TB’s a data in minutes. RBS (Remote Blob Store) and Virtualization, are just two of the many techniques discussed in this session. Realize the considerations for providing fast, automated disaster recovery for the entire SharePoint environment through SAN-based technology.
Scanning the Internet for External Cloud Exposures via SSL Certs
Optimize, Store and Protect SharePoint 2010 Server…Best Practices presented by James Baldwin
1. Optimize, Store and Protect
SharePoint 2010 Server
.
…Best Practices
James Baldwin
Strategic Solutions Group
EMC Corporation
Session W21
2. Agenda
• SharePoint architecture and performance factor
• SharePoint and FAST Search virtualization
• Storage best practices – sizing and performance
• Remote BLOB Storage
• Storage options and features
• SharePoint farm Protection
Apologises in advance…..for the word… FAST
FAST…as in Search…
FAST…as in VP
…
FAST…as in “let’s get out of here!”
4. SharePoint performance - The user experience
Domain Controller
SQL Server
Authentication
Storage
− Content/Metadata
− Search
− System
BLOB
Storage
(Optional)
BLOB
Retrieval/Creation
Document Request
Web Front End
Server
Operations
Network
•‒ CPU
Browsing to the home page
•‒ Memory
Browsing to a document library
Acceptable user
response time
Client
<3 seconds
‒ HBA/CNA
•‒ NIC
Creating a subsite Creating a list
• Uploading a document to a document library
<5 seconds
• Backing up a site
• Creating a site collection
<7 seconds
Technet: http://technet.microsoft.com/en-us/library/cc287790(office.12).aspx
5. SharePoint Farm Topologies
Small
Scale
Medium out approach
Large
Web
Web Servers Groups
H/W
Eval
Prod
RAM
4GB
8GB
CPU
4 Cores
Web/Query
User requests
Web
Crawling/Admin
App Servers Groups
Application
H/W
Eval
Prod
RAM
4GB
8GB
CPU
4 Cores
Database
H/W
Small
8 GB
4 Cores
8 Cores
App
Query
Crawl
Central Admin
/Office/Other
16 GB
CPU
Query/Crawl
Medium
RAM
App
http://technet.microsoft.com/en-us/library/cc298801.aspx
http://technet.microsoft.com/en-us/library/cc262485.aspx
All DBs
Search DBs
SharePoint DBs
Search DBs
SharePoint DBs
Content DBs
6. Key Benefits – Virtualizing SharePoint
Consolidation
• Achieve 2-10x consolidation ratio, especially for larger deployments
Performance
• Improved front end performance with more, smaller WFEs rather than few large WFEs
Availability
• VM based protection for SharePoint provides homogeneous high availability (WSFC,
VMware HA)
Business Continuity
• Simplified DR management (Geo-Clustering, vCenter Site Recovery Manager)
Maintenance
• Live migration of virtual machines (Hyper-V Live Migration, VMware vMotion)
Load Balancing
• Maximized overall performance with maximum HW utilization (SCVMM PRO , VMware DRS)
7. Virtualizing Server Roles In SharePoint
Server Roles/Priority
1st
Web Front End / Query
2nd
Application (Excel, Doc Conv, etc)
3rd
Index/Crawl
4th
SQL Server
What To Consider
CPU – User concurrency, Search requests
Scaling out is more efficient
Network – segment virtual NICs and virtual Switches
CPU – Application dependent
Scaling out is more efficient
Redundant (Non redundant in MOSS 2007)
CPU – Crawling, indexing (depends on content type/size)
Scale out (Up only with MOSS 2007)
Memory intensive
CPU – Document updates, Search, Backup
VMFS/RDM, VHD/Pass-Through
Scale up/out (Hyper-V ≤ 4 vCPU, vSphere ≤32 vCPU)
Failover Clustering, Mirroring, VMware HA
Understanding your existing workload (WFE to SQL) and requirements is better than any general best practice!!!
8. A Day in the life of SharePoint…
SQL Server CPU
The majority of load comes from systematic operations…
SharePoint Timer Job Reference - http://technet.microsoft.com/en-us/library/cc678870.aspx
Sample anonymous customer data
9. Virtualized SharePoint - Reference Architectures
Enterprise-Class SharePoint (Virtual) and FAST Search (Physical)
•
9 TB User content – (4TB SharePoint | 5TB Fileshare)
•
Virtualized SharePoint 2010 Farm
•
Virtualized Query and Content SSAs
•
5 Physical FAST Search Servers (12-core ea.)
•
•
2 Index Servers (1 Partition, 2 Search Roles)
•
•
2 Document Processors/Content Distributors
1 FAST Admin inc. Document Processor
<1 Sec search latency with 22,000 Heavy Users @ 10% Concurrency
Role
Qty
VM specs
Hyper-V Servers
3 Nodes
4-socket quad-core (16 cores), 128 GB RAM
SQL Server 2008
2
4 vCPU, 32 GB RAM
Web front ends
Application servers (Incl. Central
Admin)
Query SSA
4
4 vCPU, 6 GB RAM
2
2 vCPU, 4 GB RAM
1
4 vCPU, 8 GB RAM
Content SSA
1
4 vCPU, 8 GB RAM
Fast Search Physical Servers
5
2-socket hex-core (12 cores), 48 GB RAM
Whitepaper due for release in October timeframe
10. Virtualized SharePoint - Reference Architectures
Enterprise-Class SharePoint and FAST Search (All Virtual)
•
9 TB User content – (4TB SharePoint | 5TB Fileshare)
•
Virtualized SharePoint 2010 Farm
•
Virtualized FAST Search !
•
•
2 Index Servers (1 Partition, 2 Search Roles)
•
•
2 Document Processors/Content Distributors
1 FAST Admin inc. Document Processor
<1 Second search latency with 22,000 Heavy Users @ 10% Concurrency
Role
Qty
VM specs
Hyper-V Servers (SharePoint Farm)
3 Nodes
4-socket quad-core (16 cores), 128 GB RAM
Hyper-V Servers (FAST Search Farm)
2 Nodes
2-socket hex-core (12 cores), 48 GB RAM
SQL Server 2008
2
4 vCPU, 32 GB RAM
Web front ends
Application servers (Incl. Central
Admin)
Query SSA
4
4 vCPU, 6 GB RAM
2
2 vCPU, 4 GB RAM
1
4 vCPU, 8 GB RAM
Content SSA
1
4 vCPU, 8 GB RAM
Whitepaper due for release in October timeframe
11. SharePoint 2010 Storage Elements
SQL Server
Content
System and Configuration
Content Databases
Configuration Databases
BLOB Store
Central Admin
tempdb, master ,model , msdb
• SharePoint relies mainly on SQL but there’s important
data outside SQL System Content DBs
Volumes
8%
• Typical storage usage is not I/O heavy
20%
BLOBs
SA Data
34%
12%
• Search services in SharePoint generate most of the I/O
Search
load
Index
System DBs
SA DBs
• Content – Plan15% capacity
for
6%
5%
• Search and System – Plan for performance
% Total Capacity - Sample
Shared Service Applications
Usage & Health Data Collection - Logging
Search – Admin, Property, Crawl
Web Analytics – Staging, Reporting
User Profiles – Profile, Synchronization, Social Tagging
Managed Metadata- Term Store
State
Business Data Connectivity
Secure Store
Search Index
Index Partition/s
Query Component/s
Service Application Data
SA Volumes
System Volumes
Boot/OS/VMs
Web Parts & Features
SharePoint Binaries
12. 1TB content farm
– How much storage do I need?
Gigabytes of Storage
5000
4000
3000
2000
1000
0
2150
2 WFEs/no
Search
2910
2 WFEs/1x
Index&Query
3310
2 WFEs/2x
Index&Query
3290
2 WFEs/1x
Index/2x Query
(Striped)
4470
2 WFEs/2x
Index/4xQuery (4
Partitions,
Mirrored(1-A,D,
etc)
13. A Day in the life of SharePoint…
SQL Server Storage I/O
Plan for user load peaks, not systematic peaks…
Sample anonymous customer data
14. SharePoint 2010 Storage I/O Characteristics
* Based on average workloads in a collaboration farm (Browse/Search/Modify – 80/10/10)
SQL Server
Search
Databases
Index
Component
Query
component
8
8
32
32
16
32
16
64
64
95:5
50:50
60:40
90:10
70:30
Content
Databases
tempdb
Read (KB)
16
Write (KB)
R:W Ratio
I/O Type
SharePoint
In-box Search Servers
(property & crawl stores)
FAST Search Servers
Read (KB)
Write (KB)
R:W Ratio
FAST Index Server (Primary)
294
611
3:1
FAST Index Server (Backup)
31
710
1:30
FAST Servers
FAST
Server
13
26
1:66
(Document Processors ,etc)
SQLIO/SQLIOSim are I/O stress tools - should not be considered as “performance requirements” !
15. SharePoint 2010 Storage performance
Requirements/estimates for Search
Query
Crawler
Query Component
Crawl Component
Database Role
Microsoft’s Estimate IOPS
SQL Server
Search Admin
DB
Property
DB
Typical averages observed
Crawl Database
3,500 – 7,000 (R70:W30)
Property Database
2,000 (R30:W70)
600
Query Component
2,000 per Active/Failover pair
(Load/Write/Merge)
450
Crawl Component
300-400 IOPS
http://technet.microsoft.com/en-us/library/cc298801.aspx
Based on a Microsoft case study – Mileage may vary!!!
2,000
80-100
Crawl
DB
16. FAST Search Server 2010 for SharePoint Storage
Requirements/estimates
FAST Admin
S:FASTSearch
LUN size: 80GB
FAST server role
Document
Processor 01
Document
Processor 02
Index 01
(Primary)
S:FASTSearch
LUN size: 80GB
S:FASTSearch
LUN size: 80GB
S:FASTSearch
LUN size: 2000GB
Microsoft’s Estimate LUN size
Web Analyzer
Database (2GB per 1M items)
SharePoint or Intranet (6GB per 1M items)
Public Web content (25GB per 1M items)
Index server
(Primary)
X2.5 the size of a single index file set +50 GB
synchronization data
Index server
(Secondary)
X2.5 the size of a single index file set +50 GB
synchronization data
Based on a Microsoft case study – Mileage may vary!!!
Index 02
(Backup)
S:FASTSearch
LUN size: 2000GB
Typical averages observed
1.2 GB
910 GB
(2 TB recommended to hold temp indexing files)
899 GB
(2 TB recommended to hold temp indexing files)
http://technet.microsoft.com/en-us/library/ff381239.aspx
17. SharePoint Reference Architecture
Storage Layout
Cost driven configuration
‒ 13,000 Heavy SharePoint users
‒ 1 TB User content with RBS FILESTREAM Externalization
‒ SAN based replication (RecoverPoint)
Storage Configuration
‒ RAID5 –VM Boot Luns, Content Databases
‒ RAID10 – Search Databases, tempdb
‒ Array SSD Cache to compensate for disk latencies
http://www.emc.com/collateral/software/white-papers/h8139-protection-virtualized-sharepoint-wp.pdf
18. SharePoint Storage Sizing
Volume Sizing
% of Corpus Size
Typical sizes GB
Recommended
RAID
Virtual Machine Boot Volumes
-
80
R-5
Application Volumes
-
50 – 300+
R-5
Data File Volume
-
100 – 200
R-5
Log File Volume
10% of Data
File
10 – 20
R-5 / R-10
Data File Volume
10%
100 – 300
R-10
Log File Volume
2%
10 – 100
R-10
Storage Role
Content
Databases
tempdb
Storage and SQL Server capacity planning and configuration
Hardware and software requirements
http://technet.microsoft.com/en-us/library/cc298801.aspx
http://technet.microsoft.com/en-us/library/cc262485.aspx
19. SharePoint Storage Sizing
Volume Sizing
% of
Corpus Size
Sample size (1TB
Content)
Recommend
RAID
Crawl Store DB
0.046 × (sum of content
databases)
46GB
R-10
Property Store DB
0.015 × (sum of content
databases)
15GB
R-10
Index Component
10%
100GB
R5 / R10
Query Component
10 – 35% * 2.85
150 – 1TB
R-5 / R-10
SharePoint_Config
-
5GB
R-5
SP_AdminContent
-
15GB
R-5
- (based on % monitoring)
50 - 500GB(?)
R-5
Storage Role
SQL Search
Databases
Search Server
Volumes
SharePoint
Configuration
Databases
- Data & Log
Volume
Usage and Health Data
Collection Database
SharePoint 2010 Database sizing characteristics http://technet.microsoft.com/en-us/library/cc298801.aspx
http://technet.microsoft.com/en-us/library/cc678868(office.14).aspx
20. FAST Search server 2010 for SharePoint Storage Sizing
Volume Sizing (estimates)
Microsoft Data set
Typical result
observed (5.2
million items)
Recommend RAID
N/A
460 MB
R-10
FASTQuerySSA_PropertyStore_guid
<0.4GB
158 MB
R-10
FASTQuerySSA_CrawlStoreDB_guid
3.3 GB per million
items
15 MB
R-10
N/A
99 MB
R-10
<0.4GB
158 MB
R-10
3.3 GB per million
items
38621 MB
R-10
<0.1GB
3.00 MB
R-10
Storage Role
FASTQuerySSA_DB_guid
FAST Query
SSA Databases
FASTContent_DB_guid
FAST Query
SSA Databases
FASTContent_PropertyStoreDB_guid
FASTContent_CrawlStoreDB_guid
FAST Admin
FASTAdminConfigDB
SharePoint 2010 Database sizing characteristics http://technet.microsoft.com/en-us/library/cc298801.aspx http://technet.microsoft.com/en-us/library/cc678868(office.14).aspx
21. SharePoint Storage Best Practices
SQL Server – Storage Allocation
Use 64KB unit allocation size when formatting DB volumes
Physical volumes (RDM/Pass-through) for larger than 2TB when virtualized
Plan Database file sizes accordingly
– Don’t rely on autogrowth
File growth can cause locking, set files size and autogrowth increments appropriately
– Using RBS would keep the SQL Database files small
– Watch the SP Configuration database log file growth!
When using Thin/Virtual provisioning
– Use the “Quick Format” option
– Enable Instant file initialization*
Enhances the speed for data file creations, restores, data file growth
Assign SQL service account to “Perform Volume Maintenance Tasks” permission
– Log files are fully allocated and zeroed upon creation http://msdn.microsoft.com/en-us/library/ms175935.aspx
or growths
22. SharePoint Storage Best Practices
SQL Server - Performance
Plan for optimal disk response times
Data file Latency
Log file Latency
Read/Write Operations
Write Operations
< 10 ms
< 5 ms
Very Good
< 20 ms
5 – 10 ms
Acceptable
> 20 ms
> 15 ms
Investigate and Improve
Recommendation
Important Perfmon I/O counters
Real Meaning!
Average Disk/sec Read or Write
Disk Latency
Current Disk Queue Length*
Outstanding I/Os
Disk Reads/Writes per Second
IOPS
Average Disk Bytes/Read & Write
I/O Size (KB)
* Hard to interpret due to virtualization of storage. Consider in combination with response times
23. EMC Storage Technologies for SharePoint 2010
•
Thin Provisioned Pools
‒
‒
•
Reduces initial storage requirements/costs for content databases
Enables SharePoint administrators to grow volumes non-distruptively
FAST VP (Fully Automated Storage Tiering)
‒
Helps to handle unanticipated performance requirements
SharePoint Component
Recommended
Notes
Search Index
No
Highly changing, throw-away data
Search Query
Yes
Highly-read data with small burst write changes
tempdb
Yes
Same blocks are re-used on disk and performance of TEMPDB
directly affects SharePoint performance request
Content databases
BLOB Store
–
Yes/Maybe
No
C-Databases which are defragmented regularly or complex
Databases like NewsGator
BLOB Store has low IOPs requirements
EMC Cluster Enabler (CE)
–
–
Automated multi-site Disaster Recovery (failover) for Hyper-V/Physical/Hybrid SharePoint farms
Can use any EMC replication technology (RecoverPoint/SRDF/MirrorView)
24. EMC Storage Technologies for SharePoint 2010
•
LUN Compression/De-duplication/Thin Provisioning
‒
‒
•
Reduces initial storage requirements/costs for content databases
Enables SharePoint administrators to grow volumes non-disruptively
FAST Cache
‒
Helps to handle unanticipated performance requirements
SharePoint
Component
Performance
Improvement
Notes
Search Index re-uses the same blocks on disk as working space to process listitems
during indexing, then that storage is quiet between crawls
Search Index
High
Search Query
Medium
Query improves due to random large block reads/writes being serviced by
FASTCache, but Query storage is large and costly in FASTCache
tempdb
Medium
TempDB can benefit from FASTCache as the way SQL Server uses TempDB (database
page reuse) is ideally suited to FASTCache algorithms
Content databases
Medium
Low IOPs requirements does not require FastCache, DB Index operations see an
improvement
BLOB Store
Low
Low IOPs requirements, large-block I/O
25. BLOB Externalization (RBS/EBS)
BLOB = Binary Large Object
BLOB externalization options
‒ EBS using 3rd party provider- Mainly for MOSS 2007 but still supported with SP2010
– RBS through SQL RBS FILESTREAM provider (Local/Remote)
– RBS using feature rich 3rd party provider
Pros
– Lower TCO as BLOBs can reside on lower cost storage tier (SATA/NL-SAS)
– More flexible and efficient storage configurations (CIFS/NFS/REST)
– Performance improvement
• Cons
– Large objects retrieval (>1Mb)
– SharePoint Indexing operations
– Various SQL operations (Index defragmentation, statistics operations, consistency checks)
– Backup and restore complexity
– 2GB file size limit is not lifted by using RBS
– Not supported with System Center DPM and/or Database Mirroring
http://technet.microsoft.com/en-us/library/ff628583.aspx
http://technet.microsoft.com/en-us/library/cc262787.aspx
26. Remote BLOB Storage (RBS)
Feature
Externalization using SMB/CIFS/NFS protocols
User Interface
BLOBs tiering based on Age/Size/Type/Hierarchy policies
Orphaned BLOB Garbage Collection
BLOB store volumes automatically included in VSS Backup
SQL Server
RBS FILESTREAM
3rd Party RBS provider
No (Local volumes only)
Yes
Powershell
Central Admin
No
Yes
Basic (RBS Maintainer)
Policy-based
Yes
No
EMC has several RBS solutions
–
–
–
–
–
In partnership with several vendors for BLOB externalization (EMC Select-Metalogix StoragePoint)
EMC SourceOne also leverages RBS
Block/File/Object - VMAX, VNX, DataDomain, Isilon
Cloud Optimized Storage - Atmos/Atmos VE
Archival Storage - Centera
27. Considerations for Remote BLOB Storage
Replication, Backup and Recovery
– Native/Item level backup (stsadm based) would include BLOBs (“deep copy”)
– SQL based backup would only protect the content database
– To maintain consistency:
Backup – First Content Databases then BLOB Store
Restore – First BLOB Store then Content Databases
– For faster recovery, consider larger intervals of garbage collection jobs (Keeps previous BLOB versions)
– For DR purposes always tie RBS volumes with SQL Server volumes
Block, File or Object storage?
– Performance: Block would be faster but RBS has typically low I/O
– Storage Efficiency
– Block-level LUN Compression – increases storage efficiency, may affect backup performance
– Filesystem-Deduplication – better performance and increased dedup rate
28. Reference Architecture - BLOB Externalization
File Storage (CIFS)
Farm Profile
Total content ~1TB
4.36M documents
Avg. File Size - 220K
Storage Profile
EMC VNX5300
15K SAS (System and DBs) /7.2K NL-SAS (BLOB Store)
CIFS Share for RBS
Results
Max user capacity – 8,630 (10%)
BLOBs consumed 92% of content databases
Full crawl duration – 34 hours
8-10% user throughput (RPS) improvement using RBS
20% capacity saved with RBS FS de-duplication
29. Reference Architecture - BLOB Externalization
Block Storage
Farm Profile
Total content 3.1TB
1.2M documents
File Sizes – 200KB-5MB
16 Site Collections/40,000 Sites
Storage Profile
EMC Symmetrix VMAX
FAST VP controlled FC and SATA tiers
Results
BLOBs consumed 94% of content databases
3.1 TB Externalized in 35 Hours
Storage
Configuration
Baseline
Externalized
Concurrency
10%
Users (Heavy)
(80%-Browse/ 10% -Search/ 10%-Modify)
25,800
31,800
Response
Time
< 3 sec
30. Protecting SharePoint to Enterprise Scales
Replication Management for Microsoft SharePoint 2010
Leverage SAN based replication for Rapid full farm protection and item-level recovery….
– Hardware VSS-based coordinated SharePoint replication, Enabling farmwide consistency
– Negligible impact to farm performance even during the first
synchronization
– Utilizing EMC storage replication (Snaps/Clones/Bookmarks)
– Simple, intuitive SharePoint discovery and application set configuration
(Configure protection in <8 minutes)
– Works with SQL RBS FILESTREAM
– Full farm restore include search index consistency on recovery
– Item level recovery enabled by Kroll Ontrack Powercontrols
31. SharePoint Replication Management
Reference Architecture
Enabled by EMC Replication Manager, Kroll Ontrack PowerControls
Hybrid farm (Physical/Virtual)
1.5 TB of content (6,818,250 files)
15 SharePoint content DBs
Both SnapView Snaps
and Clones used
SharePoint action
EMC SnapView
STSADM
Clone: 3hr 11min
39hr 30min
Snapshot: 9min
(14.8 MB/s)
Clone: 11min
Not tested in this environment
Content database recovery (online)
7min
3hr 12min
(12 MB/s)
Item-level recovery
10min
Not tested in this environment (requires recovery farm)
Full farm backup (2.5 TB)
Daily incremental SharePoint backup
(~1% daily change rate)
32. Business Continuity for SharePoint 2010:
Options
SAN
SQL
Point in Time
Stretched Farm
(Partial Replication)
Mirror Farm
(Partial Replication)
Virtualized Farm
DB Mirroring
Log Shipping
(Complete Replication)
Continuous Replication
Backup
PiT Replication
33. Designing DR consistency for SharePoint
Automated DR
Consistency Group
(RP/SRDF/MV)
LUNs
Grouping
VMware
SRM
Hyper-V
Cluster Enabler
Web Front Ends
Boot + Query (optional)
Protection Group
Cluster Group
Query Servers
Boot + Query Component
Protection Group
Cluster Group
Index Servers
Boot + Index Component
Protection Group
Cluster Group
Application Servers
Boot + Application Volumes
Protection Group
Cluster Group
Protection Group
Cluster Group
Boot + Pagefile (optional)
SQL System Databases
SQL Server(s)
Configuration Databases
Search Databases
Content Databases
Content consistency
RBS BLOB Store
Search consistency
34. Protecting SharePoint
Business Continuity – vCenter SRM with RecoverPoint CRR
Proven Solution Test Results
13,080 heavy users @ 10% concurrency Production Site Outage
Sustained Performance
30% reduced cost of BLOB Storage with EMC
LUN Compression
<16 minutes to perform full end-to-end
failover
Failover Test Results
Test type
Executing recovery plan for a fully
operational farm (under load)
Shutting down
production VMs
Preparing storage
Restarting DR VMs
Total time
00:33
11:45
3:15
15:33
35. Key Takeaways
•
SharePoint is more than just SQL…
–
–
–
•
Full SharePoint and FAST Search farms virtualization has great advantages over physical/hybrid configurations
–
–
–
•
Horizontal scaling is more efficient
The best FULL farm protection when Integrated with EMC replication.
Simplifies, accelerates and automates SharePoint DR! (vCenter SRM, Hyper-V+EMC Cluster Enabler)
EMC’s SharePoint VSS based replication can significantly accelerate replication and recovery of SharePoint
–
–
–
•
Leverage EMC Proven solutions and Best Practices for SharePoint and FAST Search storage, networking and compute design
FAST, FAST Cache, VP improve efficiency & performance but require proper planning
Use RBS to improve scalability and TCO
A must for large deployments (TBs)
Protects content and index
Fast and simple Item level recovery while integrating with EMC partners (e.g. Kroll)
Block-level replication for Automated Disaster Recovery answers the difficult challenges in providing site
protection
–
–
Essential for critical SharePoint farms, where uptime and site resiliency is critical
Automates failover for the entire farm/s