SlideShare une entreprise Scribd logo
1  sur  38
Télécharger pour lire hors ligne
Optimize, Store and Protect
SharePoint 2010 Server
.

…Best Practices
James Baldwin
Strategic Solutions Group
EMC Corporation
Session W21
Agenda

• SharePoint architecture and performance factor
• SharePoint and FAST Search virtualization
• Storage best practices – sizing and performance
• Remote BLOB Storage
• Storage options and features
• SharePoint farm Protection

Apologises in advance…..for the word… FAST
FAST…as in Search…
FAST…as in VP
…
FAST…as in “let’s get out of here!”
Proven Solutions approach
1

2

3

4

Capture

Test and
Validate

Document

Publish

& Define

Durham, NC

Santa Clara, CA

Cork, Ireland

Vienna, Austria

Shanghai, China

Singapore
SharePoint performance - The user experience
Domain Controller
SQL Server
Authentication

Storage
− Content/Metadata
− Search
− System

BLOB
Storage
(Optional)
BLOB
Retrieval/Creation

Document Request

Web Front End

Server

Operations

Network

•‒ CPU
Browsing to the home page
•‒ Memory
Browsing to a document library

Acceptable user
response time
Client
<3 seconds

‒ HBA/CNA

•‒ NIC
Creating a subsite Creating a list
• Uploading a document to a document library

<5 seconds

• Backing up a site
• Creating a site collection

<7 seconds

Technet: http://technet.microsoft.com/en-us/library/cc287790(office.12).aspx
SharePoint Farm Topologies
Small

Scale
Medium out approach

Large

Web

Web Servers Groups

H/W

Eval

Prod

RAM

4GB

8GB

CPU

4 Cores

Web/Query

User requests

Web

Crawling/Admin

App Servers Groups

Application
H/W

Eval

Prod

RAM

4GB

8GB

CPU

4 Cores

Database
H/W

Small
8 GB

4 Cores

8 Cores

App

Query

Crawl

Central Admin
/Office/Other

16 GB

CPU

Query/Crawl

Medium

RAM

App

http://technet.microsoft.com/en-us/library/cc298801.aspx
http://technet.microsoft.com/en-us/library/cc262485.aspx

All DBs

Search DBs

SharePoint DBs

Search DBs

SharePoint DBs

Content DBs
Key Benefits – Virtualizing SharePoint
Consolidation

• Achieve 2-10x consolidation ratio, especially for larger deployments

Performance

• Improved front end performance with more, smaller WFEs rather than few large WFEs

Availability

• VM based protection for SharePoint provides homogeneous high availability (WSFC,
VMware HA)

Business Continuity

• Simplified DR management (Geo-Clustering, vCenter Site Recovery Manager)

Maintenance

• Live migration of virtual machines (Hyper-V Live Migration, VMware vMotion)

Load Balancing

• Maximized overall performance with maximum HW utilization (SCVMM PRO , VMware DRS)
Virtualizing Server Roles In SharePoint
Server Roles/Priority
1st

Web Front End / Query
2nd

Application (Excel, Doc Conv, etc)
3rd

Index/Crawl
4th

SQL Server

What To Consider
 CPU – User concurrency, Search requests
 Scaling out is more efficient
 Network – segment virtual NICs and virtual Switches
 CPU – Application dependent
 Scaling out is more efficient
 Redundant (Non redundant in MOSS 2007)
 CPU – Crawling, indexing (depends on content type/size)
 Scale out (Up only with MOSS 2007)
 Memory intensive
 CPU – Document updates, Search, Backup
 VMFS/RDM, VHD/Pass-Through
 Scale up/out (Hyper-V ≤ 4 vCPU, vSphere ≤32 vCPU)
 Failover Clustering, Mirroring, VMware HA

Understanding your existing workload (WFE to SQL) and requirements is better than any general best practice!!!
A Day in the life of SharePoint…
SQL Server CPU

The majority of load comes from systematic operations…

SharePoint Timer Job Reference - http://technet.microsoft.com/en-us/library/cc678870.aspx

Sample anonymous customer data
Virtualized SharePoint - Reference Architectures
Enterprise-Class SharePoint (Virtual) and FAST Search (Physical)
•

9 TB User content – (4TB SharePoint | 5TB Fileshare)

•

Virtualized SharePoint 2010 Farm

•

Virtualized Query and Content SSAs

•

5 Physical FAST Search Servers (12-core ea.)
•
•

2 Index Servers (1 Partition, 2 Search Roles)

•
•

2 Document Processors/Content Distributors
1 FAST Admin inc. Document Processor

<1 Sec search latency with 22,000 Heavy Users @ 10% Concurrency
Role

Qty

VM specs

Hyper-V Servers

3 Nodes

4-socket quad-core (16 cores), 128 GB RAM

SQL Server 2008

2

4 vCPU, 32 GB RAM

Web front ends
Application servers (Incl. Central
Admin)
Query SSA

4

4 vCPU, 6 GB RAM

2

2 vCPU, 4 GB RAM

1

4 vCPU, 8 GB RAM

Content SSA

1

4 vCPU, 8 GB RAM

Fast Search Physical Servers

5

2-socket hex-core (12 cores), 48 GB RAM

Whitepaper due for release in October timeframe
Virtualized SharePoint - Reference Architectures
Enterprise-Class SharePoint and FAST Search (All Virtual)
•

9 TB User content – (4TB SharePoint | 5TB Fileshare)

•

Virtualized SharePoint 2010 Farm

•

Virtualized FAST Search !
•
•

2 Index Servers (1 Partition, 2 Search Roles)

•
•

2 Document Processors/Content Distributors
1 FAST Admin inc. Document Processor

<1 Second search latency with 22,000 Heavy Users @ 10% Concurrency
Role

Qty

VM specs

Hyper-V Servers (SharePoint Farm)

3 Nodes

4-socket quad-core (16 cores), 128 GB RAM

Hyper-V Servers (FAST Search Farm)

2 Nodes

2-socket hex-core (12 cores), 48 GB RAM

SQL Server 2008

2

4 vCPU, 32 GB RAM

Web front ends
Application servers (Incl. Central
Admin)
Query SSA

4

4 vCPU, 6 GB RAM

2

2 vCPU, 4 GB RAM

1

4 vCPU, 8 GB RAM

Content SSA

1

4 vCPU, 8 GB RAM
Whitepaper due for release in October timeframe
SharePoint 2010 Storage Elements
SQL Server
Content

System and Configuration

Content Databases

Configuration Databases

BLOB Store

Central Admin
tempdb, master ,model , msdb

• SharePoint relies mainly on SQL but there’s important
data outside SQL System Content DBs
Volumes
8%
• Typical storage usage is not I/O heavy
20%
BLOBs
SA Data
34%
12%
• Search services in SharePoint generate most of the I/O
Search
load
Index
System DBs
SA DBs
• Content – Plan15% capacity
for
6%
5%
• Search and System – Plan for performance
% Total Capacity - Sample

Shared Service Applications
Usage & Health Data Collection - Logging
Search – Admin, Property, Crawl
Web Analytics – Staging, Reporting
User Profiles – Profile, Synchronization, Social Tagging
Managed Metadata- Term Store
State
Business Data Connectivity
Secure Store

Search Index
Index Partition/s

Query Component/s

Service Application Data
SA Volumes

System Volumes
Boot/OS/VMs

Web Parts & Features

SharePoint Binaries
1TB content farm
– How much storage do I need?
Gigabytes of Storage
5000
4000
3000
2000
1000
0

2150

2 WFEs/no
Search

2910

2 WFEs/1x
Index&Query

3310

2 WFEs/2x
Index&Query

3290

2 WFEs/1x
Index/2x Query
(Striped)

4470

2 WFEs/2x
Index/4xQuery (4
Partitions,
Mirrored(1-A,D,
etc)
A Day in the life of SharePoint…
SQL Server Storage I/O

Plan for user load peaks, not systematic peaks…

Sample anonymous customer data
SharePoint 2010 Storage I/O Characteristics
* Based on average workloads in a collaboration farm (Browse/Search/Modify – 80/10/10)

SQL Server
Search
Databases

Index
Component

Query
component

8

8

32

32

16

32

16

64

64

95:5

50:50

60:40

90:10

70:30

Content
Databases

tempdb

Read (KB)

16

Write (KB)
R:W Ratio

I/O Type

SharePoint

In-box Search Servers
(property & crawl stores)

FAST Search Servers
Read (KB)

Write (KB)

R:W Ratio

FAST Index Server (Primary)

294

611

3:1

FAST Index Server (Backup)

31

710

1:30

FAST Servers

FAST

Server

13

26

1:66

(Document Processors ,etc)

SQLIO/SQLIOSim are I/O stress tools - should not be considered as “performance requirements” !
SharePoint 2010 Storage performance
Requirements/estimates for Search

Query

Crawler

Query Component

Crawl Component

Database Role

Microsoft’s Estimate IOPS

SQL Server

Search Admin
DB

Property
DB

Typical averages observed

Crawl Database

3,500 – 7,000 (R70:W30)

Property Database

2,000 (R30:W70)

600

Query Component

2,000 per Active/Failover pair
(Load/Write/Merge)

450

Crawl Component

300-400 IOPS

http://technet.microsoft.com/en-us/library/cc298801.aspx

Based on a Microsoft case study – Mileage may vary!!!

2,000

80-100

Crawl
DB
FAST Search Server 2010 for SharePoint Storage
Requirements/estimates

FAST Admin

S:FASTSearch
LUN size: 80GB

FAST server role

Document
Processor 01

Document
Processor 02

Index 01
(Primary)

S:FASTSearch
LUN size: 80GB

S:FASTSearch
LUN size: 80GB

S:FASTSearch
LUN size: 2000GB

Microsoft’s Estimate LUN size

Web Analyzer

Database (2GB per 1M items)
SharePoint or Intranet (6GB per 1M items)
Public Web content (25GB per 1M items)

Index server
(Primary)

X2.5 the size of a single index file set +50 GB
synchronization data

Index server
(Secondary)

X2.5 the size of a single index file set +50 GB
synchronization data
Based on a Microsoft case study – Mileage may vary!!!

Index 02
(Backup)

S:FASTSearch
LUN size: 2000GB

Typical averages observed
1.2 GB
910 GB

(2 TB recommended to hold temp indexing files)

899 GB

(2 TB recommended to hold temp indexing files)
http://technet.microsoft.com/en-us/library/ff381239.aspx
SharePoint Reference Architecture
Storage Layout

Cost driven configuration
‒ 13,000 Heavy SharePoint users
‒ 1 TB User content with RBS FILESTREAM Externalization
‒ SAN based replication (RecoverPoint)

Storage Configuration
‒ RAID5 –VM Boot Luns, Content Databases
‒ RAID10 – Search Databases, tempdb
‒ Array SSD Cache to compensate for disk latencies

http://www.emc.com/collateral/software/white-papers/h8139-protection-virtualized-sharepoint-wp.pdf
SharePoint Storage Sizing
Volume Sizing

% of Corpus Size

Typical sizes GB

Recommended
RAID

Virtual Machine Boot Volumes

-

80

R-5

Application Volumes

-

50 – 300+

R-5

Data File Volume

-

100 – 200

R-5

Log File Volume

10% of Data
File

10 – 20

R-5 / R-10

Data File Volume

10%

100 – 300

R-10

Log File Volume

2%

10 – 100

R-10

Storage Role

Content
Databases

tempdb

Storage and SQL Server capacity planning and configuration
Hardware and software requirements

http://technet.microsoft.com/en-us/library/cc298801.aspx
http://technet.microsoft.com/en-us/library/cc262485.aspx
SharePoint Storage Sizing
Volume Sizing
% of
Corpus Size

Sample size (1TB
Content)

Recommend
RAID

Crawl Store DB

0.046 × (sum of content
databases)

46GB

R-10

Property Store DB

0.015 × (sum of content
databases)

15GB

R-10

Index Component

10%

100GB

R5 / R10

Query Component

10 – 35% * 2.85

150 – 1TB

R-5 / R-10

SharePoint_Config

-

5GB

R-5

SP_AdminContent

-

15GB

R-5

- (based on % monitoring)

50 - 500GB(?)

R-5

Storage Role
SQL Search
Databases

Search Server
Volumes
SharePoint
Configuration
Databases
- Data & Log
Volume

Usage and Health Data
Collection Database

SharePoint 2010 Database sizing characteristics http://technet.microsoft.com/en-us/library/cc298801.aspx
http://technet.microsoft.com/en-us/library/cc678868(office.14).aspx
FAST Search server 2010 for SharePoint Storage Sizing
Volume Sizing (estimates)
Microsoft Data set

Typical result
observed (5.2
million items)

Recommend RAID

N/A

460 MB

R-10

FASTQuerySSA_PropertyStore_guid

<0.4GB

158 MB

R-10

FASTQuerySSA_CrawlStoreDB_guid

3.3 GB per million
items

15 MB

R-10

N/A

99 MB

R-10

<0.4GB

158 MB

R-10

3.3 GB per million
items

38621 MB

R-10

<0.1GB

3.00 MB

R-10

Storage Role
FASTQuerySSA_DB_guid
FAST Query
SSA Databases

FASTContent_DB_guid
FAST Query
SSA Databases

FASTContent_PropertyStoreDB_guid
FASTContent_CrawlStoreDB_guid

FAST Admin

FASTAdminConfigDB

SharePoint 2010 Database sizing characteristics http://technet.microsoft.com/en-us/library/cc298801.aspx http://technet.microsoft.com/en-us/library/cc678868(office.14).aspx
SharePoint Storage Best Practices
SQL Server – Storage Allocation

 Use 64KB unit allocation size when formatting DB volumes
 Physical volumes (RDM/Pass-through) for larger than 2TB when virtualized
 Plan Database file sizes accordingly
– Don’t rely on autogrowth
 File growth can cause locking, set files size and autogrowth increments appropriately

– Using RBS would keep the SQL Database files small
– Watch the SP Configuration database log file growth!
 When using Thin/Virtual provisioning
– Use the “Quick Format” option
– Enable Instant file initialization*
 Enhances the speed for data file creations, restores, data file growth
 Assign SQL service account to “Perform Volume Maintenance Tasks” permission

– Log files are fully allocated and zeroed upon creation http://msdn.microsoft.com/en-us/library/ms175935.aspx
or growths
SharePoint Storage Best Practices
SQL Server - Performance

Plan for optimal disk response times
Data file Latency

Log file Latency

Read/Write Operations

Write Operations

< 10 ms

< 5 ms

Very Good

< 20 ms

5 – 10 ms

Acceptable

> 20 ms

> 15 ms

Investigate and Improve

Recommendation

Important Perfmon I/O counters

Real Meaning!

Average Disk/sec Read or Write

Disk Latency

Current Disk Queue Length*

Outstanding I/Os

Disk Reads/Writes per Second

IOPS

Average Disk Bytes/Read & Write

I/O Size (KB)

* Hard to interpret due to virtualization of storage. Consider in combination with response times
EMC Storage Technologies for SharePoint 2010
•

Thin Provisioned Pools
‒
‒

•

Reduces initial storage requirements/costs for content databases
Enables SharePoint administrators to grow volumes non-distruptively

FAST VP (Fully Automated Storage Tiering)
‒

Helps to handle unanticipated performance requirements

SharePoint Component

Recommended

Notes

Search Index

No

Highly changing, throw-away data

Search Query

Yes

Highly-read data with small burst write changes

tempdb

Yes

Same blocks are re-used on disk and performance of TEMPDB
directly affects SharePoint performance request

Content databases
BLOB Store

–

Yes/Maybe
No

C-Databases which are defragmented regularly or complex
Databases like NewsGator
BLOB Store has low IOPs requirements

EMC Cluster Enabler (CE)
–
–

Automated multi-site Disaster Recovery (failover) for Hyper-V/Physical/Hybrid SharePoint farms
Can use any EMC replication technology (RecoverPoint/SRDF/MirrorView)
EMC Storage Technologies for SharePoint 2010
•

LUN Compression/De-duplication/Thin Provisioning
‒
‒

•

Reduces initial storage requirements/costs for content databases
Enables SharePoint administrators to grow volumes non-disruptively

FAST Cache
‒

Helps to handle unanticipated performance requirements

SharePoint
Component

Performance
Improvement

Notes
Search Index re-uses the same blocks on disk as working space to process listitems
during indexing, then that storage is quiet between crawls

Search Index

High

Search Query

Medium

Query improves due to random large block reads/writes being serviced by
FASTCache, but Query storage is large and costly in FASTCache

tempdb

Medium

TempDB can benefit from FASTCache as the way SQL Server uses TempDB (database
page reuse) is ideally suited to FASTCache algorithms

Content databases

Medium

Low IOPs requirements does not require FastCache, DB Index operations see an
improvement

BLOB Store

Low

Low IOPs requirements, large-block I/O
BLOB Externalization (RBS/EBS)
 BLOB = Binary Large Object
 BLOB externalization options
‒ EBS using 3rd party provider- Mainly for MOSS 2007 but still supported with SP2010
– RBS through SQL RBS FILESTREAM provider (Local/Remote)
– RBS using feature rich 3rd party provider

 Pros
– Lower TCO as BLOBs can reside on lower cost storage tier (SATA/NL-SAS)
– More flexible and efficient storage configurations (CIFS/NFS/REST)
– Performance improvement

• Cons

– Large objects retrieval (>1Mb)
– SharePoint Indexing operations
– Various SQL operations (Index defragmentation, statistics operations, consistency checks)

– Backup and restore complexity
– 2GB file size limit is not lifted by using RBS
– Not supported with System Center DPM and/or Database Mirroring
http://technet.microsoft.com/en-us/library/ff628583.aspx
http://technet.microsoft.com/en-us/library/cc262787.aspx
Remote BLOB Storage (RBS)
Feature
Externalization using SMB/CIFS/NFS protocols
User Interface
BLOBs tiering based on Age/Size/Type/Hierarchy policies
Orphaned BLOB Garbage Collection
BLOB store volumes automatically included in VSS Backup

SQL Server
RBS FILESTREAM

3rd Party RBS provider

No (Local volumes only)

Yes

Powershell

Central Admin

No

Yes

Basic (RBS Maintainer)

Policy-based

Yes

No

 EMC has several RBS solutions
–
–
–
–
–

In partnership with several vendors for BLOB externalization (EMC Select-Metalogix StoragePoint)
EMC SourceOne also leverages RBS
Block/File/Object - VMAX, VNX, DataDomain, Isilon
Cloud Optimized Storage - Atmos/Atmos VE
Archival Storage - Centera
Considerations for Remote BLOB Storage
Replication, Backup and Recovery
– Native/Item level backup (stsadm based) would include BLOBs (“deep copy”)
– SQL based backup would only protect the content database
– To maintain consistency:
 Backup – First Content Databases then BLOB Store
 Restore – First BLOB Store then Content Databases

– For faster recovery, consider larger intervals of garbage collection jobs (Keeps previous BLOB versions)
– For DR purposes always tie RBS volumes with SQL Server volumes

Block, File or Object storage?
– Performance: Block would be faster but RBS has typically low I/O
– Storage Efficiency
– Block-level LUN Compression – increases storage efficiency, may affect backup performance
– Filesystem-Deduplication – better performance and increased dedup rate
Reference Architecture - BLOB Externalization
File Storage (CIFS)

Farm Profile

Total content ~1TB
4.36M documents
Avg. File Size - 220K

Storage Profile

EMC VNX5300
15K SAS (System and DBs) /7.2K NL-SAS (BLOB Store)
CIFS Share for RBS

Results

Max user capacity – 8,630 (10%)
BLOBs consumed 92% of content databases
Full crawl duration – 34 hours
8-10% user throughput (RPS) improvement using RBS
20% capacity saved with RBS FS de-duplication
Reference Architecture - BLOB Externalization
Block Storage

Farm Profile

Total content 3.1TB
1.2M documents
File Sizes – 200KB-5MB
16 Site Collections/40,000 Sites

Storage Profile

EMC Symmetrix VMAX
FAST VP controlled FC and SATA tiers

Results

BLOBs consumed 94% of content databases
3.1 TB Externalized in 35 Hours
Storage

Configuration

Baseline
Externalized

Concurrency

10%

Users (Heavy)
(80%-Browse/ 10% -Search/ 10%-Modify)

25,800
31,800

Response
Time
< 3 sec
Protecting SharePoint to Enterprise Scales
Replication Management for Microsoft SharePoint 2010

Leverage SAN based replication for Rapid full farm protection and item-level recovery….
– Hardware VSS-based coordinated SharePoint replication, Enabling farmwide consistency
– Negligible impact to farm performance even during the first
synchronization
– Utilizing EMC storage replication (Snaps/Clones/Bookmarks)
– Simple, intuitive SharePoint discovery and application set configuration
(Configure protection in <8 minutes)
– Works with SQL RBS FILESTREAM
– Full farm restore include search index consistency on recovery
– Item level recovery enabled by Kroll Ontrack Powercontrols
SharePoint Replication Management
Reference Architecture

Enabled by EMC Replication Manager, Kroll Ontrack PowerControls



Hybrid farm (Physical/Virtual)
1.5 TB of content (6,818,250 files)



15 SharePoint content DBs



Both SnapView Snaps
and Clones used

SharePoint action

EMC SnapView

STSADM

Clone: 3hr 11min

39hr 30min

Snapshot: 9min

(14.8 MB/s)

Clone: 11min

Not tested in this environment

Content database recovery (online)

7min

3hr 12min
(12 MB/s)

Item-level recovery

10min

Not tested in this environment (requires recovery farm)

Full farm backup (2.5 TB)
Daily incremental SharePoint backup
(~1% daily change rate)
Business Continuity for SharePoint 2010:
Options

SAN

SQL

Point in Time

Stretched Farm
(Partial Replication)

Mirror Farm

(Partial Replication)

Virtualized Farm

DB Mirroring
Log Shipping

(Complete Replication)

Continuous Replication

Backup
PiT Replication
Designing DR consistency for SharePoint
Automated DR
Consistency Group
(RP/SRDF/MV)

LUNs

Grouping

VMware
SRM

Hyper-V
Cluster Enabler

Web Front Ends

Boot + Query (optional)

Protection Group

Cluster Group

Query Servers

Boot + Query Component

Protection Group

Cluster Group

Index Servers

Boot + Index Component

Protection Group

Cluster Group

Application Servers

Boot + Application Volumes

Protection Group

Cluster Group

Protection Group

Cluster Group

Boot + Pagefile (optional)
SQL System Databases
SQL Server(s)

Configuration Databases
Search Databases
Content Databases

Content consistency

RBS BLOB Store

Search consistency
Protecting SharePoint

Business Continuity – vCenter SRM with RecoverPoint CRR

Proven Solution Test Results




13,080 heavy users @ 10% concurrency Production Site Outage
Sustained Performance
30% reduced cost of BLOB Storage with EMC
LUN Compression
<16 minutes to perform full end-to-end
failover

Failover Test Results
Test type
Executing recovery plan for a fully
operational farm (under load)

Shutting down
production VMs

Preparing storage

Restarting DR VMs

Total time

00:33

11:45

3:15

15:33
Key Takeaways
•

SharePoint is more than just SQL…
–
–
–

•

Full SharePoint and FAST Search farms virtualization has great advantages over physical/hybrid configurations
–
–
–

•

Horizontal scaling is more efficient
The best FULL farm protection when Integrated with EMC replication.
Simplifies, accelerates and automates SharePoint DR! (vCenter SRM, Hyper-V+EMC Cluster Enabler)

EMC’s SharePoint VSS based replication can significantly accelerate replication and recovery of SharePoint
–
–
–

•

Leverage EMC Proven solutions and Best Practices for SharePoint and FAST Search storage, networking and compute design
FAST, FAST Cache, VP improve efficiency & performance but require proper planning
Use RBS to improve scalability and TCO

A must for large deployments (TBs)
Protects content and index
Fast and simple Item level recovery while integrating with EMC partners (e.g. Kroll)

Block-level replication for Automated Disaster Recovery answers the difficult challenges in providing site
protection
–
–

Essential for critical SharePoint farms, where uptime and site resiliency is critical
Automates failover for the entire farm/s
Additional References
EMC Solutions for SharePoint Portal
–

http://www.emc.com/sharepoint

Technical Whitepapers
•

SharePoint in a multi-tenancy virtualized environment (Vblock)
–

•

Virtualized SharePoint 2010 (ESX 4.0, CX4-240)
–

•

http://www.emc.com/collateral/software/white-papers/h8024-virtual-sharepoint-clariion-vsphere-wp.pdf

Continuous protection for virtual SharePoint 2010 (ESX 4.1, RBS, RP, RM, CX4-120)
–

•

http://www.emc.com/collateral/software/white-papers/h8798-virtualizing-ms-apps-multi-tenancy-vblock1-wp.pdf

http://www.emc.com/collateral/software/white-papers/h8139-protection-virtualized-sharepoint-wp.pdf

SharePoint 2010 BLOB externalization (Hyper-V, Metalogix StoragePoint, VNX-5300)
–

http://www.emc.com/collateral/hardware/technical-documentation/h8185-sharepoint-vnx-metalogix-psg.pdf

• SharePoint 2007 Business Continuity (Hyper-V, Cluster Enabler, CX4-120)
–

http://www.emc.com/collateral/solutions/reference-architecture/h7041-bc-sharepoint-clariion-recoverpoint-hyperv-ref-arc.pdf

SharePoint Blogs - EMC
–
–

http://sharepointintheprivatecloud.wordpress.com
http://sustainablesharepoint.com
Q&A?
Thank you!
FAST Search Server…
Physical to Virtual Test Results

Comparison between Physical and Virtual FAST Search environments…
Description

Physical

Virtual

# Physical Servers

5

2

Total CPU number

60

20

# Document Processors

52

24

Search Response Time

<1sec

<1sec

SharePoint Content
full crawl rate (Avg. Doc Size 1.76MB)

Items/min

File Share Content
full crawl rate (Avg. Doc Size 1.56MB)

Items/min

1,513

123.1

1,667

152.5

GB/Hr

GB/Hr

770

81.7

Items/min

GB/Hr

978

82.98

Items/min

GB/Hr

Contenu connexe

Plus de European SharePoint Conference

SharePoint Adoption: Bridging the Gulf presented by Bob Kreha
SharePoint Adoption: Bridging the Gulf presented by Bob KrehaSharePoint Adoption: Bridging the Gulf presented by Bob Kreha
SharePoint Adoption: Bridging the Gulf presented by Bob KrehaEuropean SharePoint Conference
 
11 Strategic Considerations for SharePoint Migration presented by Christian B...
11 Strategic Considerations for SharePoint Migration presented by Christian B...11 Strategic Considerations for SharePoint Migration presented by Christian B...
11 Strategic Considerations for SharePoint Migration presented by Christian B...European SharePoint Conference
 
Why Are We Still Talking About SharePoint Governance? presented by Anders Skj...
Why Are We Still Talking About SharePoint Governance? presented by Anders Skj...Why Are We Still Talking About SharePoint Governance? presented by Anders Skj...
Why Are We Still Talking About SharePoint Governance? presented by Anders Skj...European SharePoint Conference
 
State Machine Workflow: Esoteric Techniques & Patterns Everyone Should Buy pr...
State Machine Workflow: Esoteric Techniques & Patterns Everyone Should Buy pr...State Machine Workflow: Esoteric Techniques & Patterns Everyone Should Buy pr...
State Machine Workflow: Esoteric Techniques & Patterns Everyone Should Buy pr...European SharePoint Conference
 
SharePoint Authentication and Authorization presented by Liam Cleary
SharePoint Authentication and Authorization presented by Liam ClearySharePoint Authentication and Authorization presented by Liam Cleary
SharePoint Authentication and Authorization presented by Liam ClearyEuropean SharePoint Conference
 
Infrastructure Best Practices for SharePoint On-Premises presented by Michael...
Infrastructure Best Practices for SharePoint On-Premises presented by Michael...Infrastructure Best Practices for SharePoint On-Premises presented by Michael...
Infrastructure Best Practices for SharePoint On-Premises presented by Michael...European SharePoint Conference
 
Forms and Workflow: Putting the Right Work in the Right Place presented by Mi...
Forms and Workflow: Putting the Right Work in the Right Place presented by Mi...Forms and Workflow: Putting the Right Work in the Right Place presented by Mi...
Forms and Workflow: Putting the Right Work in the Right Place presented by Mi...European SharePoint Conference
 
Installing SharePoint 2013 – Step by Step presented by Alan Richards
Installing SharePoint 2013 – Step by Step presented by Alan RichardsInstalling SharePoint 2013 – Step by Step presented by Alan Richards
Installing SharePoint 2013 – Step by Step presented by Alan RichardsEuropean SharePoint Conference
 

Plus de European SharePoint Conference (8)

SharePoint Adoption: Bridging the Gulf presented by Bob Kreha
SharePoint Adoption: Bridging the Gulf presented by Bob KrehaSharePoint Adoption: Bridging the Gulf presented by Bob Kreha
SharePoint Adoption: Bridging the Gulf presented by Bob Kreha
 
11 Strategic Considerations for SharePoint Migration presented by Christian B...
11 Strategic Considerations for SharePoint Migration presented by Christian B...11 Strategic Considerations for SharePoint Migration presented by Christian B...
11 Strategic Considerations for SharePoint Migration presented by Christian B...
 
Why Are We Still Talking About SharePoint Governance? presented by Anders Skj...
Why Are We Still Talking About SharePoint Governance? presented by Anders Skj...Why Are We Still Talking About SharePoint Governance? presented by Anders Skj...
Why Are We Still Talking About SharePoint Governance? presented by Anders Skj...
 
State Machine Workflow: Esoteric Techniques & Patterns Everyone Should Buy pr...
State Machine Workflow: Esoteric Techniques & Patterns Everyone Should Buy pr...State Machine Workflow: Esoteric Techniques & Patterns Everyone Should Buy pr...
State Machine Workflow: Esoteric Techniques & Patterns Everyone Should Buy pr...
 
SharePoint Authentication and Authorization presented by Liam Cleary
SharePoint Authentication and Authorization presented by Liam ClearySharePoint Authentication and Authorization presented by Liam Cleary
SharePoint Authentication and Authorization presented by Liam Cleary
 
Infrastructure Best Practices for SharePoint On-Premises presented by Michael...
Infrastructure Best Practices for SharePoint On-Premises presented by Michael...Infrastructure Best Practices for SharePoint On-Premises presented by Michael...
Infrastructure Best Practices for SharePoint On-Premises presented by Michael...
 
Forms and Workflow: Putting the Right Work in the Right Place presented by Mi...
Forms and Workflow: Putting the Right Work in the Right Place presented by Mi...Forms and Workflow: Putting the Right Work in the Right Place presented by Mi...
Forms and Workflow: Putting the Right Work in the Right Place presented by Mi...
 
Installing SharePoint 2013 – Step by Step presented by Alan Richards
Installing SharePoint 2013 – Step by Step presented by Alan RichardsInstalling SharePoint 2013 – Step by Step presented by Alan Richards
Installing SharePoint 2013 – Step by Step presented by Alan Richards
 

Dernier

Vertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsVertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsMiki Katsuragi
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr BaganFwdays
 
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024Lonnie McRorey
 
Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Enterprise Knowledge
 
DevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenDevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenHervé Boutemy
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii SoldatenkoFwdays
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningLars Bell
 
Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebDev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebUiPathCommunity
 
The Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsThe Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsPixlogix Infotech
 
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdfHyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdfPrecisely
 
Connect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationConnect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationSlibray Presentation
 
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsMark Billinghurst
 
Commit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyCommit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyAlfredo García Lavilla
 
Artificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxArtificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxhariprasad279825
 
SAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxSAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxNavinnSomaal
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc
 
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .Alan Dix
 
Powerpoint exploring the locations used in television show Time Clash
Powerpoint exploring the locations used in television show Time ClashPowerpoint exploring the locations used in television show Time Clash
Powerpoint exploring the locations used in television show Time Clashcharlottematthew16
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsRizwan Syed
 

Dernier (20)

Vertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsVertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering Tips
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan
 
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024
 
Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024
 
DevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache MavenDevoxxFR 2024 Reproducible Builds with Apache Maven
DevoxxFR 2024 Reproducible Builds with Apache Maven
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko
 
DSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine TuningDSPy a system for AI to Write Prompts and Do Fine Tuning
DSPy a system for AI to Write Prompts and Do Fine Tuning
 
Dev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio WebDev Dives: Streamline document processing with UiPath Studio Web
Dev Dives: Streamline document processing with UiPath Studio Web
 
The Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and ConsThe Ultimate Guide to Choosing WordPress Pros and Cons
The Ultimate Guide to Choosing WordPress Pros and Cons
 
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptxE-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
 
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdfHyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
Hyperautomation and AI/ML: A Strategy for Digital Transformation Success.pdf
 
Connect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck PresentationConnect Wave/ connectwave Pitch Deck Presentation
Connect Wave/ connectwave Pitch Deck Presentation
 
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR Systems
 
Commit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyCommit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easy
 
Artificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxArtificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptx
 
SAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptxSAP Build Work Zone - Overview L2-L3.pptx
SAP Build Work Zone - Overview L2-L3.pptx
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
 
From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .From Family Reminiscence to Scholarly Archive .
From Family Reminiscence to Scholarly Archive .
 
Powerpoint exploring the locations used in television show Time Clash
Powerpoint exploring the locations used in television show Time ClashPowerpoint exploring the locations used in television show Time Clash
Powerpoint exploring the locations used in television show Time Clash
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL Certs
 

Optimize, Store and Protect SharePoint 2010 Server…Best Practices presented by James Baldwin

  • 1. Optimize, Store and Protect SharePoint 2010 Server . …Best Practices James Baldwin Strategic Solutions Group EMC Corporation Session W21
  • 2. Agenda • SharePoint architecture and performance factor • SharePoint and FAST Search virtualization • Storage best practices – sizing and performance • Remote BLOB Storage • Storage options and features • SharePoint farm Protection Apologises in advance…..for the word… FAST FAST…as in Search… FAST…as in VP … FAST…as in “let’s get out of here!”
  • 3. Proven Solutions approach 1 2 3 4 Capture Test and Validate Document Publish & Define Durham, NC Santa Clara, CA Cork, Ireland Vienna, Austria Shanghai, China Singapore
  • 4. SharePoint performance - The user experience Domain Controller SQL Server Authentication Storage − Content/Metadata − Search − System BLOB Storage (Optional) BLOB Retrieval/Creation Document Request Web Front End Server Operations Network •‒ CPU Browsing to the home page •‒ Memory Browsing to a document library Acceptable user response time Client <3 seconds ‒ HBA/CNA •‒ NIC Creating a subsite Creating a list • Uploading a document to a document library <5 seconds • Backing up a site • Creating a site collection <7 seconds Technet: http://technet.microsoft.com/en-us/library/cc287790(office.12).aspx
  • 5. SharePoint Farm Topologies Small Scale Medium out approach Large Web Web Servers Groups H/W Eval Prod RAM 4GB 8GB CPU 4 Cores Web/Query User requests Web Crawling/Admin App Servers Groups Application H/W Eval Prod RAM 4GB 8GB CPU 4 Cores Database H/W Small 8 GB 4 Cores 8 Cores App Query Crawl Central Admin /Office/Other 16 GB CPU Query/Crawl Medium RAM App http://technet.microsoft.com/en-us/library/cc298801.aspx http://technet.microsoft.com/en-us/library/cc262485.aspx All DBs Search DBs SharePoint DBs Search DBs SharePoint DBs Content DBs
  • 6. Key Benefits – Virtualizing SharePoint Consolidation • Achieve 2-10x consolidation ratio, especially for larger deployments Performance • Improved front end performance with more, smaller WFEs rather than few large WFEs Availability • VM based protection for SharePoint provides homogeneous high availability (WSFC, VMware HA) Business Continuity • Simplified DR management (Geo-Clustering, vCenter Site Recovery Manager) Maintenance • Live migration of virtual machines (Hyper-V Live Migration, VMware vMotion) Load Balancing • Maximized overall performance with maximum HW utilization (SCVMM PRO , VMware DRS)
  • 7. Virtualizing Server Roles In SharePoint Server Roles/Priority 1st Web Front End / Query 2nd Application (Excel, Doc Conv, etc) 3rd Index/Crawl 4th SQL Server What To Consider  CPU – User concurrency, Search requests  Scaling out is more efficient  Network – segment virtual NICs and virtual Switches  CPU – Application dependent  Scaling out is more efficient  Redundant (Non redundant in MOSS 2007)  CPU – Crawling, indexing (depends on content type/size)  Scale out (Up only with MOSS 2007)  Memory intensive  CPU – Document updates, Search, Backup  VMFS/RDM, VHD/Pass-Through  Scale up/out (Hyper-V ≤ 4 vCPU, vSphere ≤32 vCPU)  Failover Clustering, Mirroring, VMware HA Understanding your existing workload (WFE to SQL) and requirements is better than any general best practice!!!
  • 8. A Day in the life of SharePoint… SQL Server CPU The majority of load comes from systematic operations… SharePoint Timer Job Reference - http://technet.microsoft.com/en-us/library/cc678870.aspx Sample anonymous customer data
  • 9. Virtualized SharePoint - Reference Architectures Enterprise-Class SharePoint (Virtual) and FAST Search (Physical) • 9 TB User content – (4TB SharePoint | 5TB Fileshare) • Virtualized SharePoint 2010 Farm • Virtualized Query and Content SSAs • 5 Physical FAST Search Servers (12-core ea.) • • 2 Index Servers (1 Partition, 2 Search Roles) • • 2 Document Processors/Content Distributors 1 FAST Admin inc. Document Processor <1 Sec search latency with 22,000 Heavy Users @ 10% Concurrency Role Qty VM specs Hyper-V Servers 3 Nodes 4-socket quad-core (16 cores), 128 GB RAM SQL Server 2008 2 4 vCPU, 32 GB RAM Web front ends Application servers (Incl. Central Admin) Query SSA 4 4 vCPU, 6 GB RAM 2 2 vCPU, 4 GB RAM 1 4 vCPU, 8 GB RAM Content SSA 1 4 vCPU, 8 GB RAM Fast Search Physical Servers 5 2-socket hex-core (12 cores), 48 GB RAM Whitepaper due for release in October timeframe
  • 10. Virtualized SharePoint - Reference Architectures Enterprise-Class SharePoint and FAST Search (All Virtual) • 9 TB User content – (4TB SharePoint | 5TB Fileshare) • Virtualized SharePoint 2010 Farm • Virtualized FAST Search ! • • 2 Index Servers (1 Partition, 2 Search Roles) • • 2 Document Processors/Content Distributors 1 FAST Admin inc. Document Processor <1 Second search latency with 22,000 Heavy Users @ 10% Concurrency Role Qty VM specs Hyper-V Servers (SharePoint Farm) 3 Nodes 4-socket quad-core (16 cores), 128 GB RAM Hyper-V Servers (FAST Search Farm) 2 Nodes 2-socket hex-core (12 cores), 48 GB RAM SQL Server 2008 2 4 vCPU, 32 GB RAM Web front ends Application servers (Incl. Central Admin) Query SSA 4 4 vCPU, 6 GB RAM 2 2 vCPU, 4 GB RAM 1 4 vCPU, 8 GB RAM Content SSA 1 4 vCPU, 8 GB RAM Whitepaper due for release in October timeframe
  • 11. SharePoint 2010 Storage Elements SQL Server Content System and Configuration Content Databases Configuration Databases BLOB Store Central Admin tempdb, master ,model , msdb • SharePoint relies mainly on SQL but there’s important data outside SQL System Content DBs Volumes 8% • Typical storage usage is not I/O heavy 20% BLOBs SA Data 34% 12% • Search services in SharePoint generate most of the I/O Search load Index System DBs SA DBs • Content – Plan15% capacity for 6% 5% • Search and System – Plan for performance % Total Capacity - Sample Shared Service Applications Usage & Health Data Collection - Logging Search – Admin, Property, Crawl Web Analytics – Staging, Reporting User Profiles – Profile, Synchronization, Social Tagging Managed Metadata- Term Store State Business Data Connectivity Secure Store Search Index Index Partition/s Query Component/s Service Application Data SA Volumes System Volumes Boot/OS/VMs Web Parts & Features SharePoint Binaries
  • 12. 1TB content farm – How much storage do I need? Gigabytes of Storage 5000 4000 3000 2000 1000 0 2150 2 WFEs/no Search 2910 2 WFEs/1x Index&Query 3310 2 WFEs/2x Index&Query 3290 2 WFEs/1x Index/2x Query (Striped) 4470 2 WFEs/2x Index/4xQuery (4 Partitions, Mirrored(1-A,D, etc)
  • 13. A Day in the life of SharePoint… SQL Server Storage I/O Plan for user load peaks, not systematic peaks… Sample anonymous customer data
  • 14. SharePoint 2010 Storage I/O Characteristics * Based on average workloads in a collaboration farm (Browse/Search/Modify – 80/10/10) SQL Server Search Databases Index Component Query component 8 8 32 32 16 32 16 64 64 95:5 50:50 60:40 90:10 70:30 Content Databases tempdb Read (KB) 16 Write (KB) R:W Ratio I/O Type SharePoint In-box Search Servers (property & crawl stores) FAST Search Servers Read (KB) Write (KB) R:W Ratio FAST Index Server (Primary) 294 611 3:1 FAST Index Server (Backup) 31 710 1:30 FAST Servers FAST Server 13 26 1:66 (Document Processors ,etc) SQLIO/SQLIOSim are I/O stress tools - should not be considered as “performance requirements” !
  • 15. SharePoint 2010 Storage performance Requirements/estimates for Search Query Crawler Query Component Crawl Component Database Role Microsoft’s Estimate IOPS SQL Server Search Admin DB Property DB Typical averages observed Crawl Database 3,500 – 7,000 (R70:W30) Property Database 2,000 (R30:W70) 600 Query Component 2,000 per Active/Failover pair (Load/Write/Merge) 450 Crawl Component 300-400 IOPS http://technet.microsoft.com/en-us/library/cc298801.aspx Based on a Microsoft case study – Mileage may vary!!! 2,000 80-100 Crawl DB
  • 16. FAST Search Server 2010 for SharePoint Storage Requirements/estimates FAST Admin S:FASTSearch LUN size: 80GB FAST server role Document Processor 01 Document Processor 02 Index 01 (Primary) S:FASTSearch LUN size: 80GB S:FASTSearch LUN size: 80GB S:FASTSearch LUN size: 2000GB Microsoft’s Estimate LUN size Web Analyzer Database (2GB per 1M items) SharePoint or Intranet (6GB per 1M items) Public Web content (25GB per 1M items) Index server (Primary) X2.5 the size of a single index file set +50 GB synchronization data Index server (Secondary) X2.5 the size of a single index file set +50 GB synchronization data Based on a Microsoft case study – Mileage may vary!!! Index 02 (Backup) S:FASTSearch LUN size: 2000GB Typical averages observed 1.2 GB 910 GB (2 TB recommended to hold temp indexing files) 899 GB (2 TB recommended to hold temp indexing files) http://technet.microsoft.com/en-us/library/ff381239.aspx
  • 17. SharePoint Reference Architecture Storage Layout Cost driven configuration ‒ 13,000 Heavy SharePoint users ‒ 1 TB User content with RBS FILESTREAM Externalization ‒ SAN based replication (RecoverPoint) Storage Configuration ‒ RAID5 –VM Boot Luns, Content Databases ‒ RAID10 – Search Databases, tempdb ‒ Array SSD Cache to compensate for disk latencies http://www.emc.com/collateral/software/white-papers/h8139-protection-virtualized-sharepoint-wp.pdf
  • 18. SharePoint Storage Sizing Volume Sizing % of Corpus Size Typical sizes GB Recommended RAID Virtual Machine Boot Volumes - 80 R-5 Application Volumes - 50 – 300+ R-5 Data File Volume - 100 – 200 R-5 Log File Volume 10% of Data File 10 – 20 R-5 / R-10 Data File Volume 10% 100 – 300 R-10 Log File Volume 2% 10 – 100 R-10 Storage Role Content Databases tempdb Storage and SQL Server capacity planning and configuration Hardware and software requirements http://technet.microsoft.com/en-us/library/cc298801.aspx http://technet.microsoft.com/en-us/library/cc262485.aspx
  • 19. SharePoint Storage Sizing Volume Sizing % of Corpus Size Sample size (1TB Content) Recommend RAID Crawl Store DB 0.046 × (sum of content databases) 46GB R-10 Property Store DB 0.015 × (sum of content databases) 15GB R-10 Index Component 10% 100GB R5 / R10 Query Component 10 – 35% * 2.85 150 – 1TB R-5 / R-10 SharePoint_Config - 5GB R-5 SP_AdminContent - 15GB R-5 - (based on % monitoring) 50 - 500GB(?) R-5 Storage Role SQL Search Databases Search Server Volumes SharePoint Configuration Databases - Data & Log Volume Usage and Health Data Collection Database SharePoint 2010 Database sizing characteristics http://technet.microsoft.com/en-us/library/cc298801.aspx http://technet.microsoft.com/en-us/library/cc678868(office.14).aspx
  • 20. FAST Search server 2010 for SharePoint Storage Sizing Volume Sizing (estimates) Microsoft Data set Typical result observed (5.2 million items) Recommend RAID N/A 460 MB R-10 FASTQuerySSA_PropertyStore_guid <0.4GB 158 MB R-10 FASTQuerySSA_CrawlStoreDB_guid 3.3 GB per million items 15 MB R-10 N/A 99 MB R-10 <0.4GB 158 MB R-10 3.3 GB per million items 38621 MB R-10 <0.1GB 3.00 MB R-10 Storage Role FASTQuerySSA_DB_guid FAST Query SSA Databases FASTContent_DB_guid FAST Query SSA Databases FASTContent_PropertyStoreDB_guid FASTContent_CrawlStoreDB_guid FAST Admin FASTAdminConfigDB SharePoint 2010 Database sizing characteristics http://technet.microsoft.com/en-us/library/cc298801.aspx http://technet.microsoft.com/en-us/library/cc678868(office.14).aspx
  • 21. SharePoint Storage Best Practices SQL Server – Storage Allocation  Use 64KB unit allocation size when formatting DB volumes  Physical volumes (RDM/Pass-through) for larger than 2TB when virtualized  Plan Database file sizes accordingly – Don’t rely on autogrowth  File growth can cause locking, set files size and autogrowth increments appropriately – Using RBS would keep the SQL Database files small – Watch the SP Configuration database log file growth!  When using Thin/Virtual provisioning – Use the “Quick Format” option – Enable Instant file initialization*  Enhances the speed for data file creations, restores, data file growth  Assign SQL service account to “Perform Volume Maintenance Tasks” permission – Log files are fully allocated and zeroed upon creation http://msdn.microsoft.com/en-us/library/ms175935.aspx or growths
  • 22. SharePoint Storage Best Practices SQL Server - Performance Plan for optimal disk response times Data file Latency Log file Latency Read/Write Operations Write Operations < 10 ms < 5 ms Very Good < 20 ms 5 – 10 ms Acceptable > 20 ms > 15 ms Investigate and Improve Recommendation Important Perfmon I/O counters Real Meaning! Average Disk/sec Read or Write Disk Latency Current Disk Queue Length* Outstanding I/Os Disk Reads/Writes per Second IOPS Average Disk Bytes/Read & Write I/O Size (KB) * Hard to interpret due to virtualization of storage. Consider in combination with response times
  • 23. EMC Storage Technologies for SharePoint 2010 • Thin Provisioned Pools ‒ ‒ • Reduces initial storage requirements/costs for content databases Enables SharePoint administrators to grow volumes non-distruptively FAST VP (Fully Automated Storage Tiering) ‒ Helps to handle unanticipated performance requirements SharePoint Component Recommended Notes Search Index No Highly changing, throw-away data Search Query Yes Highly-read data with small burst write changes tempdb Yes Same blocks are re-used on disk and performance of TEMPDB directly affects SharePoint performance request Content databases BLOB Store – Yes/Maybe No C-Databases which are defragmented regularly or complex Databases like NewsGator BLOB Store has low IOPs requirements EMC Cluster Enabler (CE) – – Automated multi-site Disaster Recovery (failover) for Hyper-V/Physical/Hybrid SharePoint farms Can use any EMC replication technology (RecoverPoint/SRDF/MirrorView)
  • 24. EMC Storage Technologies for SharePoint 2010 • LUN Compression/De-duplication/Thin Provisioning ‒ ‒ • Reduces initial storage requirements/costs for content databases Enables SharePoint administrators to grow volumes non-disruptively FAST Cache ‒ Helps to handle unanticipated performance requirements SharePoint Component Performance Improvement Notes Search Index re-uses the same blocks on disk as working space to process listitems during indexing, then that storage is quiet between crawls Search Index High Search Query Medium Query improves due to random large block reads/writes being serviced by FASTCache, but Query storage is large and costly in FASTCache tempdb Medium TempDB can benefit from FASTCache as the way SQL Server uses TempDB (database page reuse) is ideally suited to FASTCache algorithms Content databases Medium Low IOPs requirements does not require FastCache, DB Index operations see an improvement BLOB Store Low Low IOPs requirements, large-block I/O
  • 25. BLOB Externalization (RBS/EBS)  BLOB = Binary Large Object  BLOB externalization options ‒ EBS using 3rd party provider- Mainly for MOSS 2007 but still supported with SP2010 – RBS through SQL RBS FILESTREAM provider (Local/Remote) – RBS using feature rich 3rd party provider  Pros – Lower TCO as BLOBs can reside on lower cost storage tier (SATA/NL-SAS) – More flexible and efficient storage configurations (CIFS/NFS/REST) – Performance improvement • Cons – Large objects retrieval (>1Mb) – SharePoint Indexing operations – Various SQL operations (Index defragmentation, statistics operations, consistency checks) – Backup and restore complexity – 2GB file size limit is not lifted by using RBS – Not supported with System Center DPM and/or Database Mirroring http://technet.microsoft.com/en-us/library/ff628583.aspx http://technet.microsoft.com/en-us/library/cc262787.aspx
  • 26. Remote BLOB Storage (RBS) Feature Externalization using SMB/CIFS/NFS protocols User Interface BLOBs tiering based on Age/Size/Type/Hierarchy policies Orphaned BLOB Garbage Collection BLOB store volumes automatically included in VSS Backup SQL Server RBS FILESTREAM 3rd Party RBS provider No (Local volumes only) Yes Powershell Central Admin No Yes Basic (RBS Maintainer) Policy-based Yes No  EMC has several RBS solutions – – – – – In partnership with several vendors for BLOB externalization (EMC Select-Metalogix StoragePoint) EMC SourceOne also leverages RBS Block/File/Object - VMAX, VNX, DataDomain, Isilon Cloud Optimized Storage - Atmos/Atmos VE Archival Storage - Centera
  • 27. Considerations for Remote BLOB Storage Replication, Backup and Recovery – Native/Item level backup (stsadm based) would include BLOBs (“deep copy”) – SQL based backup would only protect the content database – To maintain consistency:  Backup – First Content Databases then BLOB Store  Restore – First BLOB Store then Content Databases – For faster recovery, consider larger intervals of garbage collection jobs (Keeps previous BLOB versions) – For DR purposes always tie RBS volumes with SQL Server volumes Block, File or Object storage? – Performance: Block would be faster but RBS has typically low I/O – Storage Efficiency – Block-level LUN Compression – increases storage efficiency, may affect backup performance – Filesystem-Deduplication – better performance and increased dedup rate
  • 28. Reference Architecture - BLOB Externalization File Storage (CIFS) Farm Profile Total content ~1TB 4.36M documents Avg. File Size - 220K Storage Profile EMC VNX5300 15K SAS (System and DBs) /7.2K NL-SAS (BLOB Store) CIFS Share for RBS Results Max user capacity – 8,630 (10%) BLOBs consumed 92% of content databases Full crawl duration – 34 hours 8-10% user throughput (RPS) improvement using RBS 20% capacity saved with RBS FS de-duplication
  • 29. Reference Architecture - BLOB Externalization Block Storage Farm Profile Total content 3.1TB 1.2M documents File Sizes – 200KB-5MB 16 Site Collections/40,000 Sites Storage Profile EMC Symmetrix VMAX FAST VP controlled FC and SATA tiers Results BLOBs consumed 94% of content databases 3.1 TB Externalized in 35 Hours Storage Configuration Baseline Externalized Concurrency 10% Users (Heavy) (80%-Browse/ 10% -Search/ 10%-Modify) 25,800 31,800 Response Time < 3 sec
  • 30. Protecting SharePoint to Enterprise Scales Replication Management for Microsoft SharePoint 2010 Leverage SAN based replication for Rapid full farm protection and item-level recovery…. – Hardware VSS-based coordinated SharePoint replication, Enabling farmwide consistency – Negligible impact to farm performance even during the first synchronization – Utilizing EMC storage replication (Snaps/Clones/Bookmarks) – Simple, intuitive SharePoint discovery and application set configuration (Configure protection in <8 minutes) – Works with SQL RBS FILESTREAM – Full farm restore include search index consistency on recovery – Item level recovery enabled by Kroll Ontrack Powercontrols
  • 31. SharePoint Replication Management Reference Architecture Enabled by EMC Replication Manager, Kroll Ontrack PowerControls   Hybrid farm (Physical/Virtual) 1.5 TB of content (6,818,250 files)  15 SharePoint content DBs  Both SnapView Snaps and Clones used SharePoint action EMC SnapView STSADM Clone: 3hr 11min 39hr 30min Snapshot: 9min (14.8 MB/s) Clone: 11min Not tested in this environment Content database recovery (online) 7min 3hr 12min (12 MB/s) Item-level recovery 10min Not tested in this environment (requires recovery farm) Full farm backup (2.5 TB) Daily incremental SharePoint backup (~1% daily change rate)
  • 32. Business Continuity for SharePoint 2010: Options SAN SQL Point in Time Stretched Farm (Partial Replication) Mirror Farm (Partial Replication) Virtualized Farm DB Mirroring Log Shipping (Complete Replication) Continuous Replication Backup PiT Replication
  • 33. Designing DR consistency for SharePoint Automated DR Consistency Group (RP/SRDF/MV) LUNs Grouping VMware SRM Hyper-V Cluster Enabler Web Front Ends Boot + Query (optional) Protection Group Cluster Group Query Servers Boot + Query Component Protection Group Cluster Group Index Servers Boot + Index Component Protection Group Cluster Group Application Servers Boot + Application Volumes Protection Group Cluster Group Protection Group Cluster Group Boot + Pagefile (optional) SQL System Databases SQL Server(s) Configuration Databases Search Databases Content Databases Content consistency RBS BLOB Store Search consistency
  • 34. Protecting SharePoint Business Continuity – vCenter SRM with RecoverPoint CRR Proven Solution Test Results    13,080 heavy users @ 10% concurrency Production Site Outage Sustained Performance 30% reduced cost of BLOB Storage with EMC LUN Compression <16 minutes to perform full end-to-end failover Failover Test Results Test type Executing recovery plan for a fully operational farm (under load) Shutting down production VMs Preparing storage Restarting DR VMs Total time 00:33 11:45 3:15 15:33
  • 35. Key Takeaways • SharePoint is more than just SQL… – – – • Full SharePoint and FAST Search farms virtualization has great advantages over physical/hybrid configurations – – – • Horizontal scaling is more efficient The best FULL farm protection when Integrated with EMC replication. Simplifies, accelerates and automates SharePoint DR! (vCenter SRM, Hyper-V+EMC Cluster Enabler) EMC’s SharePoint VSS based replication can significantly accelerate replication and recovery of SharePoint – – – • Leverage EMC Proven solutions and Best Practices for SharePoint and FAST Search storage, networking and compute design FAST, FAST Cache, VP improve efficiency & performance but require proper planning Use RBS to improve scalability and TCO A must for large deployments (TBs) Protects content and index Fast and simple Item level recovery while integrating with EMC partners (e.g. Kroll) Block-level replication for Automated Disaster Recovery answers the difficult challenges in providing site protection – – Essential for critical SharePoint farms, where uptime and site resiliency is critical Automates failover for the entire farm/s
  • 36. Additional References EMC Solutions for SharePoint Portal – http://www.emc.com/sharepoint Technical Whitepapers • SharePoint in a multi-tenancy virtualized environment (Vblock) – • Virtualized SharePoint 2010 (ESX 4.0, CX4-240) – • http://www.emc.com/collateral/software/white-papers/h8024-virtual-sharepoint-clariion-vsphere-wp.pdf Continuous protection for virtual SharePoint 2010 (ESX 4.1, RBS, RP, RM, CX4-120) – • http://www.emc.com/collateral/software/white-papers/h8798-virtualizing-ms-apps-multi-tenancy-vblock1-wp.pdf http://www.emc.com/collateral/software/white-papers/h8139-protection-virtualized-sharepoint-wp.pdf SharePoint 2010 BLOB externalization (Hyper-V, Metalogix StoragePoint, VNX-5300) – http://www.emc.com/collateral/hardware/technical-documentation/h8185-sharepoint-vnx-metalogix-psg.pdf • SharePoint 2007 Business Continuity (Hyper-V, Cluster Enabler, CX4-120) – http://www.emc.com/collateral/solutions/reference-architecture/h7041-bc-sharepoint-clariion-recoverpoint-hyperv-ref-arc.pdf SharePoint Blogs - EMC – – http://sharepointintheprivatecloud.wordpress.com http://sustainablesharepoint.com
  • 38. FAST Search Server… Physical to Virtual Test Results Comparison between Physical and Virtual FAST Search environments… Description Physical Virtual # Physical Servers 5 2 Total CPU number 60 20 # Document Processors 52 24 Search Response Time <1sec <1sec SharePoint Content full crawl rate (Avg. Doc Size 1.76MB) Items/min File Share Content full crawl rate (Avg. Doc Size 1.56MB) Items/min 1,513 123.1 1,667 152.5 GB/Hr GB/Hr 770 81.7 Items/min GB/Hr 978 82.98 Items/min GB/Hr