New Exchange Server 2013 Architecture

1. Exchange Server 2013
Architecture
Ross Smith IV
Principal Program Manager, Exchange Server

2. Disclaimer
© 2012 Microsoft Corporation. All rights reserved.
Microsoft, Office 365, and other product and service
names are or may be registered trademarks and/or
trademarks in the U.S. and/or other countries.
The information herein is for informational purposes
only and represents the current view of Microsoft
Corporation as of the date of this
presentation. Because Microsoft must respond to
changing market conditions, it should not be
interpreted to be a commitment on the part of
Microsoft, and Microsoft cannot guarantee the
accuracy of any information provided after the date of
this presentation. MICROSOFT MAKES NO
WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS
TO THE INFORMATION IN THIS PRESENTATION.

3. Agenda
Agenda
Discuss the new server
architecture in Exchange 2013
Discuss the Client Access
server role
Discuss the Mailbox
server role

4. E2007/E2010 Server Role Architecture
Enterprise Network
5 major roles
Tightly
coupled
• Functionality
• Geo affinity
Forefront
Online
Protection for
Exchange
External
SMTP
servers
Edge Transport
Routing and AV/AS
Hub Transport
Routing & policy
Mailbox
Storage of
mailbox items
• Versioning
• User partitioning
Mobile
phone
Layer 7 LB
Web
browser
Outlook
(remote user)
Phone system
(PBX or VOIP)
Outlook (local user)
Unified Messaging
Voice mail and
voice access
Client Access
Client connectivity
Web services
Line of
business
application
AD

5. What’s wrong with the existing model?
•
•
Exchange deployments are overly complicated
Doing Exchange load balancing “right” is hard and often requires
expensive solutions
– Requiring session affinity at the load balancer significantly impacts scalability
– Many hardware load balancing solutions are expensive and thus, are a luxury
many of our customers can’t afford or don’t have the expertise to deploy
•
Customers deploy based on dedicated server roles
– This means that in many cases, hardware is deployed that is unutilized (e.g.,
DIMM slots and disk slots, etc.) or under-utilized
•
Too many namespaces are required (especially in site resilient designs)

6. Exchange: The Evolution
LB
L7 LB
CAS
Ex
MBX
Ex
HT
Ex
SAN
MBX
Ex

7. Exchange: The Evolution
2000/2003
2007
2010
2013
Role differentiation through
manual configuration
Separate roles for ease of
deployment and mgmt.
segmentation
Separate HA solutions for
each role
Simplify for scale, balanced
utilization, isolation
Introduced the DAG
Integrate HA for all roles
Rich management
experience using RBAC
Simplify network
architecture
Hardware solutions for
“reliability” ($$$$)
Support cheaper storage
Leaves resources on the
ground in each role
Ex
Ex
SAN
Ex
Ex
CAS
HT
MBX
MBX
L7 LB
LB

8. The new server role
architecture

9. Exchange 2013 Architecture Theme
Use Building Blocks to facilitate deployments at all scales – from self-hosted
small organizations to Office 365
– Server role evolution
– Network layer improvements
– Versioning and inter-op principles
Benefits
–
–
–
–
Hardware efficiency
Deployment simplicity
Low friction cross-version inter-op
Failure isolation

10. Exchange 2013 Server Role Architecture
2 Building
Blocks
Database
Availability Group
• Evolution of
E2010 DAG
• Includes core
server protocols
Loosely
coupled
• Functionality
• Versioning
• User partitioning
• Geo affinity
Forefront
Online
Protection for
Exchange
External
SMTP
servers
CAS Array
CAS
CAS
MBX
CAS
MBX
MBX
CAS
MBX
Mobile
phone
Web
browser
Outlook
(remote user)
AD
MBX
CAS
Edge Transport
Routing and AV/AS
DAG
Layer 4LB
Client Access Array
• Evolution of
E2010 CAS Array
• SMTP Front-End
Enterprise Network
Outlook (local user)
Line of
business
Phone system
application (PBX or VOIP)

11. Every Server is an Island
EWS protocol
MRS proxy protocol
SMTP
Protocols,
Server Agents
MRS MRSProxy
EWS
Transport
Custom WS
MRS MRSProxy
Transport
Storage
Assistants
Assistants
RPC CA
XSO
Mail Item
XSO
Mail Item
CTS
Business Logic
RPC CA
Other API
CTS
Other API
Store
Content index
Store
Content index
ESE
File system
ESE
File system
Server1 (Vn)
Banned
E2010
Server2 (Vn+1)
EWS

12. Functional Layering
E2010 Architecture
L7LB
Load Balancer
AuthN, Proxy, Re-direct
CAS, HT, UM
AuthN, Proxy, Re-direct
Protocols, API, Biz-logic
Assistants, Store, CI
Store, CI
MBX2013
Protocols, API, Biz-logic
MBX
CAS2013
Hardware LB
E2013 Architecture

13. The key to enlightenment…
User
For a given mailbox’s connectivity, the
protocol being used is always served by
the protocol instance that is local to the
active database copy
CAS
This means that the rendering for clients like
OWA occurs on the Mailbox server
This means that Transport transcoding is
occurring on the Mailbox server etc…
DAG1
MBX-A
MBX-B

14. Client Access Protocol
Proxying

15. What is the CAS2013 role?
•
CAS2013 is comprised of three components:
•
Thin, stateless (protocol session) servers organized in a load balanced
configuration
– Client protocols (HTTP, IMAP, POP)
– SMTP
– UM Call Router
– Session affinity NOT required at the load balancer
•
•
Provides a unified namespace and authentication for clients
Where the logic “lives” to route a specific protocol request to the “correct”
destination end point
– Capable of supporting legacy servers with redirect or proxy logic
•
Is a domain-joined machine in the corporate forest

16. CAS2013 Client Protocol Architecture
OWA
Outlook
EAS
EAC
PowerShell
IMAP
SMTP
Telephony
Load
Balancer
Redirect
IIS
CAS2013
POP
IMAP
HTTP Proxy
POP
IMAP
HTTP
UM
SMTP
POP
IMAP
IIS
MBX2013
SMTP
Transport
UM
RpcProxy
RPS
OWA, EAS, EWS, ECP, OAB
RPC CA
MDB
MailQ
SIP
+
RTP

17. Outlook Connectivity
RPC/HTTP and the death of RPC/TCP
RPCProxy.dll
• What are the benefits?
– Does not require a “RPC CAS array namespace” for the DAG
– No longer have to worry about “The Exchange administrator has made a
change that requires you to quit and restart Outlook” during mailbox
moves or *over events
– Extremely reliable and stable connectivity model – the RPC session is
always on the MBX2013 server hosting the active database copy
• What changes?
– RPC end point for Outlook client is now a GUID (and SMTP suffix)
– Support for internal and external Outlook Anywhere namespaces

18. Third-Party MAPI Products
• Third-party MAPI products will need to use RPC/HTTP to connect to
CAS2013
• Exchange 2013 will be the last release that supports a MAPI/CDO
download
– Third-parties must move to EWS in the future
• The MAPI/CDO download will be updated to include support for
RPC/HTTP connectivity
– Will require third-party application configuration; either by
programmatically editing a dynamic MAPI profile or setting registry keys
– Legacy environments can continue to use RPC/TCP

19. CAS2013 Client Protocol Connectivity Flow
HTTP
HTTP
Load Balancer
HTTP Proxy
HTTP
MBX
CAS
IIS
HTTP Proxy
HTTP
MBX
Site Boundary
IIS
Site Boundary
CAS
Load Balancer
HTTP
MBX
Protocol Head
Protocol Head
Protocol Head
DB
DB
DB
Local Proxy Request
OWA Cross-Site Redirect Request
Cross-Site Proxy Request

20. Trade-Offs
Who’s it for?
Exchange Load Balancing Options
Generalist IT admin
Those with increased
network flexibility
Those who want to
maximize server
availability
Functionality
Simplicity
+ Simple, fast, no affinity LB
+ Single, unified namespace
+ Minimal networking skillset
+ Per protocol availability
+ Single, unified namespace
- One namespace per protocol
- Per Server Availability
+ Simple, fast, no affinity LB
+ Per protocol availability
- SSL termination @ LB
- Requires increase networking
skillset

21. A Single Common Namespace Example
Geographical DNS Solution
Sue
(somewhere in NA)
mail.contoso.com
DNS Resolution
Round-Robin between # of VIPs
VIP #1
DAG
Sue
DNS Resolution via Geo-DNS
Round-Robin between # of VIPs
VIP #2
VIP #3
DAG
(traveling
in APAC)
VIP #4

22. CAS2013 Client Protocol Benefits
•
Simplifies the network layer
•
Deployment flexibility
•
Simplifies upgrade and inter-op
– No longer requires session affinity at the load balancer
– Just get the traffic to CAS2013 and let it handle the affinity
– CAS2013 can be “farther away” from MBX2013 and still offer good client
performance (because it is a 1:1 proxy)
– Removes the need for RPC Client Access arrays
– CAS2013 provides more deployment flexibility; for example, consolidate to
fewer sites
– Can deploy a single world-wide namespace
– Designed to proxy to multiple Mailbox server versions, up and down level
– DAGs can be replaced with E2013 at any desired pace

23. The Mailbox Server Role

24. The Mailbox Server Role
•
•
•
A server that hosts all the components that process,
render and store the data
Clients do not connect directly to MBX2013 servers;
connectivity is through CAS2013
Evolution of E2010 DAG
– Collection of servers that form a HA unit
– Databases are replicated between servers in a given
DAG
– Servers can be in different locations, for site resiliency
– Maximum of 16 Mailbox servers
– 50 database copies / server
MBX1
MBX2
MBX16

25. The New Store Process
• Store is effectively made up of three processes
– Replication service
– Store service process/controller
– Store worker process
• Replication service initiates failovers and is responsible for issuing
mount/dismount operations
• Store service process/controller manages the store worker processes
• Each database has its own Store worker process

26. Exchange IOPS Trend
DB IOPS/Mailbox
1
+99%
0.8
0.6
reduction!
0.4
0.2
0
Exchange 2003
Exchange 2007
Exchange 2010
Exchange 2013

27. Large Mailboxes for the win!
 Large Mailbox Size 100GB+
– Aggregate Mailbox = Primary Mailbox +
Archive Mailbox + Recoverable Items
– 1-2 years of mail (minimum)
 Increased knowledge worker
productivity
 Eliminate or reduce PST reliance
 Eliminate or reduce third-party
archive solutions
 Outlook 2013 allows you to control
OST size!
– Gives more options around mailbox
deployments
1 Day
150
11 MB
1 Month
3300
242 MB
1 Year
39000
2.8 GB
2 Years
78000
5.6 GB
4 Years
156000
11.2 GB

28. New Exchange Search Infrastructure
• Leverages Search Foundation
– Common, actively developed search platform used across Office server
products
– Does consume more memory (1/6 available memory) to improve query
performance
• Provides
– Significantly improved query performance compared to E2010
– Significantly improved indexing performance compared to E2010
• Feature parity with E2010 search
• Leverages the same cmdlets like Get-MailboxDatabaseCopyStatus

29. Exchange Indexing
Reduced Processing of Body and Attachments
MBX2013
MBX2013
Transport
Transport
Content Transformation
Service
Mailbox
Store
Log
DB
Mailbox
Local Delivery
ExSearch
Reliable
Event
CTS
Index Node
Read
Content
Passive
Log
Idx
DB
Idx

30. Public Folders
Dawn of a New Age
Architectural bet
Public folders are based on the mailbox architecture
Details
Private
logon
Public Logon
Public
logon
CAS 2013
Hierarchy Mailbox
MBX2013
Content Mailbox
MBX2013
MBX2013
• Hierarchy is stored in PF mailboxes (one writeable)
• Content can be broken up and placed in multiple
mailboxes
• The hierarchy folder points to the target content mailbox
• Uses same HA mechanism as mailboxes
• No separate replication mechanism
• Single-master model
• Similar administrative features to current PFs (setting
quota, expiry, etc.)
• No end-user changes (looks just like today’s PFs)
Not all public folder usage scenarios
are best served by public folders

31. Transport Architecture

32. Transport Components on Client Access
Front-End Transport Service
• Handles all inbound and outbound external SMTP traffic for the
organization, as well as client endpoint for SMTP traffic
– Does not replace the Edge Transport Server Role
• Functions as a layer 7 proxy and has full access to protocol
conversation
• Will not queue mail locally and will be completely stateless
• All outbound traffic appears to come from the CAS2013
• Listens on TCP25 and TCP587 (two receive connectors)

33. Processing Inbound Messages
Externa
l Server
CAS
MBX
1. New SMTP Connection
2. CAS performs envelope filtering
3. CAS determines route to best MBX
server
4. Message delivery begins
1. If successful, CAS returns 250 OK
acknowledgement to external
server
2. If unsuccessful, CAS returns 421
response

34. Benefits of SMTP Front-End Service
• The SMTP Front-End Service provides:
– Protocol level filtering – performs connection, recipient, sender and
protocol filtering
– Network protection – centralized, load balanced egress/ingress point for
the organization
– Mailbox locator – avoids unnecessary hops by determining the best
MBX2013 to deliver the message
– Load balanced solution for client/application SMTP submissions
• Scales based on number of connections – just add more servers

35. Transport Components on Mailbox
•
Transport in MBX2013 has been broken down into three components
– Transport Service - Stateful and handles SMTP mail flow for the organization
and performs content inspection (Was previously referred to as “Hub
Transport”)
– Mailbox Transport Delivery Service - Receives mail from the Transport service
and deliveries to the Mailbox Database
– Mailbox Transport Submission Service - Takes mail from the Mailbox Databases
and submits to the Transport service
•
•
Mailbox Transport is stateless and does not have a persistent storage
mechanism
Mailbox Transport performs content conversion

36. Transport Components on Mailbox
Responsibilities
• Receives all inbound mail to the organization (Proxied through CAS
or direct)
• Submits all outbound mail from the organization (Proxied through
CAS or direct)
• Handles all internal message processing such as Transport Rules,
Content Filtering, and Anti-Virus
• Performs mail flow routing
• Queue messages
• Supports SMTP extensibility

37. Routing Optimizations
•
Next hop selection is broken down into distinct delivery groups:
–
–
–
–
–
•
•
•
Routable DAG
Mailbox Delivery Group
Connector Source Servers
AD Site (Hub Sites; Edge Subscriptions)
Server list (DG expansion servers)
Queuing is per delivery group, connector, or mailbox
Once message is received at final destination, Transport will deliver the
message via SMTP to Mailbox Transport on the server hosting the active
database copy
Send/Delivery-Agent Connectors can have source servers from multiple
DAGs or AD Sites, and can be proxied through CAS

38. Mail Delivery
CAS /
MBX
DAG
MBX-1
SMTP
MBX-2
Transport
SMTP
Mailbox
Transport
SMTP
Transport
Mailbox
Transport
MAPI
DB1
DB2
DB1
DB2

39. Service Availability
Improvements

40. Service Availability Improvements
• Best copy selection now includes health of services when
selecting best copy
• Failover time reductions
• Lagged copy enhancements
• Simpler site resilience strategy
• Managed Availability

41. Managed Availability
•
•
•
Monitoring and recovery
infrastructure is integrated with
Exchange’s high availability
solution
Detects and recovers from
problems as they occur and are
discovered
Is user focused – if you can’t
measure it, you cannot monitor it

42. Managed Availability
Managed Availability + Retries…“stuff breaks and the Experience does not”
LB
CAS-1
DAG
MBX-1
OWA
DB1
DB2
DB1
DB2
DB1
DB2
MBX-2
CAS-2
OWA
MBX-3
OWA
—OWA send
—OWA failure
—OWA failure detected
—OWA restart service
—OWA restart complete
—OWA verified as healthy
—OWA send
—OWA failure
—OWA failure detected
—OWA restart service
—OWA restart service failed
—Failover server’s databases
—OWA service restarts
—OWA verified as healthy
—Server becomes “good”
failover target (again)

43. Transport High Availability Improvements
• Every message is redundantly persisted before its receipt is
acknowledged to the sender
• Delivered messages are kept redundant in transport similar to
active messages
• Every DAG represents a transport HA boundary and owns its
HA implementation
– If you have a stretched DAG, you also have transport site resilience
• Resubmits due to transport DB loss or MDB *over are fully
automatic and do not require any manual involvement

44. Transport HA
CAS2013 or
MBX2013
SMTP
250 OK
R1, R2, R3
250 OK
Transport
Transport
1. Maintain a copy of the message in the queue
database but don’t acknowledge the DATA verb
2. Generate a shadow copy on another MBX2013
server in the DAG (remote site preferred)
3. Wait for acknowledgement from the shadow server
4. Send acknowledgement to SMTP client
5. Delete message from queue after SafetyNet
threshold has expired
Transport
Transport
R1
R1,R3 R3
R2,
R2
Mail.que
Mail.que
MBX Transport
Mail.que
MBX Transport
Mail.que
MBX Transport
MBX Transport
250 OK
Store
DB
2
Log
MBX1
Transport
R3
DB
1
Log
DB
2
Mail.que
MBX Transport
MBX5
DB
1
Log
DB
2
Log
DB
4
MBX6
MBX3
Store
DB
1
Log
DB
2
MBX Transport
Store
Log
DB
4
MBX4
Mail.que
MBX Transport
DB
3
Log
Transport
Mail.que
Store
DB
3
Log
Transport
MBX Transport
Store
DB
4
250 OK
Transport
Mail.que
DB
3
MBX2
Store
Site Boundary
DB
1
Store
MBX7
Store
DB
3
Log
DB
4
MBX8

45. –
–
–
–
–
Facilitates deployments at all scales – from self-hosted
small organizations to Office 365
Provides more flexibility in namespace management
All core Exchange functionality for a given mailbox is
served by the MBX2013 server where that mailbox’s
database is currently activated
Simplifies the network layer
Transport protection is built-in
–
–
All components in a given server upgraded together
No need to juggle with CAS <-> MBX versions
separately
–
–
–
Utilize CPU core increase, cheaper RAM
Utilize capacity effectively
Fewer disks/server => simpler server SKUs

46. Thank you!