2. ANALYSIS SERVICES: TODAY
Broad adoption
“Customers in the Magic Quadrant survey report that their Microsoft
average deployment sizes are now larger than any other vendor in
the survey in terms of users.”
“Use of OLAP functionality by Microsoft customers is more than
double that for the rest of the survey respondents.”
Large ecosystem
"Wide availability of skills is among the top reasons customers select Microsoft over competing
vendors.”
Highest rated infrastructure and development tools
“Microsoft customers rate its BI platform infrastructure and development tools among the
highest compared to other vendors, and a higher percentage of customers use them extensively.”
Source: Gartner Magic Quadrant for BI Platforms, 2011
3. ANALYSIS SERVICES: TOMORROW
Goal #1 Goal #2
Extend the reach of BI
Continue to provide
tools to a broader base
best-in-class tools for BI
of IT professionals and
specialists
developers
SQL Server
Analysis
Services
BI Semantic
Future
Goal #3
Model
Roadmap
Goal #4
Support the full
Provide a single model
spectrum of BI
for creating BI solutions
solutions, including
that is transparent to
personal, team, and
client tools
corporate contexts
4. BI SEMANTIC MODEL
Single model … multiple ways to
for BI … build it
Client Tools
Analysis, Reports, Scorecards,
Dashboards, Custom Apps
BI Semantic Model
Data model
Business logic Personal BI Team BI Corporate BI
and queries Created by user Created by user or IT Created by IT
Individual context Team context Organizational context
Exists as document Managed on server Actively managed on server
Data access
Data Sources
Databases, LOB Applications, Odata Feeds,
Spreadsheets, Text Files
5. BI SEMANTIC MODEL: BENEFITS
Flexibility Richness Scalability
o Tabular and multidimensional o Serves entire range of BI o VertiPaq in-memory engine
modeling solutions
o State-of-the-art compression
o Cached or passthrough storage o Rich modeling capabilities algorithms
o VertiPaq for o Complex business logic o Scales to largest enterprise
performance, MOLAP for scale servers
o Fine-grained security
o Choice of end-user BI tools o Improved SharePoint
configuration and performance
6. BI SEMANTIC MODEL
What about existing Analysis Services applications?
Existing Existing New
applications applications applications
Based on Unified Automatically converted New technology options
Dimensional Model to BI Semantic Model
2012
7. BI SEMANTIC MODEL: ARCHITECTURE
Third-party Reporting
applications Services Excel PowerPivot SharePoint
Power
View
BI Semantic Model
Multi-
Data model dimensional
Tabular
Business logic
MDX DAX
and queries
Direct
Data access ROLAP MOLAP VertiPaq
Query
Databases LOB Applications Files OData Feeds SQL Azure
8. BI SEMANTIC MODEL
Reporting BI Development
Services Studio
Power
View
Model
Client Tool
model and queries Developer
Model, business
logic, and data
BI Semantic Model access
Data model Tabular Tabular
Business logic
DAX DAX
and queries
Data access VertiPaq
SQL Server Microsoft Dynamics CRM
9. BI SEMANTIC MODEL: SCENARIOS
Power View over a Sales model
Reporting BI Development
Services Studio
Power
View
Model, busin
Model and ess logic, and
queries data access
BI Semantic Model
Data model Tabular Tabular
Business logic
DAX DAX
and queries
Data access VertiPaq
SQL Server Microsoft Dynamics CRM
10. BI SEMANTIC MODEL: SCENARIOS
Excel over a Sales model
BI Development
Excel
Studio
Model,
Model and business
queries logic, and
data access
BI Semantic Model
Multi-
Data model dimensional
Tabular
Business logic
MDX DAX
and queries
Data access VertiPaq
SQL Server Microsoft Dynamics CRM
11. BI SEMANTIC MODEL: SCENARIOS
Excel over a Finance model
BI Development
Excel
Studio
Model, busin
Model and ess logic, and
queries data access
BI Semantic Model
Multi- Multi-
Data model dimensional dimensional
Business logic
MDX MDX
and queries
Data access MOLAP
Oracle SAP
12. BI SEMANTIC MODEL: SCENARIOS
Power View over a Finance model
Reporting BI Development
Services Studio
Power
View
Model, busin
Model and ess logic, and
queries data access
BI Semantic Model
Multi-
Data model Tabular
dimensional
Business logic
DAX MDX
and queries
Data access MOLAP
Oracle SAP
13. SUMMARY
Single model for users, multiple ways of building solutions
Flexibility Richness Scalability
Personal BI Team BI Corporate BI
Created by user Created by user or IT Created by IT
Individual context Team context Organizational context
Exists as document Managed on server Actively managed on server
14. UDM IMPROVEMENTS AND ROADMAP
Industry leading OLAP engine
• Large developer/partner ecosystem
• Broad adoption from small businesses to large enterprises
SQL Server 2012 addresses top pain points
• 4GB string store limit
• XEvents and monitoring enhancements
• Performance, scale, reliability improvements
Roadmap
• Will continue to make measured investments in UDM and
MOLAP technology based on customer & partner feedback
• Primary focus going forward will be on BISM and VertiPaq
15. BISM AND UDM COEXISTENCE
BISM does not replace UDM
• Use UDM for complex OLAP applications (budgeting, forecasting, write back,
complex calculations)
• For everything else, BISM offers a simpler and high performance alternative
Existing BI solutions
• Stick with UDM…
• Consider BISM if you’re planning a major solution upgrade
• Migration from UDM to BISM will require some redesign
BISM and UDM are available side-by-side
• Instance level option
16. HOW SHOULD I BUILD MY MODEL?
Depends on the application needs for each layer
• Data model
• Business logic
• Data access & storage
Two Visual Studio (SQL Server Data Tools) project
types in SQL Server 2012
• Multidimensional project – with MDX and MOLAP/ROLAP
• Tabular project – with DAX and VertiPaq/DirectQuery
Project types could change post-SQL Server 2012
• VertiPaq in multidimensional projects, MDX scripts in tabular
projects…
• Based on customer feedback
17. DATA MODEL
Relational Multidimensional
− Familiar model, easier to − Sophisticated model, higher
build, faster time to learning curve
solution
− Advanced concepts baked
− Advanced concepts (parent- into the model and
child, many-to-many) not optimized (parent-
available natively in the child, many-to-
model… need calculations many, attribute
to simulate these relationships, key vs.
name, etc.)
− Easy to wrap a model over
a raw database or − Ideally suited for financial
warehouse for reporting & apps
analytics (planning, budgeting, forec
asting) that need the power
of the multidimensional
model
18. BUSINESS LOGIC
DAX MDX
− Based on Excel formulas − Based on understanding of
and relational concepts – multidimensional concepts
easy to get started – higher initial learning
curve
− Complex solutions require
steeper learning curve – − Complex solutions require
row/filter context, Calculate, steeper learning curve –
etc. CurrentMember, overwrite
semantics, etc.
− Calculated columns enable
new scenarios, however no − Ideally suited for apps that
named sets or calc need the power of
members multidimensional
calculations – scopes,
assignments, calc members
19. DATA ACCESS AND STORAGE
VertiPaq MOLAP
− In-memory column store… typical − Disk based store… typical 3x
10x compression compression
− Brute force memory scans… high − Disk scans with in-memory
performance by default… no subcube caching… aggregation
tuning required tuning required
− Basic paging support… data − Extensive paging support… data
volume mostly limited to physical volumes can scale to multiple
memory terabytes
DirectQuery ROLAP
− Clean pass-through of DAX queries − Acceptable pass-through of fact
& calculations… fully exploits table requests (except for distinct
backend database capabilities count )… unusable for dimension
− No support for MDX queries… no tables
support for data sources other − Supports all relational data
than SQL Server sources… no support for
aggregations
20. DESIGN TOOLS
PowerPivot BI Development
for Excel Studio
Feels like Visual
Feels like Studio
One file,
Excel Optimized
Save to It’s a project
SharePoint for BI Pros
Rapid (business
response to case, budget, dat Larger data
business es) volumes
problems Teams building BI
Solutions live
solutions Deployment
Optimized for for weeks or
scripts, versions
Excel power months
user Source
Control, TFS
Personal Team Corporate
21. NEW FEATURES LIST
• Richer Models • Optimized Usability
• KPIs • Improved Date and Text
• Descriptions filtering
• Persisted formatting • Diagram
• Advanced sorting • Measure grid
• Mark as Date Table • Various usability
enhancements
• Distinct count
• Drillthru
• Perspectives
Reporting Properties
• Hierarchies
• Multiple relationships
• Parent child
22. KEY HIGHLIGHTS
•Professional Tools
• SQL Server Data Tools in Visual Studio
2010
• Multi-dimensional and tabular projects
• Management Studio updates
• Powershell support
28. BI SEMANTIC MODEL
Flexibility
Familiar Interface
o Excel for business users, Visual Studio for BI pros
o Reduced time to deliver solution
o Experiences scale from simple apps to enterprise BI solutions
Interactive and Iterative Design
o Intuitive data-driven experience allows faster iteration in BI application
design
o Source control and seamless deployment to dev/test/prod through
Visual Studio
Sharing and Collaboration
o Business users can share by publishing to SharePoint with a single click
o BI pros can use team development features in VS
o BI apps are auto refreshed/maintained on the server
29. BI SEMANTIC MODEL
Richness
Rich Modeling Capabilities
o Multiple relationships, hierarchies
o Many-to-many, parent-child relationships
o Key performance indicators, drill-through, perspectives
o Rich data types, BLOBs, images
Sophisticated Business Logic
o Data Analysis Expressions (DAX), Excel formulas, MDX
o Relational operators (Filter, Aggregate, GroupBy, Lookup)
o Statistical, time intelligence (YTD, QTD) functions
o Rank, TopN, VisualTotals, DistinctCount, etc.
Fine-Grained Security
o Role-based security model using Active Directory
o Row-level security
30. BI SEMANTIC MODEL
Scalability
High Performance
o VertiPaq engine – in memory, column oriented store
o High performance via brute force memory scans
o No tuning, indexes, aggregates required
Optimized for Latest Hardware
o VertiPaq is optimized for latest x86 and x64 chipsets
o Designed to exploit cheap memory on latest server h/w
o Inherently multi-threaded and scales linearly with number of cores
Enterprise Scale
o Scales from desktops to highest end servers
o State-of-the-art compression algorithms reduce data volumes by 10x
or more
o Partitioning & paging to support large models
31. Analysis Services Architecture
Internet Explorer SharePoint
BI Development Studio
Power View Project Juneau
Excel Services
Reporting Services
PowerPivot for
xlsx
Excel
Analysis Services
PowerPivot for
SharePoint
(Analysis Services)
Excel
BI Semantic Model
xlsx
Third Party Apps
Personal BI Team BI Corporate BI
32. BI SEMANTIC MODEL: OPTIONS
Flexibility in how to build solutions
Scalability
Workload fits
primarily
in memory
Static
Reporting
Adhoc
Reporting & Analysis
Advanced
Calcs & Modeling
Richness
Notes de l'éditeur
Analysis Services is a very mature product.Investment in Microsoft SQL Server Analysis Services technology is a good bet.Microsoft leads in:Broad adoption from small businesses to large enterprises; Analysis Services is the leading OLAP engine out in the market.Ecosystem of developers & partners, BI tools & solutions including client tools built on top of it.Integration with the Microsoft stack (SSRS, Excel, SharePoint BI). Increasingly, we see Analysis Services being used to solve complex business problems.
We wanted to take a look at a multi-year / multi-release vision for Analysis Services Here we show the future “roadmap” of Analysis Services, from SQL Server 2012 and beyond.We want to build on the strengths and success of Analysis Services including the current ecosystem of developers & partners, BI tools & solutions. The current base of BI-focused developers and IT pros are proficient with multidimensional BI solutions; we’ll continue to provide best-in-class tools for BI specialists.We’ll expand the reach of BI to include the broader base of developers and IT pros in the Microsoft ecosystem. To do that, we need to embrace the relational data model for BI, which is well understood by developers and IT pros.At the same time, we want to carry forward existing capabilities in OLAP, so we want to bring together relational and multidimensional modeling under one BI platform. Analysis Services will provide a single model for creating BI solutions that is transparent to client tools; the goal is to extend analysis services to include a broader base of users. We want to provide flexibility in the platform to suit the diverse needs of BI applications. Our vision for this is the BI Semantic Model.
What is the BI Semantic model? One model that serves all the end user experiences for Microsoft BI. It’s a model that integrates data from a variety of data sources whether traditional, such as line-of-business applications, database systems, or nontraditional sources such as web services, Odata feeds, text files, spreadsheets, we want to be able to integrate data from a variety of sources, enrich, accelerate access to it, and serve it up using a data model experience that is appropriate for a variety of client tools, whether it’s tools that allow the user to do data visualization, interactive analysis, reporting, scorecarding, dashboarding, or a custom application. We want the model to light up those user experiences no matter how the model is built.The semantic model is also the technology that powers the entire spectrum of BI applications that you can build using the Microsoft BI stack.
This slide summarizes the benefits of the BI Semantic Model.Flexibility – to enable BI application developers to do the right thing for the needs of the application; flexibility in terms of client applications that can consume the model; flexibility in the choice of client tools that end users can use; and flexibility in terms of how you manage the data – cache, pass-through, etc.Richness – contains rich modeling constructs; has sophisticated calculation languages; fine-grained securityScalability – Performance and scale, as shown in Vertipaq (blazing-fast performance with very minimal tuning) and MOLAP (can scale to very large workloads into terabytes).
If you are concerned about the impact of the BI Semantic Model, let’s put your fears to rest, and talk about what will be possiblein the future.What happens to existing applications built on Analysis Services?The answer is very simple. When you upgrade your Analysis Services server, or your Visual Studio projects to SQL Server 2012, every existing Analysis Services cube or UDM becomes a BI semantic model. This isn’t an upgrade or migration process. Every UDM becomes a BI semantic model.The reason: the BI semantic model encompasses all of the capabilities of what the UDM offers today, and a lot more. It offers:all of the existing modeling experiences that the UDM offersall existing interfaces that the UDM offersa bunch of other experience and new interfacesThe BI semantic model is really an evolution of what UDM was. It’s a hybrid model. It supports both tabular and multidimensional modeling both for the model developer as well as the model consumer. Your current UDM projects will continue to work in the future—no change! They will automatically work with the BI Semantic Model. New projects in the future will be able to take advantage of different technologies.The BI Semantic Model decouples the model from the clients that use it. The model will be transparent to users. Because the BI semantic model takes full advantage of the capabilities present in UDM, it also carries forward all the rich client tools, and the rich ecosystem of third-party tools built on top of Analysis Services, they get carried forward too.
Let’s take a look at the conceptual architecture. At the bottom you have data sources – a variety.The BI semantic model enriches and accelerates the data and provides a model that gets exposed to all of the tools in the Microsoft BI stack, whether it’s reporting services, including Power View, Excel, SharePoint, Scorecards, PowerPivot.The model can be thought of conceptually as three layers: data model, business logic and query layer, and data access layer.The data model is the one used by the model designer/builder and the data consumer. The model developer uses Visual Studio and SQL Server Data Tools; the client tool uses the the Interfaces exposed by the model.As far as business logic and queries are concerned, the BI Semantic Model uses MDX and DAX. DAX is an expression language introduced in PowerPivot based on Excel formulas. It’s built on top of tabular concepts, basically tables and relationships, so it really lowers the threshold of entry into building sophisticated business logic into your models. In terms of data access, the model gives you the option of caching the data or having it pass through. You’re already familiar with MOLAP and ROLAP. In SQL Server 2012, we’re offering Vertipaq as a new option for caching data in the model. Vertipaq is a new in-memory store engine. It caches the data in memory, organizes it by column and uses state-of-the-art compression techniques to compress and manage the data in memory and operate on it in a compressed state and it produces blazing fast performance to BI-type queries. Vertipaq does not require any indexing or aggregation; it just does brute-force scanning of data in memory. Then we have Direct Query, which is a new clean pass-through data access we’re introducing in SQL Server 2012.
To elaborate on the hybrid nature of the BI Semantic model, I want to take a few scenarios and walk through them.
Here we have a sales model that has been developed by a BI professional.The little ellipsoid thing that looks like the infinity symbol stands for Visual Studio. All the services: Analysis Services, Reporting Services run inside the Visual Studio 2010 shell. The model developer has used Visual Studio to build a BI semantic model using the tabular experience. The model developer has enriched the model using DAX. And the model happens to be cached in the Vertipaq column store. Let’s say an end user connects to this model using Power View, which is an ad-hoc reporting tool which consumes the model using the tabular interface and sends DAX queries to the model.That’s the picture you see of a specific use case.
The exact same model shown previously (Power View over a Sales model) can be consumed by Excel.In this case, there’s nothing different about what the model developer did. On the other hand, the user sees a multidimensional view of the model. Excel is the client, consuming the model as if it were a cube, as if it were a multidimensional model. And it sends MDX queries to the model.So now you can see the hybrid nature of the model.
Let’s say the model developer has used the classic multidimensional modeling experience.The developer used MDX scripts to enrich the model with calculated members, scopes and assignments, and so forth, and data is cached in the MOLAP engine. Of course, Excel users point to it and consume a multidimensional view of the model and send MDX queries.
The same model shown previously (Excel over a Finance model) can be consumed by using Power View even though it was built using the multidimensional experience; the user can consume it using the tabular interface using DAX queries.The point: it’s truly a hybrid model. The model developer chooses to build it the way that works best for them, and the model consumer uses the option that’s best for the consumption experience.
The BI Semantic model vision is about:Flexibility – to enable BI application developers to do the right thing for the needs of the application; flexibility in terms of client applications that can consume the model; flexibility in the choice of client tools that end users can use; and flexibility in terms of how you manage the data – cache, pass-through, etc.Richness – contains rich modeling constructs; has sophisticated calculation languages; fine-grained securityScalability – Performance and scale, as shown in Vertipaq (blazing-fast performance with very minimal tuning) and MOLAP (can scale to very large workloads into terabytes).
Purpose of the Slide: Show that UDM is the most important part of our current offerings and continues to be supported.Key Points: Continuing to make enhancements to supports large and complex OLAP solutions.No immediate need to change existing solutionsNew solutions can use BISM and VertiPaqContinued leadership in top OLAP engineSQL Server 2012 has OLAP improvements
Purpose of the Slide: UDM and BISM can coexistKey Points: Follow slide talking pointsBISM is preferred solution for new development or for major revisions. Key benefits already covered earlier in presentation (simple, rich, scalable)UDM is preferred for pure OLAP solutions (planning, data mining, large and complex proejcts)Can work together side by sideExisting solutions continue to be supported.
You may be asking: How do I choose which experience and model are right for me?We’ll provide additional detailed guidance in the near future, but here is an overview. Think of the needs of your applications along the lines of the three layers that constitute the BI semantic model. Data modelBusiness logicData access and storageIn the SQL Server 2012 release, we have two projects, two experiences for building the model for model developers. And you have a multidimensional project experience and a tabular project experience. The multidimensional project allows you to use MDX, MDX scripts, with MOLAP and ROLAP storage options. The tabular experience lets you use DAX and VertiPaq/Direct Query options. These are two options we happen to offer in SQL Server 2012 (tabular is the new option, available in SQL Server 2012). The guiding principle is that customers already enjoy the multidimensional experience; we wanted to reduce the barrier to entry and provide the opportunity to create solutions really quickly and that’s what the tabular experience offers.There’s nothing in the model that limits us in terms of these restrictions. We’ve had conversations with customers where they say “I have this large MOLAP cube and I could really use that Vertipaq engine and stop doing all that aggregation designing I’m having to do.” That’s certainly something we could consider for a future release. If you’ve built a model using a tabular experience, and then you run into certain restrictions that DAX has, maybe you want to use MDX scripts, calculated members or scopes and assignments, to enrich the model, that’s certainly something we could consider adding to the modeling experiences. The point is this: there is the model and there are these Visual Studio experiences that you use to build the model. What we have in SQL Server 2012 is the first step in realizing the BI semantic model vision and it can and will evolve. Regardless of how the model building experience itself evolves, the change is seamless to client tools. Client tools connect to the model regardless, using a set of interfaces you’re already used to.
How should I build my model? Here are design considerations for the data model.For tabular/relational model – most people are familiar with it, and there is a fast time to solution with it. It’s pretty simple; advanced concepts such as many-to-many relationships, reference relationships, parent-child, they’re not concepts that are natively baked into the model. The model certainly lets you do that using calculations. If you wanted to wrap a model on top of an existing data warehouse, for example, and surface it to your end users, maybe all you want to do is clean up the column names, hide a few columns, do some joins, the tabular model is simple for that.Multidimensional – there is a higher up-front learning curve. Advanced concepts are baked into the model. Once your understanding of the model grows, you can start taking advantage of those modeling constructs. This model excels at classic OLAP applications – planning, budgeting, forecasting. There are lots of applications that need the power of the multidimensional model, the power of MDX scripting. These are the two ends of the spectrum. Most BI applications fall somewhere in between.
How should I build my model? Here are design considerations for business logic.DAX – is based on Excel formulas and relational concepts, the concept of scanning tables, looking up values from another table, relatively straightforward to get started with. It does have a learning curve once you get into more complex solutions. DAX was designed with the goal to lower the barrier to entry into building complex BI applications. Our goal with it is to keep simple problems simple. Has calculated columns, which enables a new scenario that doesn’t exist in the multidimensional experience. However, we don’t have named sets or calc members in DAX, and that may be a requirement for some BI applications, so MDX may be right for you. MDX – requires an up-front learning curve that gets steeper the more complex things get. MDX is more powerful than DAX.
How should you build your model? Here, we compare and contrast the factors related to data access and storage.Comparing VertiPaq and MOLAP…Vertipaq is an in-column store. The compression algorithms in Vertipaq are state-of-the-art; we typically see 10X compression although lower and higher levels are possible. We recommend you have as much memory on your servers as required to load the model in memory because that’s where Vertipaq excels. You don’t want to have Vertipaq start to page data in and out of discs because that’s not what it was optimized for. MOLAP on the other hand is a compressed row store for the most part. MOLAP does a little bit of column store-type implementation for dimensions but fact tables for which MOLAP was designed is still a row store. 3X compression is rule-of-thumb. MOLAP is a disc-based storage system so it scales to models that don’t fit in memory. Example: Yahoo cube which runs into tens of terabytes. MOLAP is optimized to page data between disc and memory. You want to make sure when the system has to touch the disc that it touches as little of the disc as possible, and that’s where aggregations and indexes come in. MOLAP takes care of indexes automatically, but aggregations are something that you as IT admins and pro’s need to actively manage.DirectQuery and ROLAP…DirectQuery is the complement to VertiPaq. It pushes down DAX calculations into the back-end. Currently there is no support for MDX queries, so directquery is in an early implementation that we won’t have time to completely round out in this release. We expect to broaden this, post-SQL Server 2012. For the basic scenario of doing ad-hoc reporting using Power View over a datamart or data warehouse, directquery makes a lot of sense. ROLAP is great for fact tables but not recommended for dimension tables.
This is a high-level view of how the BI semantic model flows through to PowerPivot to SharePoint, to Analysis Services. This shows that this is the same model, one model, in a workbook and it flows through to SharePoint and you can also have the same model in Analysis Services published from Visual Studio.You have end user tools that live on the user’s desktop or you have services such as Excel services and reporting services that can access the model.
