CRM User Group Belux 03/2017 - Visualizing and analyzing Dynamics 365 data with Power BI

1. Dynamics 365 and Power BI
Joris Poelmans
Solution Architect – Realdolmen
@jopxtwits http://jopx.blogspot.com

2. Why?
“Without data,
you’re just another
person with an
opinion”
W. Edwards Deming,
Data Scientist

3. Demo time

5. Data acquisition
Connect to many
data sources
Data
Transformation
Combine
multiple queries
into single model
Refreshable

6. Data acquisition – supported data sources
 Use Query functionality to connect to data from a variety of sources beyond those available
from powerbi.com
 Navigate data quickly with in-memory processing that optimizes performance
 Pare down datasets to only what’s needed by editing queries before data loading
 Ability to automatically find related tables when connecting to database sources
File Database Azure Other
• Excel
• CSV
• XML
• Text
• Folder
• SQL Server
• Direct Query for SQL
Server
• Access
• SQL Server Analysis
Services
• Oracle
• IBM DB2
• MySQL
• PostgreSQL
• Sybase
• Teradata
• SQL Database
• Direct Query for SQL
Database
• SQL Data Warehouse
• Marketplace
• HDInsight
• Blob Storage
• Table Storage
• HDInsight Spark
• DocumentDB
• Web
• SharePoint List
• Odata Feed
• Hadoop File (HDFS)
• Active Directory
• Microsoft Exchange
• Dynamics CRM Online
• Facebook
• Google Analytics
• Salesforce Objects
• Salesforce Reports
• ODBC
• appFigures
• GitHub
• QuickBooks Online
• SweetIQ
• Twilio
• Zendesk
• Spark
• Blank Query
• Mail Chimp

7. Data acquisition - getting D365 data
• Retrieve data from Dynamics 365 by accessing the Dynamics 365 Web API
• ODATA RESTful API that gives access to all data stored in Dynamics 365
• https://<baseurl>/api/data/v8.x/
• Slow to retrieve large amounts of data – use filters and pre-selection (or Data Export Service)
• Leverage content packs as alternative

8. Data acquisition – loading D365 data
• Navigator shows a list of all accessible entities
• For every entity selected a query is created
• Each query loads data into table of the data model – use filters to only
retrieve required data

9. Data acquisition - editing a query
• Each query has one or more steps
• Each step can
• Connect to a data source and retrieve data
• Apply a transformation to the data
• Last step in a query returns the final data and
loads it into the table of the data model
Common data-shaping tasks
• Remove rows
• Rename/Remove tables and
columns
• Change a data type
• Delete Relationships
• Pivot columns and group rows
• Modify a table name
• Identify and fix errors
• Merge or append queries to
combine data from multiple
queries into a single query

10. Data acquisition - D365 queries (specifics)
• Fields with “normal” data types are immediately accessible
• Optionsets
• Entity references/lookups: 2 fields _attributename_value & attributename
• Relationships are available as columns of type Table

11. Data acquisition – leveraging content packs
• Organizational Content Packs: possible to package dashboards, reports and
datasets and share them with co-workers in “Content Pack Library”
• Dynamics 365 Content Packs
• Can be downloaded and modified in Power BI Desktop

12. Demo time

13. Data Modeling with formulas and relationships
• Relationships can be managed in
Relationship view
• Automatically detect relationships
• Adjust relationship types (1:N,1:1)
• Define calculations (measures) to generate
new fields for use in reports
• Create Calculated Tables with Data Analysis
Expressions (DAX) formulas
• Use data categorization

14. Building reports
 Explore data in a variety of ways and
across multiple visualizations
o Select data elements and sort data
o Filter data and use cross-filter capabilities
o Drill into and across datasets
o Pivot and slice data
o Change visualization types
o Date sort and filters including Min & Max
functions
o Ability to refresh all views with a single click
o Copy/Paste queries to/from Excel
o Improved default sorting
o Ability to sort the slicers
 Select, transform and mashup data via a
freeform, drag-and-drop canvas

15. Share reports, visualization and data
• Data models and reports created in Power BI Desktop can be published to
Power BI Cloud Service
• Free and Pro edition
• Data model => Dataset, Report => Reports

16. Dynamics 365 integration
• Enable Power BI use through System Settings (Reporting tab)
• Embedded Power BI dashboard tiles
• For Dynamics CRM Online 2016 Update 1 (8.1) and later
• Create personal dashboards that use Power BI Dashboard
• Available in December 2016 update

17. Demo time

18. Understanding Power BI Data refresh
Daily, Weekly
(free)
Up to 8 times
per day (Pro)
Token cached
in service
Direct Query
On-premises
data through
gateway

19. Power BI Data Refresh and Direct Query
Import (cached mode) Direct query or Live connection
Refresh frequency Scheduled - hourly or daily Real-time
Performance No noticeable delay since data is already
cached
Depends on how fast the data source is as
queries are executed in real-time
Data storage in
Power BI
Since it is cached mode, data is stored in
the cloud
No data is stored in Power BI. Data is always on-
premises*
Data size Current limit of 1 GB (compressed) per
model (Free edition) and 10 GB per
licensed Pro user
The on-premises database is the limit; no Power
BI limitation
Security Can create row-level security on the PBI
dataset (import only)
Re-use on-premise row level security for
Analysis Services
*Some data is cached for optimizing first-time load performance

20. Direct Query – Supported data sources
• Cloud data sources
• SQL Azure Data Warehouse
• Spark on HDInsight
• On-premise data sources
• SQL Server Analysis Services (Tabular or Multidimensional)
• SQL Server
• SAP HANA
• Oracle
• Teradata

21. On premises data gateway
Centralized way to refresh on-
premises content in Power BI
Access control to data sources
Monitor and track usage
Live, interactive query with on-
premises data sources

22. Miscellaneous
Power BI
Architecture
Security and
identity
Alerts
Data Export
Service
Licensing

23. Power BI Architecture

24. Security and Data Authorization
• Users have access to dashboards, reports via sharing or
organizational content packs
• Scoped to user accounts, AAD security groups, or O365 modern
groups (content packs only)
• (a) Calls to data sources are made using service-level credentials in
the case of cached and Direct Query sources. Authorization in the
data source is done using the single service credentials.
• (b) for Analysis Services Live Connect, the user’s credentials are
used and user authorization is performed in Analysis Services (and
RLS can be applied).
• Row filters can be applied in the Power BI service for cached data.

25. Alerts
• Run on refresh
• Send e-mails
• Trigger a Microsoft Flow

26. Dynamics 365 Data Export Service
• Replicate data from Dynamics 365 to Azure SQL or SQL
Server on Azure VMs
• Enables more complex reporting or custom integration
scenarios
• Azure components within same subscription
• Data sent to the Data Export Service is stored
temporarily in Azure Service Bus and Azure
Storage, processed in Azure Service Fabric, and
finally synchronized to the destination DB
• Support for Data Export implemented in recent update
of the Dynamics 365 Content Packs

27. Licensing
Free
• 1 GB storage limit
• Daily refresh
• Cloud data sources
• Service content packs
• Streaming – 10,000 rows/hour
Power BI Pro
($9.99/user/mo)
• 10 GB storage limit
• Hourly refresh (sort of)
• Live Data
• On-premises data
• Modern Groups
• Organizational content packs
• Row Level Security
• Streaming – 1,000,000 rows/hour
• Data Catalog
• SharePoint Web Part

28. What we did not cover
• Power BI REST API
• Streaming data
• Custom visualizations
• Cortana integration
• Power BI Embedded and Publish to web
• Power BI Mobile Apps
• Sharing of reports and workgroups
• SharePoint integration
• Compliancy features: Power BI auditing, data classification, etc …
• Power BI Admin Center
• R support
• Power BI reports in Reporting Services Vnext
• …

29. References
• https://powerbi.microsoft.com/en-us/documentation/powerbi-admin-manage-your-data-storage-
in-power-bi/
• Row Level Security with Power BI - https://powerbi.microsoft.com/en-us/documentation/powerbi-
admin-rls/ and http://radacad.com/dynamic-row-level-security-with-power-bi-made-simple
• Power BI Security - https://powerbi.microsoft.com/en-us/documentation/powerbi-admin-power-bi-
security/ (also contains link to the Power BI architecture white paper)
• Power BI Infographics - http://community.powerbi.com/t5/Community-Blog/Power-BI-Infographic-
for-Power-BI-from-an-end-to-end-perspective/ba-p/143307
• Setup the Data Export Service with a Power BI Solution template -
https://crmchartguy.wordpress.com/2017/01/11/setup-the-data-export-service-with-a-power-bi-
solution-template/
• …
See reference list on my blog http://jopx.blogspot.com

30. “THE WAY TO GETSTARTEDIS TO
QUIT TALKING AND BEGIN DOING.”
Walt Disney