4. AllWorks Data Warehouse
A data warehouse model must consider both the available data and reporting
needs to be supported.
4
5. Data Modeling Alternatives
Even a small data warehouse
might have alternate designs
that need to be tested for
support of the needs of SSAS or
SSRS development.
5
7. A Table Value Function
The function returns a product and the price as of a give date.
The function can be used to return the price
IF EXISTS (SELECT * FROM sys.objects for a single item,
WHERE object_id = OBJECT_ID(N'[dbo].[GetStandardCostByDate]')
AND type in (N'FN', N'IF', N'TF', N'FS', N'FT'))
DROP FUNCTION [dbo].[GetStandardCostByDate] DECLARE @ProdNum NVARCHAR(25) = 'BK-R89R-58‘
GO , @CostDate DATETIME = '1/1/2008'
CREATE FUNCTION dbo.GetStandardCostByDate SELECT * FROM dbo.GetStandardCostByDate
(@ProdNum NVARCHAR(25), @CostDate AS DATETIME) (@ProdNum,@CostDate)
RETURNS TABLE
AS
RETURN
SELECT PP.ProductID
, PP.ProductNumber
, PP.Name
, ROUND(PCH.StandardCost,2) AS RoundedStandCost
. . . or it could be used to return a list based
FROM Production.ProductCostHistory PCH on addition selection criteria.
JOIN Production.Product PP
ON PCH.ProductID = PP.ProductID SELECT GET.ProductNumber
WHERE StartDate < DATEADD(DAY,1,@CostDate) , GET.Name
AND PP.ProductNumber = @ProdNum , GET.RoundedStandCost
AND ( PCH.EndDate is null FROM Production.Product PP
OR PCH.EndDate >= @CostDate CROSS APPLY dbo.GetStandardCostByDate (PP.ProductNumber, GETDATE()) GET
) WHERE GET.RoundedStandCost >= 1500
GO ORDER BY GET.RoundedStandCost DESC
Both uses are potentially valuable in .NET
and web based applications.
7
8. A Pivot Table with Dense Rank Query
Pivot tables can be produced in many of the BI layers beginning with SQL.
This example provides output that is useful in reporting and applications.
;WITH SaleByShipper AS
( SELECT DATEADD(DAY, 7 - DATEPART(WEEKDAY,OrderDate),OrderDate) AS WeekEnding
, ShipMethodID
, SUM(TotalDue) AS ShipperTotal
FROM Purchasing.PurchaseOrderHeader POH
WHERE YEAR(DATEADD(DAY, 7 - DATEPART(WEEKDAY,OrderDate),OrderDate)) = 2007
GROUP BY DATEADD(DAY, 7 - DATEPART(WEEKDAY,OrderDate),OrderDate)
, ShipMethodID
)
SELECT WeekEnding
, [1] AS XRQ
, [2] AS ZY
, [3] AS OVERSEAS
, [4] AS OVERNIGHT
, [5] AS CARGO
INTO #Pivot
FROM SaleByShipper
PIVOT ( SUM(ShipperTotal) FOR ShipMethodID IN ([1], [2], [3], [4], [5])) AS ShipDol
SELECT TOP 5 REPLACE(CONVERT(CHAR(10),WeekEnding,111),'/','-') AS WeekEnding
, ISNULL(XRQ,0) + ISNULL(ZY,0) + ISNULL(OVERSEAS,0) + ISNULL(OVERNIGHT,0) + ISNULL(CARGO,0) AS GrandTotal
, DENSE_RANK() OVER (ORDER BY ISNULL(XRQ,0) + ISNULL(ZY,0) + ISNULL(OVERSEAS,0) + ISNULL(OVERNIGHT,0) + ISNULL(CARGO,0) DESC) AS [Rank]
, XRQ
, ZY
, OVERSEAS
, OVERNIGHT
, CARGO
FROM #Pivot
8
9. Segments of a Stored Procedure
Producing a dual ranked output that has application uses for reporting.
SELECT * INTO #Vendor
FROM
( SELECT VEN.BusinessEntityID In a stored procedure
, VEN.Name AS VendorName
VendorRank
, DENSE_RANK() OVER (ORDER BY SUM(TotalDue) DESC) AS
a temporary table built using a
FROM
, SUM(POH.TotalDue) AS TotalDue
Purchasing.Vendor VEN
sub-query to build a Vendor
JOIN Purchasing.PurchaseOrderHeader POH Ranking,
ON POH.VendorID = VEN.BusinessEntityID
WHERE POH.OrderDate >= @StartDate
AND POH.OrderDate < DATEADD(DAY,1,@EndDate)
and a similar query on Products
GROUP BY VEN.BusinessEntityID
, VEN.Name are joined in a final query
) vendalias
WHERE VendorRank <= @TopVend to provide a combined ranked
SELECT V.VendorName listing of vendors and products.
, V.VendorRank
, V.TotalDue EXEC dbo.TopProductForTopVendorSales
, P.ProductName @TopVend = 3, @TopProd = 3,
@StartDate = '1/1/2007',
, P.ProductRank
@EndDate = '6/30/2008'
, P.ProductTotalDue GO
FROM #Vendor V
JOIN #Product P
ON P.VendorID = V.BusinessEntityID
ORDER BY V.VendorRank
, P.ProductRank
9
10. S Q L S E R V E R I N T E G R AT I O N S E R V I C E S
(SSIS)
10
11. The AllWorks Construction Company
A series of real world projects are undertaken for a fictional construction firm. After
creation of the database, an SQL Server Integration Services package was created to
process the data.
The first step was the creation of the
relational database using SQL.
This was followed by the development of an
ETL process using SQL Server Integration
Services.
The process was deployed to SQL Server
Agent.
The deployed SSIS process is designed to do
an initial load of the data as well as to be
used for regular, ongoing processing of data.
The source included files from Excel, CSV
and XML.
11
12. Processing Employee Time Records
This process is used for the initial load of time records, but designed for scheduled
processing that can be done daily or at any selected frequency. The process validates
Projects and Employees and produces a single report of invalid records. Records that pass
validation are checked for late time and a separate late time report is produced. Only
records passing all three tests are inserted, and reports are sent by e-mail for invalid
records and late time. Multiple source files are received and processed in a ForEach Loop
in Control Flow.
Data Flow
Control Flow
12
13. Master ETL Control Flow
A Master Control Flow was used to assure that the file processing order was sequenced to
process primary keys before use of the primary key as a foreign key. The process began
with control tables and ended with secondary transaction tables. Once processing is
complete, database maintenance tasks are done in a final step.
13
14. S Q L S E R V E R A N A LY S I S S E R V I C E S
(SSAS)
14
15. Browsing the All Works Cube Data
The AllWorks OLTP database was staged to provide the OLAP cube shown here in the
browser.
One of the features of
the cube is the ability
to click on a county
and open a map
centered on the
county.
15
16. Creating A Staging Area for OLAP Deployment
A copy of the OLTP Database was loaded to AllWorksDW and T-SQL Programs were
used to create and load Fact and Dimension tables.
CREATE FUNCTION [dbo].[WeekEndingDate]
( @InputDate date ) -- 9-20-2010
This query creates a function to find the week
RETURNS Date ending date. A similar function to find the week
AS
BEGIN
ending key was also created.
DECLARE @ReturnDate DATE
SET @ReturnDate =
dateadd(day, ( @@DateFirst - datepart(weekday,@InputDate)), @InputDate)
RETURN @ReturnDate
END
GO
Fact and Dimensions were
;WITH EndDateCTE AS
( created using a combination
SELECT DISTINCT [dbo].[WeekEndingDate](JobClosedDate) AS Dates FROM dbo.JobMaster
UNION
of:
SELECT DISTINCT [dbo].[WeekEndingDate](WorkDate) FROM dbo.JobTimeSheets
UNION Views and
SELECT DISTINCT [dbo].[WeekEndingDate](PurchaseDate)FROM dbo.JobMaterialPurchases
UNION
SELECT [dbo].[WeekEndingDate]('Oct 2, 2004') -- DEFAULT JobCloseDate
Tables created in Stored
) Procedures
INSERT INTO [AllWorksDW].[dbo].[DimDate]
SELECT dbo.WeekEndingKey(Dates)
, CONVERT(VARCHAR(12),Dates, 107)
, CAST(CAST(YEAR(Dates) as varchar(4)) + CAST(DATEPART(qq,Dates) as varchar(1)) AS INT)
, 'Q' + CAST(DATEPART(Q,Dates) as varchar(1)) + '-' + CAST(YEAR(Dates) as varchar(4)) This query was used in a
, YEAR(Dates) Stored Procedure to create
FROM EndDateCTE
the Date Dimension.
WHERE Dates > '12/31/2003'
16
17. Developing A Cube
The AllWorksOLAP cube build included basic structure, dimension usage, calculated
members and sets, KPI’s, Actions, Partitions, Aggregations, and Perspectives.
17
20. Useful Simple MDX Queries
WITH
MEMBER [Labor Hrs Last Year] AS
A query producing a weekly moving average. ( [Worked Hours]
, parallelperiod ( [Dates].[Calendar Tree].[Quarter]
WITH SET [Weeks] AS , 4
FILTER ([Dates].[Calendar Tree].[Weekend], [Overhead Cost] <> null) , [Dates].[Calendar
Tree].CurrentMember
)
MEMBER [MovingXWeekAvg] AS )
AVG (
Comparing this year to
, FORMAT_STRING = 'Standard'
LastPeriods ( 52 last year
, [Dates].[Calendar Tree].CurrentMember
) SELECT [Dates].[Quarter] * { [Worked Hours], [Labor Hrs Last Year] }
on columns,
, ([Overhead Cost]) NON EMPTY [Labor].[Employee Name].Children
) on rows
from [Labor]
SELECT {[Overhead Cost],[MovingXWeekAvg]} ON COLUMNS, WHERE [Dates].[Calendar Tree].[Q4-2005]
[Weeks] ON ROWS
FROM [Overhead]
Some really simply queries produce interesting, useful
results, like this TOPPERCENT query:
SELECT [Invoice Amount] on COLUMNS,
TOPPERCENT ([Projects].[Project].children, 30, [Invoice Amount]) ON ROWS
FROM [Global]
20
21. A More Complex MDX Example
WITH MEMBER [Internet Sales Amount Prior PD] as This much more complex example calculates the
( [Internet Sales Amount] , [Date].[Calendar].PrevMember)
, format_string = ‘currency' percentage increase in sales over the prior month.
MEMBER [% Sales Change] AS It identifies the months having the greatest increase
IIF ( [Internet Sales Amount Prior PD] <> null
, ([Internet Sales Amount] - [Internet Sales Amount Prior PD]) and ranks these.
/ [Internet Sales Amount Prior PD]
, 0) It evaluates cities within each of these months for
, format_string = 'percent'
sales increases and ranks those.
SET [TopMonth] AS
TOPCOUNT ( [Date].[Calendar].[Month], 3, [% Sales Change] ) Finally the query reports the top 3 months and top 3
MEMBER [MonthRank] AS cities within those month and reports
RANK ( [Date].[Calendar].CurrentMember, [TopMonth] )
the amount of sales for the month and prior
SET [Top3MonthsTop3Cities] AS
GENERATE( [TopMonth], ( [Date].[Calendar].CurrentMember, month,
TOPCOUNT([Customer].[City].Children, 3, [% Sales Change]) ) )
the percentage increase,
MEMBER [CityRank] as
RANK( ( [Date].[Calendar].CurrentMember, the ranking of the months, and
[Customer].[City].CurrentMember),
EXISTS( [Top3MonthsTop3Cities], [Date].[Calendar].CurrentMember))
the ranking of the cities within those months.
SELECT
{[Internet Sales Amount] , [Internet Sales Amount Prior PD],
[% Sales Change], [MonthRank], [CityRank] }
ON COLUMNS,
[Top3MonthsTop3Cities]
ON ROWS
FROM [Adventure Works]
21
23. Example Reports Developed in SSRS
The SSRS Project developed reports Promotional Sales: A simple
for deployment in SharePoint. report requiring custom MDX.
Sales by state with an Employee Sales: Actually is a
accompanying chart of the top 10 master report with 4 sub reports.
states by either sales or returns.
23
25. Excel Power Pivot Against Contoso Operation Cube
Product Category is configured to
highlight and preselect valid
subcategories.
Geography is based on a hierarchy from the cube,
so can be expanded to lower levels.
Gridlines and headings are turned of to provide a
cleaner appearance when deployed to SharePoint.
25
27. Performance Point Services
Reports were prepared directly in PPS, and reports
deployed to SharePoint from SSRS and Excel where brought
into SharePoint to be deployed to the Dashboard.
For SSRS reports, the deployment to the Dashboard via PPS
enabled the use of superior drop down and tree selections for
report parameters.
27
28. Contoso Retail Scorecard Using SSAS KPIs
KPIs for the Contoso SSAS Operation Cube are used to build
a scorecard. Indicators are modified to provide a better
web appearance.
When building the Dashboard Page containing the KPIs, the primary KPIs are linked
to a different chart that will appear in SharePoint when that KPI is selected. 28
29. Other PPS Reports
Other Charts and Reports are created directly in PPS.
The Excel Pivot Report is added so that it will be included
on the Dashboard.
SSRS Reports are added and deployed to the
SharePoint Dashboard using PPS page
controls. This not only allows inclusion on
the Dashboard, but also enables these
reports to use SharePoint selection controls
which have a better appearance and are
easier to use.
29
32. KPI Scorecard in SharePoint Dashboard
The right hand chart is set by the select KPI on
the left.
Product Gross Margin selected in KPI.
Channel Revenue selected in KPI.
Returns % selected in KPI.
32
Machine Downtime Trend selected in KPI.
33. A Deceptively Simple Chart from PPS
But it has
impressive drill
down capabilities.
33
35. SSRS Scheduled Reports
A simple informational report
was created in SSRS, deployed
in SharePoint, then versions
for Maryland and Virginia
were set up for daily
production.
35
36. Demonstration and Educational Presentations
These presentations, prepared independently, provide education and demonstrations
for anyone interest in T-SQL and MDX query syntax and use. The completed
presentations are posted on LinkedIn as SlideShare Presentations.
36
37. Experience Summary
Over 12 years experience as a analyst, data auditor and T-SQL Programmer for
data and financial system implementation and maintenance
Prior experience in accounting and system specification, selection and
implementation
MS Business Intelligence
T-SQL Programming
SQL Server Integration Services (SSIS)
SQL Server Analysis Services (SSAS)
MDX Programming
SQL Server Reporting Services (SSRS)
Excel, Excel Services and Power Pivot
SharePoint for Business Intelligence
Performance Point Services (PPS)
37
38. Recommendations
Letter of Recommendation
Robert Litsinger
The following is a letter of reference/recommendation for Robert Litsinger. Robert was enrolled in the SetFocus SQL Server Business Intelligence Master’s
Program in the third quarter of 2011 and will graduate on October 21, 2011. I was Robert’s instructor throughout the 12-week program.
Robert brought substantial experience (25+ years) into the Master’s Program, with a background as a systems analyst, database professional, and management
consultant. Robert demonstrated very early in our Master’s Program that he “knows data”. His prior work on data conversions in particular came through during
many lecture discussions.
Robert performed very well in our Master’s Program curriculum, both during lecture and project weeks. He picked up a great deal of information, performed well
on all areas of the BI stack, and often worked on lab assignments in the evenings/weekends after a long day of class. His attention to detail is exactly what every
professional instructor hopes to see. The road to learning the Microsoft BI stack is paved with countless details: Robert demonstrated he can integrate them into
his thought processes. During our final student team project, we will select Robert as the team lead.
Robert is very intelligent and articulate, and can translate business requirements into results. He has decades of experience in this area that a company will value.
He also constructed some documentation of his own as part of the learning curve for MDX programming – several of his ideas were so good that I plan to utilize
them for the next class.
Having been a hiring manager, I know that companies are equally concerned about the character of an applicant. Without any exaggeration, Robert is a model of
professionalism, especially during periods of adversity. A company will benefit from his experience, capabilities, and work ethic - and will also appreciate his
steady demeanor and common sense. He is a quality person who produces quality work. I enjoyed having him in class and appreciated his insights.
I would recommend Robert without hesitation for any SQL Server Business Intelligence developer/management position. Please contact me at
kgoff@setfocus.com, if you have any additional questions.
Kevin S. Goff | kgoff@setfocus.com
Microsoft SQL Server MVP
SetFocus SQL Server Business Intelligence Practice Manager
About SetFocus: SetFocus, LLC (www.setfocus.com) is a Microsoft Certified Gold Partner for Learning Solutions.
The Master's Program consists of intensive coverage of T-SQL, SSIS, SSAS, MDX, SSRS, PerformancePoint Server,
SharePoint, and Excel Services. Our BI curriculum is one of the most intensive curriculums in the industry. The
projects are based on actual project specifications from industry SQL Server and OLAP/Business Intelligence
applications.
38