2. OVERVIEW
1. My specialties
2. BI Projects involved in
3. Continuous improvement
4. Recommended books
5. About me
3. SPECIALTIES
Databases: Microsoft SQL Server, MySQL and
Oracle Tools/Products
Microsoft: SSMS, SSIS, SSAS, SSRS, Sharepoint,
Oracle: PL/SQL Developer and
Open Source: Jasper Reports.
Database design and programming
Data modeling
Business Intelligence
- Data Warehouse (Dan Linstedt's Data Vault Model
and Methodology, Bill Inmon's DW )
- Data Mart (Ralph Kimball Model and
Methodology)
4. BUSINESS INTELLIGENCE DEVELOPER
SQL Server and SQL programming language, BI
concepts and architecture (Data warehouse, Data
Mart, Data Vault, MDX), SSIS, SSAS, SSRS. BI
Test cases Validation, Error Handling in SSIS,
Builing, Deploying, Managing, Securing Reports
using SSRS and Report Buider.
7. BANKING PROJECT FLOW
Microsoft SQL Server 2008 R2 ,SSIS and C#
received in an input folder a set of files
(transactions between accounts and customers)
whenever a file arrives it is extracted, transformed
and loaded in databases.
the output files are generated when all expected
files arrives.
ETL process generates messages which are stored
into a log table.
9. DATABASES
three databases for main data storage
Staging: here the data suffer basic validations and
transformations
Core: the central repository of the data (like a
datawarehouse)
Output: here the data in stored in the format expected to
be in the output files
two auxiliary databases:
Configuration: store all parameters and other
configuration data on which the system is based.
AuditAndLog: store all information regarding auditing
and also all the messages which the processes
generate.
10. PROCESSES
- six interdependent processes:
F2S: File to staging: Loads the data from the input files
to the Staging DB
S2C: Staging to Core: Loads the data from the Staging
DB to the Core DB
C2O: Core to Output: Loads the data from the Core DB
to the Output DB and latter to the output files.
LOG: Log messages to external files in a format
required by the consumer systems of those file.
CFA: Check File Arrival: Is checking if the expected files
arrived or not, and notify if a file is not arrived in the
specified time-window.
PURGE: It purges old data from databases.
11. COMPANY MANAGEMENT PROJECT FLOW
SQL Server 2012, PowerPivot 2010
develop and monitoring the structure of the
company using source data collected from
Operational Systems (iTimeTrack, CRM,
SmartOffice,Costs, Budgets).
13. THE MAIN TASKS:
create Audit database structure (contains tables that are logging all the events
occurring in the system),
staging area database structure (SQL dynamic scripts to generate tables,
primary keys, indexes and views to concatenate the table columns),
Data Vault database structure (SQL scripts used to generate Hubs, Satellites and
Links) and
Data Mart database structure (SQL scripts to generate dimensions and facts tables),
create PITs and bridge tables to improve performance of the queries,
analyze the full and delta load for each table,
ETL Integration: stored procedures to import the data from source to staging area,
from staging area to Data Vault and from Data Vault to Data Mart,
create linked servers,
create SSIS packages used to import the CSV source files,
package configuration and deploy,
use PowerView from SharePoint to create reports.
15. MULTIDIMENSIONAL MODEL VS
TABULAR MODEL
Multidimensional
- complex calculations and scoping
- Many-to-Many relationships
- complex modeling solution
- access to many different external data sources
Tabular
- consistently fast query time
- planning on using Power View (an interactive data
exploration, visualization, and presentation
experience browser-based Silverlight application
launched from SharePoint Server 2010)
16.
17. INSURANCE PROJECT
involved in a maintenance BI project of a mix of
applications and technologies that are used to
gather, provide access to and analyze data and
information about company operations.
The main tasks I had provide me with the ability to
resolve database administration issues:
18. MAIN TASKS
cube deployment errors,
wrong mapping of columns in cube,
update links with the right values,
set the protection level in order to get package access,
implement business logic - create a job running if another one
is correctly processed, run the job if more than 1 business day
is past- add rights to users,
move the project on TFS,
upgrade the BI solution in order to bring performance,
robustness and a better platform to improve the solution,
implement a purge process to adjust the space allocation for
files
code review and research in reducing SQL Server deadlocks,
implement Backup and Recovery planning.
20. RESEARCH MOVING TO SQL SERVER 2012
SQL Server engine-
ColumnStore index - combined with enhanced query
processing features, improves data warehouse query
performance by hundreds to thousands of times in
some cases;
Database engine is more powerful.
Analysis Services-
New tabular model which allows analysis using
PowerPivot for Excel 2012 gives powerful tools for
analyze very quickly large amounts of data
Improving integration of PowerPivot to SharePoint
portal (if the user choose to deploy a SharePoint portal)
21. RESEARCH MOVING TO SQL SERVER 2012
Reporting Services-
PowerView: an interactive data exploration,
visualization, and presentation experience. It provides
drag-and-drop ad hoc reporting for business users such
as data analysts, business decision makers, and
information workers.
Integration Services-
The processing engine is better
Deployment and administration of packages greatly
enhanced
Reduced memory usage
The upgrade process will bring performance,
robustness and, in the future, a better platform to
improve the current BI solution.
24. PRESENTATIONS, BEST PRACTICES, TASKS
DESCRIPTION, RECOMMENDED BOOKS
ro.linkedin.com/pub/monica-opris/38/659/407
http://businessintelligencedev.blogspot.ro/
Notes de l'éditeur
SSIS’ core purpose is to move data and do something with it. This is probably my favorite tool in the SQL Server BI Stack because it is very visual and you can see where your data is going. Once the ETL is developed, you can also schedule it to run as often as you need updated information. It has many other uses, such as import/export to Excel, emailing attachments, looping through files, performing maintenance on the server.
SQL Server Analysis Services (SSAS) to create OLAP databases, also known as Cubes. These OLAP databases are high performance database structures that aggregate, slice-and-dice and organize your dimensions with hierarchies, allowing you to drill down until you get to the data you want. On top of all of that, it is really fast.