1. Employee Time Analysis Data Load Process (SSIS) 2005 *** John V. Stires October 10, 2009 Table of Contents TOC
1-1
Heading 2,2,Hdg 1,1,Hdg 2,2
I.Introduction PAGEREF _Toc244932001 3 II.Purpose PAGEREF _Toc244932002 3 III.Overview PAGEREF _Toc244932003 3 A.Counter Variables PAGEREF _Toc244932004 3 B.Process Control PAGEREF _Toc244932005 3 IV.Package Detail PAGEREF _Toc244932006 5 A.Employee Master Upload PAGEREF _Toc244932007 5 B.Employee Rates Upload PAGEREF _Toc244932008 7 C.Division Master Upload PAGEREF _Toc244932009 9 D.Client Master Upload PAGEREF _Toc244932010 11 E.Client Grouping Master Upload PAGEREF _Toc244932011 13 F.Client Groupings Client Xref Upload PAGEREF _Toc244932012 15 G.Project Master Upload PAGEREF _Toc244932013 17 H.Time Sheet Upload PAGEREF _Toc244932014 19 I.DB Maintenance PAGEREF _Toc244932015 22 Introduction Design and build a SQL Server 2005 database to track employee and customer information, timesheet and labor rates data, as well as job order information, job materials, and customer invoices for the fictitious company, Allworks, Inc. AllWorks currently stores this information in Excel Spreadsheets, XML files, and csv files. Purpose This project is designed to load data from various sources into an uniform SQL 2005 Server database. Eight tables are loaded with the external data sources each of these tables will be uploaded by a separate package. Overview There are eight packages that upload the data from various external sources. Every attempt was made to avoid abortive errors because of data issues. However, when such an event does occur, an email is sent and a message written to an error log giving notification that an error has occurred. The packages and tables addressed here are: TableUpload PackageEmployee Master UploadEmployeesEmployee Rates UploadEmployeeRatesDivision Master UploadDivisionClient Master UploadClientsClient Grouping Master UploadClientGroupingsClient Groupings Client Xref UploadClientGroupingsXClientProject Master UploadJobMasterTime Sheet UploadJobTimeSheets There are two remaining packages. There is a package, DB Maintenance, which performs maintenance on the database immediately after the successful completion of the uploading process. The other package, Master Payroll Package Control, coordinates the running of all of the other packages. Each of these packages will be discussed further in detail below. Counter Variables Total Input rows- TtlRows Changed Rows- ChangedRows Unchanged Rows- UnchangedRows Rows in Err- ErrorRows Process Control This process control drives the entire upload process. This package coordinates the execution of all of the other packages to perform the data upload. Upon successful completion or should an error occur, it sends an email. Process Control: This process control drives the entire upload process. Package Detail Each package performs its upload. When successful, it sends an email notification of that fact giving the statistics for the run in terms of the number of new rows, updated rows, etc. There is a standard naming for the counters across all packages. Others will be discussed in those packages where they are used: Employee Master Upload Input Excel spreadsheet file-Employees.XLS, Sheet: Employees Output The Employees table in the AllWorksDBStudent database. Special Processing. There is a new, derived, field created for FullName. Lookups / Validations A check is made to verify whether this employee is already in the table or not. Conditional Processing. The conditions processed are: New Rows are added to the table. Updated rows where rows are modified Error rows. Errors include input data errors where the input Employee ID is null. Package Diagram Employee Upload: This process. Uploads new Employee Data. Employee Rates Upload Input Excel spreadsheet file-Employees.XLS, Sheet: EmployeeRates Output The EmployeeRates table in the AllWorksDBStudent database. Special Processing. N/A Lookups / Validations Verify if this employee is in the Employees table. Determine if there is already a row for this employee in the EmployeeRates table. Conditional Processing. The conditions processed are: New Rows are added to the table. Updated rows where rows are modified Unchanged rows that were not modified Error rows. Errors include input data errors where the input Employee ID is null Package Diagram Employee Rates Upload: This process uploads the Employee Rates Data. Division Master Upload Input Excel spreadsheet file-ClientGeographies.XLS, Sheet: DivisionDefinitions Output The Division table in the AllWorksDBStudent database. Special Processing. N/A Lookups / Validations The Division table in the AllWorksDBStudent database. Determine if there is already a rows for this division in the Division table. Conditional Processing. The conditions processed are: New Rows are added to the table. Updated rows where rows are modified Unchanged rows that were not modified Error rows. Errors include input data errors where the input Employee ID is null Package DiagramDivision Upload: This process uploads new Division Data. Client Master Upload Input Excel spreadsheet file-ClientGeopgraphies.XLS, Sheet: Client Listing Output The Client table in the AllWorksDBStudent database. Special Processing. N/A Lookups / Validations A check is made to verify whether this client is already in the table or not. Conditional Processing. The conditions processed are: New Rows are added to the table. Updated rows where rows are modified Unchanged rows that were not modified Error rows. Errors include input data errors where the input Employee ID is null Package Diagram Client Master Upload: This process uploads new and changed Client Data. Client Grouping Master Upload Input Excel spreadsheet file-ClientGeopgraphies.XLS, Sheet: Special Groupings Output The ClientGroupings table in the AllWorksDBStudent database. Special Processing. Duplicate input records are expected and all duplicates are eliminated from the data run stream. A duplicate is where there are input records with both the Client Grouping Id and the Grouping Name the same. Lookups / Validations A check is made to verify whether this client is already in the table or not. Conditional Processing. The conditions processed are: New Rows are added to the table. Updated rows where rows are modified Unchanged rows that were not modified Error rows. Errors include input data errors where the input Employee ID is null Package Diagram Client Groupings Master Upload: This process uploads new Client Groupings Data. Client Groupings Client Xref Upload Input Excel spreadsheet file-ClientGeopgraphies.XLS, Sheet: Special Groupings Output The ClientGroupingsXClients table in the AllWorksDBStudent database. Special Processing. N/A. Lookups / Validations Verify the Client is in the Clients table. Verify if this Client Grouping ID is in the Client Groupings table. Determine if there is already a row for this ClientGroupings Xref Clients record in the ClientGroupingsXClients table. Conditional Processing. The conditions processed are: New Rows are added to the table. Error rows. Errors include input data errors where the input Client Grouping ID, Client Grouping ID is null or the Record entry is already in the table. Package Diagram Client Groupings Xref Upload: This process uploads new Client Groupings Cross Reference Data. Project Master Upload Input Excel spreadsheet file-ProjectMaster.XLS, Sheet: ProjectMaster Output The JobMaster table in the AllWorksDBStudent database. Special Processing. N/A. Lookups / Validations Verify the client ID is in the Clients table Determine if the project is already in the JobMaster Conditional Processing. The conditions processed are: New Rows are added to the table. Updated rows where rows are modified Unchanged rows that were not modified Error rows. Errors include input data errors where the input Project ID or Client ID are null, or the client ID does not exist in the Clients table. Package Diagram Project Master Upload: This process uploads new Project Master Data. Time Sheet Upload Input Excel spreadsheet file-EmpTime*.CSV Output The JobTimeSheets table in the AllWorksDBStudent database. Special Processing. There are a number file input files to be processed. The file name pattern is: EmpTime*.CSV. The variable TimeSheetFile is used to process each file through the load. Lookups / Validations Verify the Employee ID is in the Employee table Verify the Project ID is in the JobMaster table Determine if the Job Time Sheet is already in the JobTimeSheets tables Conditional Processing. The conditions processed are: New Rows are added to the table. Updated rows where rows are modified Unchanged records are not modified. Time Sheets for job not in the JobMaster Time Sheets with WorkDates after the job Closed Date, Late Time Sheets Error rows. Errors include input data errors where: Input Employee ID or Project ID or WorkDate are null The Employee ID does not exist in the Employees table Project ID does not exist in the JobMaster table. Package Diagram Control Flow Time Sheet Upload: This process uploads new Time Sheet Data. Data Flow DB Maintenance This package handles the duties of performing maintenance on the AllWorksDBStudent database. The functions it performs are: Backing up the database Rebuilding the indexes Shrinking the database to eliminate unused space Sending an email upon a successful completion Package Diagram Database Maintenance