Contenu connexe Similaire à Tera stream for datastreams Similaire à Tera stream for datastreams (20) Tera stream for datastreams2. Data Integration as Data Infrastructure
content Why TeraStream™?
Case Studies
Company Profile
Q&A
© 2012 DataStreams Corp. All Rights Reserved.
4. Enterprise Data Management Suite
Minimal implementation effort and producing high-quality
Business Intelligence output.
Real Time Monitoring
• Performance Tuning
• Emergency Management
Source Reports
Systems Integrated ODS / DW Report
Mart
MERGE SPLIT Aggregation Calculation
ETL / Real Time / Near Real Time
SORT JOIN Conversion Reformat
Data Governance Architecture
Master Data
Meta Data Data Quality Impact Analysis
Management
© 2012 DataStreams Corp. All Rights Reserved.
6. DataStreams Products
ETL Test Data Security Change Data Capture Near Real Time
Meta Data Data Quality Master Data Management Impact Analysis
Common Benefit of DataStreams Products
Simple Mgt. & Upgrade Stability
TCO Reduction One-stop support
Saving time cost by Speed Extreme Performance
© 2012 DataStreams Corp. All Rights Reserved.
8. Floods of Massive Data
To get Business intelligence on the Data.. Need Data
Integration
ERP Systems Other Operational Apps
Legacy Systems Web Logs etc
Call Center Apps RDBMSs
File based information Unstructured Data
© 2012 DataStreams Corp. All Rights Reserved.
9. Data Integration Landscape
Business Challenges
Impact
Factors Result
Multiple versions of the
Disparate data Bad decisions
“Truth”
Inaccurate data Wasted time and Lost revenue
Incomplete data resources aggregating
Lost productivity
information
Untimely data Lost market opportunity
Difficult to use Data
Fragmented Delayed Decision making Bad Citizen relationships
Integration Approach Uninformed management
This is more than 30 percent of corporate
IT budgets so data integrity is used to
emphasize what is important.
© 2012 DataStreams Corp. All Rights Reserved.
10. Well Positioned
TeraStream is very well positioned in the Data Integration
marketplace.
Powerful Performance (TeraSort™)
High-speed extraction (FACT™)
Reuse of data (ETL and Batch Hub)
Over 200 Customers
Serving multiple industries including banking,
government retail
Over a decade of experience
Intuitive GUI
Easy to Operate
Easy to maintain
Fast implementation
Easy customization
Low resource use
© 2012 DataStreams Corp. All Rights Reserved.
11. TeraStream™ Positioning
A TeraStream™ bundle, one Unload tool, one Sort tool and ETL metadata
repository
An end to end, High Performance driven data integration platform.
• Basel II
• Risk Management
• Data Warehouse
• Data Migration
• Data Synchronization
• Batch Execution
• Master Data Management
© 2012 DataStreams Corp. All Rights Reserved.
12. TeraStream™ Data Integration
Extraction, Transformation, Load with High performance
SORT Engine
Data extraction from various High Performance SORT engine resolves time bottleneck due to transform Automatic generation of scripts
can be used for loading to various
commercial DBMS in high speed
LARGE DATUM DBMSs
Text Text
Data Data
MERGE SPLIT Aggregation Calculation
High Performances SORT Engine
SORT JOIN Conversion Reformat
EXTRACT TRANSFORM LOAD
© 2012 DataStreams Corp. All Rights Reserved.
13. TeraStream™ Data Integration Benefits
3times Elapsed time improvement against its competitor
Result on Data Migration in Shinhan Bank, Korea
TeraStream™ IBM DataStage
FILE → FILE FILE → DB
FILE → DB DB → DB
thread MAX for sort =3 Parallel = 4
File manipulation : 35% CPU usage File manipulation : 58% of CPU usage.
Load : 80% of peak CPU usage Load: 58% of peak CPU usage
Conclusion Conclusion
Elapse time : 20 minutes Elapse time : 59 minutes
Wasted System Resource : 800 Wasted System Resource : 3000
( 40% Avg. CPU usage X 20 mins ) (50% Avg. CPU usage X 60 mins)
© 2012 DataStreams Corp. All Rights Reserved.
14. TeraStream™ Near Real Time Data
Integration Processing Benefits
High Performance improvement on Real Data processing
Result on DW Benchmark Shinhan Bank, Korea
(Minutes)
50
DataStage
40
30
10
03 TeraStream
02
01
0
1 10 100 200 1,000 2,000
(Ten thousand records)
© 2012 DataStreams Corp. All Rights Reserved.
15. TeraStream™ Large Data Benefits
High Performance improvement on Very Large Data processing
Result on Batch Job Benchmark National Health Insurance Corporations, Korea (2005)
55% Reduction 50% Reduction 90% Reduction
70day
11day 4day 7day
5day 2day
© 2012 DataStreams Corp. All Rights Reserved.
16. TeraStream™ Batch Processing Benefits
High Performance improvement on Batch Jobs execution
than DB processing
Result on Batch Job Benchmark Post Insurance, Korea
40,000 400,000 4,000,000
Records Records Records
12x
3x
© 2012 DataStreams Corp. All Rights Reserved.
17. TeraStream™ EBH Benefits
Massive volume of files extracted from source systems
are stored in EBH, and reuse in next step
ETL Time is reduced by avg. 56% In case of LG Telecom
Legacy ODS EDW MART
ORACLE 8.0.6 ORACLE Teradata Sybase
EBH (ETL Batch Hub)
EBH (ETL and Batch Hub)
stores temporary or result files which is shared for further table generation in EDW and DATA MART.
© 2012 DataStreams Corp. All Rights Reserved.
18. TeraStream™ Features
Intuitive User Interface
Intuitive task flow Metadata Repository
GUI for developers Impact analysis
Change history
manager
Scheduler Editor window
Scheduling by time / Mapping Creation
period / business
calendar
Configuration Management Project Version Execution log Web Monitoring File View
© 2012 DataStreams Corp. All Rights Reserved.
19. TeraStream™ Features
High speed data extraction with FACT unload engine
DBMS Supported SQL is supported High Speed Extraction
SQL
Database API optimized
Split data Filtering Export Format
Time
Timestamp
Fixed / Variable
CLOB / BLOB
NULL
FRAME
FILL
© 2012 DataStreams Corp. All Rights Reserved.
20. TeraStream™ Features
Database Connectivity and Diverse manipulation process
and functions
DBMS Supported N:N mapping File to DB or DB to File
Built-in Function User Function Supported Function Types
INSERT
DELETE
UPDATE
INSERT - UPDATE
DELETE – INSERT
EBCDIC ↔ ASCII
CLOB / BLOB
NDB (Unisys 9-bit)
HDB (IBM)
© 2012 DataStreams Corp. All Rights Reserved.
21. TeraStream™ Features
Uses various transportation method according to file
structure, security, amount of record and etc.
DBMS
File to DB data load for bulk data
Provide Bulk Load to process large data
Partition Load
Bulk Load
Parallel Load
DBMS Load scripts automatically
Automatically Generate Script by using DBMS
Load Script Schema information
Automatic Fixed / Variable
Provide DBMS Option
Encryption while transporting
Data Readable Symmetric Encryption
Maintain original data characteristic
Support various encryption pattern
© 2012 DataStreams Corp. All Rights Reserved.
22. Case Studies
© 2012 DataStreams Corp. All Rights Reserved.
23. Kookmin Bank
EDW and integrated DM installation
Various DBMS(IMS HDB, HOST DB2, Oracle, DB2 UDB) integration by using
Expected
TeraStream™
Result
Meeting batch target time of 2 hours and 30 minutes for 4TB of EBCDIC data.
Issues System configuration
• M/F and IMS HDB conversion
File process flow DB QUERY
• Processing changed data in absence of time-series
column Source system EDW
• Processing large size data within batch process IMS HDB Sybase ASIQ 12.7
time(10TB/day based on source data)
• How to process high volume files in parallel ETL
A-SOR DM
Informover
Plans IBM M/F - Seg. split
- conversion & Array split
• Converting main frame data into data in Unix - logic applied
environment (10TB → 25TB) within 18 hours.
• Various data conversion and processing including HDB, DB2 ETL
Korean character conversion
- conversion
• ETL task from accounting system server to new ODW Informover
- logic applied
server(extracting appx. 200 GB of daily changed data
within 1 hour and 30 minutes by using FACT module of
Server RDB
TeraStream™)
TS(FACT) ETL
• ETL and Batch process in unified way.
• Batch job in core banking system within 6 hours. - Logic applied
© 2012 DataStreams Corp. All Rights Reserved.
24. Samsung Electronics
Global Database Integration using NRT ETL
Efficiency is maintained despite cross transportation
Expected Bi-directional NRT integration allows the same service regardless of system type
Result and country (Time from extraction to loading is 20 minutes.)
Bi-directional remote data transportation using TeraStream
Issues System Configuration
• Registration in one country should have the same Smart Phone Smart Phone System in China
service at other country. System in
Germany
• duplicated record should be avoided due to cross
transportation.
DBs in DBs in
• 20 minutes Near Real-time Service Service
• Perfect Recovery scheme should be presented
Plans Sam To DB
UPSERT
• Rea-time data transportation between Germany and SAM To DB
NRT Extract NRT Extract UPSERT
China.
• Bi-directional synchronization between TeraStream of
Germany and China.
• 20 min. MAX loading time for transported data is
implemented using TeraStream NRT. 프로그램 성공, 실패 등 실행 결과
Web Monitoring
• Web Monitoring is developed
© 2012 DataStreams Corp. All Rights Reserved.
25. LG Telecom
LG Telecom new billing system data transfer
The working hours shortened to D +3 and D +1 in reducing the system load
Expected
On average, 56% of the effect of reducing working hours
Result
Emergency response system rework due to delay in securing and providing data
to minimize Impact
Issues System Configuration
• Solution provided by „I‟ company requires more than
twelve hours for processing every billing and call data.
ETL EDW Server
• It delays entire processes and often requires re-
processing of data. D+1
• Efficient unique key generation for entire business tasks Custo
Call
Data Billing
Legacy ODS Server mer
Plans Contacts PPS/
BSS
• Transition from old to new billing system AR SRDF ODS
Mining
Input Campaign MOLAP
- Data size: 3TB→ 3.5TB, Object: Transition in Oracle Oracle/ Variables Analysis Analysis
Billing Informatica ETL
30 minutes Mining Teradata
Analysis
• Move unchanged data among large dataset three CSM CCS CTI NCR 10Node
days prior to the new system open date. MPS PPS
DM Server
• Separate files that will be loaded to EDW and DM ERP NMS
and load them in different business tables. OLAP
TeraStream loads data transformed Mart
• Unique key generation for entire business process
Sybase ASIQ
is done first. in ODS to EDW and DM at the same time. IBM P Series
© 2012 DataStreams Corp. All Rights Reserved.
27. Company Profile
Company DataStreams Corp.
CEO Young-sang Lee
Established 2001.9.19
Data Integration Solutions Development and Sales
- ETL/data migration/high performance batch product
-Deferred (Near Real Time) data integration solution
- Real time data integration product
Data Quality Solutions Development and Sales
Business -Metadata and data quality management solutions
- Application impact analysis product
Master Data Management Solution
Data Warehouse & BI Development / SNS Analytics
Data Standardization & Quality Management Consulting
High Performance Batch Job Technical Consulting
Investor JAFCO ASIA
Headquarter Chungho-Nais B/D 6F, 1597-3, Seocho-dong, Seocho-gu, Seoul, Korea
#601, U-Spacemall #2, Sampyoung-dong, Bundang-gu, Seongnam-si,
R&D Center
Gyounggi-do , Korea
Homepage www.datastreams.co.kr
© 2012 DataStreams Corp. All Rights Reserved.
28. Business Area
DataStreams provides “data integration and quality solutions” that enable your company to
stand firmly on concrete data infrastructure. It allows you to analyze and utilize your
internal or external data to gain competitiveness by empowering with timely, relevant and
trustworthy data for your top business imperatives. Based on leading data integration and
quality solutions, DataStreams has expanded its business area to Master Data Management
and Information Analytics as well as big data performance improvement, data migration,
various data integration and metadata management.
Data Integration Solutions Meta Data & Quality
Development / Sales Management Solutions
- Metadata and data
- ETL / Data Migration / High-speed Batch Tool quality management solution
- Real-time data Integration Product Business - Application impact analysis product
- Deferred data integration solution(NRT)
Areas Data Quality Consulting
D/W BI Development
Master Data High-speed batch
Management Solution & processing technical
Consulting SNS Analytics consulting
© 2012 DataStreams Corp. All Rights Reserved.
29. Mission of DataStreams
DataStreams is evolved toward data management company based on over 13 years of
experience in data integration and quality management. DataStreams helps information
users to build concrete data infrastructure for integration and quality management. Based
on superb quality of on-hand data, DataStreams‟s mission to satisfy customers by raising
enterprise value and competitiveness will be accomplished.
No.1 end-to-end data integration and quality management solution provider
Help your enterprise by providing data governance capability Consult and build
systems how to utilize your accumulated data for long years
© 2012 DataStreams Corp. All Rights Reserved.
30. Market Share & Reputation
DataStreams is No.1 data integration and metadata management solution provider in Korea. DataStreams
has been selected for 30 Korean software company worthy of global recognition in 2012.
Market share Good Reputation
No. 1 Share in Domestic Market
Etc 6%
SAS 3%
SAP 5% DataStreams
ORACLE 28%
6%
INFOMATICA,
10%
Wise
IBM 9%
10%
Encore Gtone
BTL
7% 10%
6%
<Domestic M/S>
* Domestic M/S is researched by DataStreams (2008)
• M/S of Global Vendors is quoted by "Worldwide DI and Access Software 2006
Vendor Shares: Market and Segments," IDC.
© 2012 DataStreams Corp. All Rights Reserved.
31. Credentials
DataStreams is recognized as leading company in data integration & quality management in domestic
market. DataStreams is the only solution provider who has expertise in both solution market. DataStreams
has developed its own methodologies for master data management and data governance technologies for
EDM. As a leader, DataStreams owns good references in various industrial fields (especially, more than 80%
of market share in domestic banking industry).
Data Integration Data Quality Methodology & Experiences
Data quality technologies with metadata
Data Integration technologies Its own methodology for MDM & lot
management, data quality and application
with ETL, CDC and NRT solutions. of technical experiences
impact analysis solutions.
Progress No. 1 Market Share Professional Support
DataStreams
80%
Perfect data governance for data
administrator is progressive approach for
EDM . It is based on product line-up from 80% of M/S in domestic Banking Market, Technical support and quality assurance
ETL to real-time and metadata to impact Market Share, ETL(55%)#1 in with Over 60 Consultants and over 34
analysis solutions, Data Migration(60%)#1 in Korea R&D personnel
© 2012 DataStreams Corp. All Rights Reserved.
32. DataStreams is the SW Market Leader in Korea
TeraStream™ has No.1 market share and stellar references in ETL and Migration market in Korea.
5%
15%
ETL &
55% Migration
25%
TeraStream IBM Data Stage Informatica Other
- 31 -
© 2012 DataStreams Corp. All Rights Reserved.
33. DataStreams dominates Korean banking industry
Most of major banks in Korea use TeraStream™ as their major ETL product. TeraStream is
unique market leader in Korean banking industry.
© 2012 DataStreams Corp. All Rights Reserved.
34. The Market Leader in Korean is going global
DataStreams already has a strong global presence with many of its products and plans to roll
out its entire data management product suite internationally
Merklenet.Inc.
Accenture Gmbh Airweb
CSC Consulting
Sungard
Bisys American Air
Telefonica Soluciones Reliance Industry Comcast Highmark, Inc
Procecard Excelcom Merkle Data Tech
Techocom
Sodexhopass
Telkomsel
Aviva
Banco Colpatria
Credibanco
© 2012 DataStreams Corp. All Rights Reserved.
35. Strong Track Record of Success
2012 05 Open Pangyo Techno valley R&D Center
2005 04 TeraStream™ won New Technology Certification
from Ministry of Knowledge Economy
2011 12 Datastreams China. established
04 Acquired KDB Solution Co., Korea‟s first meta-data
11 Received SW achievement award by Korean Government management solution company.
06 Selected as an ATC(Advanced Technology Center) 03 Awarded for Product Competitiveness from SMBA
2010 12 TeraStream™ V.3.2 released
2004
08 TeraStream™, trademark registered (No.0589927)
03 CEO, Young S. Lee elected as chairman of 05 First worldwide sales contract of FACT™
KOSEA(Korea Software Enterprise Association). 04 Selected as technically innovated company of
01 MetaStream™ V.2.7 released 2004 by Small & Medium Business Administration
2009 06 QualityStream™, ImpactStream™ released
2003 10 Presented FACT™ to Oracle Open World 2003 in
San Francisco, USA
02 DeltaStream™ V.1.1 released
08 KEB selected TeraStream™ as standard Batch/
06 Contracted with Intellectual Property Office ETL solution for Next Generation Banking System.
2008 for Enterprise Data Quality Management.
03 Company‟s name change to DataStreams 2002 12 First TeraStream™ V.1.1 contract at National
Statistics Office
11 JAFCO, Japan invested 4 million USD. 07 TeraStream™ exposed in SEK2002, Seoul
07 Awarded for Excellent Venture Company by 06 TeraStream™ Version 1.1, Released
2007 Deputy Prime Minister 01 Registered TeraStream™ as a trademark
06 TeraStream™ Version 2.0 released
05 MetaStream™, awarded for Digital Business 2001 09 Innovative Data Solutions Corp., established
Innovation by SMBA
2006 12 TeraStream is selected as standard ETL for 16 2000 04 TeraStream development started
01 CoSort was successfully applied to a DW
Province Public Administration Office
project for the first time at LG company
11 TeraStream™ V.1.4 won GS(Good Software)
Certification 1999 09 CoSort Korea, established
© 2012 DataStreams Corp. All Rights Reserved.
36. Marquee Accounts in Korea and Beyond
Banks
Insurance
Securities
Card/
Finance
Government/
Public Sector/
Universities
Telco
Manufacturing
/Services
© 2012 DataStreams Corp. All Rights Reserved.
Notes de l'éditeur In 2009, Younglimwon Soft Lab set a record expanding its sales by 50%. In addition, the company is making efforts to diversify by as developing new products and improving quality through R&D investment and holding a CEO forum and a CEO workshop for corporate customers. These efforts have paid off. The company has enjoyed ratio of 90% or higher signing maintenance contracts with corporate customers. In 2010, the company recorded 20 win-backs beating larger competitors.