Webinar - The 360-Degree Approach to Managing a Data Lake
•Télécharger en tant que PPTX, PDF•
2 j'aime•1,403 vues
A discussion about how Zaloni's Data Lake 360° Solution reduces the complexity of creating a modern, integrated big data architecture for advanced analytics and data science. A demo of the latest versions of Bedrock and Mica is featured.
Recommandé
Recommandé
Contenu connexe
Dernier
Dernier (20)
En vedette
En vedette (20)
Webinar - The 360-Degree Approach to Managing a Data Lake
- 1. Webinar: The 360-Degree Approach to Managing a Data Lake August 31, 2016
- 2. Zaloni Confidential and Proprietary - Provided under NDA • Award-winning provider of enterprise data lake management solutions: Integrated data lake management platform Self-service data preparation • Data Lake Design and Implementation Services: POC, Pilot, Production, Operations, Training • Data Science Professional Services Delivering on the business of big data Financed by top-tier technology investors:
- 3. Zaloni Confidential and Proprietary - Provided under NDA Meet today’s speakers Adam Diaz, Director of Field Engineering, Zaloni Adam is a long time technologist working in the software industry for roughly twenty years. He manages the day to day operations of the Zaloni Field Engineering team responsible for product technical support of sales, marketing and partnership activities. Kelly Hopkins Schupp, VP, Data-driven Marketing, Zaloni Kelly has 20 years of experience in the enterprise software and technology industry. She serves as Zaloni’s brand steward and is deeply passionate about the impact of data-driven marketing on businesses today.
- 4. Zaloni Confidential and Proprietary - Provided under NDA Increased Agility New Insights Improved Scalability Data lakes are central to the modern data architecture
- 5. Zaloni Confidential and Proprietary - Provided under NDA Data lake – the challenges and the solution • Ingestion • Lack of Visibility • Privacy and Compliance • Quality Issues • Reliance on IT • Reusability • Rate of Change • Skills Gap • Complexity Managing: Delivering: Govern the data in the lake • Cleanse • Secure • Operationalize Enable the data lake • Ingest • Organize • Catalog Building: Engage the business • Discover • Enrich • Provision
- 6. Zaloni Confidential and Proprietary - Provided under NDA • Leverage the full power of a scale-out architecture with an actionable, scalable data lake Data Lake 360°: Zaloni’s holistic approach to actionable big data 1. Enable the lake 2. Govern the data • Improve data visibility, reliability and quality to reduce time-to-insight 3. Engage the business • Safeguard sensitive data and enable regulatory compliance • Foster a data-driven business through self-service data discovery and preparation
- 7. Zaloni Confidential and Proprietary - Provided under NDA Data lake reference architecture Consumption Zone Source System File Data DB Data ETL Extracts Streaming Transient Loading Zone Raw Data Refined Data Trusted Data Discovery Sandbox Original unaltered data attributes Tokenized Data APIs Reference Data Master Data Data Wrangling Data Discovery Exploratory Analytics Metadata Data Quality Data Catalog Security Data Lake Integrate to common format Data Validation Data Cleansing Aggregations OLTP or ODS Enterprise Data Warehouse Logs (or other unstructured data) Cloud Services Business Analysts Researchers Data Scientists
- 8. Zaloni Confidential and Proprietary - Provided under NDA Ingest Organize Enrich Engage • Managed ingestion • Metadata management • Data lineage • Data quality • Data privacy and security • Data lifecycle management • Data catalog • Self-service data preparation Manage the complete data pipeline
- 9. Zaloni Confidential and Proprietary - Provided under NDA Multiple cluster management with Bedrock Single management console for Ground and Cloud Hadoop management Bedrock Workflow Executors Bedrock Data Collectors Persistent ClusterMetadata Data Quality Data Catalog Security Files Streams 9 Object Stores Bedrock Data Management Transient Cluster
- 10. Zaloni Confidential and Proprietary - Provided under NDA Demo Zaloni Confidential and Proprietary
- 11. Challenge: To develop a multi-vendor loyalty program required gathering information from a wide variety of sources, bring it all into a unified platform, and give marketers access to the data for customer targeting. Solution: Zaloni built a data lake to house all customer data, as well as an interactive visualization platform that enabled marketers to profile customers on selection (based on demographics, geography, etc.) for analysis. Zaloni Enables 360˚ Customer View For Effective Loyalty Program GLOBAL CREDIT CARD ISSUER Industry: Financial Services Company Description: Global credit card issuer Technical Use Case: Data Lake 360˚ - Agile Analytics Business Use Case: Customer 360˚ - Behavioral Analytics and Customer Segmentation Big Data Technologies: Zaloni Professional Services, MapR, Elasticsearch Deployment: On premises Zaloni Confidential and Proprietary - Provided under NDA At a Glance 1Mcustomer sign-ups 22M users collecting data from 1M+hits/day on the customer Results in 3 weeks system
- 12. Zaloni Underpins Data Platform for News and Entertainment Network MEDIA & ENTERTAINMENT COMPANY Industry: Media and Entertainment Company Description: News and Entertainment Network Technical Use Case: Data Lake 360˚: Agile Analytics, Data Discovery Portal Business Use Case: Improve Customer Experience, Drive Dynamic Ad Pricing Big Data Technologies: Bedrock, Mica, Pivotal, Spark, Hive, MapReduce Deployment: On premises Zaloni Confidential and Proprietary - Provided under NDA At a Glance Challenge: The company needed a data platform for advertising analytics and audience measurement in order to drive dynamic pricing and various other use cases during key sporting event-based entertainment broadcasting. Solution: Zaloni Bedrock and Mica were used to create an enterprise-wide data lake for managed ingestion of multiple data sources, standardization and consumption using a scale out Hadoop platform. 310% MORE DATA PROCESSED 220% IMPROVEMENT in performance Results 0 DAYS DATA DELIVERY MISSED 9.5B AD CLICK RECORDS processed in 17 days
- 13. Zaloni Confidential and Proprietary - Provided under NDA Wherever you are in your journey, we can help…. Products & Solutions: Data Lake 360° White Paper Bedrock Overview Mica Overview To Learn More: info@zaloni.com Contact Us Form 919-323-4050 x7085 Custom & Packaged Services: Readiness Assessments Quick Starts Custom implementations
Notes de l'éditeur
- When we refer to a “creating” a modern data architecture, there is an assumption that the existing data architecture is broken or somehow lacking. In this case that is part of the story. Existing EDW’s and the ETL processes used to managed them are stable and productive processes and technologies. There are tried and true data governance, data quality, and data stewardship services in place to help manage the creation and delivery of the data that feeds existing BI and Data Visualization technologies. This process worked well in the world of structured data that was being fed from data created primarily within an organization in known formats. But as organizations have looked to gain competitive advantage, reduce costs, and increase revenue, they realized that they needed to take advantage of not only their internal existing data sources, but add in social data, machine generated data, or 3rd party data services that have evolved over the years. This deluge of data volumes and variety have put a strain on the traditional data management approach. With the advent of big data technologies like hadoop and commodity hardware reducing compute and storage costs, organizations have looked to data lakes as a new architecture whereby “all” data is stored in it’s raw format, data discovery techniques allow data scientists, data analysts and knowledge workers of ally types to engage in the data management and analytics process. The benefits can be huge as there are more and different types of insights can be gained, time to value for gaining these insights can be reduced.
- The world’s largest global credit card issuer wanted to create an innovative customer retention program with rewards that could be “cashed-in” across multiple brands and industries, including larger retailers, financial institutions, car rental companies, global telecom providers and insurance companies. THE CHALLENGE To develop a more targeted multi-vendor loyalty program, the customer needed an enriched understanding of its members. The company needed to gather information from a wide variety of sources, including data from multiple merchants and financial services companies and their loyalty programs, bring it all into a unified platform, and give marketers access to the data needed for customer targeting. THE SOLUTION Zaloni architected and built a managed data lake to serve as the single repository for all customer data. In addition, Zaloni developed a fluid and interactive visualization platform that offered different dimensional views on the data and enabled marketers to profile customers on selection (based on demographics, geography, etc.) for analysis. The data could be accessed by the loyalty reporting and analytics team that needed to use the scores and recommendations and work on the overall process flow. Counter application Developed a responsive big data analytics platform that enabled campaign marketing teams to do interactive visualization of data for analysis and summarize trending insights in order to provide relevant offers to loyalty program members. Population planner Provided an application that streamlined marketing processes by tying marketing campaign information to a specific card member. This automated closed loop allowed the team to learn customer behavior and target specific card member segments with intelligent marketing campaigns. THE RESULTS Zaloni’s custom-built application enabled the credit card issuer to launch an effective, multi-vendor customer loyalty program that increased customer engagement, boosted revenue and nurtured loyalty over the long-term. The solution also facilitated data governance, data access, data integration and comprehensive analytics. • ENHANCED 360˚customer view • 1 MILLION+ customer sign-ups in three weeks • 22 MILLION+ customers are members of the customer loyalty program today • 1 MILLION hits/day on customer system
- Media and entertainment companies are using data to create and deliver more targeted content for customers, as well as increase operational efficiency, with the end goal of generating more revenue and increasing profitability. The customer needed to shift towards becoming a more data-driven organization at all levels of the business. THE CHALLENGE The company wanted to give its data users, including C-suite executives, business analysts, data analysts, data scientists and “data enthusiasts” self-service access to its enterprise-wide data. The intent was to enable faster time to insight into some of its metrics such as volume, quality, and business KPIs. ZALONI’S SOLUTION Zaloni dug in to understand the media company’s datasets in order to customize the data ingestion and processing framework. Zaloni put several Hadoop-based optimizations and data quality checks in place to ensure these processes could meet service-level agreements (SLAs), particularly for spikes in data volumes expected during certain media events. In addition, future enhancements included connectors to non-HDFS databases, sandbox provisioning and analytics home page enhancements. Specifically, the solution provided: Enterprise-wide data catalog Zaloni’s self-service data preparation platform, Mica, provided end users with information on enterprise-wide data catalogs and made it easy to self-service and search for datasets and refine, transform, enrich and extract data. Mica leveraged Bedrock for data organization, metadata management and operationalization of the execution of logic on the Hadoop cluster. Data governance Facilitated data governance with metadata management that tracked data lineage, data profiling and data quality using Bedrock. While all the customization was done, Bedrock kept track of all the operational metadata for better governance. Data security and access Leveraged Hadoop’s ACL feature to ensure only valid users had access to sensitive datasets. Also, Bedrock obfuscated and tokenized all personally identifiable information (PII), making the data available for reporting and analytics. Further, close integration with the customer’s enterprise active directory (AD) enabled data access governance and security on even the workflows that ran on the cluster. THE RESULTS The customer has successfully migrated more than 20 of its critical datasets into the data lake. Further, the Mica self-service platform has empowered the analytics and business intelligence teams, formerly dependent on IT or support teams, to create and prepare the data themselves. • FASTER TIME TO INSIGHT via direct business access to data • Improved ingestion speeds for 20 critical data sets