Analyzing Billions of Data Rows with Alteryx, Amazon Redshift, and Tableau
•
5 j'aime•4,177 vues
This document discusses amaysim's implementation of Amazon Redshift, Alteryx, and Tableau for data analytics. It provides an overview of each tool and how amaysim uses them together in their business intelligence stack. Key points include: - Amaysim uses Redshift for data warehousing, Alteryx to prepare and blend data, and Tableau for visualization and self-service analytics. This allows for analysis within hours rather than weeks. - With a small analytics team, the tools empower line of business users to solve their own problems quickly. This increases workforce productivity. - Lessons learned include democratizing analytics, making tools relevant to different stakeholders, and celebrating successes to drive cultural
Recommandé
Recommandé
Contenu connexe
Tendances
Tendances (20)
En vedette
En vedette (20)
Similaire à Analyzing Billions of Data Rows with Alteryx, Amazon Redshift, and Tableau
Similaire à Analyzing Billions of Data Rows with Alteryx, Amazon Redshift, and Tableau (20)
Plus de DATAVERSITY
Plus de DATAVERSITY (20)
Dernier
Dernier (20)
Analyzing Billions of Data Rows with Alteryx, Amazon Redshift, and Tableau
- 1. Analyzing billions of rows with Amazon Redshift,Alteryx and Tableau May 3, 2016
- 2. Today’s presenters Adrian Loong National Manager, RXP Services, Analytics Consultant & Alteryx ACE Brandon Chavis Solutions Architect, Amazon Web Services Raman Kaler Alliance Marketing Manger, Alteryx
- 3. Download a FREE Trial: www.alteryx.com/alteryxfortableau© 2016 Alteryx, Inc. | Confidential Alteryx - the leading platform for self-service data analytics Prep, blend, and analyze all your data using a repeatable workflow Deliver deeper insights in hours, not weeks Deploy and share analytics at scale
- 4. © 2016 Alteryx, Inc. | Confidential Download a FREE Trial: www.alteryx.com/alteryxfortableau Share The leading platform for self-service data analytics Enrich Output All Popular Formats Prep & Blend Analyze Input All Relevant Data
- 5. © 2016 Alteryx, Inc. | Confidential Download a FREE Trial: alteryx.com/trial5 Emerging Market (>$1B) Fast Moving (>50% growth) Business User Focused Data Prep & Blending Tiny (<$100M) Middling Growth Niche, Open Source Data Science Operational ETL Large Market (>$5B) Slow Moving (<5% growth) IT Driven Three Data Integration Markets
- 6. Relational data warehouse Massively parallel; Petabyte scale Fully managed HDD and SSD Platforms $1,000/TB/Year; starts at $0.25/hour Amazon Redshift a lot faster a lot simpler a lot cheaper What is Amazon Redshift?
- 7. Amazon Redshift is easy to use • Provision in minutes • Monitor query performance • Point and click resize • Built in security • Automatic backups
- 8. Amazon Redshift architecture Leader Node Simple SQL end point Stores metadata Optimizes query plan Coordinates query execution Compute Nodes Local columnar storage Parallel/distributed execution of all queries, loads, backups, restores, resizes Start at just $0.25/hour, grow to 2 PB (compressed) DC1: SSD; scale from 160 GB to 326 TB DS2: HDD; scale from 2 TB to 2 PB Ingestion/Backup Backup Restore JDBC/ODBC 10 GigE (HPC)
- 9. The Amazon Redshift view of data warehousing • 10x cheaper • Easy to provision • Higher DBA productivity • 10x faster • No programming • Easily leverage BI tools, Hadoop, Machine Learning, Streaming • Analysis in-line with process flows • Pay as you go, grow as you need • Managed availability & DR Enterprise Big Data SaaS
- 10. The legacy view of data warehousing ... Global 2,000 companies Sell to central IT Multi-year commitment Multi-year deployments Multi-million dollar deals
- 11. … Leads to dark data This is a narrow view Small companies also have big data (mobile, social, gaming, adtech, IoT) Long cycles, high costs, administrative complexity all stifle innovation 0 200 400 600 800 1000 1200 Enterprise Data Data in Warehouse
- 12. Please note: All data used for this presentation is sample data only. It is used to demonstrate the capability of AWS, Alteryx and Tableau, and no actual amaysim data will be shared.
- 13. 1 2 3 4 Introduction Business challenges Best of breed BI stack - AWS Redshift, Alteryx & Tableau Learnings
- 14. amaysim is an Australian mobile virtual network operator established in November 2010. About me Adrian Loong – Business Intelligencemanager • Responsible for analytics strategy and execution across Finance, Retail Sales, Marketing and HR • Expertise: • Decision support / Management consulting • Visualisation, Financial analytics, Budgeting & Forecasting • CPA . About amaysim
- 15. Strategy - Data driven decisions in real time 1. Knowledge is power 2. Smart management = happy customers & happy shareholders 3. Get a real competitive edge - stop looking back and start predicting the future 4. A 3 year rolling forecast in real time – generate real company wealth by being reliable
- 16. How Amaysim has benefited from their analytics program Workforce Productivity • We have a 3 person analytics team covering wide span of functions (Finance, Marketing – Customer retention & acquisition, Sales – Retail and Online, Data warehousing, HR) • By enabling line of business users to quickly build on a baseline of analytics, they can easily solve their own specific business problems quickly and do not have to wait on Business intelligence teams Reduced time to insight • We are able to get the data we need faster. Projects that would have taken 2-3 weeks are now down to a day Data driven decision making • Line of business users are able to get direct access to their own data in an easy visualization enabling them to able to solve problems faster. • People can look at the data, do some discovery, and then arrive at an answer
- 17. 1 2 3 4 Introduction Business challenges Best of breed BI stack - AWS Redshift, Alteryx & Tableau Learnings
- 18. amaysim has a wide variety of data sources and a lot of data being generated daily Wide variety of data sources A lot of data is generated daily Source Systems Phone call Internet data traffic SMS Over 10 billion rows of data with nearly 20 million rows added daily Databases
- 19. Competition is intense in the mobile sector Major carriers Mobile Virtual Network Operators
- 20. We only have a small team covering a wide span of functions Retail Sales Finance Customer acquisition HR Photo Source: http://www.timeshighereducation.co.uk/news/business- schools-not-first-port-of-call-for-managerial-recruits/2013863.article Customer retention
- 21. Traditional tools are unable to keep up with speed and velocity of business demands, tools like redshift, tableau and alteryx enable analysis within hours Traditional BI : Keep calm and please wait Analysis at the speed of thought V S We need tools that empower us to analyse at the speed of thought
- 22. 1 2 3 4 Introduction Business challenges Best of breed BI stack - AWS Redshift, Alteryx & Tableau Learnings
- 23. Business intelligence stack Source Data IT ELT/ET L Data Warehouse Business ETL Visualizer & Reporting • ECC • CRM • CSC • Google Analytics • Sales POS • The list goes on… • Alteryx Desktop selected & deployed • Gives our business the agility we need • Business driven rather than IT driven • Ability to do predictive analytics • Tableau Desktop & Server selected and deployed • Self serve BI • Business dashboarding • Ability to do predictive analytics Time to load Reliability of Data Query performance Flexibility for slice/dice Visualization performance Data definitions / governance Data exploration • CDC used for real time data replication • Slower to build • More reliable • Redshift selected for reliability, processing power and scalability
- 24. Redshift provides the speed and robustness to store and analyze vast volumes of data. Alteryx fuels the Tableau visualisation allowing us to quickly gain insights in Tableau Source system 1 Livechat Source system 2 Zendesk Amazon Redshift Datamart External data / Marketing data Blend & enrich Visualization • Clean the data • Apply business rules • Validate business rules in teams 3 4 2 1 Comments Tableau can be used to directly visualise and analyse big data directly from Redshift 1 Alteryx can be used to blend data that isn't in the Redshift database 2 Alteryx can apply & validate more complex business rules before visualizing outputs in Tableau Continuously iterate between Alteryx and Tableau when discovering more data 4 2 3 1 3 4 4
- 25. Alteryx enables us to combine different data sources in a visual workflow before visualisation in Tableau Server Source systems Databases Servers Workstations
- 26. Tableau server Tableau dashboards Tableau is excellent for collaboration
- 27. 1 2 3 4 Introduction Business challenges Best of breed BI stack - AWS Redshift, Alteryx & Tableau Learnings
- 28. Democratize the opportunity Source: http://www.heynataliejean.com/2009/01/end-of-nerd-era.html Most business leaders want to use data to make better decisions.. • You don’t have to be an IT specialist to use Alteryx • Give users access to the tools, use Alteryx & Tableau to show them how much faster you can spot a trend • 1:1 Training sessions for different stakeholders tailored to their needs
- 29. Make it relevant to different stakeholders Senior management • More interested in insights, dashboards and outcomes • Spend time showing them dashboards built and how opportunities to improve business performance (eg: revenue generation, expense reduction). Functional specialists & leaders • Alteryx relevant to functional leaders • Alteryx to “clean” the data before visualizing in Tableau • How can it be used to streamline processes
- 30. Celebrate success : Keep building and Iterating Existing tools require analysts to have “coding” experience Few disparate analysts using SQL to deliver analytical solutions Where we are now Using ‘Best’ of Breed products • Visualization : Tableau • Data blending: Alteryx • Warehousing : Redshift (WIP) Using Alteryx & Tableau to deliver : • Revenue assurance • Sim Sales • Customer disconnections & churn • Port-outs by carrier • CDR analysis Where we will be Self serve analytics for all business users • Real time P&L with Slice and dice with visualization • Assurance over logic processing (Audit, Commissions) Advanced predictive analytics • Churn propensity • Customer behavior Where we started
- 31. 1. Democratize theopportunity 2. Makeitrelevanttodifferent stakeholders 3. Culturalchangetakestime
- 32. 32 Next steps: Get the Alteryx Starter Kit: www.alteryx.com/alteryxfortableau
Notes de l'éditeur
- Traditional data integration is dominated by a few large, legacy vendors. This IT-driven, operational ETL model promised to create a single version of the truth inside a data warehouse so that was the only place a data analyst ever had to go to get the data they need. But, our research and industry reports suggest this model only succeeded in a small fraction of companies. Buying for operational ETL today is only from companies who are trying to maintain this model at all costs – hence the slow growth of this segment. On the left we see a tiny market for data scientists who use complex programming languages like Perl, Python, and Pandas for data integration. These are powerful programming environments, but their steep learning curves inhibit use by data analysts who don’t have the time, or need, to invest in learning them. And finally, on the right you see the rapidly growing data prep and blending market led by Alteryx. Gartner correctly called out this market as the place where the majority of spending for business user data integration is taking place. Data analysts appreciate the ability of Alteryx to allow them to prep and blend whatever data they want to use in their analysis – not what another department thinks they need. And do it with an intuitive user interface that doesn’t require any coding. Once the data is prepped and blended, Alteryx users can use the same workflow for analytics – predictive, statistical, and spatial – and deliver deeper insights in hours, not the weeks that are required with the other approaches.
- For those unfamiliar with Amazon Redshift, it is a fast, fully managed, petabyte-scale data warehouse for less than $1000 per terabyte per year. fast, cost effective, easy to use (launch cluster in a few minutes, scale with the push of a button)
- Redshift is not only cheaper but also easy to use. Provisioning takes 15 minutes.
- Amaysim is Australia’s award winning low cost mobile provider operating on the Optus network. We are dedicated to delivering simplicity, fairness and low prices Currently amaysim is one of the largest mobile virtual network operator (MVNO) with over 700,000 customers and one of the few telcos in Australia with a Net promoter score in mid 50-60s. In additional we have also won money magazine's best of best awards for 2012 and 2013 As for me, my name is Adrian and I am the BI manager for amaysim. I’m responsible for the analytics strategy and execution across finance, retail sales, marketing and HR. My expertise includes decision support and consulting and visualization.
- At amaysim had a bold vision for BI.. We want to deliver insight which enables business leaders and users of all levels to make better data driven decisions in realtime It’s not about the size of data you have ; rather its about the ability to deliver continuous insight in a timely fashion. AWS, Alteryx and Tableau enable us to ask and answer questions in that continuous iterative flow which is so important. Every question leads to more questions. Delivering a dashboard is not the end rather it is the beginning with our Alteryx workflows and tableau dashboards continually being refined daily to help answer more questions. Most importantly we use AWS, Alteryx and Tableau to provide recommendations to improve business performance… We believe that the tools we have combined with our agility is what gives us that competitive edge
- Many of you will ask ? Why have an analytics program. Well we have seen benefits in 3 mains areas. 1. Workforce Productivity We have a 3 person analytics team covering wide span of functions (Finance, Marketing – Customer retention & acqusition, Sales – Retail and Online, HR) By enabling line of business users to quickly build on a baseline of analytics, they can easily solve their own specific business problems quickly and do not have to wait on Business intelligence teams 2. Reduced time to insight With alteryx, instead of gathering requirements, we instead ask a question on what is the business problem we are trying to solve. Alteryx enables us to get to the data we need faster. Projects that would have taken 2-3 weeks are now down to a day 3. Data driven decision making Line of business users are able to get direct access to their own data in an easy visualization enabling them to able to solve problems faster. People can look at the data, do some discovery, and then arrive at an answer
- Amaysim has a large variety of data sources, some of which have complicated data structures like XML and JSON We ingest over 20m rows of data, each time a phone call, SMS or internet traffic is sent a row of data is generated
- We exist in a competitive landscape where we compete for business against the 3 major carriers as well as a host of Mobile virtual network operators
- In addition, we only have a small team of 3 analysts servicing a wide variety of audience
- We exist in a competitive landscape where we compete for business against the 3 major carriers as well as a host of Mobile virtual network operators
- So the question is why do we need both IT ETL and data blending capabilities? The answer is really the speed and agility that alteryx gives us, it allows us to answer questions in minutes and hours rather than days of writing requirements. It enables anyone to be an analyst and is not limited to just the BI team; in doing so we lift the productivity of the entire company by enabling line of business analysts to ask and answer their own questions
- At amaysim we have adopted a best of breed solutions within BI from datawarehousing (redshift) to data blending (alteryx) to visualisation (tableau). Like many companies, we have many different data sources, both internal and external which the business needs to analyse at short notice and act on. Alteryx is a great tool which allows non-technical people to clean the data, apply complex rules, validate rules with stakeholders in an iterative manner before fuelling the tableau visualisation. Redshift which is an massively parallel database in the cloud gives us the speed and robustness to store and analyse vast volumes of data. Finally we use both Tableau desktop to slice and dice clean data and tableau server to really drive collboaration amongst business leaders.
- Here’s another look on our setup of how we pull in multiple data sources including complex XML and JSON combine it with data in our database using Alteryx before visualisation in Tableau
- Finally we use both Tableau server and Tableau desktop as a collaboration mechanism at Amaysim to deliver a variety of dashboards, data sources and insights to our business leaders Our internal tableau server has a few key areas Exec dashboard Finance reporting Marketing reporting Customer service reporting Retail sales IT operations I’ve covered the few areas on how tableau can be used with other best of breed products to enable business success.
- So how do you bring about cultural change in a business? This slide says it all.. How many business leaders look like that ?? Analytics is for everyone, not just IT or BI specialists! Remember that You don’t have to be an IT specialist to use tools like Alteryx and Tableau! Democraticise the opportunity.. Most business leaders want to combine analytics with industry experience to make better decisions. Often the best way to show how alteryx & tableau works is to sit down 1 on 1 with a business leader and show them how alteryx can used to combine data together before visualising in Tableau. You can often build a dashboard in under 10 minutes! That way you can test your ideas in a dynamic flowing manner
- Another important point to note when enabling business leaders is to understand and make tableau relevant for different stakeholders.. A person in senior management often has different needs to a functional specialist or leader Senior management who are more interested in insights and how it can be used drive improved P&L performance. Spend time with them understanding their needs and discuss if insights gleaned from tableau can result in improved revenue performance or expense reduction. (eg: revenue generation, expense reduction). Functional leaders however may be more interested in a narrower field for example how they can spot trends which improve operational performance eg: HR leaders want to see metrics around staff turnover, absenteeism, leave accruals Frontline leaders want to see metrics around how many calls are handled per hour, # livechats closed Sales may want to see metrics around how many sales occur in a particular region or store.
- The 3rd key ingredient of cultural change is to celebrate success. At amaysim where we started, business intelligence was certainly not scalable as it required analysts to have “coding” experience. There were a few disparate analysts using SQL to deliver analytical solutions. This way was certainly not scalable as business users were not empowered to analyze their business. Where we are today, we are using Alteryx & Tableau to deliver insights and not just dashboards to management and are proactively trying to find ways to improve financial performance. Each time we use Alteryx and Tableau to produce a new dashboard gets built, we don’t just send senior management a link, instead we take time to sit down one on one with management to explain what we are seeing in the dashboard and any trends (both positive and negative) which can be used to improve business performance.. This personalised approach is a key ingredient to success we have had. We often gather a group of cross functional leaders for half an hour to click through and see if we can find any trends.. Where we will be is to have self serve analytics for all business users. Realtime P&L with slice and dice capabilitites using Tableau data sources. We will be using advanced predictive analytics to model and mitigate churn analysing customer behaviour and helping them find best plans to suit their needs.
- I hope that this presentation will inspire you to build a culture of creative analytics which is both empowering and satisfying and brings out the best in people. Democratize the Opportunity - Gather people of interest together from Sales, product, marketing Make it relevant to different stakeholders - Include everyone & break down silos Celebrate success - Keep people involved and excited