Recommended
Recommended
More Related Content
Similar to Mike keating - News Int - 18th BDL meetup
Similar to Mike keating - News Int - 18th BDL meetup (20)
Recently uploaded
Recently uploaded (20)
Mike keating - News Int - 18th BDL meetup
- 1. Big Data at News International! Welcome ! Big Data at News International! ! Big Data Event - 29/05/2013! Mike Keating: Product Owner! @mikerkeating! Slide 01/14!
- 2. Big Data at News International! Introduction! • Where to start! • Data and decisions! • Our technology choices! • Lessons! ! Big Data Event - 29/05/2013!Slide 02/14!
- 3. Big Data at News International! Where to start…! Big Data Event - 29/05/2013!Slide 03/14! Lots of data, suppliers, technology and teams! ! Take control, bring digital data into one place! Link data sets and look for new approaches! ! Make data/ information/ knowledge/ insight available! Build awareness and decision making capability!
- 4. Big Data at News International! Making the basics available via dashboards! Big Data Event - 29/05/2013!Slide 04/14!
- 5. Big Data at News International! Understanding our content and consumption! Big Data Event - 29/05/2013!Slide 07/14! Visits per Visitor ViewsperVisit Behaviours Across Website Sections
- 6. Big Data at News International! Understanding our content in a social world! Big Data Event - 29/05/2013!Slide 08/14!
- 7. Big Data at News International! Tracking subscrip.on growth across products! Big Data Event - 29/05/2013!Slide 05/14! Days Following First Subscription Subscriptions Product Growth Following Launch
- 8. Big Data at News International! Analysing attributes that indicate churn! Big Data Event - 29/05/2013!Slide 06/14!
- 9. Big Data at News International! Designing products based on patterns in navigation ! The iPhone – Octopus Naviga4on The Website – Flower Petal Naviga4on The iPad Edi4on -‐ Linear Naviga4on Big Data Event - 29/05/2013!Slide 10/14!
- 10. Big Data at News International! Our technology choices – what?! Big Data Event - 29/05/2013!Slide 11/14! Infrastructure: AWS EC2, S3, RDS, EMR, Cloudformation, Vagrant! Ops: Jenkins, Anthill Pro, Maven, Nexus, Zabbix, CloudWatch! Code & Config: Puppet, Github! Data Retrieval: Java & Python! Data Pipeline: Java Map Reduce, Apache Crunch, ! Spark, MRUnit, Celery, RabbitMQ, Python, Flume! Data Storage: HDFS, HBase, AWS S3, MySQL, Redis! Data Schema: Avro! Data Access: Python APIs, Tornado, S3, Hive & AWS EMR! Analysis: R, Pandas, Excel -> analyst’s choice! Visualisation: R, D3, Highcharts, Google Charts! Products: APIs, JS+HTML+CSS!
- 11. Big Data at News International! Our technology choices – why?! Big Data Event - 29/05/2013!Slide 11/14! • Team: Build on team’s skillsets and knowledge. ! • Recruitment: Be conscious of “hire-ability”! • Open Source: Big wins from usage; great communities; contribute back! • Versions: use what works; work with alpha releases but not as production code! • Consistency: Try and use what you do today, e.g. AWS! • Flexibility: Use a “better” product where it makes sense!
- 12. Big Data at News International! Our technology choices – who?! Big Data Event - 29/05/2013!Slide 11/14! • Tech Lead: Architecture; Design; Hands-On! • Delivery Manager: Project Management w Agile! • Hadoop: build from Java Map Reduce ! • Python: Tornado, Native Python, real-time processing! • Data Science: Hive, R, Modeling! • DevOps: AWS, Vagrant, Puppet! • Experience: Practical experience of Hadoop in production! • Capability: Ability to learn new tech, design and build! • Demo: contributions to projects, working examples!
- 13. Big Data at News International! Lessons! Big Data Event - 29/05/2013!Slide 12/14! • Building awareness & common knowledge! • Building on existing teams, systems and their work! • Looking for extra capability and output! • Focus on visuals – it needs to be sharable/ visible! • Working with range of teams to share outputs! • Making good tech choices!
- 14. Big Data at News International! Thanks! Big Data Event - 29/05/2013!Slide 13/14! Big Data Team – DevOps, Hadoop, Python, UI, Analysts, Test! ! Technology Teams – Design, Production Ops, Perf Testing, Security, Products, Platforms, Service Desk! ! Editorial, Marketing, Commercial, Finance Teams!
- 15. Big Data at News International! ! ! Thanks! Big Data at News International Contact Us News Interna.onal Technology @techatni Mike Kea.ng; Product Owner @mikerkea.ng Jobs via hBp://joinnitech.co.uk/ Big Data Event - 29/05/2013!Slide 14/14!