Ce diaporama a bien été signalé.
Le téléchargement de votre SlideShare est en cours. ×

Agile Mumbai 2022 - Balvinder Kaur & Sushant Joshi | Real-Time Insights and AI for better Products, Customer experience and Resilient Platform

Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
Publicité
Publicité

Consultez-les par la suite

1 sur 27 Publicité

Agile Mumbai 2022 - Balvinder Kaur & Sushant Joshi | Real-Time Insights and AI for better Products, Customer experience and Resilient Platform

Agile Mumbai 2022

Real-Time Insights and AI for better Products, Customer experience and Resilient Platform

Balvinder Kaur
Principal Consultant, Thoughtworks

Sushant Joshi
Product Manager, Thoughtworks

Agile Mumbai 2022

Real-Time Insights and AI for better Products, Customer experience and Resilient Platform

Balvinder Kaur
Principal Consultant, Thoughtworks

Sushant Joshi
Product Manager, Thoughtworks

Publicité
Publicité

Plus De Contenu Connexe

Plus par AgileNetwork (20)

Plus récents (20)

Publicité

Agile Mumbai 2022 - Balvinder Kaur & Sushant Joshi | Real-Time Insights and AI for better Products, Customer experience and Resilient Platform

  1. 1. © 2021 Thoughtworks | Confidential Real time insights and AI for better products, customer experience and resilient platform Balvinder Khurana and Sushant Joshi www.agilemumbai.com
  2. 2. Who we are 2 Balvinder Khurana Principal Consultant Data Architect and Global data community lead Sushant Joshi Product Principal @sushantjoshi https://sushant-joshi.medium.com Balvinder has 15 years of experience in building large- scale custom software and big data platform solutions for complicated client problems. She has extensive experience in Analysis, Design, Architecture, and Development of Web based Enterprise systems and Analytical systems using Agile practices like Scrum and XP. Balvinder currently works as a Data Architect and Global Data Community Lead for Thoughtworks Sushant is a Product Principal at ThoughtWorks. His work includes working with clients to assess product-market-fit, create goal aligned roadmaps and product delivery. He brings in his ever-curious mindset, business knowledge, and interdisciplinary thinking to solve problems that form our surroundings. His primary focus area is - product discovery - through which he helps address key product risks in the early stages of the product Sushant is passionate about Indian digital ecosystems. He is working with Indian companies to create better products.
  3. 3. Start your day with the your business dashboard! 3
  4. 4. Start your day with the your business dashboard! 4
  5. 5. Problem Landscape 5 Retail Bank KYC - Compliance to Customer Service The Ask
  6. 6. Few examples: How much amount was disbursed yesterday in Mumbai? How many car loans were sanctioned this week? What’s the health of APIs and underlying systems for last 3 hours? What are we trying to achieve 6 “Data intelligence delivering business value” What are the reasons for drop offs? Is it system or user? Are my APIs overloaded? Average time for disbursement Opportunity loss in the Funnel at different stages Login Offers Risk Checks Sanction 2FA Disbursement Which offers are attractive? How many customers are moving further after viewing offers How long it takes to send the OTP and customer action? Do users need alternate mechanism?
  7. 7. Explosion of personas and explosion of requirements 7 Revenue Generated Price Sensitivity Number of Users Customer micro- Segment Business Deployment status Load on a service Downtime for Service Developers Service Availability Service Traceability Routing Security team Insights What can I understand? Data Scientists Customer 360 Customer Propensity Customer-product fit Customer facing executives 7
  8. 8. 8 Isolated Solutions Web application Mobile application User click stream Social media Market data System level metrics Customer support Competitor data Logs Real time monitoring of all infrastructure components and service issues monitored using Prometheus and charts created on Grafana. Many other tools like EFK stack, kafka etc. are used. Developers/ IT Support Understand system health and avoid failures Periodic data is provided (sometimes manually) after pulling out of tools like Kafka, GTM in form of excel. Product Owners No drop-outs, all journeys should be completed Business/ C-level execs Data pulled out on-demand (manually) and shared via email/excel sheets. How is my business performing, where are the leakages
  9. 9. 9 Business ● Siloed ○ Systems ○ Tools ● Different ○ Targets ○ Maturity ○ Objectives People & Process ● Dependent ○ On a central data team ● Manual effort ○ Lack of standard processes ○ Duplicate effort ● Different ○ Tech Stack, Architecture & tools ○ Data granularity, formats & architectures ○ Data exploration Scopes ● Siloed ○ Data (Storage) ○ Business Units Data & Technology Limitations and pain points
  10. 10. 10 ● Consolidation ● Coordination ● Manual synthesis ● Low confidence Piecemeal Solutions
  11. 11. 11 Shift in mindset Solutions of Yesterday created the problems of Today Photo by Garidy Sanders on Unsplash
  12. 12. 12 Persona and business domains, wants/requirement s Levers of mindset shift Traditionally data is looked from the weekly or for monthly leadership review. So someone requests and presents what needs to be presented From requests to predefined datasets serving insights & exploration Design, Data discoverability & presentability Product Mindset 12 © 2021 Thoughtworks
  13. 13. Bringing it all together - The “Superdata” Solution 13
  14. 14. 14 14 What it is really? Make it quick and easy to explore a hypothesis (business or technical), accept or disprove it, and move on to find the root cause. Platform which enables people to use their skills, extend their senses, support their intuitions. The Superdata solution a.k.a Command center
  15. 15. 15 Data Platform Cloud DW Data Lake (Cloud Object Storage) arts Data Marts ODS Tech Solution Dashboards Data Service API’s Reports Data Ingestion & Integration Batch Ingestion Unstructured Source Ingestion API Ingestion Streaming Ingestion Orchestration Service 15 Source Systems Bank applications data Tele channels data Physical data from Branches Social media data Partners data 15 15 DevOps/DataOps Data Governance Data Catalog Data Quality Security Business Events Data Transformation & Ml ELT Stream Processing ML Toolkits* Deep Learning* ETL *Future Scope
  16. 16. ©ThoughtWorks 2021 Commercial in Confidence SOURCE SYSTEMS Tele channels data Physical data from branches Social media data Partners data Bank applications data Whitelisted, one way hashed data CONFIGURABLE SECURITY LAYERS Data filter service GEMALTO CCKM GEMALTO HSM TOKEN LOOKUP STORE DATA INGESTION / INTEGRATION ON LAYER DATA TRANSFORMATION LAYER DATA CONSUMPTION LAYER Cloud DW Data Lake (Cloud Object Storage) arts Data Marts ODS ODS Cloud Native Encryption at Storage DATA PLATFORM DATA INGESTION / INTEGRATION ON LAYER DATA TRANSFORMATION LAYER DATA CONSUMPTION LAYER PII / SPDI Layer (Only for PII fields) Data Encryption Component Data Decryption Component Tokenization Layer (Only for PII fields) Tokenizer De Tokenizer Gemalto Virtual HSM KMS VPC Security in Layers
  17. 17. Data Quality Framework Intermediate Data Quality Fit for Purpose Data Quality Ad-hoc Analysis Data Discovery Metadata Service Repository & Indexing Service Ownership of DQ Fit for Purpose Data Quality Purpose Fit for Purpose Data Quality Purpose Baseline Data Quality / Sensible Defaults Metrics Definition Rules Authoring Rules Execution Engine KPIs and Dashboards Metrics Definition Continuous data quality improvement*
  18. 18. Dashboards Data Service API’s Reports 18 Tech Stack Data Platform Cloud DW Data Lake (Cloud Object Storage) Data Marts ODS Data Ingestion & Integration Batch Ingestion Unstructured Source Ingestion API Ingestion Streaming Ingestion Orchestration Service Data Transformation & Ml ELT Stream Processing ML Toolkits Deep Learning ETL Source Systems Bank Applications Data Tele Channels Data Physical Store Data Social Media Data Partners data DevOps/DataOps Data Governance Data Catalog Data Quality Security Business Events
  19. 19. 19 19 Data Platform Cloud DW Data Lake (Cloud Object Storage) arts Data Marts ODS Dashbo ards Data Service API’s Report s Data Ingestion & Integration Batch Ingestion Unstructured Source Ingestion API Ingestion Streaming Ingestion Orchestration Service Data Transformation & Ml ELT Stream Processi ng ML Toolkits Deep Learning ETL Source Systems Bank Applications Data Tele Channels Data Physical Store Data Social Media Data Partners data DevOps/DataOps Data Governance Data Catalog Data Quality Security Business Events Serve Data as a Product Auto Loan/Cam paign Customer Personal Loan /Network analysis Credit Card/Finan ce Social Media Customer Profile Domain driven data boundaries The boundaries cut across the platform - from source to consumption!
  20. 20. Principles guiding building blocks of Data Mesh 20 Domain ownership Data as a product Self-serve data infrastructure Federated computational governance {G} {G} {G}
  21. 21. 21 Use cases served through platform ● Increased self-service ● Anomaly detection and alerting ● Personalization and Nudges ● Domain driven boundaries for data ● Adaptive journey completions
  22. 22. Impact and learnings 22
  23. 23. 23 Business ● Responsive ○ Real Time ○ Self service ○ Responding to customer behavior quicker ● Insights ○ Stimulating ○ Proactive Process & People ● Transparent ○ Democratization ○ Standardized processes ○ Data driven process planning ● Empowered ○ Own, create and analyse ○ Touch multiple business aspects ● Governance & Quality ○ Accurate ○ Secure ● Technology ○ Resilient ○ Rapidly evolving ○ Loosely coupled ○ Configuration driven Data & Technology Impact “Can we ask for this data” to “Can we pull up this data” “we are not able to track numbers, since last couple of hours dashboard is showing the same numbers”
  24. 24. 24 Our Learning ● Data-as-a-first-class-citizen ● Extend domain boundaries to data platform ● Avoid tight coupling with upstream and downstream ● Timely scaling considerations
  25. 25. Thank you Sushant Joshi Twitter: @sushantjoshi https://sushant-joshi.medium.com Balvinder Khurana LinkedIn: https://www.linkedin.com/in/balvinder-khurana/ https://khuranabalvinder.medium.com/ 25 www.agilemumbai.com
  26. 26. Size, Scale and Complexity 26 Web & Apps Tele channels Physical stores Digital Ecosystem partners Kiosks, agents etc Sales & marketing operations Products Technology Risk & compliance Strategy Enterprise functions Social media Retail Loans Retail Deposits ... Regulators Investors Aggregators Partners Product based Customer segment based Channel Corporate Customer risk Fraud Underwriting Liquidity HR Employee Facilities Legal How customers interact with the organisation Constraining and driving forces
  27. 27. Data Product Data Platform Architecture Quantum Fundamental unit of architecture Self-Serve Data Product Domain Polyglot Data Output Ports Polyglot Data Input Ports Discoverable Addressable Self-describing Trustworthy Interoperable governed by global open standard Secure enforce globally configured access control at each data product output port Control Ports Stats Logs, metrics Self discovery Management Defined and Monitored silos 27

Notes de l'éditeur

  • Sushant & Balvinder

    1 min : Intro : Slide 4 n
    4 min : Business context : Slide 5 to 8 : quick passthrough
    5 min : Explain the situation and mindset shift : slide 8 to 12
    2 min : slide for mindset shift : Slide 13 & 14
    1 min : ask and summersing the approach in our own words : slide 17
    10 mins min : solution + tech + security + quality + tech stack upto Slide 22
    2 min : domain data product : Slide 23
    2 min : Impact on people and processes : slide 21
    2 min : Impact and learnings Closure
    29 min / 35 min
    10 min : Q&A
  • Sushant
    Every executive likes information at their fingertips. In the form which will help them do real time probing and take decisions in time.
    We hear this advice from everyone, know where you stand.
    They look for Actionable Insights available real time rather than monthly or periodic reports
  • Sushant
    Imagine you needing to catch a flight, first thing you would want to know is how long it will take you to reach airport, traffic is unpredictable.
    They look for Actionable Insights available real time rather than monthly or periodic reports
  • Sushant & Balvinder (Techview of The Ask)

    THE BANK and the operating environment
    Pre Covid days - Digital lending is on everyone’s agenda / Some are exploring , toying with the idea
    Situation at the banking world
    Engagement is a problem and Fintechs are vying for the pie
    Payment infrastructure is coming to an age / wallets
    Customer engagement is coming at the center of the strategy
    KYC Resulting into
    KYC not for compliance but for acquiring, retaining and serving right

    THE ASK
    Define data strategy and roadmap for a data platform on cloud
    Self-service data platform which can onboard multiple products and systems in future

    To know the customer you need to know your systems wells


    CLIENT BACKGROUND
    Leading Bank in India who had embarked on the ambitious digital journey to bring in all retail loans under one roof, provide better customer experience and eliminate waste in the proces
    This was also the time startups have started making inroads in banks’ business quite well.
    GOAL
    Unified and clear view of customer actions
    Self-service insights into customer and system behavior in real-time (at scale) to
    Identify value pockets through behaviour based segments
    Create business ecosystem to power real-time offers based on data insights
    Provide clear view of business for timely actions and course corrections to optimize identified metric such as Risk, Account Profitability

  • Sushant

    Actionable Insights through Real Time, Self Serviced Information of everything that’s happening on the platform
    WHAT
    Generalise.
    We are directly jumping on the domain oriented
  • Parameters - Data democratization, self-service


    Sushant

    WHO
    Double click on objective through lenses of stakeholders

    Balvinder will come on this slide
  • Balvinder
    HOW
    Because of all the limitations mentioned earlier, each stakeholder group started attempting to solve their problems individually
    Organisation goal was not aligned and individuals were opting for solutions which would make sense to them and feasible with in the limited resources they have.


    The intent (and hence call to action), granularity and scope of information needed is different. So not just the tools, but also the data that is consumed - is isolated

    For each group:
    Stakeholder - ask - solution - data

    Add data stakeholder group
  • Balvinder
    But it was not easy to reach to answers to the questions for each stakeholder group. There are so many hurdles on the way.
    Business
    Siloed and fragmented systems
    Different targets and no common agreed goals for building data world view
    Varying maturity of business and tech orgs
    Who is focussed on Customer happiness
    Do we want 99.99% availability but still pissed off customer

    Data
    Siloed and locked useful data into various tools and owned business orgs operating in silos
    Disparate data sources
    Difficult to Correlate Quickly for Monitoring or finding Business relevance
    Limited scope for exploration
    Unified architectures (of consuming platform) are not possible
    Technology
    Competing or non-compatible tech stacks
    Learn individual tools
  • Balvinder
    Human/emotional dimension
    What happens because of the frustration

    What got you here wont take you far - change in the mindset is needed

    Consolidated insights was still a problem
    It needed co-ordinating for information availability and then synthesis by someone who may not have the best of understanding of how that information is collected
    Patchy solution for a group of people was still serving only limited section and broad based acceptance and hence data availability was a challenge
  • Sushant
    https://unsplash.com/photos/-X1CDIau79o
  • Sushant
  • Sushant & Balvinder
  • Balvinder
    Inline with regulated environment
    Scalability
    Configurability
    Self-service consumption
    Security
    Banking and regulated world
    Financial data
    Approach to security
    Data Quality
  • Data Quality framework

    hierarchical data quality framework from the perspective of data users. This framework consists of big data quality dimensions, quality characteristics, and quality indexes
    ROI of data quality
    Define, Measure, Analyze, Design/Improve, and Verify/Control

  • Balvinder
    Inline with regulated environment
    Scalability
    Configurability
    Self-service consumption
    Security
    Banking and regulated world
    Financial data
    Approach to security
  • Balvinder
    Bootstrap the platform so easily, scale to the sources and scale to the consumers and kept on incrementally materializing data driven value which was differentiating
    Decompose data products around domains, distribute the ownership. The principle we have been applying to web services world to create microservices.
  • Balvinder & sushant
    A decentralised socio technical approach in managing and accessing analytical data at scale
    Getting value from data at scale, in complex organization in an environment thats constantly changing, looks at both the organizational responsibilities and the architecture

    Domain Ownership
    The genesis of the first principle is pushing towards the Domain expertise continuum. Domain teams to manage and own not only their operational data but also the analytical data.
    “No longer a by-product of that domain but an first class product of that domain” which they share with other domains to enable the value driven outcomes for the organization
    So that the ecosystem creating and consuming data can scale out as the number of sources of data, number of use cases, and diversity of access models to the data increases; simply increase the autonomous nodes on the mesh.

    Data as product
    For a distributed data platform to be successful, domain data teams must apply product thinking with similar rigor to the datasets that they provide; considering their data assets as their products and the rest of the organization's data scientists, ML and data engineers as their customers.
    So that data users can easily discover, understand and securely use high quality data with a delightful experience; data that is distributed across many domains.


    Self serve infrastructure
    So that the Portfolio teams can create and consume data products autonomously using the platform abstractions, hiding the complexity of building, executing and maintaining secure and interoperable data products.
    Infrastructure is centrally managed, yet it is provisioned per data product to support its autonomous operation in a multi-tenancy fashion. It’s important that deployment or update of one data product doesn’t impact other data products, from an infrastructure perspective.

    Federated computational governance
    So that data users can get value from aggregation and correlation of independent data products - the mesh is behaving as an ecosystem following global interoperability standards; standards that are baked computationally into the platform.
  • Sushant

    Slide 1- 7 : 8-9 mins
    Slide 8-10: 5-6 mins
    Slide 11-14: 3-4 mins
    Slide 15-20: 7 mins
    Slide 21-22: 3 min
    QnA : 15 mins
  • Sushant(business, process & people) and Balvinder (Data & tech)
    How engineering and data platforms can be used to derive real-time business and system insights that help in proactive decisions (Data intelligence delivering business value).

    Approach towards creating self-service data analysis and visualization platform (Impact of Data Intelligence in Software Development Life Cycle).

    Artificial intelligence can help you respond to customer behavior quicker. (Automated intelligence delivering business value).


    Understand how holistic data view can help into multiple aspects of business - operation, process, and delivery (Data intelligence delivering business value).

  • Sushant
    Data has a better Idea
  • Sushant
    How does a large enterprise look like
    General picture
    Thought process
    Complexity of channels X complexity of Products
    Each product has it’s own way of selling and operations
    Customer segments also need specific handling such as HNI, premium, priority sector etc
    Each product type takes it’s own shape in terms of strategy, risks and balance

    Large enterprise have multi speed departments, that dictates inherent need for different systems and customised processes for suitability.
    This leads to each business group optimising people, tech and processes based on their goals
    Which results in
    Silos
    Fragmented systems
    Disparate data sources owned business orgs (operating in silos)
    Competing tech stacks // non compatible tech stacks
    Different targets and no common agreed goals for building data world view
    Varying maturity of business and tech orgs




    Siloed attempts to solve the challenges made it a further big crises as big picture was missing

    No one talked about it explicitly and what it means for the data solution
  • Explain and lead to definition - Six dimensions of data product

×