Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
1© 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved |
1© 2020 Amazon Web Services, Inc. or its affili...
2© 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved |
Predictive
Maintenance
Manufacturing,
Automotiv...
3© 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved |
Bias and Explainability: Challenges
1 Without d...
4© 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved |
Amazon
SageMaker
Clarify
Detect bias in ML mode...
5© 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved |
SageMaker Clarify works across the ML lifecycle...
6© 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved |
How SageMaker Clarify works
Amazon SageMaker
Cl...
7© 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved |
SageMaker Clarify – Detect Bias During Data Pre...
8© 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved |
SageMaker Clarify – Check Your Trained Model fo...
9© 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved |
SageMaker Clarify – Monitor Your Model for Bias...
10© 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved |
SageMaker Clarify – Understand Your Model
Mode...
11© 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved |
SageMaker Clarify – Monitor Your Model for Dri...
12© 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved |
Demo: https://youtu.be/cQo2ew0DQw0
13© 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved |
SageMaker Clarify Use Cases
Regulatory
Complia...
14© 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved |
SageMaker Clarify – Pricing & Availability
Sag...
15© 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved |
Best Practices
• Fairness as a Process:
• The ...
16© 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved |
Fairness and Explainability by Design in the M...
17© 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved |
Thank You!
For more information on Amazon Sage...
Prochain SlideShare
Chargement dans…5
×

Amazon SageMaker Clarify

Amazon SageMaker Clarify (https://aws.amazon.com/sagemaker/clarify/) provides machine learning developers with greater visibility into their training data and models so they can identify and limit bias and explain predictions. SageMaker Clarify detects potential bias during data preparation, after model training, and in your deployed model by examining attributes you specify. For instance, you can check for bias related to age in your initial dataset or in your trained model and receive a detailed report that quantifies different types of possible bias. SageMaker Clarify also includes feature importance graphs that help you explain model predictions and produces reports which can be used to support internal presentations or to identify issues with your model that you can take steps to correct.

For more information on Amazon SageMaker Clarify, please refer these links: (1) https://aws.amazon.com/sagemaker/clarify (2) https://aws.amazon.com/blogs/aws/new-amazon-sagemaker-clarify-detects-bias-and-increases-the-transparency-of-machine-learning-models (3) https://github.com/aws/amazon-sagemaker-clarify (4) Discussion and demo: https://youtu.be/cQo2ew0DQw0

Acknowledgments: Amazon SageMaker Clarify core team, Amazon AWS AI team, and partners across Amazon

Livres associés

Gratuit avec un essai de 30 jours de Scribd

Tout voir
  • Soyez le premier à commenter

Amazon SageMaker Clarify

  1. 1. 1© 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | 1© 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | Detect bias in ML models and understand model predictions Krishnaram Kenthapadi Principal Scientist, Amazon AWS AI Amazon SageMaker Clarify
  2. 2. 2© 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | Predictive Maintenance Manufacturing, Automotive, IoT Demand Forecasting Retail, Consumer Goods, Manufacturing Fraud Detection Financial Services, Online Retail Credit Risk Prediction Financial Services, Retail Extract and Analyze Data from Documents Healthcare, Legal, Media/Ent, Education Computer Vision Healthcare, Pharma, Manufacturing Autonomous Driving Automotive, Transportation Personalized Recommendations Media & Entertainment, Retail, Education Churn Prediction Retail, Education, Software & Internet https://aws.amazon.c om/sagemaker/gettin g-started Amazon SageMaker Customer ML Use cases
  3. 3. 3© 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | Bias and Explainability: Challenges 1 Without detection, it is hard to know if bias has entered an ML model: • Imbalances may be present in the initial dataset • Bias may develop during training • Bias may develop over time after model deployment 2 Machine learning models are often complex & opaque, making explainability critical: • Regulations may require companies to be able to explain model predictions • Internal stakeholders and customers may need explanations for model behavior • Data science teams can improve models if they understand model behavior
  4. 4. 4© 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | Amazon SageMaker Clarify Detect bias in ML models and understand model predictions Detect bias during data preparation Identify imbalances in data Evaluate the degree to which various types of bias are present in your model Check your trained model for bias Understand the relative importance of each feature to your model’s behavior Explain overall model behavior Understand the relative importance of each feature for individual inferences Explain individual predictions Provide alerts and detect drift over time due to changing real-world conditions Detect drift in bias and model behavior over time Generated automated reports Produce reports on bias and explanations to support internal presentations
  5. 5. 5© 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | SageMaker Clarify works across the ML lifecycle Collect and prepare training data Train and tune model Evaluate and qualify model Deploy model in production Monitor model in production Measure Bias Metrics Measure and Tune Bias Metrics Measure Explainability Metrics Catalog Model Metrics Measure Bias Metrics Measure Explainability Metrics Monitor Bias Metric Drift Monitor Explainability Drift SageMaker Data Wrangler SageMaker Training Autopilot Hyperparameter Tuning SageMaker Processing SageMaker Hosting SageMaker Model Monitor
  6. 6. 6© 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | How SageMaker Clarify works Amazon SageMaker Clarify
  7. 7. 7© 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | SageMaker Clarify – Detect Bias During Data Preparation Bias report in SageMaker Data Wrangler
  8. 8. 8© 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | SageMaker Clarify – Check Your Trained Model for Bias Bias report in SageMaker Experiments
  9. 9. 9© 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | SageMaker Clarify – Monitor Your Model for Bias Drift Bias Drift in SageMaker Model Monitor
  10. 10. 10© 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | SageMaker Clarify – Understand Your Model Model Explanation in SageMaker Experiments
  11. 11. 11© 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | SageMaker Clarify – Monitor Your Model for Drift in Behavior Explainability Drift in SageMaker Model Monitor
  12. 12. 12© 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | Demo: https://youtu.be/cQo2ew0DQw0
  13. 13. 13© 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | SageMaker Clarify Use Cases Regulatory Compliance Internal Reporting Operational Excellence Customer Service
  14. 14. 14© 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | SageMaker Clarify – Pricing & Availability SageMaker Clarify is generally available SageMaker Clarify is available at no additional cost as part of Amazon SageMaker SageMaker Clarify is available in all AWS Regions where SageMaker is available
  15. 15. 15© 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | Best Practices • Fairness as a Process: • The notions of bias and fairness are highly application dependent and the choice of the attribute(s) for which bias is to be measured, as well as the choice of the bias metrics, may need to be guided by social, legal, and other non- technical considerations. • Building consensus and achieving collaboration across key stakeholders (such as product, policy, legal, engineering, and AI/ML teams, as well as end users and communities) is a prerequisite for the successful adoption of fairness-aware ML approaches in practice. • Fairness and explainability considerations may be applicable during each stage of the ML lifecycle.
  16. 16. 16© 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | Fairness and Explainability by Design in the ML Lifecycle
  17. 17. 17© 2020 Amazon Web Services, Inc. or its affiliates. All rights reserved | Thank You! For more information on Amazon SageMaker Clarify, please refer: • https://aws.amazon.com/sagemaker/clarify • https://aws.amazon.com/blogs/aws/new-amazon-sagemaker-clarify-detects- bias-and-increases-the-transparency-of-machine-learning-models • https://github.com/aws/amazon-sagemaker-clarify Acknowledgments: Amazon SageMaker Clarify core team, Amazon AWS AI team, and partners across Amazon

×