More Related Content Similar to Amazon의 머신러닝 솔루션: Fraud Detection & Predictive Maintenance - 남궁영환 (AWS 데이터 사이언티스트 SA) (20) More from Amazon Web Services Korea (20) Amazon의 머신러닝 솔루션: Fraud Detection & Predictive Maintenance - 남궁영환 (AWS 데이터 사이언티스트 SA)1. AWS Online Series:
Data, Analytics, and ML Edition
Amazon의 머신러닝 솔루션:
Fraud Detection & Predictive Maintenance
남궁영환, AWS 데이터 사이언티스트 SA
2. 강연 중 질문하는 방법
Go to Webinar “Questions” 창에 자신이 질문한
내역이 표시됩니다. 기본적으로 모든 질문은
공개로 답변 됩니다만 본인만 답변을 받고 싶으면
(비공개)라고 하고 질문해 주시면 됩니다.
본 컨텐츠는 고객의 편의를 위해 AWS 서비스 설명을 위해 온라인 세미나용으로 별도로 제작, 제공된 것입니다. 만약 AWS
사이트와 컨텐츠 상에서 차이나 불일치가 있을 경우, AWS 사이트(aws.amazon.com)가 우선합니다. 또한 AWS 사이트
상에서 한글 번역문과 영어 원문에 차이나 불일치가 있을 경우(번역의 지체로 인한 경우 등 포함), 영어 원문이 우선합니다.
AWS는 본 컨텐츠에 포함되거나 컨텐츠를 통하여 고객에게 제공된 일체의 정보, 콘텐츠, 자료, 제품(소프트웨어 포함) 또는 서비스를 이용함으로 인하여 발생하는 여하한 종류의 손해에
대하여 어떠한 책임도 지지 아니하며, 이는 직접 손해, 간접 손해, 부수적 손해, 징벌적 손해 및 결과적 손해를 포함하되 이에 한정되지 아니합니다.
고지 사항(Disclaimer)
3. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Agenda
• Machine Learning on AWS
• AWS Solutions and Machine Learning
• Fraud Detection using Machine Learning
• Predictive Maintenance using Machine Learning
• Summary
3 out of 30
5. The vision of AWS for
Artificial Intelligence & Machine Learning
“ 전세계 모든 데이터 과학자와 개발자가
손쉽게 활용할 수 있는
편리한 인공지능 & 머신러닝 환경을 제공”
6. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS ML Stack: 가장 깊고 폭넓은 역량과 기술의 집약
ML FRAMEWORKS
& INFRASTRUCTURE
A I S E R V I C E S
REKOGNITION
IMAGE
POLLY TRANSCRIBE TRANSLATE COMPREHEND L E X
REKOGNITION
VIDEO
Vision Speech Language Chatbots
AMAZON
SAGEMAKER
BUILD TRAIN
FORECAST
Forecasting
TEXTRACT PERSONALIZE
Recommendations
DEPLOY
Pre-built algorithms & notebooks
Data labeling (GROUND TRUTH)
One-click model training & tuning
Optimization (N E O )
One-click deployment & hosting
M L S E R V I C E S
Frameworks Interfaces Infrastructure
EC2 P3
&
P3DN
EC2 C5 FPGAs GREENGRASS ELASTIC
INFERENCE
Reinforcement learning
Algorithms & models
(AWS MARKETPLACE FOR MACHINE LEARNING)
(App developers with
little knowledge of ML)
(ML developers and
data scientists)
(ML researchers and
academics)
INFERENTIA
6 out of 30
7. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AI/ML on AWS: More ML happens on AWS than anywhere else
7 out of 30
9. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Solutions - Machine Learning & Artificial Intelligence
Help you solve common problems and build faster using the AWS platform.
Fraud Detection Using
Machine Learning
Built by AWS
Learn how to build an architecture that
uses Amazon SageMaker to detect
potentially fraudulent activity and flag
that activity for review.
Predictive Maintenance
Using Machine Learning
Built by AWS
Learn how to build an architecture that
uses Amazon SageMaker to detect
potential equipment failures and
provide recommended actions.
AI-Driven Social Media
Dashboard
Built by AWS
Deploy a solution that captures multi-language
tweets in near real-time, translate them,
and stores both the raw and enriched
datasets durably in a data lake.
Machine Learning for
Telecommunication
Built by AWS
Learn how to build a framework for an end-to-
end machine learning process, including ad-hoc
data exploration, data processing and feature
engineering, and model training and evaluation.
Predictive data science with
Amazon SageMaker and
a Data Lake on AWS
Built by Pariveda and AWS
Builds a data lake environment for building,
training, and deploying machine learning
models with Amazon SageMaker.
Media Analysis Solution
Built by AWS
Learn how to use serverless, AWS-native
artificial intelligence services to automatically
extract valuable metadata from your media
files.
and more …
and more …
9 out of 30
11. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Fraud Detection using Machine Learning - AWS Solutions
Why detect fraud?
Associated high cost
Damaging customer trust
To avoid
Added operational burden
How fraud detection has been
typically done today
Rules based
• Static
• Complex
• Time Consuming
10 out of 30
12. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Fraud Detection using Machine Learning - AWS Solutions
https://www.slideshare.net/awskorea/aws-sk-kb-aws-summit-seoul-2019 11 out of 30
13. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Fraud Detection using Machine Learning - AWS Solutions
Why use machine learning?
• Machine learning (ML) can provide a more flexible approach to fraud detection
• ML models are trained to automatically recognize fraud patterns in datasets
• With AWS, it is easy to implement and deploy an ML model for fraud detection
ü Amazon SageMaker makes it easy to train and deploy ML models quickly
https://aws.amazon.com/solutions/fraud-detection-using-machine-learning/
Implementation Guide
https://s3.amazonaws.com/solutions-reference/fraud-detection-using-machine-learning/latest/fraud-detection-using-machine-learning.pdf
(step by step)
12 out of 30
14. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Fraud Detection using Machine Learning - AWS Solutions
End-to-end solution for fraud detection
Automates
detection of
potentially
fraudulent
activity
Includes model
training on a
sample dataset
Leverage
Amazon
SageMaker for
ML training and
deployment
Flags fraudulent
activity for
review and
visualize
processed
events
13 out of 30
15. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Fraud Detection using Machine Learning - AWS Solutions
Solution Architecture
Amazon
QuickSight
Amazon S3 bucket
(example dataset)
Amazon S3 bucket
(processed transactions)
Amazon
SageMaker
Amazon Kinesis
Data Firehose
AWS Lambda
function
Amazon
CloudWatch Event
(time-based)
Optional
14 out of 30
16. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Fraud Detection using Machine Learning - AWS Solutions
Amazon SageMaker - Features
LABEL Ground Truth
BUILD Notebooks
TRAIN &
TUNE
Jobs &
Model Tuner
EndpointsDEPLOY
Training
Environment
Estimator
Prediction
Environment
Predictor
Training Data
Trained
Model
Fraud
request response
.fit( )
.deploy( )
.predict( )
Notebook
Instance
endpoint
Preprocessed
event
Amazon SageMaker SDK
15 out of 30
17. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Fraud Detection using Machine Learning - AWS Solutions
Amazon SageMaker:
Built-in Algorithms
• Linear Learner
• Image Classification Algorithm
• Object Detection Algorithm
• Factorization Machines
• Principal Component Analysis (PCA)
• K-Means Algorithm
• K-Nearest Neighbors (KNN)
• Latent Dirichlet Allocation (LDA)
• Neural Topic Model (NTM)
• Random Cut Forest
• XGBoost Algorithm
• BlazingText
• Sequence2Sequence
• DeepAR Forecasting
Amazon SageMaker:
Training and Deployment
• The training code has been deployed
to our AWS account
• Let’s train the model and deploy the
model live using Amazon SageMaker
16 out of 30
18. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Fraud Detection using Machine Learning - AWS Solutions
Enable the processing pipeline
• The model is now trained and
deployed to an endpoint.
• Let’s trigger our AWS Lambda
function to start processing real time
events
Visualizing the Predictions
Amazon
QuickSight
AWS
Lambda
(Optional)
17 out of 30
19. Predictive Maintenance using
Machine Learning
- Predictive Maintenance at aws marketplace
- Predictive Maintenance using Machine Learning (AWS Solutions)
- Reference Architecture for Predictive Maintenance
20. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Predictive Maintenance
• Businesses are constantly seeking faster ways to
take advantage of the value of sensor-based
information and transform it into predictive
maintenance insights that people can act on
quickly
• Captures the condition of industrial equipment
in order to identify potential breakdowns before
impacting production
ü Predicting equipment failure
ü Real-time anomaly detection
ü Predicting pressure spikes
ü Asset health monitoring
• Industrial IoT: remote maintenance and monitoring
• Smart home and city: device operation management
Relevant Use cases
Anomaly
detection
models
Input
gaussian
periodic
correlation
independent
Mahalanobis
S-H-ESD
One Class SVM
Sparse Coding
Recommended
parameters
Breakout
LOF
Gaussian Process
Data
Characteristics
Algorithm Parameter
Output
Recommended
parameters
Recommended
parameters
Recommended
parameters
Recommended
parameters
Recommended
parameters
Recommended
parameters
19 out of 30
21. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Predictive Maintenance using Machine Learning
- AWS Solutions
Why use machine learning?
• To enable you to execute automated data processing on your dataset
• ML models detects potential equipment failures and provides recommended
actions
• Easy to implement and deploy an ML models with AWS
ü Amazon SageMaker makes it easy and quick to train and deploy ML models to predict
remaining useful life (RUL)
ü Amazon CloudWatch Events, AWS lambda for triggering events
https://aws.amazon.com/ko/solutions/predictive-maintenance-using-machine-learning/
Implementation Guide
https://docs.aws.amazon.com/ko_kr/solutions/latest/predictive-maintenance-using-machine-learning/welcome.html
https://s3.amazonaws.com/solutions-reference/predictive-maintenance-using-machine-learning/latest/predictive-maintenance-using-machine-learning.pdf
20 out of 30
22. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Predictive Maintenance using Machine Learning
- AWS Solutions
Customization
• By default, ‘turbofan degradation dataset’ used for model training
• For modeling of your own dataset you must
ü Modify the included notebook to point the model to our dataset
ü Convert your dataset to an Apache MXNet Gluon dataset
ü Modify the solutions’ AWS Lambda function to process and transform your sensor data
during inference
• CloudWatch Events rule can be modified to trigger an interval for your
specific needs
https://aws.amazon.com/ko/solutions/predictive-maintenance-using-machine-learning/
Regional Deployment
• Please check the region where Amazon SageMaker is available
21 out of 30
23. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Predictive Maintenance using Machine Learning
- AWS Solutions
Solution Architecture
Amazon SageMaker
MXNet model
Amazon SageMaker
Batch Transform
AWS Lambda
function
Amazon
CloudWatch Event
(time-based)
Amazon S3 bucket
turbofan degradation
dataset( )
Amazon SageMaker
training instance
Amazon SageMaker
notebook instance
transform
create transform job
training data
trained model
and source code
model training
code
testdata
predictions
22 out of 30
24. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Reference Architecture for Predictive Maintenance
Amazon SNS 에서 anomaly detection 알림
토픽을 생성하고 여러분이 생성한 모델에서
트리거가 일어나도록 설정
8
AWS IoT Analytics 데이터의 분석 결과를
QuickSight 를 이용하여 시각화7
AWS IoT Analytics 에서 AWS IoT SiteWise
Data store 로부터 컨테이너 데이터셋 생성 후
이를 Docker c컨테이너에 연결(link).
6
Docker 이미지 빌드 후 Amazon ECR 에 추가
5
AWS IoT SiteWise 에서 생성된 데이터셋을
위해 AWS IoT Analytics로부터 신규 Jupyter
노트북을 생성 (목적: Predictive Maintenance
머신 러닝 모델을 생성)
4
Factory Machines의 모니터링을 위한 메트릭
정의3
AWS IoT SiteWise 내에 뷰(View)를 생성하고
Factory Machines를 Assets 로 정의.2
Factory Machines OPC-UA 서버에 접속하기
위해 AWS IoT SiteWise Gateway 를 배포1
This reference architecture enables you to do Predictive Maintenance using
AWS IoT SiteWise and AWS IoT Analytics.
Factory AWS Cloud
AWS IoT AnalyticsAWS IoT SiteWise
AWS IoT
Greengrass
OPC-UA
Factory
Machines Amazon QuickSight
Jupyter Notebook
Amazon ECR
docker container
Amazon SNS
Anomaly
Notification
1
2 3
4
5
6
7
8AWS IoT SiteWise
Connector
Industrial Gateway
https://d1.awsstatic.com/architecture-diagrams/ArchitectureDiagrams/aws-industrial-PdM-ML-RA.pdf
23 out of 30
25. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Reference Architecture for Predictive Maintenance
Amazon SNS 에서 anomaly detection 알림
토픽을 생성하고 여러분이 생성한 모델에서
트리거가 일어나도록 설정
8
AWS IoT Analytics 데이터의 분석 결과를
QuickSight 를 이용하여 시각화7
AWS IoT Analytics 에서 AWS IoT SiteWise
Data store 로부터 컨테이너 데이터셋 생성 후
이를 Docker c컨테이너에 연결(link).
6
Docker 이미지 빌드 후 Amazon ECR 에 추가
5
AWS IoT SiteWise 에서 생성된 데이터셋을
위해 AWS IoT Analytics로부터 신규 Jupyter
노트북을 생성 (목적: Predictive Maintenance
머신 러닝 모델을 생성)
4
Factory Machines의 모니터링을 위한 메트릭
정의3
AWS IoT SiteWise 내에 뷰(View)를 생성하고
Factory Machines를 Assets 로 정의.2
Factory Machines OPC-UA 서버에 접속하기
위해 AWS IoT SiteWise Gateway 를 배포1
This reference architecture enables you to do Predictive Maintenance using
AWS IoT SiteWise and AWS IoT Analytics.
https://d1.awsstatic.com/architecture-diagrams/ArchitectureDiagrams/aws-industrial-PdM-ML-RA.pdf
Factory AWS Cloud
AWS IoT AnalyticsAWS IoT SiteWise
AWS IoT
Greengrass
OPC-UA
Factory
Machines
Jupyter Notebook
Amazon ECR
docker container
Amazon SNS
Anomaly
Notification
1
2 3
4
5
6
7
8
Amazon Kinesis
Data Firehose
Amazon S3
Greengrass Connector
Kinesis Firehouse
AWS IoT SiteWise
Connector
Industrial Gateway
9
Amazon QuickSight
9
Amazon S3 에 OPC-UA 태그가 없는 데이터를
전송하기 위해 AWS IoT Greengrass 상에서
Kinesis Firehose Greengrass Connector 의
환경 설정을 수행
24 out of 30
26. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Reference Architecture for Predictive Maintenance
Amazon Kinesis Data Analytics 에서 이상
탐지 애플리케이션을 생성8
AWS Athena 데이터의 결과를 QuickSight 를
이용하여 시각화7
AWS IoT GreenGrass Edge Gateway에 앞에서
만든 머신 러닝 모델을 배포
6
Amazon SageMaker를 이용하여 PdM 머신
러닝 모델 구축
5
Amazon S3 Data Lake 의 AWS Glue 데이터
카탈로그에서 Amazon Athena의 쿼리를
이용하여 데이터를 추축
4
Amazon S3 Data Lake에 Factory Machines
data를 저장하기 위해 Kinesis Firehose
스트림 생성
3
Factory Machines의 MQTT topics를 이용하여
이벤트를 트리거하기 위해 AWS IoT Core 의
룰을 설정
2
Factory Machines 과 통신하기 위해 AWS IoT
GreenGrass Connectors 를 이용하여 AWS
IoT Greengrass 환경 설정 수행
1
This reference architecture enables you to do Predictive Maintenance using Machine Learning
and/or anomaly detection with Amazon Kinesis Data Analytics
https://d1.awsstatic.com/architecture-diagrams/ArchitectureDiagrams/aws-industrial-PdM-ML-RA.pdf
9
Kinesis Data Analytics의 결과를 Amazon
SNS 가 이상 탐지 결과로 통보할 수 있도록
AWS Lambda 설정을 수행
Factory AWS Cloud
AWS IoT Greengrass AWS IoT CoreFactory Machines
Amazon
SageMaker
Amazon
QuickSight
Amazon Kinesis
Data Firehose
Amazon S3
Lambda function
Amazon Athena
Amazon Kinesis
Data Analytics
Amazon Kinesis
Data Streams
Amazon Kinesis
Data Firehose
Amazon SNSAWS
Lambda
ML
Inference
1
2
3
4
5
6 7
8 9
25 out of 30
28. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
AWS Solutions – Machine Learning & AI
• AWS 솔루션즈 아키텍트에 의한 검증 결과
• 운영 효율성, 안정성, 보안, 비용 효율성을 보장하도록 설계
• 상세 아키텍처, 배포 안내서, 수동/자동 배포용 지침도 함께 제공
• Fraud Detection using Machine Learning
ü 잠재적 사기 행위의 탐지 및 검토 활동을 플래그 하는 아키텍처 구축 방법
• Predictive Maintenance using Machine Learning
ü 장비의 이상 동작, 멈춤, 고장에 관한 잠재적 징후를 탐지하고 예방 및 조치에
권장하는 작업을 제공하는 아키텍처 구축 방법
“AWS의 플랫폼을 사용하여 공통된 문제의 해결과 빠른 구축을 지원”
27 out of 30
30. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
aws-korea-marketing@amazon.com
twitter.com/AWSKorea
facebook.com/amazonwebservices.ko
youtube.com/user/AWSKorea
slideshare.net/awskorea
twitch.tv/aws
캠페인 온라인 세미나: Data, Analytics, and ML Edition
참석해주셔서 대단히 감사합니다.
저희가 준비한 내용, 어떻게 보셨나요?
더 나은 세미나를 위하여 설문을 꼭 작성해 주시기 바랍니다.