AWS Neptune - A Fast and reliable Graph Database Built for the Cloud

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Dickson Yue
Solutions Architect, Amazon Web Services
How Amazon Neptune and Graph
Databases can transform your business

purpose
noun ˈpər-pəs
The reason for which something is done or created
or for which something exists

Traditional application characteristics
HR Payroll
CRM ERP
…
Users 100s-1000s
Data volume GB-TB
Locality HQ
Performance Seconds
Request Rate Tens of thousands
Access Internal servers, PCs
Scale Up
Economics Pay up front
Developer Access days/weeks/months

Cloud application characteristics
Users 1M+
Data volume TB-PB-EB
Locality Global
Performance Milliseconds-Microseconds
Request Rate Millions
Access Mobile, IoT, Devices
Scale Up-Out-In
Economics Pay as you go
Developer access Instant API access

AWS Databases and Analytics
Broad and deep portfolio, purpose-built for builders
DW | Big Data Processing | Interactive
Business Intelligence & Machine Learning
Data Movement
Database Migration Service | Snowball | Snowmobile | Kinesis Data Firehose | Kinesis Data Streams
QuickSight
Relational Databases
RDS
Aurora
Data Lake
S3/Glacier Glue
(ETL & Data Catalog)
SageMaker
Non-Relational Databases Analytics
DynamoDB
ElastiCache
(Redis, Memcached)
Neptune
(Graph)
Redshift EMR Athena
Kinesis
Analytics
Elasticsearch
Service
Real-time
Comprehend

Database characteristics
Referential integrity
with strong
consistency,
transactions, and
hardened scale
GraphKey-value Document
;
Time SeriesRelational
Low-latency key
based queries with
high throughput
and fast ingestion
of data
Indexing and
storing documents
with support for
query on any
property
Creating and
navigating
relations between
data easily and
quickly
Time-stamped
data with large
range-scans for
summarization
and processing
Complex query
support via SQL
Purchase
Transaction
Simple query
methods with
filters
Shopping cart
Simple query with
filters, projections
and aggregates
Product info
Easily express
queries in terms of
relations
Product
recommendation
Computational
support for
summarized
results
Clickstream
analysis

DIFFERENT APPROACHES FOR HIGHLY CONNECTED DATA
Purpose-built for a business process
Purpose-built to answer questions about
relationships

Use Cases For Highly Connected Data
Social Networking
Life Sciences Network & IT OperationsFraud Detection
Recommendations Knowledge Graphs

Social Network

Recommend New Connections
Social Network
gremlin>
g.V().has('name','Terry').as('user’)
.bothE('FRIEND').has('strength', gt(1)).otherV().aggregate('friends’)
.bothE('FRIEND').has('strength', gt(1)).otherV()
.where(neq('user’))
.where(without('friends’))
.groupCount().by('name').order(local).by(values, decr)
==>{Colin=2, Henry=2, Chloe=1}
Terry
Terry’s friend
Terry’s friend’s friend
excl Terry & Terry’s friend

t
Recommendation
? ? ? ? ? ? ?
Terry bought t:{…. }
User also who bought t:{…. }
also who bought o:{…….}
t :
o :

Recommendation
gremlin>
g.V().has("User","name","Terry").as("Terry").
out("Bought").aggregate("self").
in("Bought").where(neq("Terry")).dedup().
group().
by().by(out("Bought").
where(within("self")).count()).as("g").
select(values).
order(local).
by(decr).limit(local, 1).as("m").
select("g").unfold().
where(select(values).as("m")).select(keys).
out("Bought").where(without("self")).
groupCount().
order(local).
by(values, decr).
by(select(keys).values("name")).
unfold().select(keys).values("name")
==>Drone ==>IP camera ==>Kindle ==>Tent
Largest common set of purchase
What they bought

Recommendation

Knowledge graph applications
What museums should Alice
visit while in Paris?
Who painted the Mona Lisa?
What artists have paintings
in The Louvre?

Challenges of existing graph databases
Difficult to maintain
high availability
Difficult to scale
Limited support for
open standards
Too expensive

Amazon Neptune
Fully managed graph database
FAST RELIABLE OPEN
Query billions of
relationships with
millisecond latency
6 replicas of your data
across 3 AZs with full
backup and restore
Build powerful
queries easily with
Gremlin and SPARQL
Supports Apache
TinkerPop & W3C
RDF graph models
EASY

Leading graph models and frameworks
Open Source Apache TinkerPop™
Gremlin Traversal Language
W3C Standard
SPARQL Query Language
RESOURCE DESCRIPTION
FRAMEWORK (RDF)
PROPERTY GRAPH

Amazon Neptune High Level Architecture
Bulk load
from S3
Database
Mgmt.

Fully managed service
Easily configurable via the Console
Multi-AZ High Availability, ACID
Support for up to 15 read replicas
Supports Encryption at rest
Supports Encryption in transit (TLS)
Backup and Restore, Point-in-time
Recovery
B E N E F I T S

Amazon Neptune Read Replicas
Availability
Failing database nodes are automatically
detected and replaced
Failing database processes are
automatically detected and recycled
Replicas are automatically promoted to
primary if needed (failover)
Customer specifiable fail-over order
AZ 1 AZ 3AZ 2
Primary
Node
Primary
Node
Primary
Master
Node
Primary
Node
Primary
Node
Read
Replica
Primary
Node
Primary
Node
Read
Replica
Cluster
and
Instance
Monitoring
Performance
• Customer applications can scale out read
traffic across read replicas
• Read balancing across read replicas

Customers previewing Amazon Neptune

Navigate a web of global tax policies
“Our customers are increasingly required to navigate a complex web of global tax policies and
regulations. We need an approach to model the sophisticated corporate structures of our
largest clients and deliver an end-to-end tax solution. We use a microservices architecture
approach for our platforms and are beginning to leverage Amazon Neptune as a graph-based
system to quickly create links within the data.”
said Tim Vanderham, chief technology officer, Thomson Reuters Tax & Accounting

DEMO

First-Party Fraud
• Fraudsters apply for credit
• No intention of repaying
• Appear normal until they
“burst out”
• Clear out accounts
• Fraud ring
• Share bits of identity (SSN,
address, telephone)
• Coordinated “burst out”

Show me the code!!

Fraud ring
gremlin> g.V().
......1> hasLabel('Applicant').
......2> has('first_name','Terry').
......3> has('last_name','Wilder').
......4> out('CREDIT').
......5> out('IDENTITY').
......6> in('IDENTITY').
......7> in('CREDIT').dedup().as('ring').
......8> project('ring', 'identity').
......9> by(select('ring').values().fold()).
.....10> by(select('ring').out('CREDIT').out('IDENTITY').dedup().valueMap().fold())
==>{ring=[Terry, Wilder], {phone_number=[0208 674 5742], type=[Phone Number]}, {type=[Social Security Number], ssn=[224-23-1221]}]}
==>{ring=[Bill, Darrow], {phone_number=[0208 674 5742], type=[Phone Number]}, {type=[Social Security Number], ssn=[555-23-4545]}]}
==>{ring=[Lucy, Phillips], {phone_number=[0208 674 5742], type=[Phone Number]}, {type=[Social Security Number], ssn=[224-23-1221]}]}
==>{ring=[Colin, Smith], {phone_number=[07074 633 7654], type=[Phone Number]}, {type=[Social Security Number], ssn=[224-23-1221]}]}

Thank You
?univ

AWS Neptune - A Fast and reliable Graph Database Built for the Cloud

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à AWS Neptune - A Fast and reliable Graph Database Built for the Cloud

Similaire à AWS Neptune - A Fast and reliable Graph Database Built for the Cloud (20)

Plus de Amazon Web Services

Plus de Amazon Web Services (20)

AWS Neptune - A Fast and reliable Graph Database Built for the Cloud