Building Modern Apps with DynamoDB Transactions

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Building Modern Applications Using
Amazon DynamoDB Transactions
Yossi Levanoni
Senior Manager, Software Development
Amazon DynamoDB
D A T 3 7 4

Agenda
1. Introduction to atomicity, consistency, isolation, and durability (ACID) in
DynamoDB, and the new transactional API
2. Modern application use cases for the transactional API
3. Important things to know when integrating the transactional API into
your application: metering, concurrency control, and interaction with
other features

What you will learn
• How the new transactional API works
• How you can use the new API in your application through case
studies demonstrating transactional design patterns
• How the new API interacts with DynamoDB features

Why is this important?
The transactional API can be used to
implement real-world transactional scenarios
The transactional API can be used to simplify
the implementation of cloud application flows

Prerequisites
You should be familiar with DynamoDB concepts such as tables,
items, partition keys and sort keys, indexes, Time To Live, and
streams. We will only briefly introduce these concepts in this
session before using them in the use cases.

Transactional API at a glance
TransactWriteItems
Transact up to 10 items
Evaluate conditions
If all conditions are simultaneously true, perform write operations
TransactGetItems
Transact up to 10 items
Return a consistent, isolated snapshot of all items

Atomicity, consistency, isolation, and durability (ACID) in
DynamoDB
• Before: ACID support for single-item operations
• NEW: ACID support for multi-item operations
Nontransactional Transactional
New
TransactGetItems
TransactWriteItems
Existing
BatchGetItem, BatchWriteItem
Query, Scan
GetItem, PutItem, UpdateItem,
DeleteItem

Transactions at DynamoDB scale
• Apply transactions to items in:
• Same partition key, or across partition keys
• Same table, or across tables
• Same Region only
• Same account only
• DynamoDB tables only
• Transactions scale like the rest of DynamoDB
• Limitless concurrent transactions
• Low latency, high availability
• Scales horizontally
• Your responsibility: Design to avoid hot key

Path of success with NoSQL transactions
Design choices
• Transact any items in your account
• Service APIs vs. client-side library
• Request/response-based
• Limit of 10 items
How they help you
• Scale your data and relations within it
• Reduces complexity, and improves cost and
performance
• Keeps business logic in your application and
not in the database layer
• Consistent low latency, never deadlock
• Consistent low latency, high availability
• Addresses vast majority of use cases

User profile management (use case #1)
• Problem statement
• Users identified by user name and optionally email
• No two users can share the same identifier (user name or email)
• Users can change their email
• What is demonstrated
• Enforcing unique constraints
• Strongly consistent global indexing
• Heterogeneous tables

User profile management
• Solution
• Users table contains two kinds of records
• Profile record, keyed by opaque user name and containing user details
• Alias records, keyed by user email and referencing the profile record
Id UserNameRef Email Phone #
“John123” “john@example.com” “444-555-0110”
“john@example.com” “Joe123”
“jane456” “444-555-0199”

Insert new profile
data = await dynamoDb.transactWriteItems({ TransactItems: [
{ Put: {
TableName: 'Users',
Item: { Id: { S: 'John123' }, Email: {S: 'john123@example.com'}, ...},
ConditionExpression: 'attribute_not_exists(Id)‘ }},
{ Put: {
TableName: 'Users',
Item: { Id: { S: 'john123@example.com' }, UserNameRef: {S: 'John123'}},
ConditionExpression: 'attribute_not_exists(Id)',}}
] }).promise();

Update email address
data = await dynamoDb.transactWriteItems({ TransactItems: [
{ Put: { TableName: 'Users',
Item: { Id: { S: 'John123' }, Email: {S: 'john123@example.org'}, ...},
ConditionExpression: 'attribute_exists(Id) and Email = :old_email',
ExpressionAttributeValues: {':old_email' :{'S': 'john123@example.com'}}}},
{ Put: { TableName: 'Users',
Item: { Id: { S: 'john123@example.org' }, UserNameRef: {S: 'John123'}},
ConditionExpression: 'attribute_not_exists(Id)', }},
{ Delete: { TableName: 'Users',
Item: { Id: { S: 'john123@example.com' }}}).promise();

Hotel reservation management (use case #2)
• Guests create room reservations
• Guests check in to rooms, fulfilling a reservation
• Guests check out of rooms, closing a reservation
• Ensuring idempotency – operations with side effects happen at most once
• Ensuring multi-item changes are only triggered when valid (consistency)
• Ensuring multi-item changes happen all together (atomicity)

Hotel reservation management data model
Guests
Id (primary key)
Name, Email, etc.
Reservations (set)
OccupiesRooms (set)
Reservations
Id (primary key)
GuestId
FulfilledByRoom
State
Rooms
Id (primary key)
RentedToReservation
State

Hotel reservation management data model
Record Type Id (PK) Attributes
Guest “John”
Reservations : { “500”, “501” }
OccupiesRooms : { “20014” }
Reservation “500”
GuestId: “John”
State: “PENDING”
GuestId: “John”
State: “FULFILLED”
FulfilledByRoom: “20014”
Room “20014”
State: “OCCUPIED”
RentedToReservation : “501”

Hotel reservation management: Create a reservation
• Idempotency – how to ensure only a single reservation is created
upon retries?
• Solution
• Client generates a unique reservation ID
• Client generates TransactWriteItems request with the following operations
• Conditional Put for the new reservation
• Update the customer reservations set: Add the new reservation
• Client transaction is now idempotent and can be retried without
double booking

Hotel reservation management: Check-in
Customer “John123”
Reservations : { “567” }
OccupiesRooms : {}
CustomerId: “John123”
State: “PENDING”
FulfilledByRoom: null
Room “20014”
State: “FREE”
RentedToReservation : null
Before

Hotel reservation management: Check-in
Room “20014”
After

Hotel reservation management: Checkout
Room “20014”
Before

Hotel reservation management: Checkout
State: “CLOSED”
Room “20014”
State: “FREE”
After

Concurrency control
• Optimistic
• No deadlocks possible
• Low latency
• Optimized for scale-out
• Your responsibility
• Design for scale-out
• Avoid unnecessary conflicts

Metering transactions
Transactions perform 2x the work of nontransactional
counterparts
Transactions are metered 2x their constituent operations

Why didn’t my transaction succeed?
• Per-item failure reasons
• Precondition failure
• Insufficient capacity
• Transactional conflicts
• Other reasons
• Transaction is still in progress
• Service error
• Malformed request
• Permissions

Retrying transactions: ClientRequestToken
• Ensures SDK can retry without “replaying” the same intent
multiple times
• ClientRequestToken – idempotency token at the SDK or API level
• Autogenerated, you can override
• Applies for 10 minutes
• Provides information about transaction state and outcome
• Sometimes doesn’t remove the need for application-level idempotency or
intent design

Application-level retries
• When the SDK gives up
• Build a new view of the world
• Retry with new preconditions and desirable end state
• How to build a snapshot of the world? Three options:
• Set ReturnValuesOnConditionCheckFailure = ALL_OLD
• Perform TransactGetItems
• Work from an eventually consistent view (results of queries, etc.)

Integrating with other DynamoDB features
• General rule about DynamoDB data
• Committed state only
• But might not be isolated
• Examples
• Streams – Committed writes, sharded
• Backups and point-in-time recovery (PITR) – Subset of a committed transaction is
possible
• Global secondary index (GSI) – Committed writes are indexed individually
• Global tables – Transactional API disabled by default; writes are replicated
individually
• Amazon DynamoDB Accelerator (DAX) support is on the roadmap

Access control
• Individual operations are authorized separately
• (NEW) dynamodb:ConditionCheck
• dynamodb:UpdateItem
• dynamodb:PutItem
• dynamodb:DeleteItem
• No separate permissions for top-level new APIs

Attachment management (use case #3)
• A social media site allows attaching media to posts
• Posts need to share media efficiently
• Posts can have multiple attachments
• Unreferenced media needs to be deleted
• External resource metadata and lifetime management
• Using Time To Live (TTL) and streams
• Reference counting using transactions

Attachment management data model
Attachment item
S3Ref (PK) – Reference to an object in Amazon S3
RefCount – Number of posts referencing this attachment
TTL – Time when this attachment is eligible to be deleted
Post item
Id (PK) – Post ID
Attachments (set) – Set of attachment IDs

Attachment management invariants
1. RefCount = number of posts referencing the attachment
2. If RefCount > 0, then S3Ref refers to a valid Amazon S3 object
3. TTL is set if, and only if, RefCount = 0

Attachment management: Create attachment and link it
to a post
• Generate unique Amazon S3 object name
• Create attachment record in DynamoDB
• Set RefCount = 0,
• Set TTL = 24 hours from now
• Upload object to Amazon S3
• In a TransactWriteItems Request
• Check that the attachment is not already referenced by the post
• Increase RefCount by 1, remove TTL
• Add attachment reference to post

Attachment management: Clone attachment reference
• Copy the reference of an attachment from one post to another
• In a TransactWriteItems Request:
• Check that the attachment is not already referenced by the new post
• Increase RefCount by 1, remove TTL
• Add attachment reference to new post

Attachment management: Delete unused external
resources
• Attachment records will be deleted a day after their reference
count drops to zero
• In AWS Lambda, process stream records representing items
deleted by TTL
• Delete the corresponding objects from Amazon S3

Thank you!
Yossi Levanoni
Contact us through the AWS forums or
@dynamodb on Twitter
Get the Database Freedom T-shirt in our
swag booth!

Building Modern Apps with DynamoDB Transactions

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à Building Modern Apps with DynamoDB Transactions

Similaire à Building Modern Apps with DynamoDB Transactions (20)

Plus de Amazon Web Services

Plus de Amazon Web Services (20)

Building Modern Apps with DynamoDB Transactions