The document introduces Amazon Textract, a new AI service from Amazon for extracting text, tables, and other data from scanned documents. It discusses the need for processing large volumes of documents in industries like finance, insurance, and legal. The document outlines challenges with current document processing methods like manual entry and OCR. It then describes Amazon Textract's features for text, table, and form extraction and how it can simplify extracting structured data from documents without templates.

2. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Introducing
Amazon Textract
A I M 3 6 3
Ranju Das
General Manager
Amazon Textract
Wendy Tse
Sr. Product Manager
AWS
John Newton
Chief Technology Officer
Alfresco

3. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Documents are important
Primary tool of record keeping, communicating, collaborating, and transacting
Finance
Insurance
Real estate
Accounting
Tax management
Medical
Legal
Business management
Education
And many more…

4. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
16.3M US mortgage applications ($2.1T) in 2016
*

5. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
About 240M W2 tax forms will be processed for FY2018 in the
US
*

6. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Need for processing documents

7. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
How documents are processed today
Optical Character Recognition
(OCR)
Manual
processing
Rules and
template-based extraction

8. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Challenges for processing documents
Manual processing
Expensive Error-prone Time-consuming

9. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Challenges for processing documents
Manualprocessing
Output
1. Exempt is true
2. 28 is true
3. CPP/QPP is true
4. RPC/RRQ is true

10. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Challenges for processing documents
Optical Character Recognition (OCR)
Error-prone Flat bag of wordsSimple documents only

11. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Challenges for processing documents
Optical Character Recognition (OCR)
Output
Extract data quickly & No code templates to
accurately maintain

12. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Challenges for processing documents
Optical Character Recognition (OCR)
Output
Start Date End Date Employer Name Position Held Reason for leaving
1/15/2009 6/30/2013 Any Company Head Baker Family Relocated

13. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Challenges for processing documents
Output
Full Name Date of Birth Gender
John X Doe 01 01 1971
Male
First Middle Last MM DD YYYY
Female

14. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Challenges for processing documents
Rules and template-based extraction
Limited by
accuracy of OCR
Significant development and
management overhead
Templates
are brittle

15. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Challenges for processing documents
Rules and template-based extraction
The well-known W2 US tax form has 100s of variants each year

16. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
It looks easy, but …
…not a single corresponding pixel value in common

17. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark

18. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Amazon Textract features
Text extraction Table extraction Form extraction

19. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Amazon Textract—Text Extraction
Blocks: PAGE, PARAGRAPH, LINE, WORD
is washed by waves, and cooled

20. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Name Description
Blocks List of blocks identified
from the document
ID Unique ID of the unit
Relationships CHILD
Block type PAGE, PARAGRAPH, LINE, WORD
Pages Contains number of
pages in the document
Amazon Textract—Text Extraction API
DetectDocumentText
Name Description
Document Blob or Amazon S3 object

21. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Amazon Textract—Table Extraction
Blocks: PAGE, TABLE, CELL
For each ’block’ you get:
• Text
• Confidence score
• Block relationships (e.g. cells within a table)

22. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Name Description
Document Blob or Amazon S3 object
FeatureTypes TABLES
Name Description
Blocks List of blocks identified
from the document
ID Unique ID of the unit
Relationships CHILD
Block type PAGE, TABLE, CELL
Pages Contains number of
pages in the document
Amazon Textract—Table Extraction API
AnalyzeDocumentwith “table” as FeatureTypes parameter

23. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Amazon Textract—Form Extraction
Blocks: PAGE, KEY_VALUE_SET
For each ’block’ of your document:
• Form field name (key) and field value (value) association
• Confidence score
• Page number
• Block relationships

24. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Name Description
Document Blob or Amazon S3 object
FeatureTypes FORMS
Name Description
Blocks List of blocks identified
from the document
ID Unique ID of the unit
Relationships KEY, VALUE, CHILD
Block type PAGE, KEY_VALUE_SET
Pages Contains number of
pages in the document
Amazon Textract—Forms Extraction API
AnalyzeDocumentwith “forms” as FeatureTypes parameter

25. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Supports single-page documents
such
as images (e.g.,
mobile capture)
For multi-page documents,
up to 3,000 pages
Amazon Textract
Sync and async

26. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Amazon Textract—
TextExtractionsimplified
Output
Extract data quickly &
accurately
No code or templates to
maintain

27. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Output {
Start Date: 1/15/2009
End Date: 6/30/2013
Employer Name: Any Company
Position Held: Head Baker
Reason for leaving: Family relocated
}
Amazon Textract—
TableExtractionsimplified

28. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Output Full Name:
First: John
Middle: X
Last: Doe
Date of Birth:
MM: 01
DD: 01
YYYY: 1971
Gender:
Male: True
Female: False
Amazon Textract—
FormExtractionsimplified

29. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark

30. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Text Extraction: OCR reimagined
Orientation

31. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Text Extraction: OCR reimagined
Structure variability

32. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Text Extraction: OCR reimagined
Document variability

33. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Beyond OCR: Segmentation and rectification
Photometric

34. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Beyond OCR: Segmentation and rectification
Geometric

35. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Beyond OCR: Table and cell detection
Understand document structure and context to find tables

36. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Beyond OCR: Table and cell detection
Understand cells even without explicit boundaries

37. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Beyond OCR: Table and cell detection
Variable-sized rows and columns

38. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Beyond OCR: Field name (key) and value Extraction
Full Name:
First: John
Middle: X
Last: Doe
Date of Birth:
MM: 01
DD: 01
YYYY: 1971
Gender:
Male: True
Female: False
Detect phrases or groups of words
Output

39. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Beyond OCR: Inferring key/value association
Detect structures of the same form without templates

40. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Beyond OCR: Inferring key/value association
Key and value association

41. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Beyond OCR: Inferring key/value association
Infer empty values
Full Name:
First: John
Middle: null
Last: Doe
Date of Birth:
MM: 01
DD: 01
YYYY: 1971
Gender:
Male: True
Female: False
Output

42. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark

43. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Reference architecture—Index and search documents
Input
Uploaded document
images such as tax
forms, credit
applications, or
medical notes
Amazon S3
Uploaded
documents are
stored in data lake
AWS Lambda
A Lambda function is
triggered to initiate
document analysis
using the
Hieroglyph API
Amazon Textract
Automatically
extract text,
including key-value
pairs and tables
Amazon
Elasticsearch
Service
Extracted data and
confidence scores
are indexed to
enable document
search
Output
Perform contextual
search on millions of
documents or
integrate data into
your document
management system

44. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Reference architecture—Form capture
Input
Customer uses mobile
app to capture a photo of
a W2 form
Amazon Textract
The Amazon Textract API
is integrated into the end-
user application to
automatically extract text
from the W2 form and
auto-populate the
form fields
Customer Application
Customers experience
real-time capture of their
tax information by taking
a photo instead of
manual data entry
Database
User submitted data is
loaded into a
database for use in tax
preparation

45. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Reference architecture—Extract for NLP
Quickly turnextracted text/datainto actionableinsights
Input
Uploaded document
images of medical
notes, explanation of
benefits, and
patient forms
Amazon S3
Uploaded
documents are
stored in S3
NLP
Use natural language
processing to extract
insights from
medical documents
Amazon
Elasticsearch
Service
Easily search
through extracted
data and text
insights
Output
Discover medical
insights to improve
patient care
Amazon Textract
Automatically
extract words and
lines of text, and
tables

46. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark

47. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Amazon Textract
Launch customers

48. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
John Newton
CTO & Co-Founder
Alfresco Software, Inc
Brad Christus
Sr. Director, Global Presales
Alfresco Software, Inc

49. More than 80% of business information is locked in
unstructured content

50. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Agenda
• Who is Alfresco?
• The need for intelligent OCR
• Alfresco + Amazon Textract
• Use case example

51. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Who is Alfresco?

52. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
The need for intelligent OCR
• Paper is expensive
• Customers want mobile capture
• Manual data entry is slow and error-prone
• Access to embedded information speeds decisions and handling
• Extracting key-value and tabular data necessary to integrate documents
to line of business systems
• Storing documents with extracted information aids retrieval and proper
compliance

53. Amazon Textract allows us to move beyond just Digital
Transformation to Automatic Transformation

54. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Alfresco Digital Business Platform & Amazon Textract
JSON
JPG
Amazon
Textract

55. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Use cases
Enterprise
document library
Loan
applications
Claims & case
processing
Transaction &
logistics records
Research &
analysis
Real-time
video
Medical &
personnel records
Government records
& archives
Discovery &
litigation
Media
management

56. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Deep Dive: Financial services
Accelerate loan applications
On average, it takes a person 6-8 weeks to close a mortgage loan
Any incorrect data could mean weeks of delay and hours of wasted employee time
• With Alfresco and Amazon Textract, you can
• Automate the loan application process
• Extract data across multiple sources with high levels of accuracy
• Increase security by automating the data entry layer
• Decrease loan processing times

57. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Amazon Textract
Benefits

58. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Per 100 pages or images
processed
Up to 1M
pages/month
1M+
pages/month
Text Detection $0.15 $0.06
Table Extraction
(Text Detection included)
$1.50 $1.00
Key-Value Detection
(Text Detection included)
$5.00 $4.00
All $6.50 $5.00
Amazon Textract
Pricing

59. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Features Free for first three months
Text Detection Up to 1,000 pages
or images
Table Detection
Up to 100 pages
or images
Key-Value Detection
Text, Table, and
Key-Value Detection
Amazon Textract
FreeTier

60. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Textract
Regions
US West (Oregon)
US East (N. Virginia)
US East (Ohio) EU (Ireland)

61. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Amazon Textract
Preview
https://pages.awscloud.com/textract-preview.html
LEARN MORE
or
SIGN UP

62. Thank you!
© 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark
Ranju Das
ranjudas@amazon.com
Wendy Tse
tsewendy@amazon.com
John Newton
john.newton@alfresco.com

63. © 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.