Introduction to handwriting detection & recognition using Microsoft Cognitive Services. Specifically Custom Vision service and Computer Vision's Handwriting Recognition service(preview). At the same time, demo on how Handwriting Recognition service works and as well as, conceptually how handwriting detection can be work on automation process. Read more on @ www.techconnect.io by Marvin, Heng Twitter: @hmheng Blog: www.techconnect.io

1. Handwriting Recognition
With Microsoft Cognitive
Service

2. “To him,
turning innovative ideas into
reality is great excitement
and achievement.”
Microsoft MVP | Tech + AI Enthusiast
Marvin Heng
https://www.techconnect.io
@hmheng

3. What’s Microsoft
Cognitive Service

4. Microsoft Cognitive Service
Vision Speech Language Knowledge Search
Computer Vision
Face
Video Indexer
Content Moderator
Custom Vision
Speech to Text
Speaker Recognition
Text to Speech
Speech Translation
Text Analytics
Bing Spell Check
Language Understanding
Translator Text
Content Moderator
QnA Maker Bing Web Search
Bing Custom Search
Bing Video Search
Bing Image Search
Bing Visual Search
Bing Entity Search
Bing News Search
Bing Autosuggest

5. Why Microsoft
Cognitive Service?

6. So Why?
Roll your own with REST APIs
No on-premises infrastructure. Hosted on Microsoft Azure cloud.
Easy
Flexible
Tested
Easy integration with all platforms.
Built by experts in their field from Microsoft Research, Bing, and Azure
Machine Learning
Quality documentation, sample code and community support

7. Bringing it all
together
The Seeing App
Computer Vision,
Image Speech
Recognition,
NLP and
ML from Microsoft
Cognitive Services

8. Computer Vision

9. Analyzing Image
{ "tags": [ "train", "platform", "station",
"building", "indoor", "subway", "track",
"walking", "waiting", "pulling", "board",
"people", "man", "luggage", "standing",
"holding", "large", "woman", "yellow",
"suitcase" ], "captions": [ { "text": "people
waiting at a train station", "confidence":
0.8330993 } ] }
[ { "name": "train", "confidence": 0.9975446 },
{ "name": "platform", "confidence":
0.995543063 }, { "name": "station",
"confidence": 0.9798007 }, { "name": "indoor",
"confidence": 0.9277198 }, { "name":
"subway", "confidence": 0.838939548 }, {
"name": "pulling", "confidence": 0.4317156 } ]
Description
Tags

10. Read Text in Images
CLOSED
WHEN ONE DOOR CLOSES, ANOTHER
OPENS.ALL YOU HAVE TO DO IS WALK IN
Preview

11. Analyze Video in Near Real-Time

12. Recognize Celebrities and Landmarks
"categories": [
{
"name": "people_",
"score": 0.86328125,
"detail": {
"celebrities": [
{
"name": "Satya Nadella",
"faceRectangle": {
"left": 240,
"top": 294,
"width": 135,
"height": 135
},
"confidence": 0.99999833106994629
}
],
"landmarks": null
}
}
],
JSON

13. Read handwritten text from images
It does not matter
how slowly you go
so as long as
you do not stop
fighting
Preview

14. Demo

15. Detecting
Handwriting in an
Application Form

16. Approach
Handwriting
Detection Crop
Computer Vision
Marvin Heng
29/10/2018
Custom Vision
Handwriting
Recognition

17. Demo

18. Next

19. Next…
To use Custom ML with Python / Azure ML to improve handwriting
detection
Handwriting-To-Text recognition with custom trained model
Train on same application form template to improve detection

20. Thank You

21. THE
JOURNEY
CONTINUES TO
DEVELOP…

22. www.techconnect.io
Twitter @hmheng
Microsoft MVP | Tech + AI Enthusiast
Marvin Heng
https://www.techconnect.io
@hmheng

23. Handwriting Recognition
With Microsoft Cognitive
Service