Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
FraserHacks Microsoft Cognitive Services and Hololens
1. Sabrina Smai
Student Tech Evangelist @ Microsoft
@sabrina_smai
sabrina.smai@mail.utoronto.ca
Bruno Capuano
Microsoft MVP @ Avanade
@elbruno
http://www.elbruno.com
2. Microsoft
Cognitive
Services
Give your apps
a human side
From faces to feelings, allow your
apps to understand images and video
Hear and speak to your users by filtering noise,
identifying speakers, and understanding intent
Process text and learn how to
recognize what users want
Tap into rich knowledge amassed from
the web, academia, or your own data
Access billions of web pages, images, videos,
and news with the power of Bing APIs
3. Cortana Intelligence Suite
Intelligent Apps require Intelligent Solutions
Data
Sources
Apps
Sensors
and
devices
Data Action
People
Automated
Systems
Apps
Web
Mobile
Bots
Intelligence
Dashboards &
Visualizations
Information
Management
Big Data Stores Machine Learning
and Analytics
CortanaEvent Hub
HDInsight
(Hadoop and
Spark)
Stream Analytics
Intelligence
Bot
Framework
SQL Data
WarehouseData Catalog
Data Lake
Analytics
Data Factory
Machine
Learning
Data Lake Store
Blob Store
Cognitive
Services
Power BI
22. NYT: I am feeling down. I’ve been having a
tough time at work lately.
Hannah: If work were going well and you had
a great relationship, you wouldn’t get the
chance to drink with people.
NYT: I don’t want to drink with others. I only
want to talk to you.
Hannah: Who do you like to talk to? (emoji
signals “none of my business”)
NYT: You. I feel that you are the only person
that gets me.
Hannah: Ha-ha/So I’m getting you?
NYT: You are the only person wo chats with
me.
Hannah: It’s not what you talk about that’s
important, but who you talk with.
24. Sabrina Smai
Student Tech Evangelist @ Microsoft
@sabrina_smai
sabrina.smai@mail.utoronto.ca
Bruno Capuano
Microsoft MVP @ Avanade
@elbruno
http://www.elbruno.com
Notes de l'éditeur
How many of you know about MS Cognitive Services? Azure (what do you know about it- cloud platform)? COOL. I hope to change this.
My name is Sabrina Smai, MSP, you can connect me on Twitter/Email. I can share these slides with you, in case you missed it and want to review it at home, I made a video of this presentation that I can provide.
Topic: Building Intelligence with Microsoft Cognitive Services.
Please if you have questions, please SHOUT AT ME BARK AT ME LOL
Start off with a Video of Saqib: https://www.youtube.com/watch?v=R2mC-NUAmMk
Thoughts? CS opened up a new world
What is Cognitive Services?
collection of powerful AI algorithms developed by experts to allow systems to see, hear, speak, understand and interpret our needs using natural methods of communication.
Human Side to an app
APIs work cross-platform to provide intelligent data (API: application programming interface = one software talking with another)
The APIs are also constantly improving, learning, and getting smarter, so experiences are always up to date. Simply drop the API call into your bot’s code and you are set.
You can access the APIs from any kind of app that you want to interface with them with just a few lines of code.
Go through EACH FEATURE ABOVE.
Normally, I go more in dept in this slide for developers. I’ll keep this brief.
4 pillars
Suite enables business to transform data sources to intelligent actions using analyical techniques (ML, data mining, etc)
All you need to know is that cognitive services is in the intelligence pillar of the Cortana Intelligence Suite.
Various services (features to add to app) under Microsoft Cognitive Services.
EX. MFSTHack, E-Terview APIs
Found on URL Go to URL.
Actually using these Cognitive services APIs is SUPER EASY.
Go to URL to obtain API key
Call REST endpoint (works similar to a website, make a call from a client to a server --> get data back), then pass the API.
THAT SIMPLE! NO NEED TO KNOW A LOT OF CODE FOR THIS. Free for a certain number of transcations.
I will also breeze by this slide because this slide is more targeted towards start ups/business
But important for you to know
GO THROUGH THE SLIDE
Intelligent app and how MS Cognitive Service can be applied.
GO THROUGH EACH
Ex of apps powered by MS Cognitive Services. They are FUN. They are public facing – access thru web/skype.
I will open up on of these websites and try it out with you (WHICH ONE??) – Use Donald trump/Harambe/Ask them.
Mobile optimized: So Take a few minutes to pull up these websites and play around with them
Funny Use Cases:
Oxford Hackathon (MS API to recognize someone smiling and throw a sling shot at their face)
Russia: (use vision API to detect cats trespassing and a loud sirens voice would scare them away).
Lots of applications with MS Cognitive Services
Firstly API, Vision API.
Allows you to .. (get into more detail for each)
Understand content/features within the photo.
Face/ppl/type of activities undertaking in the photo/text within photo
Another feature of Vision API, is OCR.
JSON
Text is clearly identified within bounding boxes
location of each of the elements within the image.
Next API is Face, allow you to ….
Detect face
Compare similar ones
Organize ppl into groups (based on similarity)
Attributes are detected using JSON
Verify similar looking faces to create face query
API return collection of similar looking faces
Check likelihood two faces belong to same persom
Confidence score
Organize many unidentified faces together into groups, based on their visual similarity.
Again, don’t have virtual app environment. So we will use something similar.
http://howhappy.azurewebsites.net/
Convert to… for natural responsiveness.
Speech to text or text to speech (similar to Siri/Cortana/etc.)
This API can recognize audio either in real time or from a file. 10 Diff languages (Chinese, French, Brazil, Portaguese, etc)
Fact: When you use this API demo you consent to providing your voice input data to Microsoft for service improvement purposes
Microsoft Language Understanding Intelligent Service (LUIS) service are used to generate the intent. MS was working on it for 4yrs. Natural languages process for bots to be more intelligent.
VIDEO: https://www.luis.ai/
Weather? What is the weather? What's it like outside? (single form)
this is what LUIS aims to do, to use natural language understanding for detecting intent).
How many of you are familiar with WeChat (messaging app popular in China).
Xiaoice (small ice) is a great example of a chat bot that incorporates MS APIs. It is a great demo app created by MS to display natural language processing through the power of Ai.
You can see that Xiaoice can understands emotions thru text and intent.The more ppl use it the smarter she becomes (take a second to read this conversation).