Fun with Text - Hacking Text Analytics

Fun With Text
Hacking Text Analytics
Cohan Sujay Carlos
Aiaioo Labs
Bangalore

I shall chuck three buzz-words at you:
• One type of big data is unstructured data
• One type of unstructured data is text
• The analysis of text is text analysis
Is text analytics important?

You didn’t answer my question.
Is text analytics important?
Hell, I don’t know!
But what are you doing reading this if it isn’t?

• One Machine Learning Tool
• How to reduce Text Analytics Tasks to
steps which you can solve with merely
this one Machine Learning Tool
What we’re going to cover

The One ML Tool
Get ready for the one ML tool you’ll need to
hack text analytics
DRUM ROLL !!!!!!!

The One
Get ready for the one ML tool you’ll need to
hack text analytics
The Classifier 

That ML Tool is the Classifier
What is a Classifier?
Something that performs classification.
Classification = categorizing
Classification = deciding
Classification = labelling
Classification = Deciding = Labelling

5’11”
5’ 8”
Classify these door heights as: Short or Tall ?
5’8”
5’11”
6’2”
6’6”
5’ 2”
6’8”
6’9”
6’10”

5’11” Short
5’ 8” Short
For classification, you always start with
some labelled data points.
5’8”
5’11”
6’2” Tall
6’6” Tall
5’ 2” Short
6’8”
6’9”
6’10” Tall

5’11” Short
5’ 8” Short
5’8” Short
5’11” Short
6’2” Tall
6’6” Tall
5’ 2” Short
6’8” Tall
6’9” Tall
6’10” Tall
If you were doing analysis, you’d ask a
human to come up with a rule like:
A door below 6’ is Short else it’s Tall

5’11” Short
5’ 8” Short
In ML, you just provide some examples.
The computer discovers the rule.
5’8”
5’11”
6’2” Tall
6’6” Tall
5’ 2” Short
6’8”
6’9”
6’10” Tall

5’11” Short
5’ 8” Short
5’8”
5’11”
6’2” Tall
6’6” Tall
5’ 2” Short
6’8”
6’9”
6’10” Tall
The ML algorithm learns something like:
A door below 6’ is Short else it’s Tall

5’11” Short
5’ 8” Short
5’8” Short
5’11” Short
6’2” Tall
6’6” Tall
5’ 2” Short
6’8” Tall
6’9” Tall
6’10” Tall
You will learn to create an ML algorithm
that learns something like this and that
works with text.

Topic Classification
Can you tell which is about Politics and which is
about Sports?
The United Nations
Security Council today
Manchester United
beat Barca to reach

Can you train an ML algorithm to tell which is
about Politics and which is about Sports?
Manchester United
beat Barca to reach
Politics Sports
The United Nations
Security Council today

Start by taking some samples of documents on
politics and some samples of sports documents.
We’re using really short documents so you can
do all the calculations manually & see that this
ML algorithm really works!
Politics Sports
The United Nations
The United States and
Manchester United
Manchester and Barca

Step 1: Learn Multinomial Probabilities
The United States and
Politics
Manchester United
Manchester and Barca
Sports
P(United/Politics) = 2/7
The United Nations
P(Nations/Politics) = 1/7
P(States/Politics) = 1/7
P(Manchester/Sports) = 2/5
P(United/Sports) = 1/5
P(Barca/Sports) = 1/5
P(Politics) = 7/12 P(Sports) = 5/12
P(The/Politics) = 2/7 P(and/Sports) = 1/5
P(and/Politics) = 1/7

One ML Algorithm
Step 1 was easy!!!
Are you ready for Step 2 ?

Step 2: There’s no step 2
This is a Naïve Bayesian Classifier!!!

One ML Algorithm
Let’s put the classifier to work:
Let’s see if it can classify the following
documents :
1. United Nations
2. Manchester United
We are using deliberately short documents!

United Nations
P(Politics|United Nations)
>
P(Sports|United Nations)
So, the classifier has returned the category POLITICS

P(Sports|Manchester United)
>
P(Politics|Manchester United)
So, the classifier has returned the category SPORTS
Manchester United

Manchester United
Politics
United Nations
Sports
We have successfully used an ML algorithm to
tell us which document is about Politics and
which is about Sports !!!

Can We Solve Other Problems?
Now, we have an ML tool in our toolkit – a Naïve
Bayesian classifier.

So, if we can represent a text analysis problem
as a classification task, then we can solve it
using ML.

So, if we can represent a text analysis problem
as a classification task, then we can solve it
using ML.
So, let us learn how to represent text analysis
problems as classification tasks.

Problem 1: Sentence Segmentation
The problem of identifying the end-points of
sentences is called sentence segmentation. It
can be reduced to a classification problem.

Sentence Segmentation
Yes, Mr. Anurag. You need a D.L. to drive.
Sentence segmentation
Classify each ‘.’, ‘!’ and ‘?’ into:
1) sentence terminator and
2) not a sentence terminator.

Hurray! We have turned the problem
of sentence segmentation into a
classification problem.
Once you have modeled a text
analytics problem as an ML problem,
there is one more step you need to
perform to get a working solution.

FIND THE FEATURES!

Features
Which country’s flag is this?
Features are the clues you need for
good decision making.

Features
Which country’s flag is this?
One important feature for solving this
decision problem is colour.

Features for Topic Classification
For topic classification, the features
are the individual words in the text.

What are the features you would use?
That’s when you start reading the
research papers!!!

What are the features you would use?
Is_This_Character_a_Dot
Is_This_Within_Quotes
Is_Next_Letter_Capitalized
Is_Prev_Letter_Capitalized
Number_of_Words_in_Sentence_so_Far
Is_Next_Word_a_Name

Train an NB classifier on some text
whose characters are marked as
sentence terminators or not.
It will learn to assign characters to the
categories word terminator and not a
word terminator.

Problem 2: Tokenization
Now, get a D.L., Mr. Anurag.
For almost all analyses, you have to identify the
individual words in the text.
How can you break a sentence into words?
Now , get a D.L. , Mr. Anurag .

Tokenization
Tokenization
Classify each ‘.’, ‘!’, ‘,’, ‘ ’, ‘?’ etc. into:
1) word terminator and
2) not a word terminator.

Tokenization or Word Segmentation
We have turned the problem of word
segmentation into a classification
problem.
Train a classifier on text whose
characters are marked as word
terminators or not. It will learn to
assign characters to the categories
word terminator and not a word
terminator.

Problem 3: Part of Speech (POS) Tagging
Anurag needs a D.L.
Anurag/NNP needs/VBZ a/DT D.L./NN
How do you tag the words in a sentence?

Anurag needs a D.L.
Anurag/NNP needs/VBZ a/DT D.L./NN
POS Tagging
Run from the first to the last word in the
sentence, classifying each word in the sentence
into a part-of-speech category.

We have turned the problem of POS Tagging into a
Classification problem where you label each word as:
1. Noun
2. Verb
3. Adjective
4. Adverb
5. Interjection
6. Conjunction (and, or, if, neither … nor)
7. Pronoun (I, it, you, them)
8. Preposition (in, of, out)

Problem 4: Named Entity Recognition
Anurag is looking for a Hyundai car in Bangalore
Anurag = Person
Hyundai car = Vehicle
Bangalore = Place
How do you extract the named entities in a sentence
using a classifier?

Anurag = Person
Hyundai car = Vehicle
Bangalore = Place
Anurag/Person is/Other looking/Other
for/Other a/Other Hyundai/Vehicle car/Vehicle
in/Other Bangalore/Place

We have turned the problem of Named Entity
Recognition (NER) into a Tagging problem
where you label each word as:
1. Person
2. Vehicle
3. Place

But we know that we can turn the problem of
Tagging into a Classification problem over the
same labels:
1. Person
2. Vehicle
3. Place

Anurag/Person is/Other looking/Other
for/Other a/Other Hyundai/Vehicle car/Vehicle
in/Other Bangalore/Place
Named Entity Recognition
Run from the first to the last word in the
sentence, classifying each word in the sentence
into a Named Entity Recognition category.

Hey We Have One More ML Tool
We just built one more very useful ML tool!
The Extractor 

Extraction
What is an Extractor?
Something that performs extraction.
Extraction = recognizing
Extraction = finding
Extraction = locating
Extraction = Finding = Locating

Problem 5: Relation Extraction
Tim Cook is the new CEO of Apple Computers
Relation: CEO_of
Tim Cook (Person) Apple Computers (Org)
How do you identify relations between entities in a
sentence using a classifier?

Extracting Meaning
Text: Tim Cook is the new CEO of Apple Computers
Analysis: Tim/Person Cook/Person is the new CEO
of Apple/Org Computers/Org
Relation Extraction
Step 1:

Extracting Meaning
Relation Extraction
Step 1:
Step 2:
Relation Extraction
{Tim/Person Cook/Person, Apple/Org Computers/Org} =>
CEO_of

Extracting Meaning
Relation Extraction
Step 1:
Step 2:
Relation Extraction
Run through all pairs of named entities, classifying each
pair into CEO_of or Other.

But look at what we’re extracting!
Do you realize what we’re extracting?
Meaning !!! 

Extracting Meaning
Text: I am looking for a Hyundai car in B’lore
Syntactic Analysis: I/Pronoun am/BE looking/V
for a/DT Hyundai/NP car/NN in/PP Bangalore/NP
Semantic Analysis: I/Person am looking for a
Hyundai/Vehicle car/Vehicle in Bangalore/Place

But is this comprehensive?
Are you telling me that this is all I have to do to
extract every possible sort of simple meaning?
Yeah!!! 
All utterances fall into two main categories …

Intentional: I want to buy a computer.
Information: There was heavy snowfall in Sikkim.
Two Kinds of Utterances
360 Degree Text
Analysis

I want to buy a computer.
How do you deal
with intentional
utterances
360 Degree Text
Analysis

Intentional Utterance
Raw Text: Are you sad that Steve Jobs died?
Analysis: This person is inquiring about
someone’s emotions concerning Steve Jobs
Intention Analysis
Intention Holder: I
Intention: inquire

Subjective Utterance
Raw Text: I am sad that Steve Jobs died
Analysis: This person holds a positive opinion
on Steve Jobs
Sentiment Analysis
Sentiment Holder: I
Object of Sentiment: Steve Jobs
Polarity of Sentiment: positive

How do you deal
with informational
utterances
360 Degree Text
Analysis
There was heavy snowfall in Sikkim.

Approaches to
Extracting Meaning
Raw Text: There is heavy snowfall in Sikkim.
Analysis: Snowfall event
Event Analysis
Event: snowfall

Approaches to
Extracting Meaning
Raw Text: Bangalore is the capital of K’taka
Analysis: capital_of relation exists
Fact Analysis
Entity: Bangalore/Place
Karnataka/Place
Relation: Bangalore capital_of K’taka

Extracting Meaning
Text: I am looking for a Hyundai car in Bangalore
Semantic Analysis: I/Person am looking for a
Hyundai/Thing car/Thing in Bangalore/Place
Entity Extraction
Entities:
I Person
Hyundai car Thing
Bangalore Place
Fact Analysis

Extracting Meaning
Relation Extraction
Relation: CEO_of
Tim Cook (Person) Apple Computers(Org)
Fact Analysis

So, I can do anything (badly?)
I got it! I got it! I can do anything.
But how do I know how well I am doing it ? 

Measurement
Measurement:
This measurement thing is very
important in any design process,
because …

Measurement
How do you measure the performance of a
classifier?
Measurement:
… it lets you compare two designs and
decide which is better.

Before we go on …
… promise me …
… that you will never forget what I am about to
tell you … 

Measurement
Break up the data points into training
and test parts (usually an 80:20 split)
and never test on your training data.

Measurement
Why not test on the training data?
Break up the data points into training
and test – usually an 80:20 split.
Train on 80%
Test on the remaining 20%

Measurement
Why not develop on the test data?
Or if you are still developing features,
break up the data points into training,
development and test – usually a
70:10:20 split.
Train on 70%
Develop on 10%
Test on the remaining 20%

That applies to college as well …
… you won’t get accurate measurements …
… if you test students using questions that
appeared in the question bank!
… and you will be encouraging students to learn
things by rote!

Classification Quality Metric - Accuracy
Correct Answers
Total Number of Questions

Politics Documents classified as Politics
+ Sports Documents classified as Sports
Total Number of Documents
If your categories are Politics and Sports

Gold - Politics Gold – Sports
Observed - Politics TN (True Negative) FN (False Positive)
Observed - Sports FP (False Negative) TP (True Positive)
Point of View = Sports = (+ve)

Accuracy
Gold – Sports (1000) Gold – Politics (1000)
Observed – Sports TN = 990 FN = 100
Observed – Politics FP = 10 TP = 900
Point of View - Politics
= ?

Accuracy
= 94.5%

Classification Quality Metric - Recall
How many Politics document did it find
Total number of Politics documents
in the test data

Recall
= ?

Recall
= 90%

Recall
Observed – Sports TP = 990 FP = 100
Observed – Politics FN = 10 TN = 900
Point of View - Sports
= ?

Recall
= 99%

Classification Quality Metric - Precision
How many were really Politics documents
Total number of documents the
classifier identified as Politics

Precision
= ?

Precision
= 95%

Precision
= ?

Precision
= 99%

Are we done?
… kinda …
The more training you have, the better you will
get at text analysis … so keep learning!

How to Learn More?
Grab the UC Berkeley Natural Language
Processing Course’s slides. The course’s
name is CS 294.
Start reading the ACL conference’s
research papers (ACL = Association of
Computational Linguistics)

Now, are we done?
… kinda …
Have Fun!
… and I have an exercise for you …

Did you know
… that you can use a classifier …
… to do clustering?
The exercise is … think about it …

THE END
Fun With Text
Hacking Text Analytics
Cohan Sujay Carlos
Aiaioo Labs
Bangalore
The bottom

Fun with Text - Hacking Text Analytics

Recommandé

Recommandé

Contenu connexe

Similaire à Fun with Text - Hacking Text Analytics

Similaire à Fun with Text - Hacking Text Analytics (20)

Plus de aiaioo

Plus de aiaioo (9)

Dernier

Dernier (20)

Fun with Text - Hacking Text Analytics