This document discusses conditional random fields (CRFs) and their use for named entity recognition and event extraction tasks. It notes that CRFs address the label bias problem of hidden Markov models (HMMs) and maximum entropy Markov models (MEMMs) by conditioning on observations and considering many feature types. CRFs model transition probabilities between states and state probabilities conditioned on observations, using edge features between labels as well as vertex features of the observations.
2. BioNLP'09
Event rather than entity
Most entities are given
3 tasks
− Event detection and characterization
− Event argument recognition
− Negations and speculations
3. Example
quot;I kappa B/MAD3 masks the nuclear localization
signal of NFkappa B p65 and requires the
transactivation domain to inhibit NFkappa B
p65 DNA binding. quot;
Event: negative regulation
Trigger: masks
Theme1: the first p65
Cause: MAD3
Site: nuclear localization signal
4. Example
quot;In contrast, NFkappa B p50 alone fails to
stimulate kappa Bdirected transcription, and
based on prior in vitro studies, is not
directly regulated by I kappa B. quot;
Event: regulation
Theme1: this p50
Trigger: regulated
Negation: true for this event
Speculation: none
5. HMM and MEMM
(X1, X2, ...)
Observations
(Y1, Y2, ...)
labels
p(Xi , Yi)
X ranges over observation sequence
Y ranges over and label sequence
Requires independence assumption
i.e. each item is labelled independently
7. MMEM Label Bias Problem
Probability given the current state
− Transitions leaving a state compete against
each other
not all states
− Perstate normalization
− Probability bias towards states with few transitions
− Demonstrated experimentall
8. Label Bias Example
Training data:
− A B C D
− A B D D
− A B C E
− A B D C
Model says:
− C > D 50%
− C > E 50%
Why predict E when D is much more common?
9. CRF Solution
Model probability of transitions and probability
of states
CRFs
− Models probability of transition between states
− Probability is conditional on current observation
− Not normalised
− Considers many quot;featuresquot; of observations
10. Features
quot;edge featuresquot; as well as quot;vertex featuresquot;
− Word is capitalized
− Word ends in quot;ingquot;
− Label is quot;proper nounquot;
Features are important!