Understanding human information •Access and understand virtually any source of information on-premise and in the cloud •A strategic pillar of HP’s HAVEnBig Data platform •Non-disruptive, manage-in-place approach complements any organization

1. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
HPIDOL
Speaker’s name
Month day, 2014

2. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.2
Video surveillance
Wire tapping
Internet of Things
Facebook likes
Tweets
Drones
Online shopping Search queries Tweets
RBMS Social sentiment
CRM
Web logs
User clickstreams
Business data feeds
Mobile
SMS/MMS
User generated content
Apps YouTube
Service logs
The dawn of the information era

3. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.3
Improve customer
relationship
Extend life
expectancy
Deliver better,
smarter products
Ensure governance
& compliance
Protect and save
lives
HP IDOL makes data matter
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

4. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.4
Understanding meaning is the key to solving
information challenges
Risk modeling Fraud detection Competitive advantage Behavior analysis Knowledge delivery
?
Volume VelocityVariety Veracity
Fin Services ManufacturingLife Sciences Hospitality GovernmentTelecom RetailEntertainment Energy HealthcareMedia
Future challenges

5. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.5
Understanding human information
• Access and understand virtually any source of information on-premise and in the cloud
• A strategic pillar of HP’s HAVEn Big Data platform
• Non-disruptive, manage-in-place approach complements any organization
Social Media Video Audio Email Texts Mobile
Transactional Data IT/OTDocuments Search Engine Images
Harnessing the power

6. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.6
86%of corporations cannot deliver
the right information, at the
right time, to support enterprise
outcomes all of the time³
³Source: Coleman Parkes Survey
November 2012
Keyword, metatags, database technologies often fail
Legacy technologies fall short
• Manual process does not scale
• Multiple definitions of the same word
• Not real-time
• Inaccurate and subjective
• Limited definitions, no relativity
• No idea distancing
• Interoperability of tagging
• Retroactive reporting

7. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.7
How does HP Autonomy approach human information?
Continuous learning based on incoming data and contextAdaptive
Mathematical, language independent technologyProbabilistic
Extract main concepts present in informationConcept
Combination of proprietary technology and proven industry
standard methodologiesModeling

8. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.8
HP IDOL: Key enabling technology
• Mathematically based
• 15 years and over $280M in R&D
• >170 Patents
• Language independent
• Built for infrastructure
• All file types, all media types
(voice/video)
• Scalable and with security
• Platform/OS /device agnostic
• Managed in place

9. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.9
Powered by IDOL, the OS for human information
Social Media Video Audio Email Texts Mobile Transactional
Data
Documents IT/OT Search Engine Images
Apps for Exploratory
Information Analytics
Apps for Information
Governance and Management
Apps for Marketing
Optimization
HP Autonomy connectors
Developer/Partner
External/CloudHP Autonomy Enterprise
Applications
The OS for human
information
Repositories
Information
types
OS service layers 500+ functions
DigitalSafe SharePoint Hadoop
CRM
Jive
Exchange
Relational DB
ACA AeD
WorkSite HP Records Mgr MediaBin
Data Protector Connected LiveVault
Driven by advanced analytics to understand data in context from any source

10. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.10
Over 500 IDOL functions to augment your intelligence
Automatic hyperlinking
Conceptual search
Keyword search
Fieldtext search
Phrase search
Phonetic search
Field modulation
Fuzzy matching
Implicit profiling
Explicit profiling
Community and expertise
network
Agents
Intent-based ranking
Alerting
Social feedback
Eduction
Automatic clustering
Clustering 2D/3D
Autoclassification
Auto language detection
Sentiment analysis
Automatic taxonomy
generation
Automatic query guidance
Highlighting
Parametric refinement
Summarization
Real-time predictive query
Metadata extraction
Automatic tagging
Faceted navigation
Inquire
Search your data
Investigate
Analyze your data
Interact
Personalize your data
Improve
Enhance your data

11. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.11
Search your data
• Conceptual, Keyword or Object
• Extensive Field combinations
• Full Meta Search
• Linearly Scalable
• Fault Tolerant
• Disaster Recovery Friendly
• All Information
• Real-Time Data
• Audio and Video
• Mapped Security
• Fully Extendable
• Leverages Existing Security
Accuracy
Robust Architecture
Reach
Security
Inquire
“Search your data”
Investigate
“Analyzeyour
data”
Interact
“Personalizeyour
data”
Improve
“Enhance your
data”

12. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.12
Inquire
“Search your data”
Investigate
“Analyzeyour
data”
Interact
“Personalizeyour
data”
Improve
“Enhance your
data”
Analyze your data
Quickly evaluate the relevance of information
• Automatic Query Guidance (providing top themes from query results in real time)
• Concept navigation via advanced visualizations (node graphs, theme tracking, topic maps,
broadcast analysis)
• Intelligent summarization (simple, concept and context)
• Intelligent highlighting (search terms, phrases, concepts, context, fidelity to query grammar)
• Concept streaming (Real-time summaries from audio that are contextual to queries and intent)
• Intelligent de-duplication, including “near” de-duplication
Use structure to navigate the data
• Structured, semi-structured and XML support
• Parametric search (unlimited nesting and association support)
• Directed navigation (create compelling navigation for users)

13. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.13
Personalize your data
We are what we…
Inquire
“Search your data”
Investigate
“Analyzeyour
data”
Interact
“Personalizeyour
data”
Improve
“Enhance your
data”

14. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.14
Personalize your data
Explicit profiling (agent):
user-defined
•Define your interest using:
- Natural language descriptions
- Keyword/ Boolean rules
- Refine by example
•Automatically monitor information
•Customizable
•Share interests with knowledge community
Implicit profiling: capturing
behavior data
• Fully automatic
• Ongoing monitoring of data consumption
and contribution
• Multi-faceted profiles
• Always up-to-date
Expertise
CommunitiesAgents
Profiles
Dynamic communities of interest
•Expert identification
•Define business rules to guide relationships
•Automatically form and
manage community
•Collaboration Networks
•Document rating
•Consumer groups
Expertise Expertise
Inquire
“Search your data”
Investigate
“Analyzeyour
data”
Interact
“Personalizeyour
data”
Improve
“Enhance your
data”

15. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.15
Inquire
“Search your data”
Investigate
“Analyzeyour
data”
Interact
“Personalizeyour
data”
Improve
“Enhance your
data”
Exploratory analytics that help you discover the “unknown unknowns”
Enhance your data
Managed classification
• Create categories using business rules or training
Automatic classification and clustering
• Automatically determine categories based on patterns and relationships in information
• Spot analysis of all themes and grouping
• Time sensitive analysis; What’s hot? What’s New?
Eduction
• Apply structure to unstructured data by extracting key fields and entities
• Hundreds of entities supported, including names, addresses, credit card information, sentiment, intent, etc
Audio analysis
• Speaker independent speech to text, speaker identification, audio events, language identification, etc
Image and video analysis
• Next generation image classification (is this a car?/find more like “this”)
• On-screen OCR, logo detection, intelligent scene analysis, Color and texture analysis,
story segmentation, etc

16. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.16
HP Autonomy solutions family, powered by IDOL
IDOL
Compliance
Litigation Readiness
Storage Optimization
Database Archiving
eDiscovery
Supervision
Legal Hold
Enterprise Search &
Analytics
Voice of the Customer
Voice of the Worker
Media Intelligence
Video Surveillance
Big Data Analytics
Knowledge Mgmt
Content Access
& Extraction
Records Mgmt
Legal Content Mgmt
Business Process Mgmt
Document Mgmt
Records Mgmt
Legacy Clean Up
Server Data Protection
Virtual Machine Data
Protection
Remote & Branch
Office Data Protection
Endpoint Device
Data Protection
Cloud Data Protection
Enterprise
Content Mgmt
Archiving &
eDiscovery
Data
Protection
Web Experience Mgmt
Web Optimization
Search Engine Marketing
Marketing Analytics
Contact Center Mgmt
Rich Media Mgmt
Aurasma - Augmented
Reality Mobile Experience
Digital Marketing
Experience
Information
Analytics
Information Management & Governance Marketing
Optimization
Hybrid
OEM
Software
Cloud for human information

17. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.17
HP technology powered by IDOL
Enterprise Group
HP StoreAll
HP StoreOnce
HP Gen 8 Appliances
Enterprise Services
HP Social Command Center
HP Information Governance
PPS
HP Flow
HP Live Photo
HP Connected Backup
IDOL + Hadoop
IDOL + Vertica
HAVEn
Big Data
IDOL + ArcSight
Security HP Labs
Compass

18. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Foundationalmethodology

19. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.19
Strong information and weak information
Key Words are small amounts of very strong information without contextLarger amounts of weaker information is what humans refer to as “context”
“Mercury”
Is it a planet?Is it an element?Is it a car?With high certainty; its and element!
“A heavy element and the only metal that is liquid at standard conditions for
temperature and pressure with the symbol Hg and atomic number 80, commonly
known as quicksilver”

20. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.20
Uses pattern-matching and probabilistic modeling to form an understanding of content
HP IDOL understands the meaning of information
Fundamentally language-independent
• Treats words as symbols
Allows incoming data to dictate the model,
not pre-defined rules or dictionaries
• Adapts to changing definitions
Optimized with language packs
• Eduction, sentiment analysis, speech analytics
Information Theory
and Bayesian Inference

21. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.21
Best-in-class combination of approaches
XML and Boolean+
Natural language processing
Probabilistic
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

22. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.22
If we toss a coin 100 times and get heads every time,
what’s the probability of getting a head on the 101st?
Traditional probability says:
50%
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

23. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.23
If we toss a coin 100 times and get heads every time,
what’s the probability of getting a head on the 101st?
Adaptive intelligence: prior information changes the model of understanding
Bayesian Inference says:
99+%
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

24. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.24
What is in front of this wake?...
With high probability we can say
there is a……..
BOAT!

25. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.25 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

26. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.26
Let’s play hang man
_ _ _ e _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ e _ _ _ _ _ _ _ _ _ _ _ _ _ t _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ e _ _ _ i _ _ _ _ i _ i _ _ t i _ _ _ _ i _ _ i _ _ _ i _ _ __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ x _ _ _ _ _ _ _ _ _ _ _ _ _
e
t
a
i
n
o
s
r
l
d
h
c
u
m
f
p
y
g
w
v
b
k
x
j
q
z
Supercalifragilisticexpialidocious_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

27. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Platformfeatures

28. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.28
Language independence
• Free from linguistic restraints and rules
• Automatically adapts to changing definitions
• Over 170 live customer languages
• Single, multibyte and Unicode languages
• Optional language packs for localization
Department of Homeland Security - Requires extremely
precise handling of foreign languages, including Chinese
and Arabic
Open V – China’s largest online video website

29. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.29
HP IDOL powers the largest systems in the world
Scalability
Millions of users
• Dept of Defense: 2.5 million users
Billions of documents
• Large bank: Over 1 bn emails
• Pharma: 50 terabytes of data in
discovery repository alone
High throughput
• Bloomberg: Alert on 46m emails per day

30. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.30
Mapped security
• Fully integrated Kerberos authentication together with
Secure Socket Layer (SSL) encryption across all transactions
• Compliance with all major Security Standards, including
US DoD5015.2, UK TNA2002, Australia’s VERS, ISO 15489
• Full-range of customizable security functionality:
– Discretionary access control (ACL based)
– Mandatory access control (Based on metadata)
– Kerberized access to IDOL
– SSO authentication using Windows Active Directory
Single supplier to US Department
of Homeland Security

31. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.31
Intelligent compaction
• Pause and resume the operation without causing
corruption
• Monitor the progress
• Skip large sections of the index when appropriate to
expedite the operation
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

32. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.32
http://host:ACIPort/action=admin
HP IDOL admin
Answer common questions and ease
common actions:
• “Why is this query slow?”
• “What’s using up so much memory in my engine?”
• “Is my engine operating as expected?”
• “I need to perform some light maintenance
(DREREPLACEs, etc) but don’t want to bother
writing a perl script.”

33. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Architecture

34. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.34
HP IDOL connector overview
Connector actions
• Synchronize (fetch)
• View
• Identifiers, Collect, Hold, ReleaseHold
• Insert, Delete, Update
Repository Connector
Connector
framework
server
IDOL
LUA w/IDOL
extensions
Document
Format
detection
Pre-import
processing
KeyView
filtering
Post-import
processing
LUA w/IDOL
extensions
Index into
IDOL
Repository
Connector
Connector
framework server
IDOL
Repository
Connector
Repository
Connector
DIH
IDOL IDOL

35. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.35
HP IDOL data ingestion pipeline
LUA scripting engine is
available within connectors
KeyView file format process,
Eduction and LUA scripting
engine are available within CFS
Repository
Connector
Connector
framework server
Content
Repository
Connector
Repository
Connector
DIH
IDOL Proxy
Index tasks
OCR
Audio/Video
Category
APA Agents

36. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.36
Providesaflexiblewayofbatching,scheduling,routing,andaggregatinginformationintoIDOLservers
Distributed Index Handler (DIH)
Features
• Consistent hashing
• Batch indexing
• Index routing
• Virtual databases
• Categorization-based indexing
• Time-based indexing
Benefits
• Seamless integration with backend modules
• Resilience
• Scalability
• Flexibility
IDOL Server 1 IDOL Server 2
Distributed
Index Handler
(DIH)
Connector
Mirror/non
mirror index

37. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.37
Intelligent query distribution
Distributed Action Handler (DAH)
Features
• Arbitrary distribution
• Mirrored configuration
• Non-mirror configuration
• Load balancing
• Fail-over
Benefits
• Linear scaling
• Improved performance
• Reduced processing time
• Robustness
IDOL 1 IDOL 2 IDOL 3 IDOL 4
1
DAH1
N 1 N
DIH1
N N
DAH3 DAH2
DIH3 DIH2

38. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.38
Globally distributed system

39. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Advancedfunctionsindepth

40. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.40
HP IDOL retrieval methods
Conceptual
• Natural language
• Conceptual matching
• Unstructured refinement
Business rules
• Boolean
• Keyword
Parametric
• Structured refinement
Inquire
“Search your data”
Investigate
“Analyzeyour
data”
Interact
“Personalizeyour
data”
Improve
“Enhance your
data”

41. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.41
Over100operators forBoolean search
AND
OR
NOT
NEAR
NEARn
DNEAR
DNEARn
WNEAR
WNEARn
BEFORE
AFTER
EOR
WHEN
WHENn
vAND
vSUBSTRING
vMATCHES
NEAR
NEAR/n
SENTENCE
PARAGRAPH
BEFORE
AFTER
ORDER
SOUNDEX
MANY
[n] WORD
CASE
PHRASE
. >
. >=
. <
. <=
. !=
. =
LANG/x
TODAY
YESTERDAY
NOW
NOW+n
NOW-n
term
term*
term?
vOR
vNOT
vACCRUE
vANY
vALL
vIN
vWHEN
vCONTAINS
vENDS
vSTARTS
vSUBSTRING
vCONTAINS
vENDS
vSTARTS
FREETEXT
STEM
TYPO
TYPO/n
YES-NO
PRODUCT
SUM
COMPLEMENT
LOGSUM
LOGSUM/n
MULT
MULT/n
FREQ
term~
term[100]
term[*1.5]
"term"
"term phrase"
term:field
"term
phrase":field
~term
FUZZY()
FUZZYnn()
SOUNDEX()
APCMMOD[]
term[~]
Inquire
“Search your data”
Investigate
“Analyzeyour
data”
Interact
“Personalizeyour
data”
Improve
“Enhance your
data”

42. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.42
Conceptual search
High recall and precision
• Return documents that do not contain query terms
but are conceptually related
Input sentences or entire document as query
• Extracts main concepts in the query to deliver the
most relevant results
Inquire
“Search your data”
Investigate
“Analyzeyour
data”
Interact
“Personalizeyour
data”
Improve
“Enhance your
data”

43. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.43
Automatic hyperlinking
• Automatically retrieves conceptually related content
• Searches automatically done for the user
• Increase productivity and reduce duplicate work
Inquire
“Search your data”
Investigate
“Analyzeyour
data”
Interact
“Personalizeyour
data”
Improve
“Enhance your
data”

44. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.44
Inquire
“Search your data”
Investigate
“Analyzeyour
data”
Interact
“Personalizeyour
data”
Improve
“Enhance your
data”
Add context to short queries by grouping results into concepts
Automatic query guidance
Query
”Madonna”
Results: Documents
containing ”Madonna”
Query
search
Documents about:
1. Singer
2. Italian Renaissance
3. Madonna Further
suggestions…
Most likely
meaning…
Result
documents
Conceptual
clustering

45. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.45
Summarization
Quick summary
(N+ lines)
Context summary
(What is this doc about with relation
to query terms?)
Concept Summary
(What do I look for with regards
to interest rates?)
Inquire
“Search your data”
Investigate
“Analyzeyour
data”
Interact
“Personalizeyour
data”
Improve
“Enhance your
data”
Information Theory and
Bayesian Inference

46. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.46
Directed navigation Inquire
“Search your data”
Investigate
“Analyzeyour
data”
Interact
“Personalizeyour
data”
Improve
“Enhance your
data”
Narrow search with facets

47. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.47
Visualization of main topics Inquire
“Search your data”
Investigate
“Analyzeyour
data”
Interact
“Personalizeyour
data”
Improve
“Enhance your
data”

48. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.48
Understanding the customer at the level of a dialog
Contextual segmentation
Geo+Demo+Psychographic
segments Behavioralsegments
Functions
Performance
FeatureDriven
Reviews
News
Adverts
Socialmedia
Buzzdriven
18-35yrs
35-65
Seniors
HaveKids
Male
Female
Semantic
segments
LargeScreen
Lotsofstorage
HighResDisplay
Wouldgiveit5
stars
GreatValuefor
Price
Inquire
“Search your data”
Investigate
“Analyzeyour
data”
Interact
“Personalizeyour
data”
Improve
“Enhance your
data”

49. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.49 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Intent-based ranking
Search results personalized and targeted based on user
and context
Profile developed through complete behavior
analysis… implicit or explicit profiling
Gather data from content consumption,
content contribution, interaction with
colleagues, etc.
Inquire
“Search your data”
Investigate
“Analyzeyour
data”
Interact
“Personalizeyour
data”
Improve
“Enhance your
data”

50. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.50
Fostercollaborationbyautomaticallymatchingandconnectingemployeeswithsimilar needs
Connect with your colleagues
Experts
Communities
Files
Social
Inquire
“Search your data”
Investigate
“Analyzeyour
data”
Interact
“Personalizeyour
data”
Improve
“Enhance your
data”

51. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.51
Product performance issues
Clustering
Side letters
Off balance
sheet transactionsAutomatically
partition the data
so that similar
information is
clustered together
Inquire
“Search your data”
Investigate
“Analyzeyour
data”
Interact
“Personalizeyour
data”
Improve
“Enhance your
data”

52. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.52 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Topical sentiment analysis
Decomposition and classification within a sentence to
pull out specific topics
“I stayed at the Marriott last week, and though the
mattresses were very nice, the service was awful.”
Is this Positive? Negative? Neutral?
How much Positive? How much Negative?
Inquire
“Search your data”
Investigate
“Analyzeyour
data”
Interact
“Personalizeyour
data”
Improve
“Enhance your
data”

53. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.53
Hundreds of conceptual entities
Eduction
Quickly narrow search results with auto-identified facets and
conceptual entities such as employee names from documents
Validate or customize entities
• Is this a valid credit card number?
• What are all docs that contain SSNs?
• If area code is 415, output as Home Office
Pinpoint accuracy for multibyte languages such as CJK, Thai and
some European languages
Names
Places
IP addresses
Companies
Events
Relationships
Medicines
Airports
Cars
Social Security numbers
Phone numbers
Credit cards
Dates
Holidays
Job titles
Currencies
… many more
Inquire
“Search your data”
Investigate
“Analyzeyour
data”
Interact
“Personalizeyour
data”
Improve
“Enhance your
data”

54. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.54
Eduction Inquire
“Search your data”
Investigate
“Analyzeyour
data”
Interact
“Personalizeyour
data”
Improve
“Enhance your
data”
<Organization>
• National Security Agency
<Names>
• President Obama
• Vladimir Putin
• Edward Snowden
<Places>
• Moscow
• St. Petersburg
• Washington
• Syria
• Russia

55. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.55
Search video as easily as text
Transform rich media into intelligent assets
Inquire
“Search your data”
Investigate
“Analyzeyour
data”
Interact
“Personalizeyour
data”
Improve
“Enhance your
data”
Live video or playback
from archived footage
On-screen text
recognition
Face identification
Automatically generated
transcript using speech
recognition
Speaker identification
Timecode
synchronization
Automatic keyframe
generation
Automate
Automatically create metadata,
keyframes, transcriptions
Understand
Understand video footage and audio
streams in real time
Act
Apply advanced analytics such as
clustering and categorization, and link
with other file types

56. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.56
Most advanced speech technology
Convert spoken words to text
• Acoustic + Language Model
• Speech-to-Text and IDOL’s conceptual understanding
Eliminate manually adding metadata to A/V clips
Phonetic approaches have major problems
• No Conceptual or Contextual Language Understanding
• Keyword-Based
Model of language disambiguates similar terms
• U.S. President “Bush”
• “bush” as in a large plant
Inquire
“Search your data”
Investigate
“Analyzeyour
data”
Interact
“Personalizeyour
data”
Improve
“Enhance your
data”

57. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.57
Limitations of phonetic search
Phonetic sounds do not have a unique match
Only capable of keyword matching
• “Cambridge University”
• /k ey m b r ih jh y uw n ih v er s ax t iy/
• The University of Cambridge
• Cambridge colleges
• Kings College
• Trinity Hall
/k/ /ae/ /t/
“cat”
“category”
“scatty”
“catalogue”
?
Inquire
“Search your data”
Investigate
“Analyzeyour
data”
Interact
“Personalizeyour
data”
Improve
“Enhance your
data”

58. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.58
Accurate speech technology
Language independent, statistical algorithms to recognize speech +
Language dependent, acoustic and language models for each supported language
How is the voice being recorded?
Telephone models, Hz Rates
Broadcast Models
What language +
common phases,
product names, etc.
Trained dictionary
with vocabulary and conceptual
understanding
Recognized hypothesis
Inquire
“Search your data”
Investigate
“Analyzeyour
data”
Interact
“Personalizeyour
data”
Improve
“Enhance your
data”
Front end
processing
Front end
processing
Front end
processing
Front end
processing
Front end
processing
Speech

59. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.59
Statistical models of speech and language
Speech-to-text technology
P(W) = probability of word string W
P(A|W) = probability of a acoustic sequence A given W
Use Bayes rule to find the word string w that has the highest probability given the acoustic sequence
Inquire
“Search your data”
Investigate
“Analyzeyour
data”
Interact
“Personalizeyour
data”
Improve
“Enhance your
data”
Language model
W = arg max P(W|A) = arg max P(W) P(A|W)
P(A)
Acoustic model

60. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.60
Language model provides
probability of word sequence
Forms a conceptual understanding of language
“Can I help you?” vs. “Can eye help you?”
Trained from large text corpora
(Hundreds of millions of words)
Defines words that can be recognized
Use training text, e.g., broadcast news
Encompasses topic information, colloquial phrases, etc.
Adaptable for particular customer
Specialist vocabulary, e.g., product names
Inquire
“Search your data”
Investigate
“Analyzeyour
data”
Interact
“Personalizeyour
data”
Improve
“Enhance your
data”

61. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.61
Acoustic model analyzes the sounds
that comprise a spoken language
Inquire
“Search your data”
Investigate
“Analyzeyour
data”
Interact
“Personalizeyour
data”
Improve
“Enhance your
data”
Audio analyzed to extract energy at various frequencies
Dependent on audio format
Complex statistical techniques model both the sounds and audio
characteristics

62. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.62
Image technology: Text
Document field extraction
Inquire
“Search your data”
Investigate
“Analyzeyour
data”
Interact
“Personalizeyour
data”
Improve
“Enhance your
data”
<item>
<price>$6.23</pric
e>
<date>10/2/2012<
/date>
<purpose>Lunch</
purpose>
…
</item>
OCR: Read text from images
1D and 2D barcode reading
ISBN (“9870140189865”) PDF-417 (“LASTNAME, FIRSTNAME,…”)
Data Matrix
(“The Future of Ticketing…”)
Many more (about 20
barcode types)
Image artifacts such as wrinkled paper
Avoid non-text parts of the image
Column understanding

63. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.63
Image technology: 2D objects
Registered image Test image
Generic Logo recognition
Registered
Logos
Test image
Inquire
“Search your data”
Investigate
“Analyzeyour
data”
Interact
“Personalizeyour
data”
Improve
“Enhance your
data”

64. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.64
Image technology: Human analysis Inquire
“Search your data”
Investigate
“Analyzeyour
data”
Interact
“Personalizeyour
data”
Improve
“Enhance your
data”
Primaryclothing color= white
Not nude
Primaryclothing color= white
Not nude
Primaryclothing color= black
Not nude
Face detection
Face analysis
Found “President Obama” face

65. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.65 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Hadoop/HDFS connector
Ingest Hadoop data into IDOL for advanced retrieval
Extract metadata, enrich and conduct advanced
analytics for files stored in Hadoop
Push enterprise documents into Hadoop (chat data,
ODBC, documents) for MapReduce analysis
Collect documents in Hadoop for legal collection

66. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
What’snew

67. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.67
HP IDOL 10
Extending leadership in human information analytics
More powerful Easier to operate Reliable
• Analyze sentiment at a granular level
• Automatically extract 100s of entities
for improved search
• Enhance your Hadoop investment
• Deliver search results personalized for
each user
• Improve audio and image analysis
• Increase query speed by up to 30%
• Quickly answer performance-related
questions with our new visual dashboard,
IDOL Admin
• Dynamically expand capacity without re-
indexing for improved performance and no
downtime
• Increase your indexing speed by as much as
47x with improved data transmission
• Recover intelligently from system
failures with improved self-
diagnosis of indices
• Securely delete content from your
index
• Prevent the loss of documents
during the indexing process

68. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.68
Latest innovations in IDOL 10
Core IDOL algorithm
enhancements
• Improved compaction
• Improved ability to repair indices
• Improved query speed
• Incremental backup and point-in-
time restore
IDOL architecture
improvements
• Indexing flow control
• IDOL Admin
Speech
• Multi-CPU support
Eduction
• Improved handling of multi-byte
languages
• New grammars
• Degrees of sentiment analysis
• 3x performance improvement in
sentiment analysis
Image
• Object detection
• Unified analysis
…and many more!

69. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.69
Key strategic themes of HP IDOL development
Platform for search based applications
Enable internal and external partners to more easily leverage IDOL as a platform to build applications
Strengthen core functionality
Improve existing areas (e.g., sentiment), and continue growing in new areas (e.g., image)
Simplified consumption
Easier to install with more robust features
Consumable from private and public cloud for rapid web services
Next-generation enterprise search
Reinvent enterprise search in the era of cloud, mobile, and social computing
Big Data / analytics
Enable IDOL as content analytics platform in the broader Big Data / Information
Analytics ecosystem; integrate Hadoop

70. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Usecases

71. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.71
Insert slides from relevant decks

72. © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Thankyou