View the companion webinar at: http://embt.co/1L8V6dI Some claim that, in the age of Big Data, data modeling is less important or even not needed. However, with the increased complexity of the data landscape, it is actually more important to incorporate data modeling in order to understand the nature of the data and how they are interrelated. In order to do this effectively, the way that we do data modeling needs to adapt to this complex environment. One of the key data modeling issues is how to foster collaboration between new groups, such as data scientists, and traditional data management groups. There are often different paradigms, and yet it is critical to have a common understanding of data and semantics between different parts of an organization. In this presentation, Len Silverston will discuss: + How Big Data has changed our landscape and affected data modeling + How to conduct data modeling in a more ‘agile’ way for Big Data environments + How we can collaborate effectively within an organization, even with differing perspectives About the Presenter: Len Silverston is a best-selling author, consultant, and a fun and top rated speaker in the field of data modeling, data governance, as well as human behavior in the data management industry, where he has pioneered new approaches to effectively tackle enterprise data management. He has helped many organizations world-wide to integrate their data, systems and even their people. He is well known for his work on "Universal Data Models", which are described in The Data Model Resource Book series (Volumes 1, 2, and 3).

1. © 2015 Universal Data Models, LLC - All Rights Reserved – Not to be copied or distributed without permissions 1
Aug 26th, 2015 Webinar,
By Len Silverston, Universal Data Models, LLC
Sponsored by Embarcadero Technologies

2. © 2015 Universal Data Models, LLC - All Rights Reserved – Not to be copied or distributed without permissions 2
Purpose
Share Keys
to Big Data Modeling
and How to Collaborate

3. © 2015 Universal Data Models, LLC - All Rights Reserved – Not to be copied or distributed without permissions 3
Agenda
• Big Data Overview
• Data Modeling in Big Data
• Collaboration Principles

4. © 2015 Universal Data Models, LLC - All Rights Reserved – Not to be copied or distributed without permissions 4
Big Data
Big data is a broad term for data sets so large or
complex that traditional data
processing applications are inadequate. Wikipedia
3Vs – Volume, Velocity, Variety
By 2020 - 44 zettabytes!
Mostly unstructured

5. © 2015 Universal Data Models, LLC - All Rights Reserved – Not to be copied or distributed without permissions 5
Unstructured Data
Information that either does not have a pre-
defined data model or is not organized in a pre-
defined manner. Wikipedia
How can data have
no structure?
Is "unstructured" data
merely unmodeled?*
* Structure, Models and Meaning’ Seth Grimes, Information Week,

6. © 2015 Universal Data Models, LLC - All Rights Reserved – Not to be copied or distributed without permissions 6
New Landscape - NoSQL
KEY VALUE
DATABASES
GRAPH
DATABASES
DOCUMENT
STORES
MongoDB
MUMPS Datab
ase
ObjectDatabas
e++
OrientDB
PostgreSQL
Qizx
RethinkDB
Rocket U2
Sedna
SimpleDB
Solr
TokuMX
OpenLink
Virtuoso
OpenLinkVirtuoso
Oracle Spatial and
Graph
Oracle NoSQL
Database
OrientDB
OQGRAPH
Profium Sense
R2DF
ROIS
Semblent
Lionsgate
sones GraphDB
SPARQLCity
Sqrrl Enterprise
Stardog
Teradata Aster
Titan
TripleBit
VelocityGraph
VertexDB
VivaceGraph
Weaver
WhiteDB
OhmDB
Redis
XAP
KV - solid-state drive or
rotating disk[edit]
Aerospike
BigTable
CDB
Clusterpoint Database
Server
Couchbase Server
FairCom c-treeACE
GT.M
Hibari
Keyspace
LevelDB
LMDB
MemcacheDB (using
Berkeley DB or LMDB)
MongoDB
NoSQLz
Coherence
Oracle NoSQL Database
OpenLink Virtuoso
Tarantool
Tokyo Cabinet
Tuple space
KV - eventually
consistent
Apache Cassandra
Dynamo
Oracle NoSQL Database
Project Voldemort
Riak
OpenLink Virtuoso
KV – ordered
Berkeley DB
FairCom c-treeACE/c-
treeRTG
FoundationDB
HyperDex
IBM Informix C-ISAM
InfinityDB
LMDB
MemcacheDB
NDBM
KV - RAM[edit]
Aerospike
Coherence
Hazelcastmemcached
OpenLink Virtuoso
BaseX
Cloudant
Clusterpoint
Database
Couchbase
Server
CouchDB
CrateIO
DocumentDB
Elasticsearch
eXist
HyperDex
Informix
Jackrabbit
Lotus
Notes (IBM
Lotus Domino)
MarkLogic
AllegroGraph
ArangoDB
Blazegraph
Bitsy
BrightstarDB
Cayley
DEX/Sparksee[2]
Filament
GraphBase
Graphd
Graph Engine[3]
Grapholytic
Horton
HyperGraphDB
IBM System G Native
Store
InfiniteGraph
InfoGrid
jCoreDB Graph
Neo4j
OntotextGraphDB
Orly

7. © 2015 Universal Data Models, LLC - All Rights Reserved – Not to be copied or distributed without permissions 7
In this age of Big Data,
is there ‘less of a need’ or
‘more of a need’ for data modeling?
(or ‘no need’ or the ‘same need’)

8. © 2015 Universal Data Models, LLC - All Rights Reserved – Not to be copied or distributed without permissions 8
Why Model?
8
DATA
 Understand
 Design?
 Common semantics?

9. © 2015 Universal Data Models, LLC - All Rights Reserved – Not to be copied or distributed without permissions 9
UNDERSTAND
OVERSTAND
What are customers saying about our products?
What exactly do we mean by a customer?
Is a prospect that has signed a contract but not paid yet, a customer?
Is a person that only bought from us over 10 years ago a customer?
Is an organization that bought a minor item from us a customer?
Is sales volume based on orders, invoices, payments, or GL posts?
What are we predicting our sales volume to be this quarter?

10. © 2015 Universal Data Models, LLC - All Rights Reserved – Not to be copied or distributed without permissions 10
REQUIRES
TEXTCON
TEXT

11. © 2015 Universal Data Models, LLC - All Rights Reserved – Not to be copied or distributed without permissions 11
Traditional
MODEL
(and DESIGN)
LOAD
EXPLORE/
QUERY
DATA EXPLORE
‘Schema on write’

12. © 2015 Universal Data Models, LLC - All Rights Reserved – Not to be copied or distributed without permissions 12
Big Data
LOAD QUERY MODEL
NoSQL STORE
EXPLORE
But Fast and Agile!
‘Schema on read’

13. © 2015 Universal Data Models, LLC - All Rights Reserved – Not to be copied or distributed without permissions 13
Data modeling in Big Data
Customer
-
-
-
NoSQL DATABASE
Documents
-
-
-
Product
-
-
-
Key values
-
-
-
Conceptual/
business data model
Understanding
Logical/physical
data model
Architecture/Design
RELATIONAL DATABASE
(i.e., Data
warehouse/data mart)
May transfer into structured database
(using models)

14. © 2015 Universal Data Models, LLC - All Rights Reserved – Not to be copied or distributed without permissions 14
Big Data Modeling Considerations
• Changes nature of modeling
– Later
– Modeling for understanding
• Design considerations - performance and scalable
• Changes where physical structures reside: in code
• Shifting functions to programming
–Performance
–Security
–Integrity
• Lately, SQL interfaces over NoSQL

15. © 2015 Universal Data Models, LLC - All Rights Reserved – Not to be copied or distributed without permissions 15
When to Model First,
When To Explore First
Explore First
 When format cannot be predicted
in advance (Rapidly changing data
structures)
 When you need to keep ‘data as is’
 Continually new sources of data
 Don’t know if valuable
(exploratory)
 Huge amounts of information (e.g.
streaming terabytes per minute)
E.g. Cyber terrorism, Sentiment
Analysis
Model First
 More predictable data structure
 When there is some flexibility to
modify/conform data
 Stable and known sources
 Know that it’s valuable
 Reasonable amount of information
for relational
E.g. Customer demographics, Product
info, Sales History

16. © 2015 Universal Data Models, LLC - All Rights Reserved – Not to be copied or distributed without permissions 16
What Does ‘Agile’ Mean?
• Customer solution – deliver value
• Flexible
• Fast
• Iterative
• Sustainable – constant
pace
• Quality design
(and efficient)
• Human Factors
– Communication - face to face
– Collaborative
– Trust
– Motivation
– Ongoing reflecting and adjusting
Quotes from principles behind the Agile
Manifesto can be found at
http://agilemanifesto.org/principles.html
“Our highest priority is to
satisfy the customer through
early and continuous
delivery of valuable
software.”
“Welcome changing
requirements, even late in
development. Agile
processes harness change
for the customer's
competitive advantage”
“Deliver working software
frequently, from a couple of
weeks to a couple of
months, with a preference
to the shorter timescale”
“Business people and
developers must work
together daily throughout
the project.”
“Working software is the
primary measure of
progress.”
“Agile processes promote
sustainable development.
The sponsors, developers,
and users should be able to
maintain a constant pace
indefinitely.”
“Continuous attention to
technical excellence and good
design enhances agility.”
“The most efficient and
effective method of conveying
information to and within a
development team is face-to-
face conversation.”
“Build projects around
motivated individuals. Give
them the environment and
support they need, and trust
them to get the job done.”
“Simplicity--the art of
maximizing the amount of
work not done--is essential.”
“At regular intervals, the
team reflects on how to
become more effective,
then tunes and adjusts its
behavior accordingly.”
“"The best architectures,
requirements, and designs
emerge from self-organizing
teams.”
Can we do this in
data modeling?

17. © 2015 Universal Data Models, LLC - All Rights Reserved – Not to be copied or distributed without permissions 17
What is Agile - What is NOT Agile?
• Agile data modeling
– Deliver flexible, quality data
models that facilitate
sustainability and deliver value, in
a quick, iterative, collaborative
way, building trust and
motivation
• NOT agile data modeling
– Quick & dirty
– Excuse to develop more silos
– Without quality or understanding
PERSON ORGANIZATION
PARTY
PARTYROLE
PARTYRELATIONSHIP
PARTYCONTACTMECHANISM
FACILITY
CASE
WORKEFFORT
ROLETYPEWORKEFFORTROLE
CONTACTMECHANISM
SUPPLIERCUSTOMER WORKER PARTNER
CONTACTMECHANISMTYPE
PROJECTPROGRAM TASK
FIXEDASSET
ASSIGNMENT
WORKEFFORT
ASSOCATION
COMMUNICATIONEVENT
FIXEDASSET
PRODUCT
GOOD SERVICE

18. © 2015 Universal Data Models, LLC - All Rights Reserved – Not to be copied or distributed without permissions 18
How can we perform
agile data modeling?
1. Re-use
2. Quick broad brush data
model
3. Correct model for
correct purpose
4. Prioritize
5. Deliver
6. Understand motivations
7. Have lots of choices
available
See Article: “Data Modeling’s Role in Agile Development”
http://tdwi.org/articles/2010/07/07/data-modeling-agile-development.aspx
RE-USE!

19. © 2015 Universal Data Models, LLC - All Rights Reserved – Not to be copied or distributed without permissions 19
EPISODE TYPE
EPISODE TYPE ID
DESCRIPTION
HEALTH CARE EPISODE
HEALTH CARE EPISODE ID
PATIENT PARTY ID (FK)
PATIENT ROLE TYPE ID (FK)
INCIDENT ID (FK)
EPISODE TYPE ID (FK)
EPISODE CREATE DATE
HEALTH CARE VISIT
HEALTH CARE VISIT ID
PATIENT PARTY ID (FK)
PATIENT ROLE TYPE ID (FK)
CONTACT MECHANISM ID (FK)
FACILITY ID (FK)
FROM DATE
THRU DATE
INCIDENT
INCIDENT ID
INCIDENT TYPE ID (FK)
INCIDENT DATE
DESCRIPTION
EMPL RELATED IND
INCIDENT TYPE
INCIDENT TYPE ID
DESCRIPTION
PATIENT
PARTY ID (FK)
ROLE TYPE ID (FK)
SYMPTOM
SYMPTOM ID
HEALTH CARE EPISODE ID (FK)
SYMPTOM TYPE ID (FK)
DESCRIPTION
SYMPTOM TYPE
SYMPTOM TYPE ID
DESCRIPTION
VISIT REASON
VISIT REASON ID
HEALTH CARE VISIT ID (FK)
SYMPTOM ID (FK)
HEALTH CARE EPISODE ID (FK)
DESCRIPTION
HEALTH CARE DELIVERY
HEALTH CARE DELIVERY ID
HEALTH CARE VISIT ID (FK)
HEALTH CARE EPISODE ID (FK)
HEALTH CARE OFFERING ID (FK)
FROM DATE
THRU DATE
DELIVERY NOTES
Re-use to
understand
Universal Data Models

20. © 2015 Universal Data Models, LLC - All Rights Reserved – Not to be copied or distributed without permissions 20
We must understand the
data and therefore
continue to develop data
models, even in this 'Big
Data' era.
Come on! Get into
the new mindset of
today’s Big Data! We
need to do things
differently today!
How can you use the data without
first understanding it?
Also, it’s important that we all
use common semantics.
Are you trying to slow
us down and continue
to try to enforce
bureaucracy?!
Data Modeler Data Scientist

21. © 2015 Universal Data Models, LLC - All Rights Reserved – Not to be copied or distributed without permissions 21
Biggest Issue:
“Mine”
Data ‘Mine’ing

22. © 2015 Universal Data Models, LLC - All Rights Reserved – Not to be copied or distributed without permissions 22
Key to Big Data Modeling:
Data ‘Ours’ing

23. © 2015 Universal Data Models, LLC - All Rights Reserved – Not to be copied or distributed without permissions 23
Keys To Collaboration
• Shared Purpose
• Understand Motivations
• Develop Trust
• Listen
• Manage Conflict

24. © 2015 Universal Data Models, LLC - All Rights Reserved – Not to be copied or distributed without permissions 24
Vision?
Mission?
1. Shared Purpose

25. © 2015 Universal Data Models, LLC - All Rights Reserved – Not to be copied or distributed without permissions 25
Can you state the exact mission
statement of your organization?
(without looking it up first!)
Be Honest
BE HONEST

26. © 2015 Universal Data Models, LLC - All Rights Reserved – Not to be copied or distributed without permissions 26
Position Versus Interest
INTERESTS A
POSITION A
INTERESTS B
POSITION B

27. © 2015 Universal Data Models, LLC - All Rights Reserved – Not to be copied or distributed without permissions 27
2. Understand Motivations

28. © 2015 Universal Data Models, LLC - All Rights Reserved – Not to be copied or distributed without permissions 28
Motivational Model - Sponsorship Map

29. © 2015 Universal Data Models, LLC - All Rights Reserved – Not to be copied or distributed without permissions 29
What Does The Business Really Need?
• Insight?
• Buying behavior?
• Assessment?
• Prescriptions?
• Predictions?

30. © 2015 Universal Data Models, LLC - All Rights Reserved – Not to be copied or distributed without permissions 30
Who is the MOST important person to
know their motivations?
A. The Most Influential Sponsor?
B. Your Boss?
C. Your Most Difficult Person Who Is the
Greatest Obstacle in Your Effort?
D. Yourself?

31. © 2015 Universal Data Models, LLC - All Rights Reserved – Not to be copied or distributed without permissions 31
Core Elements of TrustKeys to Trust3. Trust

32. © 2015 Universal Data Models, LLC - All Rights Reserved – Not to be copied or distributed without permissions 32
• Character
–Integrity
–Intent
– Vulnerability/openness
• Competence
–Capabilities
–Results
From “The Speed Of Trust” By Stephen M. R. Covey
Keys to Trust

33. © 2015 Universal Data Models, LLC - All Rights Reserved – Not to be copied or distributed without permissions 33
To Listen, ACCEPT
(A) ware, attention, alert
(C) are
(C) onfirm, check
(E) mpathize
(P) urpose
(T) otally (with all senses)
4. Listen and ACCEPT

34. © 2015 Universal Data Models, LLC - All Rights Reserved – Not to be copied or distributed without permissions 34
Keys to Trust5. Conflict Management

35. © 2015 Universal Data Models, LLC - All Rights Reserved – Not to be copied or distributed without permissions 35
What is the first thing to do in a conflict?
A. Define your strategy for winning?
B. Understand their perspective?
C. Don’t react?
D. Figure out a win-win?
E. Something else?

36. © 2014 Universal Data Models, LLC - All Rights Reserved 36
From “Getting Past No: Negotiating with Difficult People”, William Ury
Step 1.
Don’t React - Observe

37. © 2015 Universal Data Models, LLC - All Rights Reserved – Not to be copied or distributed without permissions 37
Don’t React
Event
Feelings/
Thoughts Emotional
Physical
Stories
Reaction
Freeze
Flight
Fight
Mess
Step 1. Don’t react

38. © 2015 Universal Data Models, LLC - All Rights Reserved – Not to be copied or distributed without permissions 38
Respond
Event
Feelings/
Thoughts
Emotional,
Physical,
Stories
Data
Stop – observe.
Data?
Questions?
Response
Intelligent
Actions
Step 1. Don’t react

39. © 2015 Universal Data Models, LLC - All Rights Reserved – Not to be copied or distributed without permissions 39
Big Data Modeling
Model To Understand – Even if after viewing data
Collaboration is the key
Find Common Purpose
Understand Motivations
Develop Trust
Listen
Conflict Management

40. © 2015 Universal Data Models, LLC - All Rights Reserved – Not to be copied or distributed without permissions 40
What Will You Do With This?

41. © 2015 Universal Data Models, LLC - All Rights Reserved – Not to be copied or distributed without permissions 41
Questions or More Info?
www.universaldatamodels.com
lsilverston@univdata.com
Twitter: @lensilverston
For info on template Models:
www.embarcadero.com/products
/er-studio-universal-data-models

42. © 2015 Universal Data Models, LLC - All Rights Reserved – Not to be copied or distributed without permissions 42
Creative Commons Image Attributions
Much thanks to those who provided the creative common images in this presentation.
Thanks for:
Stars https://www.flickr.com/photos/tom_hall_nz/17317951241/sizes/sq/ All rights reserved
by Kiwi Tom
Sky https://www.flickr.com/photos/cubagallery/9679210392 © All rights reserved
by ►CubaGallery
License