Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
Data Modelling Zone 2019 - data modelling and JSON
1. George McGeachie, Stuttgart 2019
You’re a Data Modeller, so why
should you be interested in JSON?
Sounds a bit techie to me, so why should a data
modeller like me be interested in it?
2. George McGeachie
• Co-author of “Data Modeling Made Simple with
PowerDesigner”, PowerDesigner trainer, and data
modelling tool junkie.
4. What is JSON?
Java Script Object Notation
• a simple open-standard human-readable format for data
• used for several purposes, such as publishing public datasets and for
passing data between architectural layers and components
5. What is JSON?
{
"name": "moodle-core-blocks",
"builds": {
"moodle-core-blocks": {
"jsfiles": [
"blocks.js",
"manager.js",
"blockregion.js"
]
}
}
}
{"signifier" : "This is a PowerDesigner
model export",
"Object Type" : "Logical Data Model",
"id" : "55358DE6-A3A1-4BE0-8685-
18C0ACBA10A1",
"description" : "",
"note" : "",
"asset" : "f054830b-65e3-43c8-b535-
58f9fe7f5caf",
"community" : "",
"domain" : "",
"Extraction Date and Time" :
"1533765902",
"Extracted By" : "George McGeachie",
"Extracted from File" : "",
"Repository Version" : "",
"Model-Level Entities" : "",
"Packages" :
7. What is JSON?
(Java Script
Object
Notation)
Sounds a bit techie to me, so why should
a data modeller like me be interested in
it?
The same questions were asked about
XML almost 20 years ago.
Today’s technical environment is a lot
different from 20 years ago, and the
capabilities of modelling and design tools
have also changed a lot since then.
What hasn’t changed is the hierarchical
data structure of XML and JSON data –
does anyone remember how COBOL
programs organise their data?
11. A simple data model
AtributesEntities
Model
Model Name
Object Type
ID
Description
Entity
Entity Name
Attribute Count
Attribute
Attribute Name
12. COBOL IS HIERARCHICAL
01 MODEL-EXPORT
03 MODEL-HEADER
05 MODEL-NAME PIC X(50)
05 OBJECT-TYPE PIC 99
05 ID PIC X(36)
03 MODEL-DETAIL
05 DESCRIPTION PIC X(1024)
05 ENTITY-COUNT PIC 9999
03 ENTITIES OCCURS VARYING BY ENTITY-COUNT
05 ENTITY-DETAIL
07 ENTITY-NAME PIC X(50)
07 ATTRIBUTE-COUNT PIC 9999
05 ATTRIBUTES OCCURS VARYING BY ATTRIBUTE-COUNT
07 ATTRIBUTE-NAME
….
Atributes
Entities
Model
Model Name
Object Type
ID
Description
Entity
Entity Name
Attribute Count
Attribute
Attribute Name
13. XML IS HIERARCHICAL
This is the same model
<xs:element name="MODEL">
<xs:complexType>
<xs:sequence>
<xs:element name="ENTITY">
<xs:complexType>
<xs:sequence>
<xs:element name="ATTRIBUTE">
<xs:complexType>
<xs:attribute name="ATTRIBUTE_NAME">
</xs:attribute>
</xs:complexType>
</xs:element>
</xs:sequence>
<xs:attribute name="ENTITY_NAME">
</xs:attribute>
<xs:attribute name="ATTRIBUTE_COUNT">
</xs:attribute>
</xs:complexType>
</xs:element>
</xs:sequence>
<xs:attribute name="MODEL_NAME" use="required">
</xs:attribute>
Model
/MODEL
Model Name
Object Type
ID
Description
Entity Count
S
Entity
Entity Name
Attribute Count
S
Attribute
Attribute Name
AtributesEntities
Model
Model Name
Object Type
ID
Description
Entity
Entity Name
Attribute Count
Attribute
Attribute Name
14. The same model in JSON
{ "$schema": "http://json-schema.org/schema#",
"title": "JSON",
"objects": {
"model": {
"title": "Model",
"properties": {
"model_name": {
"title": "Model Name"
},
...
},
"relationship_1": {
"type": "array",
...
},
"primary key": ["model_name"],
"required":["model_name"]
},
"entity": {
"title": "Entity",
"properties": {
...
},
...
},
"relationship_2": {
"type": "array",
"object": { "rel": "Attribute",
"href": "#/objects/attribute" },
"minItems": 0,
"uniqueItems": true
},
...
},
"attribute": {
"title": "Attribute",
"properties": {
"model_name": {
"title": "Model Name"
},
... } } }
AtributesEntities
Model
Model Name
Object Type
ID
Description
Entity
Entity Name
Attribute Count
Attribute
Attribute Name
15. Have you ever?
• Generated COBOL layouts from data models?
• Reverse-engineered data models from COBOL layouts?
• Generated XML Schemas from data models?
• Reverse-engineered data models from XML Schemas?
A developer once
told me he’d be
happy to base XML
Schemas on our
models if we could
generate the
schemas for him
(our tool didn’t do
that)
21. Did you see the talk
given by Pascal
Desmarets (from
Hackolade) this
week?
22.
23. How’s your data governance and lineage today?
Do you know what data you have in
your organisation, where it is, how it got
there, and what happened to it along
the way?
• Service-oriented architecture
• Enterprise Services
• ETL
• Procedural Code
• Etc.
Are you in control of the definition of
the data you move around??
Do you have database columns
containing XML or JSON?
• Do you know what data is in those
columns?
• Are you managing them with schemas?
This is a metadata question!
24. Are you ready for a
challenge?
• Does your data modelling /
design tool allow you to design,
document or manage JSON
structures?
• Do you have a data lineage tool
to do that for you?
25. I don’t know of any single tool that does
everything I would want it to do
• Model JSON
• Reverse-engineer JSON
• Generate a JSON Schema
• Validate a JSON document against a Schema or a model
• Model the JSON or XML structure inside a database column
• Link a JSON schema / structure to a message or chunk of code that
creates or consumes it
• Keeps the usage metadata up to date, so I can trust what I’m looking
at
Model
Model Name
Object Type
ID
Description
Entity Count
Entity
Entity Name
Attribute Count
Attribute
Attribute Name
