Using Oracle database and Oracle SQL to retrieve XML and JSON from for example webservices, store XML and JSON in the database, query XML and JSON with SELECT as relational data, create XML and JSON in various manners from relational data. Presentation given at OUGN Spring Conference 2016.
Non Text Magic Studio Magic Design for Presentations L&P.pdf
Read, store and create xml and json
1. BASEL BERN BRUGG DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. GENF
HAMBURG KOPENHAGEN LAUSANNE MÜNCHEN STUTTGART WIEN ZÜRICH
BASEL BERN BRUGG DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. GENF
HAMBURG KOPENHAGEN LAUSANNE MÜNCHEN STUTTGART WIEN ZÜRICH
Read, Store and Create XML and JSON
OUGN Spring Seminar 10-12 March 2016
Kim Berg Hansen
Senior Consultant
2. About me
Read, Store and Create XML and JSON2 3/30/2016
• Danish geek
• SQL & PL/SQL developer since 2000
• Developer at Trivadis AG since 2016
http://www.trivadis.dk
• Oracle Certified Expert in SQL
• Oracle ACE
• Blogger at http://www.kibeha.dk
• SQL quizmaster at
http://plsqlchallenge.oracle.com
• Likes to cook
• Reads sci-fi
• Chairman of local chapter of
Danish Beer Enthusiasts
3. About Trivadis
Read, Store and Create XML and JSON3 3/30/2016
Trivadis is a market leader in IT consulting, system integration, solution engineering
and the provision of IT services focusing on and
technologies in Switzerland, Germany, Austria and Denmark.
We offer our services in the following strategic business fields:
Trivadis Services takes over the interacting operation of your IT systems.
O P E R A T I O N
8. HTTP GET with HttpUriType
Read, Store and Create XML and JSON8 3/30/2016
select httpuritype(
'maps.googleapis.com/maps/api/directions/xml'
|| '?'||'origin=' || utl_url.escape(convert(
'Endelavevej 55, DK-5500 Middelfart'
, 'UTF8'
))
|| '&'||'destination=' || utl_url.escape(convert(
'Paul-Dessau-Strasse 6, D-22761 Hamburg'
, 'UTF8'
))
|| '&'||'mode=driving'||'&'||'alternatives=true'||'&'||'units=metric'
|| '&'||'region=eu'||'&'||'language=en'||'&'||'sensor=false'
).getxml() directions
from dual;
Retrieve XML directions from Google Maps API – No SSL (https) supported
9. HTTP GET with HttpUriType
Read, Store and Create XML and JSON9 3/30/2016
<?xml version="1.0" encoding="UTF-8"?>
<DirectionsResponse>
<status>OK</status>
<route>
<summary>E45 and A7</summary>
<leg>
<step>
<travel_mode>DRIVING</travel_mode>
<start_location>
<lat>55.4892447</lat>
<lng>9.7517085</lng>
</start_location>
<end_location>
<lat>55.4878639</lat>
<lng>9.7539612</lng>
</end_location>
<polyline>
<points>wvtqIesoz@k@j@k@iCh@i@tBsBVY`B_BpAkA</points>
</polyline>
<duration>
<value>65</value>
<text>1 min</text>
</duration>
.
.
.
<geocoded_waypoint>
<geocoder_status>OK</geocoder_status>
<type>street_address</type>
<place_id>ChIJRUp9K2iWTEYRmzjwsDRYoto</place_id>
</geocoded_waypoint>
<geocoded_waypoint>
<geocoder_status>OK</geocoder_status>
<type>street_address</type>
<partial_match>true</partial_match>
<place_id>ChIJZ1x_bZWFsUcRx7t3-L0HwsI</place_id>
</geocoded_waypoint>
</DirectionsResponse>
Retrieve XML directions from Google Maps API – XMLTYPE object output
10. HTTP GET with HttpUriType
Read, Store and Create XML and JSON10 3/30/2016
select httpuritype(
'maps.googleapis.com/maps/api/directions/json'
|| '?'||'origin=' || utl_url.escape(convert(
'Endelavevej 55, DK-5500 Middelfart'
, 'UTF8'
))
|| '&'||'destination=' || utl_url.escape(convert(
'Paul-Dessau-Strasse 6, D-22761 Hamburg'
, 'UTF8'
))
|| '&'||'mode=driving'||'&'||'alternatives=true'||'&'||'units=metric'
|| '&'||'region=eu'||'&'||'language=en'||'&'||'sensor=false'
).getclob() directions
from dual;
Retrieve JSON directions from Google Maps API – No SSL (https) supported
12. HTTP GET with APEX_WEB_SERVICE
Read, Store and Create XML and JSON12 3/30/2016
declare
directions clob; directions_xml xmltype;
begin
directions := apex_web_service.make_rest_request(
p_url => 'http://maps.googleapis.com/maps/api/directions/xml'
, p_http_method => 'GET'
, p_parm_name => apex_util.string_to_table(
'origin:destination:mode:alternatives:units:region:language:sensor' )
, p_parm_value => apex_util.string_to_table(
'Endelavevej 55, DK-5500 Middelfart:Paul-Dessau-Strasse 6, D-22761
Hamburg:driving:true:metric:eu:en:false' )
);
dbms_output.put_line(substr(directions,1,4000));
directions_xml := xmltype(directions);
end;
/
Retrieve XML (or JSON) directions from Google Maps API – SSL (https) supported via Wallet
Data always as CLOB whether XML or JSON – If XMLTYPE needed, use constructor
13. HTTP GET with APEX_WEB_SERVICE
Read, Store and Create XML and JSON13 3/30/2016
declare
parm_name wwv_flow_global.vc_arr2; parm_value wwv_flow_global.vc_arr2; directions clob;
begin
parm_name(1) := 'origin'; parm_value(1) := 'Endelavevej 55, DK-5500 Middelfart';
parm_name(2) := 'destination'; parm_value(2) := 'Paul-Dessau-Strasse 6, D-22761 Hamburg';
parm_name(3) := 'mode'; parm_value(3) := 'driving';
parm_name(4) := 'alternatives'; parm_value(4) := 'true';
parm_name(5) := 'units'; parm_value(5) := 'metric';
parm_name(6) := 'region'; parm_value(6) := 'eu';
parm_name(7) := 'language'; parm_value(7) := 'en';
parm_name(8) := 'sensor'; parm_value(8) := 'false';
directions := apex_web_service.make_rest_request(
p_url => 'http://maps.googleapis.com/maps/api/directions/json'
, p_http_method => 'GET' , p_parm_name => parm_name , p_parm_value => parm_value
);
dbms_output.put_line(substr(directions,1,4000));
end;
/
Retrieve JSON (or XML) directions from Google Maps API
Alternative call syntax building parameter name/value pairs manually
14. HTTP SOAP Webservice with APEX_WEB_SERVICE
Read, Store and Create XML and JSON14 3/30/2016
declare
envelope clob; weather xmltype;
begin
select xmlroot( xmlelement( "soap:Envelope"
, xmlattributes( 'http://www.w3.org/2001/XMLSchema-instance' as "xmlns:xsi"
, 'http://www.w3.org/2001/XMLSchema' as "xmlns:xsd"
, 'http://schemas.xmlsoap.org/soap/envelope/' as "xmlns:soap" )
, xmlelement( "soap:Body" , xmlelement( "GetCityWeatherByZIP"
, xmlattributes('http://ws.cdyne.com/WeatherWS/' as "xmlns")
, xmlelement("ZIP", '60611')
) ) ), version '1.0').getclobval()
into envelope from dual;
weather := apex_web_service.make_request(
p_url => 'http://wsf.cdyne.com/WeatherWS/Weather.asmx'
, p_action => 'http://ws.cdyne.com/WeatherWS/GetCityWeatherByZIP'
, p_version => '1.1'
, p_envelope => envelope );
dbms_output.put_line(weather.getclobval());
end;
/
Retrieve weather in Chicago, USA – Data always as CLOB even though contains XML
15. Further information on callouts
Read, Store and Create XML and JSON15 3/30/2016
My presentation from ODTUG KScope14:
– External Data via HTTP, FTP and Web Services
– http://bit.ly/kibeha_http_ws_slides
17. XMLTYPE column
Read, Store and Create XML and JSON17 3/30/2016
create table addresses (
addr_id integer primary key
, address varchar2(4000)
, geocoded xmltype
)
xmltype geocoded store as securefile binary xml;
insert into addresses values (
1
, 'Endelavevej 55, DK-5500 Middelfart'
, httpuritype(
'http://maps.googleapis.com/maps/api/geocode/xml?address='||
utl_url.escape('Endelavevej 55, DK-5500 Middelfart')
).getxml()
);
Column object type XMLTYPE
18. XMLTYPE column
Read, Store and Create XML and JSON18 3/30/2016
select geocoded
from addresses;
Column object type XMLTYPE
<?xml version="1.0" encoding="UTF-8"?>
<GeocodeResponse>
<status>OK</status>
<result>
<type>street_address</type>
<formatted_address>Endelavevej 55, 5500 Middelfart,
Denmark</formatted_address>
<address_component>
<long_name>55</long_name>
<short_name>55</short_name>
<type>street_number</type>
</address_component>
.
.
.
<place_id>ChIJRUp9K2iWTEYRmzjwsDRYoto</place_id>
</result>
</GeocodeResponse>
19. CLOB column with JSON
Read, Store and Create XML and JSON19 3/30/2016
create table addresses2 (
addr_id integer primary key
, address varchar2(4000)
, geocoded clob check(geocoded is json)
);
insert into addresses2 (addr_id, address)
values (2, 'Endelavevej 55, DK-5500 Middelfart')
;
update addresses2
set geocoded = httpuritype(
'http://maps.googleapis.com/maps/api/geocode/json?address='||
utl_url.escape(address)
).getclob();
Any CLOB can contain JSON - Use IS JSON check constraint to be certain
20. CLOB column with JSON
Read, Store and Create XML and JSON20 3/30/2016
select geocoded
from addresses2;
Content like any other CLOB
{
"results" : [
{
"address_components" : [
{
"long_name" : "55",
"short_name" : "55",
"types" : [ "street_number" ]
},
.
.
"formatted_address" : "Endelavevej 55, 5500 Middelfart, Denmark",
"geometry" : {
"location" : {
"lat" : 55.4892307,
"lng" : 9.7519265
.
.
21. XMLTYPE table
Read, Store and Create XML and JSON21 3/30/2016
create table geocoded_addresses of xmltype
xmltype store as securefile binary xml;
insert into geocoded_addresses values (
httpuritype(
'http://maps.googleapis.com/maps/api/geocode/xml?address='||
utl_url.escape('Endelavevej 55, DK-5500 Middelfart')
).getxml()
);
Object table of type XMLTYPE
22. XMLTYPE table
Read, Store and Create XML and JSON22 3/30/2016
select sys_nc_rowinfo$
from geocoded_addresses;
Table now contains objects, not scalar columns
<?xml version="1.0" encoding="UTF-8"?>
<GeocodeResponse>
<status>OK</status>
<result>
<type>street_address</type>
<formatted_address>Endelavevej 55, 5500 Middelfart,
Denmark</formatted_address>
<address_component>
<long_name>55</long_name>
<short_name>55</short_name>
<type>street_number</type>
</address_component>
.
.
.
<place_id>ChIJRUp9K2iWTEYRmzjwsDRYoto</place_id>
</result>
</GeocodeResponse>
23. XMLSchema
Read, Store and Create XML and JSON23 3/30/2016
begin
dbms_xmlschema.registerschema(
schemaurl=>'/nsroot/demo/note/'
, schemadoc=>
'<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:element name="note">
<xs:complexType>
<xs:sequence>
<xs:element name="to" type="xs:string"/>
<xs:element name="from" type="xs:string"/>
<xs:element name="heading" type="xs:string"/>
<xs:element name="body" type="xs:string"/>
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:schema>'
, local=>true , gentypes=>true , genbean=>false , gentables=>false
);
end;
/
Schema defines structure of XML – schema is registered under URL “path” in repository
24. XMLSchema
Read, Store and Create XML and JSON24 3/30/2016
create table notes of xmltype
xmltype store as object relational
xmlschema "/nsroot/demo/note" element "note";
-- Raises error: ORA-30937: No schema definition for 'something' (namespace '') in parent '/'
insert into notes values (
xmltype('<something>Not valid schema</something>')
);
-- Works, because XML fits schema definition
insert into notes values ( xmltype(
'<note><to>Jack</to><from>Jill</from><heading>Be my Valentine?</heading>
<body>Let us meet at Luigis Restaurant</body></note>'
) );
XMLTYPE table stored as object relational, not binary XML
XMLSCHEMA keyword may be used for binary XML too and enforce XML structure
25. XMLSchema stored as object
Read, Store and Create XML and JSON25 3/30/2016
select sys_nc_rowinfo$
from notes;
XML re-assembled from object relational data
Whitespace between elements no longer as original
<note>
<to>Jack</to>
<from>Jill</from>
<heading>Be my Valentine?</heading>
<body>Let us meet at Luigis Restaurant</body>
</note>
26. XMLSchema stored as object
Read, Store and Create XML and JSON26 3/30/2016
select dbms_metadata.get_ddl('TYPE', 'note928_T') ddl
from dual;
-- Output:
CREATE OR REPLACE EDITIONABLE TYPE "SCOTT"."note928_T" AS OBJECT (
"SYS_XDBPD$" "XDB"."XDB$RAW_LIST_T"
, "to" VARCHAR2(32767 CHAR)
, "from" VARCHAR2(32767 CHAR)
, "heading" VARCHAR2(32767 CHAR)
, "body" VARCHAR2(32767 CHAR)
) FINAL INSTANTIABLE;
CREATE TABLE autocreated object type named <xml name><sequence>_T
27. XMLSchema stored as object
Read, Store and Create XML and JSON27 3/30/2016
select sys_nc00008$, sys_nc00009$, sys_nc00010$, sys_nc00011$
from notes;
SYS_NC00008$ SYS_NC00009$ SYS_NC00010$ SYS_NC00011$
--------------- --------------- -------------------- -----------------------------------
Jack Jill Be my Valentine? Let us meet at Luigis Restaurant
Object type table contains hidden columns with relational data
28. XMLSchema with annotations
Read, Store and Create XML and JSON28 3/30/2016
dbms_xmlschema.registerschema(
schemaurl=>'/nsroot/demo/note/' , schemadoc=>
'<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xdb="http://xmlns.oracle.com/xdb">
<xs:element name="note" type="NoteType" xdb:defaultTable="NOTES"/>
<xs:complexType name="NoteType" xdb:SQLType="NOTE_T">
<xs:sequence>
<xs:element name="to" type="HeaderType" xdb:SQLName="TO"/>
<xs:element name="from" type="HeaderType" xdb:SQLName="FROM"/>
<xs:element name="heading" type="HeaderType" xdb:SQLName="HEADING"/>
<xs:element name="body" type="BodyType" xdb:SQLName="BODY"/>
</xs:sequence>
</xs:complexType>
<xs:simpleType name="HeaderType">
<xs:restriction base="xs:string"><xs:minLength value="0"/><xs:maxLength value="30"/></xs:restriction>
</xs:simpleType>
<xs:simpleType name="BodyType">
<xs:restriction base="xs:string"><xs:minLength value="0"/><xs:maxLength value="4000"/></xs:restriction>
</xs:simpleType>
</xs:schema>'
, local=>true , gentypes=>true , genbean=>false , gentables=>true );
Put xdb:*** annotations in XML schema along with length restrictions
Object type and table generating will use the annotated names
29. XMLSchema with annotations
Read, Store and Create XML and JSON29 3/30/2016
insert into notes values ( xmltype(
'<note><to>Jack</to><from>Jill</from><heading>Be my Valentine?</heading>
<body>Let us meet at Luigis Restaurant</body></note>'
) );
select sys_nc_rowinfo$
from notes;
<note>
<to>Jack</to>
<from>Jill</from>
<heading>Be my Valentine?</heading>
<body>Let us meet at Luigis Restaurant</body>
</note>
NOTES table here created by “gentables=>true”
Inserting and selecting data same result as before
30. XMLSchema with annotations
Read, Store and Create XML and JSON30 3/30/2016
select dbms_metadata.get_ddl('TYPE', 'NOTE_T') ddl
from dual;
-- Output:
CREATE OR REPLACE EDITIONABLE TYPE "SCOTT"."NOTE_T" AS OBJECT (
"SYS_XDBPD$" "XDB"."XDB$RAW_LIST_T"
, "TO" VARCHAR2(30 CHAR)
, "FROM" VARCHAR2(30 CHAR)
, "HEADING" VARCHAR2(30 CHAR)
, "BODY" VARCHAR2(4000 CHAR)
) NOT FINAL INSTANTIABLE;
Register schema autocreated object type named as specified in annotations
Table still contains invisible SYS named columns, but now with better datatypes
32. XMLQUERY
Read, Store and Create XML and JSON32 3/30/2016
select xmlquery(
'/GeocodeResponse/result/formatted_address'
passing a.geocoded
returning content
) formatted
from addresses a;
FORMATTED
--------------------------------------------------------------------------------
<formatted_address>Endelavevej 55, 5500 Middelfart, Denmark</formatted_address>
XMLQUERY returns XML snippet
33. XMLCAST
Read, Store and Create XML and JSON33 3/30/2016
select xmlcast(
xmlquery(
'/GeocodeResponse/result/formatted_address'
passing a.geocoded
returning content
)
as varchar2(50)
) formatted
from addresses a;
FORMATTED
-----------------------------------------------------
Endelavevej 55, 5500 Middelfart, Denmark
XMLCAST is one way to turn the snippet into only the content
34. XMLTABLE
Read, Store and Create XML and JSON34 3/30/2016
select geo.*
from addresses a
, xmltable(
'*'
passing a.geocoded
columns
status varchar2(20) path '/GeocodeResponse/status'
, lat number path '/GeocodeResponse/result/geometry/location/lat'
, lng number path '/GeocodeResponse/result/geometry/location/lng'
, formatted_address varchar2(50) path '/GeocodeResponse/result/formatted_address'
) geo;
STATUS LAT LNG FORMATTED_ADDRESS
-------------------- ---------- ---------- --------------------------------------------------
OK 55.4892307 9.7519265 Endelavevej 55, 5500 Middelfart, Denmark
Pass XMLTABLE function some XML and define columns with XPath expressions
35. XMLTABLE
Read, Store and Create XML and JSON35 3/30/2016
select geo.*
from addresses a
, xmltable(
'/GeocodeResponse/result'
passing a.geocoded
columns
lat number path 'geometry/location/lat'
, lng number path 'geometry/location/lng'
, formatted_address varchar2(50) path 'formatted_address'
) geo;
LAT LNG FORMATTED_ADDRESS
---------- ---------- --------------------------------------------------
55.4892307 9.7519265 Endelavevej 55, 5500 Middelfart, Denmark
Can tell XMLTABLE to operate only on a sub-node
36. XMLTABLE
Read, Store and Create XML and JSON36 3/30/2016
select geo.*
from addresses a
, xmltable(
'/GeocodeResponse/result'
passing a.geocoded
columns
formatted_address varchar2(50) path 'formatted_address'
, street_number_name varchar2(10) path 'address_component/long_name'
) geo;
ERROR at line 3:
ORA-19279: XPTY0004 - XQuery dynamic type mismatch: expected singleton sequence -
got multi-item sequence
Multiple elements “address_component/long_name” in XML => error
37. XMLTABLE
Read, Store and Create XML and JSON37 3/30/2016
select geo.*
from addresses a
, xmltable(
'/GeocodeResponse/result'
passing a.geocoded
columns
adr_components xmltype
path 'address_component'
) geo;
Column can be XMLTYPE to allow us to inspect why the error
<address_component>
<long_name>55</long_name>
<short_name>55</short_name>
<type>street_number</type>
</address_component>
<address_component>
<long_name>Endelavevej</long_name>
<short_name>Endelavevej</short_name>
<type>route</type>
</address_component>
<address_component>
<long_name>Middelfart</long_name>
<short_name>Middelfart</short_name>
<type>locality</type>
<type>political</type>
</address_component>
<address_component>
<long_name>Middelfart Municipality</long_name>
<short_name>Middelfart Municipality</short_name>
<type>administrative_area_level_2</type>
<type>political</type>
..
38. XMLTABLE
Read, Store and Create XML and JSON38 3/30/2016
select geo.formatted_address , com.*
from addresses a
, xmltable(
'/GeocodeResponse/result'
passing a.geocoded
columns
formatted_address varchar2(50) path 'formatted_address'
, adr_components xmltype path 'address_component'
) geo
, xmltable(
'/address_component'
passing geo.adr_components
columns
type varchar2(30) path 'type'
, long_name varchar2(30) path 'long_name'
) com;
ORA-19279: XPTY0004 - XQuery dynamic type mismatch: expected singleton sequence - got multi-item sequence
Chain two XMLTABLE calls – except “type” still has too many
39. XMLTABLE
Read, Store and Create XML and JSON39 3/30/2016
select geo.formatted_address , com.*
from addresses a
, xmltable(
'/GeocodeResponse/result'
passing a.geocoded
columns
formatted_address varchar2(50) path 'formatted_address'
, adr_components xmltype path 'address_component'
) geo
, xmltable(
'/address_component'
passing geo.adr_components
columns
type varchar2(30) path 'type[1]'
, long_name varchar2(30) path 'long_name'
) com;
type[1] allows us to specify we just want the first if there are multiple
40. XMLTABLE
Read, Store and Create XML and JSON40 3/30/2016
FORMATTED_ADDRESS TYPE LONG_NAME
------------------------------------------ ------------------------------ -------------------------
Endelavevej 55, 5500 Middelfart, Denmark street_number 55
Endelavevej 55, 5500 Middelfart, Denmark route Endelavevej
Endelavevej 55, 5500 Middelfart, Denmark locality Middelfart
Endelavevej 55, 5500 Middelfart, Denmark administrative_area_level_2 Middelfart Municipality
Endelavevej 55, 5500 Middelfart, Denmark country Denmark
Endelavevej 55, 5500 Middelfart, Denmark postal_code 5500
Output of previous slide
41. XMLTABLE
Read, Store and Create XML and JSON41 3/30/2016
select geo.*
from addresses a
, xmltable(
'/GeocodeResponse/result'
passing a.geocoded
columns
street_number_name varchar2(10) path 'address_component[type="street_number"]/long_name'
, street_name varchar2(20) path 'address_component[type="route"]/long_name'
, city_name varchar2(20) path 'address_component[type="locality"]/long_name'
, zip_name varchar2(10) path 'address_component[type="postal_code"]/long_name'
, country_name varchar2(20) path 'address_component[type="country"]/long_name'
) geo;
STREET_NUM STREET_NAME CITY_NAME ZIP_NAME COUNTRY_NAME
---------- -------------------- -------------------- ---------- --------------------
55 Endelavevej Middelfart 5500 Denmark
[ ] in XPath expressions also allows syntax to query for specific values
43. DIRECTIONS EXAMPLE - 2
Read, Store and Create XML and JSON43 3/30/2016
...
xmltable(
'if (fn:empty(/DirectionsResponse/route))
then <route><leg></leg></route>
else /DirectionsResponse/route'
passing
directions.response
columns
routenum for ordinality
, routename varchar2(100) path 'summary'
, routexml xmltype path '/'
) route, xmltable(
'for $l in /route/leg return $l'
passing
route.routexml
columns
legnum for ordinality
, seconds number path 'duration/value'
, meters number path 'distance/value'
, startaddr varchar2(100) path 'start_address'
, endaddr varchar2(100) path 'end_address'
, legxml xmltype path '/'
) leg,
...
44. DIRECTIONS EXAMPLE - 3
Read, Store and Create XML and JSON44 3/30/2016
...
xmltable(
'for $s in /leg/step return $s'
passing
leg.legxml
columns
stepnum for ordinality
, seconds number path 'duration/value'
, meters number path 'distance/value'
, instructions varchar2(400) path 'html_instructions'
) step
order by route.routenum
, leg.legnum
, step.stepnum;
45. DIRECTIONS EXAMPLE - Output
Read, Store and Create XML and JSON45 3/30/2016
ROUTENAME INSTRUCTIONS STEPTIME STEPKM
---------- ------------------------------------------------------------ -------- ----------
A7 and A5 Head <b>northwest</b> on <b>Endelavevej</b> 00:01:05 .309
A7 and A5 Turn <b>right</b> to stay on <b>Endelavevej</b> 00:00:27 .124
A7 and A5 Turn <b>left</b> onto <b>Bornholmsvej</b> 00:00:42 .396
A7 and A5 Turn <b>left</b> onto <b>Vandværksvej</b> 00:04:36 3.274
A7 and A5 At the roundabout, take the <b>1st</b> exit onto <b>Jyllands 00:00:26 .242
vej</b>
A7 and A5 At the roundabout, take the <b>1st</b> exit and stay on <b>J 00:00:33 .35
yllandsvej</b>
A7 and A5 At the roundabout, take the <b>2nd</b> exit onto the <b>E20< 00:00:38 .546
/b> ramp
A7 and A5 Merge onto <b>E20</b> 00:04:10 7.229
A7 and A5 Keep <b>left</b> at the fork to stay on <b>E20</b>, follow s 00:08:57 15.359
igns for <b>E45s</b>/<b>Flensborg</b>/<b>Esbjerg</b>/<b>Kold
ing</b>
A7 and A5 Keep <b>left</b> at the fork to continue on <b>E45</b>, foll 00:47:31 83.159
ow signs for <b>Kolding</b><div style="font-size:0.9em">Ente
ring Germany</div>
A7 and A5 Continue onto <b>A7</b>/<b>E45</b> 01:35:25 155.339
A7 and A5 Keep <b>left</b> at the fork to stay on <b>A7</b>, follow si 00:09:33 15.067
gns for <b>HH. Othmarschen</b>
A7 and A5 Keep <b>left</b> at the fork to stay on <b>A7</b> 00:03:39 6.845
...
46. JSON_TABLE
Read, Store and Create XML and JSON46 3/30/2016
select geo.*
from addresses2 a
, json_table(
a.geocoded
, '$' columns (
status varchar2(10) path '$.status'
, nested path '$.results[*]' columns (
lat number path '$.geometry.location.lat'
, lng number path '$.geometry.location.lng'
, nested path '$.address_components[*]' columns (
long_name varchar2(30) path '$.long_name'
, nested path '$.types[*]' columns (
component_type varchar2(30) path '$'
) ) ) ) ) geo;
Similar but slightly different syntax as XMLTABLE
Supports NESTED PATH as alternative to chaining XMLTABLE
47. JSON_TABLE
Read, Store and Create XML and JSON47 3/30/2016
STATUS LAT LNG LONG_NAME COMPONENT_TYPE
---------- ---------- ---------- ------------------------- ------------------------------
OK 55.4892307 9.7519265 55 street_number
OK 55.4892307 9.7519265 Endelavevej route
OK 55.4892307 9.7519265 Middelfart locality
OK 55.4892307 9.7519265 Middelfart political
OK 55.4892307 9.7519265 Middelfart Municipality administrative_area_level_2
OK 55.4892307 9.7519265 Middelfart Municipality political
OK 55.4892307 9.7519265 Denmark country
OK 55.4892307 9.7519265 Denmark political
OK 55.4892307 9.7519265 5500 postal_code
Output of previous slide
48. JSON short-cut dot-notation
Read, Store and Create XML and JSON48 3/30/2016
select a.geocoded.status
, a.geocoded.results.formatted_address
, a.geocoded.results.address_components.long_name
from addresses2 a;
STATUS RESULTS RESULTS
------ -------------------------------------------------- --------------------------------------------------
OK Endelavevej 55, 5500 Middelfart, Denmark ["55","Endelavevej","Middelfart","Middelfart Munic
ipality","Denmark","5500"]
Requires table alias – Datatype will just be generic varchar2
Can return JSON arrays for multiple elements – Column names quirky
49. JSON short-cut dot-notation
Read, Store and Create XML and JSON49 3/30/2016
select a.addr_id
, a.geocoded.STATUS
, a.geocoded.RESULTS.formatted_address
from addresses2 a;
ADDR_ID STATUS RESULTS
---------- ------ --------------------------------------------------
2
Dot-notation is case sensitive – even though looks like regular SQL
50. JSON short-cut dot-notation with JSON_TABLE
Read, Store and Create XML and JSON50 3/30/2016
select a.geocoded.results.formatted_address as formatted , geo.*
from addresses2 a
, json_table(
a.geocoded.results.address_components
, '$[*]' columns (
long_name varchar2(30) path '$.long_name'
, nested path '$.types[*]' columns ( component_type varchar2(30) path '$' )
) ) geo;
FORMATTED LONG_NAME COMPONENT_TYPE
------------------------------------------ ------------------------- ------------------------------
Endelavevej 55, 5500 Middelfart, Denmark 55 street_number
Endelavevej 55, 5500 Middelfart, Denmark Endelavevej route
Endelavevej 55, 5500 Middelfart, Denmark Middelfart locality
Endelavevej 55, 5500 Middelfart, Denmark Middelfart political
Endelavevej 55, 5500 Middelfart, Denmark Middelfart Municipality administrative_area_level_2
Endelavevej 55, 5500 Middelfart, Denmark Middelfart Municipality political
Endelavevej 55, 5500 Middelfart, Denmark Denmark country
Endelavevej 55, 5500 Middelfart, Denmark Denmark political
Endelavevej 55, 5500 Middelfart, Denmark 5500 postal_code
Combine JSON dot-notation and JSON_TABLE
51. JSON_TABLE vs. XMLTABLE - Comparison of syntax
Read, Store and Create XML and JSON51 3/30/2016
JSON_TABLE
With underscore
CLOB with IS JSON check
Arg 1: PASSING object
Arg 2: Path to data
Arg 3: Column defs/paths in ( )
Simple JSON dot path with $ = root
Support for NESTED JSON arrays
Cannot query paths dependent on
values
XMLTABLE
Without underscore
XMLTYPE object type
Arg 1: Path / XQuery exp to data
Arg 2: PASSING object
Arg 3: Column defs/paths – no ( )
Full XQuery syntax with / = root
Need multiple successive XMLTABLE
XPath expressions allow query like
address_component[type="country"]/...
52. JSON_TABLE
Read, Store and Create XML and JSON52 3/30/2016
select * from ( select geo.* from addresses2 a, json_table( a.geocoded, '$' columns (
nested path '$.results[*]' columns (
lat number path '$.geometry.location.lat'
, lng number path '$.geometry.location.lng'
, nested path '$.address_components[*]' columns (
long_name varchar2(15) path '$.long_name'
, nested path '$.types[*]' columns (
component_type varchar2(15) path '$'
) ) ) ) ) geo
) pivot ( max(long_name) name
for component_type in (
'street_number' as street_number
, 'route' as street
, 'locality' as city
, 'postal_code' as zip
, 'country' as country ) );
LAT LNG STREET_NUMBER_N STREET_NAME CITY_NAME ZIP_NAME COUNTRY_NAME
---------- ---------- --------------- --------------- --------------- --------------- ---------------
55.4892307 9.7519265 55 Endelavevej Middelfart 5500 Denmark
No support for “value lookup” – need SQL to handle that
53. JSON_TABLE and APEX_JSON
Read, Store and Create XML and JSON53 3/30/2016
select a.geocoded, apex_json.to_xmltype(a.geocoded) xmlgeo from addresses2 a;
GEOCODED XMLGEO
------------------------------------------------------------ ------------------------------------------------------------
{ <?xml version="1.0" encoding="WINDOWS-1252"?>
"results" : [ <json>
{ <results>
"address_components" : [ <row>
{ <address_components>
"long_name" : "55", <row>
"short_name" : "55", <long_name>55</long_name>
"types" : [ "street_number" ] <short_name>55</short_name>
}, <types>
{ <row>street_number</row>
"long_name" : "Endelavevej", </types>
"short_name" : "Endelavevej", </row>
"types" : [ "route" ] <row>
}, <long_name>Endelavevej</long_name>
{ <short_name>Endelavevej</short_name>
"long_name" : "Middelfart", <types>
"short_name" : "Middelfart", <row>route</row>
... ...
Alternative to PIVOT we can turn JSON into XML with APEX_JSON
54. JSON_TABLE and APEX_JSON
Read, Store and Create XML and JSON54 3/30/2016
select geo.*
from addresses2 a
, xmltable(
'/json/results/row'
passing apex_json.to_xmltype(a.geocoded)
columns
street_number_name varchar2(10) path 'address_components/row[types/row="street_number"]/long_name'
, street_name varchar2(20) path 'address_components/row[types/row="route"]/long_name'
, city_name varchar2(20) path 'address_components/row[types/row="locality"]/long_name'
, zip_name varchar2(10) path 'address_components/row[types/row="postal_code"]/long_name'
, country_name varchar2(20) path 'address_components/row[types/row="country"]/long_name'
) geo;
STREET_NUM STREET_NAME CITY_NAME ZIP_NAME COUNTRY_NAME
---------- -------------------- -------------------- ---------- --------------------
55 Endelavevej Middelfart 5500 Denmark
Then we can use XMLTABLE with greater syntax on the converted JSON
56. XMLROOT, XMLELEMENT
Read, Store and Create XML and JSON56 3/30/2016
select xmlroot(
xmlelement("DeptName", d.dname)
, version '1.0'
) department
from scott.dept d
where d.deptno = 30;
DEPARTMENT
--------------------------------------------------------------------------------
<?xml version="1.0"?>
<DeptName>SALES</DeptName>
XMLROOT creates XML header with version info
XMLELEMENT creates an element with start & end tag and content between
57. XMLELEMENT
Read, Store and Create XML and JSON57 3/30/2016
select xmlroot(
xmlelement(
"Department"
, xmlelement("DeptNo", d.deptno)
, xmlelement("DeptName", d.dname)
)
, version '1.0'
) department
from scott.dept d
where d.deptno = 30;
Nesting XMLELEMENT – multiple XMLELEMENT within one XMLELEMENT
DEPARTMENT
-------------------------------
<?xml version="1.0"?>
<Department>
<DeptNo>30</DeptNo>
<DeptName>SALES</DeptName>
</Department>
58. XMLFOREST
Read, Store and Create XML and JSON58 3/30/2016
select xmlroot(
xmlelement(
"Department"
, xmlforest(
d.deptno as "DeptNo"
, d.dname as "DeptName"
)
)
, version '1.0'
) department
from scott.dept d
where d.deptno = 30;
Shortcut for multiple XMLELEMENT
DEPARTMENT
---------------------------------------
<?xml version="1.0"?>
<Department>
<DeptNo>30</DeptNo>
<DeptName>SALES</DeptName>
</Department>
59. Multiple rows
Read, Store and Create XML and JSON59 3/30/2016
select xmlroot(
xmlelement(
"Department"
, xmlforest(
d.deptno as "DeptNo"
, d.dname as "DeptName"
)
)
, version '1.0'
) department
from scott.dept d;
Here we get one XML object for each department – one in each row of output
DEPARTMENT
-------------------------------------
<?xml version="1.0"?>
<Department>
<DeptNo>10</DeptNo>
<DeptName>ACCOUNTING</DeptName>
</Department>
<?xml version="1.0"?>
<Department>
<DeptNo>20</DeptNo>
<DeptName>RESEARCH</DeptName>
</Department>
<?xml version="1.0"?>
<Department>
<DeptNo>30</DeptNo>
<DeptName>SALES</DeptName>
</Department>
<?xml version="1.0"?>
<Department>
<DeptNo>40</DeptNo>
<DeptName>OPERATIONS</DeptName>
</Department>
60. XMLAGG
Read, Store and Create XML and JSON60 3/30/2016
select xmlroot(
xmlelement(
"Departments"
, xmlagg(
xmlelement(
"Department"
, xmlforest(
d.deptno as "DeptNo"
, d.dname as "DeptName"
) )
order by d.deptno
) )
, version '1.0'
) department
from scott.dept d;
XMLAGG aggregates “Department” elements within “Departments” element
DEPARTMENT
--------------------------------------
<?xml version="1.0"?>
<Departments>
<Department>
<DeptNo>10</DeptNo>
<DeptName>ACCOUNTING</DeptName>
</Department>
<Department>
<DeptNo>20</DeptNo>
<DeptName>RESEARCH</DeptName>
</Department>
<Department>
<DeptNo>30</DeptNo>
<DeptName>SALES</DeptName>
</Department>
<Department>
<DeptNo>40</DeptNo>
<DeptName>OPERATIONS</DeptName>
</Department>
</Departments>
61. Nested XMLAGG
Read, Store and Create XML and JSON61 3/30/2016
select xmlroot( xmlelement( "Departments"
, xmlagg( xmlelement( "Department"
, xmlforest(
d.deptno as "DeptNo"
, d.dname as "DeptName"
, ( select xmlagg( xmlelement( "Employee"
, xmlforest(
e.empno as "EmpNo“
, e.ename as "EmpName"
) )
order by e.empno )
from scott.emp e
where e.deptno = d.deptno ) as "Employees"
) )
order by d.deptno
) )
, version '1.0'
) department
from scott.dept d
where d.deptno = 10;
XMLAGG nested as subquery within XMLAGG
<?xml version="1.0"?>
<Departments>
<Department>
<DeptNo>10</DeptNo>
<DeptName>ACCOUNTING</DeptName>
<Employees>
<Employee>
<EmpNo>7782</EmpNo>
<EmpName>CLARK</EmpName>
</Employee>
<Employee>
<EmpNo>7839</EmpNo>
<EmpName>KING</EmpName>
</Employee>
<Employee>
<EmpNo>7934</EmpNo>
<EmpName>MILLER</EmpName>
</Employee>
</Employees>
</Department>
</Departments>
62. Nested XMLAGG
Read, Store and Create XML and JSON62 3/30/2016
select xmlroot( xmlelement( "Departments"
, xmlagg( xmlelement( "Department"
, xmlforest(
d.deptno as "DeptNo"
, max(d.dname) as "DeptName"
, xmlagg( xmlelement( "Employee"
, xmlforest(
e.empno as "EmpNo"
, e.ename as "EmpName"
) ) order by e.empno
) as "Employees"
) )
order by d.deptno
) )
, version '1.0'
) department
from scott.dept d join scott.emp e
on e.deptno = d.deptno
where d.deptno = 10
group by d.deptno;
Join and GROUP BY – Inner XMLAGG is “group” aggregate – Outer XMLAGG is “total”
<?xml version="1.0"?>
<Departments>
<Department>
<DeptNo>10</DeptNo>
<DeptName>ACCOUNTING</DeptName>
<Employees>
<Employee>
<EmpNo>7782</EmpNo>
<EmpName>CLARK</EmpName>
</Employee>
<Employee>
<EmpNo>7839</EmpNo>
<EmpName>KING</EmpName>
</Employee>
<Employee>
<EmpNo>7934</EmpNo>
<EmpName>MILLER</EmpName>
</Employee>
</Employees>
</Department>
</Departments>
63. XMLATTRIBUTES
Read, Store and Create XML and JSON63 3/30/2016
select xmlroot( xmlelement( "Departments"
, xmlagg( xmlelement( "Department"
, xmlattributes(d.deptno as "DeptNo")
, xmlforest(
max(d.dname) as "DeptName"
, xmlagg( xmlelement( "Employee"
, xmlattributes(e.empno as "EmpNo")
, xmlelement("EmpName", e.ename)
) order by e.empno
) as "Employees"
) )
order by d.deptno
) )
, version '1.0'
) department
from scott.dept d join scott.emp e
on e.deptno = d.deptno
where d.deptno = 10
group by d.deptno;
Typically some elements are turned into attributes – makes more “terse” XML
<?xml version="1.0"?>
<Departments>
<Department DeptNo="10">
<DeptName>ACCOUNTING</DeptName>
<Employees>
<Employee EmpNo="7782">
<EmpName>CLARK</EmpName>
</Employee>
<Employee EmpNo="7839">
<EmpName>KING</EmpName>
</Employee>
<Employee EmpNo="7934">
<EmpName>MILLER</EmpName>
</Employee>
</Employees>
</Department>
</Departments>
64. Object types
Read, Store and Create XML and JSON64 3/30/2016
create type emp_t as object (empno number, empname varchar2(10));
create type emp_tt as table of emp_t;
create type dept_t as object (
deptno number
, deptname varchar2(15)
, employees emp_tt
);
create type dept_tt as table of dept_t;
create type depts_t as object (departments dept_tt);
Created object types can be basis for XML
65. XMLTYPE Constructor for Object Types
Read, Store and Create XML and JSON65 3/30/2016
select xmltype( depts_t(
cast( collect(
dept_t(
d.deptno
, max(d.dname)
, cast( collect(
emp_t(e.empno, e.ename)
) as emp_tt )
)
) as dept_tt )
) ) department
from scott.dept d
join scott.emp e
on e.deptno = d.deptno
where d.deptno = 10
group by d.deptno;
Object types can be passed as argument to XMLTYPE constructor
<DEPTS_T>
<DEPARTMENTS>
<DEPT_T>
<DEPTNO>10</DEPTNO>
<DEPTNAME>ACCOUNTING</DEPTNAME>
<EMPLOYEES>
<EMP_T>
<EMPNO>7782</EMPNO>
<EMPNAME>CLARK</EMPNAME>
</EMP_T>
<EMP_T>
<EMPNO>7934</EMPNO>
<EMPNAME>MILLER</EMPNAME>
</EMP_T>
<EMP_T>
<EMPNO>7839</EMPNO>
<EMPNAME>KING</EMPNAME>
</EMP_T>
</EMPLOYEES>
</DEPT_T>
</DEPARTMENTS>
</DEPTS_T>
66. XMLTYPE Constructor for REF CURSOR
Read, Store and Create XML and JSON66 3/30/2016
select xmltype( cursor(
select d.deptno "DeptNo"
, d.dname "DeptName"
from scott.dept d
where d.deptno = 10
)) department
from dual;
Cursor variables can be passed to XMLTYPE constructor
<?xml version="1.0"?>
<ROWSET>
<ROW>
<DeptNo>10</DeptNo>
<DeptName>ACCOUNTING</DeptName>
</ROW>
</ROWSET>
67. Nested REF CURSOR
Read, Store and Create XML and JSON67 3/30/2016
select xmltype( cursor(
select d.deptno "DeptNo"
, d.dname "DeptName"
, cursor(
select e.empno "EmpNo"
, e.ename "EmpName"
from scott.emp e
where e.deptno = d.deptno
) "Employees"
from scott.dept d
where d.deptno = 10
)) department
from dual;
Cursor variables can contain nested cursor variables
<?xml version="1.0"?>
<ROWSET>
<ROW>
<DeptNo>10</DeptNo>
<DeptName>ACCOUNTING</DeptName>
<Employees>
<Employees_ROW>
<EmpNo>7782</EmpNo>
<EmpName>CLARK</EmpName>
</Employees_ROW>
<Employees_ROW>
<EmpNo>7839</EmpNo>
<EmpName>KING</EmpName>
</Employees_ROW>
<Employees_ROW>
<EmpNo>7934</EmpNo>
<EmpName>MILLER</EmpName>
</Employees_ROW>
</Employees>
</ROW>
</ROWSET>
68. DbUriType
Read, Store and Create XML and JSON68 3/30/2016
select dburitype(
'/SCOTT/DEPT'
).getxml() departments
from dual;
DbUriType accesses schema/table via XMLDB
<?xml version="1.0"?>
<DEPT>
<ROW>
<DEPTNO>10</DEPTNO>
<DNAME>ACCOUNTING</DNAME>
<LOC>NEW YORK</LOC>
</ROW>
<ROW>
<DEPTNO>20</DEPTNO>
<DNAME>RESEARCH</DNAME>
<LOC>DALLAS</LOC>
</ROW>
<ROW>
<DEPTNO>30</DEPTNO>
<DNAME>SALES</DNAME>
<LOC>CHICAGO</LOC>
</ROW>
<ROW>
<DEPTNO>40</DEPTNO>
<DNAME>OPERATIONS</DNAME>
<LOC>BOSTON</LOC>
</ROW>
</DEPT>
69. DbUriType
Read, Store and Create XML and JSON69 3/30/2016
select dburitype(
'/SCOTT/EMP/ROW[ENAME="KING"]'
).getxml() employees
from dual;
Data can be queried using XPath expressions
<?xml version="1.0"?>
<ROW>
<EMPNO>7839</EMPNO>
<ENAME>KING</ENAME>
<JOB>PRESIDENT</JOB>
<HIREDATE>17-NOV-81</HIREDATE>
<SAL>5000</SAL>
<DEPTNO>10</DEPTNO>
</ROW>
70. APEX_JSON
Read, Store and Create XML and JSON70 3/30/2016
begin
apex_json.initialize_clob_output;
apex_json.open_array('Departments');
for d in (select deptno, dname from scott.dept where deptno = 10) loop
apex_json.open_object('Department');
apex_json.write('DeptNo', d.deptno);
apex_json.write('DeptName', d.dname);
apex_json.open_array('Employees');
for e in (select empno, ename from scott.emp e where e.deptno = d.deptno) loop
apex_json.open_object('Employee');
apex_json.write('EmpNo', e.empno);
apex_json.write('EmpName', e.ename);
apex_json.close_object;
end loop;
apex_json.close_array;
apex_json.close_object;
end loop;
apex_json.close_array;
dbms_output.put_line(apex_json.get_clob_output);
apex_json.free_output;
end;
/
71. APEX_JSON
Read, Store and Create XML and JSON71 3/30/2016
"Departments":[
"Department":{
"DeptNo":10
,"DeptName":"ACCOUNTING"
,"Employees":[
"Employee":{
"EmpNo":7782
,"EmpName":"CLARK"
}
,"Employee":{
"EmpNo":7839
,"EmpName":"KING"
}
,"Employee":{
"EmpNo":7934
,"EmpName":"MILLER"
}
]
}
]
Output of previous slide
72. APEX_JSON
Read, Store and Create XML and JSON72 3/30/2016
declare
c sys_refcursor;
begin
open c for select d.deptno "DeptNo"
, d.dname "DeptName"
, cursor( select e.empno "EmpNo"
, e.ename "EmpName"
from scott.emp e
where e.deptno = d.deptno ) "Employees"
from scott.dept d
where d.deptno = 10;
apex_json.initialize_clob_output;
apex_json.open_object;
apex_json.write('Departments', c);
apex_json.close_object;
dbms_output.put_line(apex_json.get_clob_output);
apex_json.free_output;
end;
Besides building manually, APEX_JSON also supports cursor variable
73. APEX_JSON
Read, Store and Create XML and JSON73 3/30/2016
{
"Departments":[{"DeptNo":10,"DeptName":"ACCOUNTING","Employees":[{"EmpNo":7782,"EmpN
ame":"CLARK"},{"EmpNo":7839,"EmpName":"KING"},{"EmpNo":7934,"EmpName":"MILLER"}]}]
}
This time output of ref cursor in a single line
75. Read, Store and Create XML and JSON75 3/30/2016
Links
This presentation PowerPoint http://bit.ly/kibeha_xmljson_pptx
Script with all examples from this presentation http://bit.ly/kibeha_xmljson_sql
76. Questions & Answers
Kim Berg Hansen
Senior Consultant
kim.berghansen@trivadis.com
3/30/2016 Read, Store and Create XML and JSON76
http://bit.ly/kibeha_xmljson_pptx
http://bit.ly/kibeha_xmljson_sql