Working With XML in IDS Applications

- 1 -
B07: Working With XML in IDS Applications
Keshava Murthy,
Architect, IBM, rkeshav@us.ibm.com

- 2 -
International Informix Users Group.2009 IIUG Informix Conference 2
XML in Application Environment
•XML is ubiquitous and solves many problems
• Application Integration
• Data Centric Applications
• Document Centric Applications
• Heterogeneity
• XML and RDBMS
• Persistence
• XML in and XML out
• Application requirements
• XML support in IDS
• What’s planned?
<? xml version=“1.0” ?>
<purchaseOrder id=“12345” >
<customer id=“A6789”>
<name>John Smith Co</name>
<address>
<street>1234 W.
Main</street>
<city>Toledo</city>
<state>OH</state>
<zip>95141</zip>
</address>
</customer>
…

- 3 -
Websphere
App server
XML
Planning
Optimization Engine
Informix Dynamic Server
(IDS)
JDBC
ODBC
Carrier Integration
Business Logic
JDBC based
Database Layer
Lane Tracking
Supplier integration
Fetch Data
TP: Bid winners
TPE: Shipment
MQ based Event
handling
Storage
WebsphereMQ
Procurement
Optimization Engine
Typical Transportation Application SOA

- 4 -
•Application design follows SOA
• Separation of business optimization services from Integration Services
• XML documents exchanged follows tXML (transport XML)
• tXML documents are shred to relational tables
• efficient storage
• query performance
• Application queries RDBMS to create and publish tXML documents/reports
• tXML rarely changes - schema evolution isn’t a requirement, but performance is.
Source: IBM SWG Architecture

- 5 -
HTTP/SOA
P
WebSphere Application
Server
IDS Web Service
Provider
J2EE
Applications
Direct
Applications
XML Publishing &
Extraction with XPATH
support
XSLT Transformation
XML Schema Repository
And Shredding Functions
XML aware FullText Index
(with XPath Support)
Websphere
MQ
Websphere
MQ
Integration
ESQL/C, ODBC,
.NET applications
JDBC/JCC
EGL Apps
WORF/DAD
X
Apache Tomcat
IDS XML Plans

- 6 -
Functions
invoking the
XSLT4C and
XML4C libraries
within IDS
IDS Client
IDS Client
Extract functions
extract()
extracevalue()
existsnode()
idsXMLParse()
Publishing functions
genxml()
genxmlelem()
genxmlqueryhdr()
…
XSLT4C
Processor
XML4C Parser
ICU
XSLT4C and
XML4C
What’s in 11?
Transform functions
XSLTransform()
XSLTransformAsCLOB()
XSLTransformAsBLOB()
New in 11.5

- 7 -
What’s in 11?
• XML Publishing Functions
 Genxml() and GenxmlClob()
 Genxmlqueryhdr() and GenxmlqueryhdrClob()
 Genxmlelem() and genxmlelemclob()
• XPath functions for pattern matching and extraction within
XML document
 Extract() and Extractclob()
 ExtractValue()
 ExistsNode()

- 8 -
Genxml – simple example
select genxml(mycustrpt, 'mycustomers')
FROM (select c.customer_num, c.customer_name.last, o.order_num from customer c, orders o
WHERE c.customer_num = o.customer_num
and order_date = '05/20/1998') as mycustrpt;
genxml <mycustomers customer_num="104" expression="Kilmore" order_num="1001"/>
1 row(s) retrieved.
genxml <mycustomers customer_num="104" expression="Kilmore" order_num="1001"/>
1 row(s) retrieved.
FROM (select first 5 c.customer_num, c.customer_name.last, o.order_num from customer c, orders o
and order_date < '01/01/1999') as mycustrpt;
genxml <mycustomers customer_num="104" last="Kilmore" order_num="1001"/>
<mycustomers customer_num="101" last="Mangal" order_num="1002"/>
<mycustomers customer_num="104" last="Kilmore" order_num="1003"/>
<mycustomers customer_num="106" last="Puente" order_num="1004"/>
<mycustomers customer_num="116" last="Westerberg" order_num="1005"/>
1 row(s) retrieved.

- 9 -
Genxml – simple example
select genxmlelem(mycustrpt, 'mycustomers')
genxmlelem <mycustomers>
<row>
<customer_num>104</customer_num>
<expression>Kilmore</expression>
<order_num>1001</order_num>
</row>
</mycustomers>
1 row(s) retrieved.
select genxmlelem(mycustrpt, 'mycustomers')
… same result as above

- 10 -
Extract() and ExtractValue()
SELECT extract(col2, '/personnel/person[3]/name/given')
FROM tab;
SELECT extractvalue(col2,
'/personnel/person[3]/name/given') FROM tab;
execute function
extract(“<person><name><fname>john</fname><lnam
e>Kelly</lname></person>”,
“/person/name/lname”);
select col1 from tab where existsnode(col2,
'/personnel/person/*/email') = 1;

- 11 -
ExtractClob() and ExtractValueClob()
SELECT extractclob(col2, '/personnel/person[3]/name/given')
FROM tab_clob_neg;
SELECT extractvalueclob(col2,
'/personnel/person[3]/name/given') FROM tab_clob_neg;

- 12 -
ExistsNode()
select docid from books
where existsnode(booklist, '/bookstore/book[author="Kurt
Cagle"]/title') = 1;

- 13 -
Transform Functions based on
XSLT: New in 11.5
• eXtensible Stylesheet Language Tranformations
• XSL and XSLT is part of W3C standard
• XSLT is used to transform XML documents
 XML to XML (transform to confirm different schema/standard)
 XML to HTML
 XML to PDF
• XML is device independent representation of data
• XSLT helps you to transform to customize display
• XSLT helps you to transform from one XML schema into
another.
• XSLT enables loose integration thru transformation
• XSLT enables same data to be publishable to multiple targets
with unique requirements.

- 14 -
Functions
invoking the
XSLT4C and
XML4C libraries
within IDS
IDS Client
IDS Client
Extract functions
extract()
extracevalue()
existsnode()
idsXMLParse()
Publishing functions
genxml()
genxmlelem()
genxmlqueryhdr()
…
XSLT4C
Processor
XML4C Parser
ICU
XSLT4C and
XML4C
What’s new in 11.5?
Transform functions
XSLTransform()
XSLTransformAsCLOB()
XSLTransformAsBLOB()
New in 11.5

- 15 -
XSLT Processor
Xerces and
Xalan
XML
Document
HTML
Document
Any other
document
Input
XML
document
XSL Style
Sheet
Document
XSLT: what is it?

- 16 -
Using XSLT Transformation
> select * from t where id = 1;
id 1
info <?xml version='1.0' encoding='ISO-8859-1' ?><doc>Hello world!</doc>
style <?xml version='1.0'?> <xsl:stylesheet xmlns:xsl='http://www.w3.org/1999/
XSL/Transform' version='1.0'> <xsl:output encoding='US-ASCII'/> <xsl:te
mplate match='doc'> <out><xsl:value-of select='.'/></out> </xsl:template
> </xsl:stylesheet>
1 row(s) retrieved.
-- info and style simply lvarchar columns.
-- xsltransform() returns lvarchar. Max 32K
> select xsltransform(info, style) from t where id = 1;
(expression) <?xml version="1.0" encoding="UTF-8"?>
<out>Hello world!</out>
1 row(s) retrieved.

- 17 -
Using XSLT Transformation
-- Transforming XML documents stored in tables
select XSLTransform(s.style_doc, x.xml_doc)
FROM style_sheets s, xml_docs x
WHERE s.style_title = 'ecommerce_ROOT.xsl'
AND x.xml_transaction_id = 12345;
-- Transforming dynamically generated XML documents
select XSLTransform(s.style_doc,
genxml(“transaction”, row(o.trnid, o.custid,
o.addr)))
FROM style_sheets s, orders o
WHERE s.style_title = 'ecommerce_ROOT.xsl'
AND o.trnid = 12345;

- 18 -
XSLT Transform Functions
•Three XSLT functions, but they’re overloaded
• XSLTransform()
• XSLTransformAsCLOB()
• XSLTransformAsBLOB()
• They’re overloaded
• The functions differ in parameters and return types

- 19 -
XSLT Transform Functions
Function name Parameter1 (input XML
doc)
Parameter2 (input Stylesheet Returning type
XSLTransform lvarchar lvarchar lvarchar
XSLTransform CLOB lvarchar lvarchar
XSLTransform lvarchar CLOB lvarchar
XSLTransform CLOB CLOB lvarchar
XSLTransform BLOB lvarchar lvarchar
XSLTransform lvarchar BLOB lvarchar
XSLTransformAsCLOB lvarchar lvarchar CLOB
XSLTransformAsCLOB CLOB lvarchar CLOB
XSLTransformAsCLOB lvarchar CLOB CLOB
XSLTransformAsBLOB lvarchar Lvarchar BLOB
XSLTransformAsBLOB BLOB varchar BLOB
XSLTransformAsBLOB lvarchar BLOB BLOB

- 20 -
Text Search Index for XML

- 21 -
BTS Secondary Access Method
• To create an BTS index, specify the bts secondary access method
with the USING clause on the CREATE INDEX
• A BTS index is required to perform text searches using the
bts_contains() predicate
• Required to specify the opclass that is specific to the data type of the
column being indexed
• Requires an extent space specified with the IN Clause (no default)
• Example:
CREATE INDEX books_idx
ON books(description bts_lvarchar_ops)
USING bts
IN bts_extspace;
Acknowledgement: Slides on BTS functionality were created by
Mark Ashworth, Developer of BTS datablade.

- 22 -
Selecting Rows
• Selecting rows using the BTS is done with the bts_contains()
UDF
bts_contains(column, query-parse-string)
• The bts_contains() is a Boolean UDF. The first parameter,
column, is the column being search and the second
parameter, query-parse-string, is the Lucene query string
• A BTS index must be created on the column being searched
with bts_contains() UDF

- 23 -
• XML document: lusitania.xml
<?xml version="1.0"?>
<ship>
<Name type="oceanliner">RMS Lusitania</Name>
<owners> Cunard Line</owners>
<builders> John Brown & Company, Limited</builders>
<maiden_voyage> 7 September, 1907</maiden_voyage>
<sank>
<date> 7 May, 1915 </date>
<how> by German submarine U-20. </how>
</sank>
</ship>
• XML document: andrea_doria.xml
<ship>
<Name type="oceanliner">SS Andrea Doria</Name>
<owners> Italia Line</owners>
<builders> Ansaldo, Sestri Ponente yards,
Italy</builders>
<maiden_voyage> 14 January, 1953</maiden_voyage>
<sank>
<date> 26 July, 1956 </date>
<how> after colliding with SS Stockholm </how>
</sank>
</ship>
XML Sample Documents

- 24 -
xmlpaths
• Xmlpaths can be absolute or relative
 Absolute paths start with a / and relative paths do not
 very basic X Path support with location paths
 like a directory path in a filesystem

- 25 -
Relative xmlpaths in the example xml
documents
• Relative Paths
ship/Name
ship/owners
ship/builders
ship/maiden_voyage
sank/date
sank/how
ship/sank/date
ship/sank/how
 And all the tags previously listed
(Name, owners, builders,
maiden_voyage, date, how)
<ship>
<Name type="oceanliner">RMS Titanic</Name>
<owners> White Star Line, Liverpool, England</owners>
<builders> Harland & Wolff, Belfast, Ireland</builders>
<maiden_voyage> 10 April, 1912</maiden_voyage>
<sank>
<date> 15 April, 1912 </date>
<how> after hitting an iceberg on April 14, 1912 </how>
</sank>
</ship>
ship
Name owner builders maiden_voyage sank
date how

- 26 -
Absolute xmlpaths in the example xml
documents
• Absolute Paths
/ship/Name
/ship/owners
/ship/builders
/ship/maiden_voyage
/ship/sank/date
/ship/sank/how
<ship>
<Name type="oceanliner">RMS Titanic</Name>
<owners> White Star Line, Liverpool, England</owners>
<builders> Harland & Wolff, Belfast, Ireland</builders>
<maiden_voyage> 10 April, 1912</maiden_voyage>
<sank>
<date> 15 April, 1912 </date>
<how> after hitting an iceberg on April 14, 1912 </how>
</sank>
</ship>
ship
Name owner builders maiden_voyage sank
date how

- 27 -
Query Parser String (revisited) -
Fields
• You can search any field by typing the field name followed
by a colon ":" and then the term you are looking for.
• For example:
bts_contains(document, 'Name:rms and owner:"White
Star Line"')
 Here we search the field "Name" for the word "rms" and the field
"owner" for the phrase "White Star Line"

- 28 -
An example using xmlpath_processing
and xmltags• Index the xmlpaths: ship/Name and ship/sank/how
create index ships_bts on ships(document bts_clob_ops)
using
bts(xmltags="(ship/Name,ship/sank/how)",xmlpat
h_processing="yes")
in bts_extspace;
• For the titanic.xml document, the following fields would be created:
ship/Name:RMS Titanic
ship/sank/how:after hitting an iceberg on April 14, 1912
• Query example to find ships that sank because of icebergs
select name from ships
where bts_contains(document, 'ship/sank/how:iceberg*');
name
Titanic
1 row(s) retrieved.

- 29 -
An example using xmlpath_processing
and all_xmltags
• Index the all absolute xmlpaths:
using bts(all_xmltags="yes",xmlpath_processing="yes")
in bts_extspace;
• For the titanic.xml document, the following fields would be created:
/ship/Name:RMS Titanic
/ship/owners:White Star Line, Liverpool, England
/ship/builders:Harland & Wolff, Belfast, Ireland
/ship/maiden_voyage:10 April, 1912
/ship/sank/date:15 April, 1912
/ship/sank/how:after hitting an iceberg on April 14, 1912
• Query example to find ships that sank because of icebergs:
select name from ships
where bts_contains(document, '/ship/sank/how:iceberg*');
name
Titanic
1 row(s) retrieved.

- 30 -
include_namespaces parameter
(cont.)• Given the xml fragment:
<book>
<book:title>Graph Theory</book:title>
<author>Stewart</author>
<date>January 14, 2006</date>
</book>
• For the given xml fragment, the following three fields
would be created:
/book/book:title:graph theory
/book/author:stewart
/book/date:january 14, 2006
• Query:
bts_contains(xml_data, '/book/book:title:theory')
 Note, because the colon is used to delineate a field, the colon
used to signify the namespace must be escaped with a
backslash (‘’)

- 31 -
bts_index_fields() function
• Given a path to an index path (like bts_index_compact), it returns a list of fields that currently exist in
the index
• For example:
using bts(all_xmltags="yes",xmlpath_processing="yes")
in bts_extspace;
Index created.
execute function
bts_index_fields('/local18/clucene_index/testdb/ashwort
h/ships_bts'));
(expression) /ship/Name
/ship/builders
/ship/maiden_voyage
/ship/owners
/ship/sank/date
/ship/sank/how
1 row(s) retrieved.

- 32 -
bts_index_fields() function (cont.)
• Another example, with xmlpath_processing disabled and include_contents enabled:
using bts(all_xmltags="yes",xmlpath_processing="no",
include_contents="yes")
in bts_extspace;
Index created.
execute function
bts_index_fields('/local18/clucene_index/testdb/ashworth/ships_bts')
);
(expression) Name
builders
contents
maiden_voyage
owners
date
how
1 row(s) retrieved.

- 33 -
Searching XML Documents as part of
Basic Text Search• Customer can now generate their own stopword lists
• Can indicate which xml tags to index at index create time.
 Values get indexed, not tags.
 Also options to index all tags as well as treat tags as text.
• Can now limit searches to only certain XML tags
• “XPath" queries supported in search.
 Search string can contain a path to search /books/title/chapter/....
 Path can be relative or fixed.
• tags can be completely ignored
 XML doc treated as a set of values
 Looses the ability to search by tags
• You can specify whether the namespace should be required in searches
• Markup in sentences can be ignored
 "foo <bold> bar </bold>" can either be indexed as the single "foo bar" or as "foo" and "bar". This matters for proximity
searches for instance.
• New function provided which returns all the unique tags in the XML doc.

- 34 -
HTTP/SOA
P
WebSphere Application
Server
IDS Web Service
Provider
J2EE
Applications
Direct
Applications
XML Publishing &
Extraction with XPATH
support
XSLT Transformation
XML Schema Repository
And Shredding Functions
XML aware FullText Index
(with XPath Support)
Websphere
MQ
Websphere
MQ
Integration
ESQL/C, ODBC,
.NET applications
JDBC/JCC
EGL Apps
WORF/DAD
X
Apache Tomcat
Informix Dynamic ServerIDS XML Plans

- 35 -
Thank You
rkeshav@us.ibm.com

Working With XML in IDS Applications

Recommandé

Recommandé

Contenu connexe

En vedette

En vedette (20)

Similaire à Working With XML in IDS Applications

Similaire à Working With XML in IDS Applications (20)

Plus de Keshav Murthy

Plus de Keshav Murthy (20)

Dernier

Dernier (20)

Working With XML in IDS Applications

Notes de l'éditeur