Oracle database - Get external data via HTTP, FTP and Web Services

External Data via HTTP, FTP
and Web Services
Kim Berg Hansen
T. Hansen Gruppen A/S

Network Access Control List
 FTP_UTIL_PKG
 UTL_HTTP
 HttpUriType
 UTL_DBWS
 JPublisher
 APEX_WEB_SERVICE
Topics

Network ACL
begin
dbms_network_acl_admin.create_acl(
acl => 'kscope_demo.xml'
, description => 'KScope14 demo ACL'
, principal => 'SCOTT'
, is_grant => true
, privilege => 'connect'
);
dbms_network_acl_admin.add_privilege(
, principal => 'HR'
, is_grant => true
);
commit;
end;
/
begin
dbms_network_acl_admin.assign_acl(
, host => 'kscope14.com'
, lower_port => 80
, upper_port => 80
);
, host => '*.googleapis.com'
);
commit;
end;
/
Create one or more ACLs
Grant users connect privilege
Assign hosts / ports to ACLs

 Network Access Control List
FTP_UTIL_PKG
 UTL_HTTP
 HttpUriType
 UTL_DBWS
 JPublisher
Topics

 Tim Hall, ORACLE-BASE
http://www.oracle-base.com/articles/misc/ftp-from-plsql.php
 Alexandria PL/SQL Code Library
https://code.google.com/p/plsql-utils/
FTP_UTIL_PKG

 Get, put, delete, rename, mkdir, rmdir
 Ascii, binary
 Our use:
● We buy data regarding cars on the danish market
● Data supplier generates files on FTP server monthly
● Scheduled job uses FTP_UTIL_PKG to get files and
imports via external table
FTP_UTIL_PKG

FTP GET file to directory
create directory ftp_downloads as
'/mnt/vol1/ftp_downloads';
grant read, write on directory
ftp_downloads to scott;
create or replace package body car_data
is
procedure get_carinfo_file
is
l_conn utl_tcp.connection;
begin
l_conn := ftp_util_pkg.login(
p_host => 'ftp.oursupplier.dk'
, p_port => '21'
, p_user => 'mycompany'
, p_pass => 'myPas$w0rd'
);
util_ftp.ascii(p_conn => l_conn);
util_ftp.get(
p_conn => l_conn
, p_from_file => '/carinfo.txt'
, p_to_dir => 'FTP_DOWNLOADS'
, p_to_file => 'carinfo.txt'
);
util_ftp.logout(l_conn);
end get_carinfo_file;
... continues ...
Server directory and
grants to work with files
(as SYS or other dba)
FTP protocol calls:
login, get, logout

External table and materialized view
create table carinfo_ext (
crm_artcode number
, approval_id number
, variant number
, year number
, population number
) organization external (
type oracle_loader
default directory ftp_downloads
access parameters (
records delimited by newline
territory 'DENMARK'
skip 1
fields terminated by 't'
lrtrim missing field values are null (
crm_artcode
, approval_id
, variant
, year
, population
)
)
location ('carinfo.txt')
);
create materialized view carinfo_mview
build deferred
refresh complete on demand
as
select *
from carinfo_ext;
create index carinfo_mview_key on
carinfo_mview ( crm_artcode );
External table defined on
the file we got by FTP
Materialized view defined
on the external table

FTP GET and refresh mview
...continued...
procedure fresh_carinfo
is
begin
get_carinfo_file;
dbms_refresh.refresh('CARINFO_MVIEW');
end fresh_carinfo;
end car_data;
/
declare
jobno binary_integer;
begin
dbms_job.submit(
job => jobno
, what => 'car_data.fresh_carinfo'
, next_date =>
add_months(trunc(sysdate,'MM'),1) + interval
'3' day + interval '22' hour
, interval =>
q'[add_months(trunc(sysdate,'MM'),1) +
interval '3' day + interval '22' hour]'
);
commit;
dbms_output.put_line(jobno);
end;
/
When we need fresh data,
we first call procedure that
FTP downloads the file,
then refresh the mat.view
from the file Fresh data every 3rd day of month
Use Scheduler rather than jobs if
more complex rules are needed

 Not quite as easy to do SFTP protocol
 AskTom discussion:
● https://asktom.oracle.com/pls/apex/f?
p=100:11:0::::P11_QUESTION_ID:25564058003464
33753
 Load SFTP java libraries – examples:
● http://michailgalvidis.blogspot.com/2010/03/sftp-
from-oracle.html
● http://dbashavefeelingstoo.wordpress.com/2011/07/2
2/sftp-with-plsql/
SFTP

 FTP_UTIL_PKG
UTL_HTTP
 HttpUriType
 UTL_DBWS
 JPublisher
Topics

 Fairly low-level
 Can do SOAP, REST, practically whatever
- manual code whatever protocol on top of http
 GET, POST, …
 Cookies, redirects, SSL (https), authentication
 Set parameters for connections, charsets, …
 In many ways a ”browser emulator”
 Our use: http POST to SMS service provider
UTL_HTTP

 Eric van Roon
http://www.evrocs.nl/evenementen/kscope14-seattle/
 HTTP GET of KScope14 HTML session list
 UTL_HTTP read HTML a line at a time
 Pipelined function
 String functions search for content
 Pipe sessions allow SELECT from webpage
UTL_HTTP scraping HTML webpage

function pagesource(p_url in st_maxchar, p_proxy in st_maxchar default null)
return ta_ero_sourcelines pipelined
is
t_http_req utl_http.req;
t_http_resp utl_http.resp;
t_line st_maxchar;
t_lineno integer := 0;
r_return to_ero_sourcelines := to_ero_sourcelines();
begin
if p_proxy is not null then
utl_http.set_proxy(p_proxy);
end if;
t_http_req := utl_http.begin_request (p_url);
utl_http.set_header (t_http_req,'User-Agent','Mozilla/4.0');
t_http_resp := utl_http.get_response (t_http_req);
loop
utl_http.read_line (t_http_resp, t_line, true);
t_lineno := t_lineno + 1;
r_return.lineno := t_lineno;
r_return.text := t_line;
pipe row (r_return);
end loop;
exception
when utl_http.end_of_body then
utl_http.end_response(t_http_resp);
when no_data_needed then
utl_http.end_response(t_http_resp);
end pagesource;
HTTP request, header
and get response
Read response line by line
Close when everything
read or "client cursor"
stops reading the pipe

procedure retrieve_field_values(p_itemtext in st_maxchar, p_fieldvalues out to_ero_kscope_sessions)
is
cn_start_indicator_title constant st_maxchar := gc_start_of_new_item||gc_eol;
cn_end_indicator_title constant st_maxchar := '<a name="';
cn_start_indicator_presenterid constant st_maxchar := 'presenterlist?last_id=';
cn_end_indicator_presenterid constant st_maxchar := '">';
cn_start_indicator_presenter constant st_maxchar := '">';
cn_end_indicator_presenter constant st_maxchar := '</a>';
cn_start_indicator_company constant st_maxchar := ''||gc_eol;
cn_end_indicator_company constant st_maxchar := '';
cn_start_indicator_copresenter constant st_maxchar := 'Co-presenter(s):';
cn_end_indicator_copresenter constant st_maxchar := '</div>';
cn_start_indicator_date constant st_maxchar := 'When: '||gc_eol;
cn_end_indicator_date constant st_maxchar := ',';
...
begin
get_stripped_value (p_itemtext => t_itemtext
,p_start_indicator => cn_start_indicator_title
,p_end_indicator => cn_end_indicator_title
,p_value => r_sessions.title );
get_stripped_value (p_itemtext => t_itemtext
,p_start_indicator => cn_start_indicator_presenterid
,p_end_indicator => cn_end_indicator_presenterid
,p_value => t_presenterid );
...
p_fieldvalues := r_sessions;
end retrieve_field_values;
Look for certain html elements
Parse one by one

select *
from table(ero$kscope_sessions.kscope_sessions('http://kscope14.com/seminarlist'))
where lower(presenter) like '%hansen%'
order by starttime;
TOPIC SUBTOPIC STARTTIME ENDTIME SESSION_ID PRESENTER COMPANY COPRES
------------------- ----------- ---------------- ---------------- ---------- ---------- ----------------- ------
TITLE
----------------------------------------------------------------------------------------------------------------
DESCRIPTION
----------------------------------------------------------------------------------------------------------------
Developer's Toolkit Languages 2014-06-24 11:15 2014-06-24 12:15 Session 7 Kim Hansen T. Hansen Gruppen
Analytic Functions: Advanced Cases
Analytic functions can often help you, using data across rows in ways to allow you to solve tasks that otherwise
might have called for slowly using the data one row at a time procedurally. This presentation will show several
such cases of using analytic functions to avoid slow procedural data handling. All cases are from actual produc
tion system and will hopefully serve as inspiration.
Developer's Toolkit No Subtopic 2014-06-25 14:00 2014-06-25 15:00 Session 14 Kim Hansen T. Hansen Gruppen
External Data via HTTP, FTP, and Web Services
There are many ways you can access external data from within the Oracle database. This presentation will show di
fferent methods using HTTP or FTP protocol getting raw data, files, or XML via proprietary calls or web-service
calls (either REST or SOAP). Topics include HttpUriType, UTL_HTTP, UTL_DBWS, APEX_WEB_SERVICE API, and more.
Just two presentations by a presenter named Hansen

select presenter, title
from table(ero$kscope_sessions.kscope_sessions('http://kscope14.com/seminarlist'))
where session_id = 'Session 14'
order by presenter, title;
PRESENTER TITLE
--------------- ------------------------------------------------------------------------------------------------
Ashley Chen Hands-On Training: Developing RESTful APIs with Oracle REST Data Services for On-Premises or the
Charles Beyer Bulletproofing Excel and Smart View
Donald Clarke How to Integrate EBS 11i with APEX 4.2
Eric Erikson Why Aren't You Using Calc Manager with HFM Yet?
Heli Helskyaho Nine Use Cases for Oracle SQL Developer Data Modeler
Jackie Womble Production-Level Control of APEX Troubleshooting and Performance Tracing
Jake Turrell Planning Experts Panel
Jason Jones How to Integrate Essbase with Oracle and Third-Party Applications
Jeremy Harms Sorry, McGinley: Even Advanced OBIEE Modeling and Front-End Features Show Big Ten Football < SEC
John Flack Magic Managed Beans: A Primer for Jack and Jill ADF Developer
John Kozar Common APEX Responsive Design Issues and How to Correct Them
Jonathan Lewis Developers' Guide to Cost-Based Optimization
Kenny Vernon Numbers Don't Always Tell the Whole Story: Allowing Users to Enter Comments on OBIEE Dashboards
Kim Hansen External Data via HTTP, FTP, and Web Services
Kyle Hailey Agile Data Platform: Revolutionizing Database Cloning
Michael Casey Using the Open Interface Adapter in FDM Enterprise Edition 11.1.2.3
Paul Hoch Driver-Based Planning: When More Isn't Better
Scott LeBeau Implementing Advanced Financial Reporting Concepts for Rolling Trend Reports
Tiffany Briseno Hyperion Financial Reporting: Tips, Tricks, and Best Practices at TeleTech
19 rows selected.
Uh oh… Jonathan Lewis in same timeslot…

• Save site certificate using browser
• Create wallet on db server and add certificate:
mkdir /home/oracle/admin/wallet
orapki wallet create -wallet /home/oracle/admin/wallet -pwd myW4lletPasswd -auto_login
orapki wallet add -wallet /home/oracle/admin/wallet -trusted_cert -cert
"/mnt/share/savedcertificate.crt" -pwd myW4lletPasswd
• Set wallet for utl_http:
utl_http.set_wallet('file:/home/oracle/admin/wallet', 'myW4lletPasswd');
• Wallet will now be used for https:// urls
• http://www.oracle-base.com/articles/misc/utl_http-and-ssl.php
SSL (https) with UTL_HTTP

 FTP_UTIL_PKG
 UTL_HTTP
HttpUriType
 UTL_DBWS
 JPublisher
Topics

 Very simple for HTTP GET of content
 No SSL, No POST, No Cookies, No…
 Object type with member methods
● GetBlob()
● GetClob()
● GetXml()
 Not browser simulator
 But easy to get a resource from the web
HttpUriType

 Our use:
 HttpUriType('url').GetBlob()
● Retrieve employee picture from intranet website
 HttpUriType('url').GetClob()
● Get simple up/down status from payment provider
 HttpUriType('url').GetXml()
● Google Maps routes to find time needed for delivery
routes from shop to shop in fireworks season
HttpUriType

declare
directions xmltype;
begin
directions := httpuritype(
'maps.googleapis.com/maps/api/directions/xml'
|| '?origin=' || utl_url.escape(convert(
'100 Oracle Pkwy, Redwood City, CA'
, 'UTF8'
))
|| '&destination=' || utl_url.escape(convert(
'1400 6th Ave, Seattle, Washington'
, 'UTF8'
))
|| '&mode=driving&alternatives=true&units=metric'
|| '&region=us&language=en&sensor=false'
).getxml();
end;
/
REST-like Google Maps
Simply pass complete URL to HttpUriType
In SQL*Plus use SET DEFINE OFF to avoid substitution variable prompting
Or use || '&' || 'destination... as single '&' will not be treated as variable

<?xml version="1.0" encoding="UTF-8"?>
<DirectionsResponse>
<status>OK</status>
<route>
<summary>I-5 N</summary>
<leg>
<step>
<travel_mode>DRIVING</travel_mode>
<start_location>
<lat>37.5288827</lat>
<lng>-122.2666866</lng>
</start_location>
<end_location>
<lat>37.5288322</lat>
<lng>-122.2601636</lng>
</end_location>
<polyline>
<points>oz`dFxegiVCBADCHKTQX_@ZSHO@SDU?QAOCMGMIOMQSISKWKSKOSOKGKEOCiCMUCU@]?Q?OAMCOEMGQIQKKMU_@MWK]G_@C[AS?U?O?
QBk@Dg@Jg@^wAZ}@L]BODQDYBW@]@O?m@A}A@q@@Q?MBIBMFUDIJMRQHEBCJELCLA`@?T@F?x@@PAn@OZOLKXWLMFEd@Ob@IB?
LCPCPGRMPQN?BABAFCJGFAFCZE</points>
</polyline>
<duration>
<value>170</value>
<text>3 mins</text>
</duration>
<html_instructions>Head northwest on Oracle Pkwy toward
Marine Pkwy</html_instructions>
<distance>
<value>1247</value>...
XML results of call

select r.routenum, l.legnum
, numtodsinterval(l.seconds, 'second') time, l.meters / 1000 distance_km
, extractvalue(x.x,'/DirectionsResponse/status') status
, convert(r.routename,'WE8ISO8859P1','UTF8') routename
, convert(l.startaddr,'WE8ISO8859P1','UTF8') startaddr
, convert(l.endaddr,'WE8ISO8859P1','UTF8') endaddr
from xmltable(
'/DirectionsResponse'
passing httpuritype(
, 'UTF8'
))
, 'UTF8'
))
).getxml()
columns x xmltype path '/'
) x,
...
Process with XMLTABLE …
httpuritype().getxml()
in passing clause for
xmltable for easy
parsing of the result

...
xmltable(
'if (fn:empty(/DirectionsResponse/route)) then <route><leg></leg></route>
else /DirectionsResponse/route'
passing x.x
columns routenum for ordinality
, routename varchar2(100) path 'summary'
, r xmltype path '/'
) r,
xmltable(
'for $l in /route/leg return $l'
passing r.r
columns legnum for ordinality
, seconds number path 'duration/value'
, meters number path 'distance/value'
, startaddr varchar2(100) path 'start_address'
, endaddr varchar2(100) path 'end_address'
) l
order by r.routenum, l.legnum;
… and some Xquery …
Xquery here used to
make dummy empty
route if Google returned
an error status

ROUTENUM LEGNUM TIME DISTANCE_KM STATUS ROUTENAME
-------- ------ ------------------------------ ----------- ------ --------------------
STARTADDR ENDADDR
------------------------------------------------ ---------------------------------------
1 1 +000000000 12:37:44.000000000 1334.2 OK I-5 N
100 Oracle Parkway, Redwood City, CA 94065, USA 1400 6th Avenue, Seattle, WA 98101, USA
2 1 +000000000 15:19:09.000000000 1521.3 OK I-5 N
100 Oracle Parkway, Redwood City, CA 94065, USA 1400 6th Avenue, Seattle, WA 98101, USA
… to get route data
Google shows two alternative routes
- one 1334 km in a bit more than 12½ hours
- one 1521 km in a bit less than 15½ hours

select r.routenum, l.legnum, s.stepnum
, numtodsinterval(s.seconds, 'second') time, s.meters / 1000 distance_km
, convert(s.html_instructions,'WE8ISO8859P1','UTF8') html_instructions
from xmltable(
'/DirectionsResponse'
passing httpuritype(
, 'UTF8'
))
, 'UTF8'
))
).getxml()
columns x xmltype path '/'
) x, xmltable(
'if (fn:empty(/DirectionsResponse/route)) then <route><leg></leg></route>
else /DirectionsResponse/route'
passing x.x
columns routenum for ordinality
, routename varchar2(100) path 'summary'
, r xmltype path '/'
) r, xmltable(
'for $l in /route/leg return $l'
passing r.r
columns legnum for ordinality
, startaddr varchar2(100) path 'start_address'
, endaddr varchar2(100) path 'end_address'
, l xmltype path '/'
) l, xmltable(
'/leg/step'
passing l.l
columns stepnum for ordinality
, html_instructions varchar2(1000) path 'html_instructions'
) s
order by r.routenum, l.legnum, s.stepnum;
More XMLTABLE for route steps
Put on yet another XMLTABLE
to get individual steps

RO LE STE TIME DISTANC HTML_INSTRUCTIONS
-- -- --- ------------------------------ ------- --------------------------------------------------------------------------------
1 1 1 +000000000 00:02:50.000000000 1.2 Head northwest on Oracle Pkwy toward Marine Pkwy
1 1 2 +000000000 00:00:58.000000000 .7 Turn right onto Marine Pkwy
1 1 3 +000000000 00:21:28.000000000 33.6 Merge onto US-101 N via the ramp to San Francisco
1 1 4 +000000000 00:00:33.000000000 .6 Take the Interstate 80 exit toward Bay Bridge
1 1 5 +000000000 00:14:30.000000000 19.0 Merge onto I-80 E
1 1 6 +000000000 00:39:50.000000000 69.2 Keep left to stay on I-80 E<div style="font-size:0.9em">Partial to
ll road</div>
1 1 7 +000000000 00:28:11.000000000 55.0 Take the exit onto I-505 N toward Winters/Redding
1 1 8 +000000000 10:48:17.000000000 1154.3 Merge onto I-5 N<div style="font-size:0.9em">Passing through Oregon</div>
<div style="font-size:0.9em">Entering Washington</div>
1 1 9 +000000000 00:00:24.000000000 .4 Take exit 165 on the left for Seneca St
1 1 10 +000000000 00:00:43.000000000 .3 Slight right onto 6th Ave<div style="font-size:0.9em">Destination
will be on the right</div>
2 1 1 +000000000 00:02:50.000000000 1.2 Head northwest on Oracle Pkwy toward Marine Pkwy
2 1 2 +000000000 00:00:58.000000000 .7 Turn right onto Marine Pkwy
2 1 3 +000000000 00:21:28.000000000 33.6 Merge onto US-101 N via the ramp to San Francisco
2 1 4 +000000000 00:00:33.000000000 .6 Take the Interstate 80 exit toward Bay Bridge
2 1 5 +000000000 00:14:30.000000000 19.0 Merge onto I-80 E
2 1 6 +000000000 00:52:11.000000000 91.9 Keep left to stay on I-80 E<div style="font-size:0.9em">Partial to
ll road</div>
2 1 7 +000000000 00:10:46.000000000 19.5 Keep left to stay on I-80 E
2 1 8 +000000000 02:04:42.000000000 220.0 Keep right to stay on I-80 E, follow signs for Reno<div sty
le="font-size:0.9em">Entering Nevada</div>
2 1 9 +000000000 01:13:38.000000000 129.5 Take exit 15 to merge onto US-395 N toward Susanville<div s
tyle="font-size:0.9em">Entering California</div>
2 1 10 +000000000 00:00:06.000000000 .1 Slight left to stay on US-395 N
2 1 11 +000000000 00:03:12.000000000 4.6 Continue onto CA-36 W
2 1 12 +000000000 00:00:20.000000000 .2 Turn right onto Riverside Dr
2 1 13 +000000000 00:00:47.000000000 .6 Continue straight onto Johnstonville Rd
2 1 14 +000000000 00:02:52.000000000 3.3 Turn left onto Skyline Rd
2 1 15 +000000000 01:31:15.000000000 140.0 Turn right onto CA-139 N
2 1 16 +000000000 00:51:31.000000000 88.5 Turn left to stay on CA-139 N<div style="font-size:0.9em">Entering
Oregon</div>
2 1 17 +000000000 00:19:24.000000000 27.6 Continue onto OR-39 N
2 1 18 +000000000 00:06:36.000000000 9.4 Turn left onto OR-140 W

 "Proper" REST webservice exposed like Oracle
Rest Data Services with documentcentric URLs
● http://myserver.net/myprefix/hr/employees
 GET data with HttpUriType('url').GetXml()
or
 GET data with HttpUriType('url').GetClob()
and use 12.2 JsonTable() to parse result
 Not POST, PUT, DELETE etc.
RESTful webservicecalls

 FTP_UTIL_PKG
 UTL_HTTP
 HttpUriType
UTL_DBWS
 JPublisher
Topics

 Helps setup SOAP webservice calls
● Handles SOAP envelope, service, port, operation
 Can do SSL (https) with wallets
 Extra installation (not in default install)
 Uses java classes
 Our use:
● Payment authorization via provider’s SOAP
webservice that uses SSL
UTL_DBWS

 Starting point:
http://www.oracle.com/technetwork/database/d
atabase-083829.html
 Download:
http://download.oracle.com/technology/sample_
code/tech/java/jsp/dbws-callout-utility-10131.zip
 MOS (metalink) note 838892.1:
FAQ on UTL_DBWS / Jpublisher callout
UTL_DBWS

UTL_DBWS privileges
grant javauserpriv to scott;
BEGIN
SYS.DBMS_JAVA.GRANT_PERMISSION(
grantee => 'SCOTT'
,permission_type =>
'SYS:java.net.SocketPermission'
,permission_name => '*'
,permission_action => 'connect, resolve'
);
grantee => 'SCOTT'
,permission_type =>
'SYS:java.util.PropertyPermission'
,permission_name => '*'
,permission_action => 'read,write'
);
grantee => 'SCOTT'
,permission_type =>
'SYS:java.lang.RuntimePermission'
,permission_name => 'getClassLoader'
,permission_action => ''
);
grantee => 'SCOTT'
,permission_type =>
,permission_name => 'createClassLoader'
);
grantee => 'SCOTT'
,permission_type =>
'SYS:java.util.logging.LoggingPermission'
,permission_name => 'control'
);
grantee => 'SCOTT'
,permission_type =>
,permission_name => 'setFactory'
);
commit;
END;
/ Not ACL, but java privileges

unzip dbws-callout-utility-10131.zip
cp sqlj/lib/* $ORACLE_HOME/sqlj/lib
cd $ORACLE_HOME/sqlj/lib
loadjava -user scott/tiger -resolve -verbose -force -genmissing dbwsclientws.jar
dbwsclientdb11.jar
sqlplus scott/tiger
@utl_dbws_decl.sql
@utl_dbws_body.sql
Alternative for installing for public use:
loadjava -user tools/tooluserpsw -resolve -verbose -force -genmissing -synonym -grant public
dbwsclientws.jar dbwsclientdb11.jar
Create packages in tools schema and grant privileges as necessary
UTL_DBWS install

declare
l_namespace varchar2(1000) := 'http://ws.cdyne.com/WeatherWS/';
l_wsdlurl varchar2(1000) := 'http://wsf.cdyne.com/WeatherWS/Weather.asmx?WSDL';
l_endpointurl varchar2(1000) := 'http://wsf.cdyne.com/WeatherWS/Weather.asmx';
l_zipcode varchar2(10) := '98101'; -- Seattle
l_service utl_dbws.service;
l_call utl_dbws.call;
l_service_qname utl_dbws.qname;
l_port_qname utl_dbws.qname;
l_operation_qname utl_dbws.qname;
l_request xmltype;
l_response xmltype;
begin
l_service_qname := utl_dbws.to_qname(l_namespace, 'Weather');
l_service := utl_dbws.create_service(httpuritype(l_wsdlurl), l_service_qname);
l_port_qname := utl_dbws.to_qname(l_namespace, 'WeatherSoap12');
l_operation_qname := utl_dbws.to_qname(l_namespace, 'GetCityWeatherByZIP');
l_call := utl_dbws.create_call(
l_service
, l_port_qname
, l_operation_qname
);
utl_dbws.set_property(l_call, 'SESSION_MAINTAIN', 'FALSE');
utl_dbws.set_property(l_call, 'SOAPACTION_USE', 'TRUE');
utl_dbws.set_property(l_call, 'SOAPACTION_URI', l_endpointurl);
utl_dbws.set_property(l_call, 'ENCODINGSTYLE_URI', 'http://schemas.xmlsoap.org/soap/encoding/');
utl_dbws.set_property(l_call, 'OPERATION_STYLE', 'document');
...

...
utl_dbws.set_target_endpoint_address(l_call, l_endpointurl);
select xmlroot(
xmlelement(
"GetCityWeatherByZIP"
, xmlattributes('http://ws.cdyne.com/WeatherWS/' as "xmlns")
, xmlelement(
"ZIP"
, l_zipcode
)
)
, version '1.0'
)
into l_request
from dual;
l_response := utl_dbws.invoke(call_Handle => l_call, request => l_request);
utl_dbws.release_call(l_call);
utl_dbws.release_service(l_service);
utl_dbws.release_all_services();
dbms_output.put_line(l_response.getclobval());
end;
/
Select from DUAL to avoid error:
PLS-00122: AS as separator is allowed
only with specific built-in functions

-- l_request –-
<?xml version="1.0"?>
<GetCityWeatherByZIP xmlns="http://ws.cdyne.com/WeatherWS/">
<ZIP>98101</ZIP>
</GetCityWeatherByZIP>
-- l_response --
<GetCityWeatherByZIPResponse xmlns="http://ws.cdyne.com/WeatherWS/">
<GetCityWeatherByZIPResult>
<Success>true</Success>
<ResponseText>City Found</ResponseText>
<State>WA</State>
<City>Seattle</City>
<WeatherStationCity>Renton</WeatherStationCity>
<WeatherID>3</WeatherID>
<Description>Mostly Cloudy</Description>
<Temperature>72</Temperature>
<RelativeHumidity>37</RelativeHumidity>
<Wind>VRB6</Wind>
<Pressure>29.98R</Pressure>
<Visibility/>
<WindChill/>
<Remarks/>
</GetCityWeatherByZIPResult>
</GetCityWeatherByZIPResponse>

• Add certificate to wallet in path:
$ORACLE_HOME/javavm/lib/security/cacerts
• Java based UTL_DBWS can now use wallet for https:// urls
• MOS (metalink) note 443438.1
SSL (https) with UTL_DBWS

 FTP_UTIL_PKG
 UTL_HTTP
 HttpUriType
 UTL_DBWS
JPublisher
Topics

 Install SQLJ from client installation
See JPublisher user guide for details
● http://docs.oracle.com/cd/E11882_01/java.112/e10587/t
 Serverside load java and UTL_DBWS
See previous slides
 Can be clientside only on development PC if so
desired – jpub really is just code generator
JPublisher

 Might use local copy of wsdl file rather than
URL to original WSDL file
● For example to annotate WSDL file with Oracle
datatypes or fix WSDL file xml schema errors
 Our use:
● Other cardata from dataprovider we FTP from
● Parcel registering with parcel service
● Parcel track & trace with parcel service
JPublisher for SOAP callout

SET JAVA_HOME=%ORACLE_HOME%jdk
SET TNS_ADMIN=%ORACLE_HOME%networkadmin
SET PATH=%JAVA_HOME%bin;%JAVA_HOME%jrebin;%PATH%
SET CLASSPATH=%ORACLE_HOME%sqljlibdbwsa.jar;%ORACLE_HOME%jdklibdt.jar;^
%ORACLE_HOME%jdklibtools.jar;^
%ORACLE_HOME%jlibjssl-1_1.jar;%ORACLE_HOME%jdbclibojdbc5.jar;^
%ORACLE_HOME%sqljlibruntime12.jar;%ORACLE_HOME%jliborai18n.jar;^
%ORACLE_HOME%sqljlibtranslator.jar;%ORACLE_HOME%javavmlibaurora.zip;^
%ORACLE_HOME%rdbmsjlibxdb.jar;%ORACLE_HOME%libxsu12.jar:^
%ORACLE_HOME%jlibjndi.jar;%ORACLE_HOME%rdbmsjlibaqapi.jar;^
%ORACLE_HOME%rdbmsjlibjmscommon.jar;%ORACLE_HOME%libxmlparserv2.jar
SET LOCAL=MYORCL
jpub -user=scott/tiger -sysuser=sys/syspwd -dir=weather -package=weather
-plsqlpackage=weather -proxywsdl=http://wsf.cdyne.com/WeatherWS/Weather.asmx?WSDL
For testing without loading into the database:
jpub -user=scott/tiger -sysuser=sys/syspwd -proxyopts=noload -dir=weather -package=weather
-plsqlpackage=weather -proxywsdl=http://wsf.cdyne.com/WeatherWS/Weather.asmx?WSDL
On windows client with jpub

weatherobject := weather.GetCityWeatherByZIP('98101');
dbms_output.put_line(weatherobject.Description);
select w.weatherobject.Success
, w.weatherobject.ResponseText
, w.weatherobject.State
, w.weatherobject.City
, w.weatherobject.WeatherStationCity
, w.weatherobject.WeatherID
, w.weatherobject.Description
, w.weatherobject.Temperature
, w.weatherobject.RelativeHumidity
, w.weatherobject.Wind
, w.weatherobject.Pressure
, w.weatherobject.Visibility
, w.weatherobject.WindChill
, w.weatherobject.Remarks
from (
select weather.GetCityWeatherByZIP('98101') weatherobject
from dual
) w
/
Use of generated objects PL/SQL or SQL
PL/SQL use object type
Object in SQL as well
Beware this may call the
webservice 14 times as SQL
engine may rewrite to call
package function once for
every column

 FTP_UTIL_PKG
 UTL_HTTP
 HttpUriType
 UTL_DBWS
 JPublisher
APEX_WEB_SERVICE
Topics

 Useful even if not using APEX
 SOAP
● Manual SOAP envelope handling
 REST
● GET, POST, PUT, DELETE, etc.
 SSL made simple with wallet parameters
 Uses network ACL, not java privileges
 Possibly more "supported" in future
APEX_WEB_SERVICE

Add a bit to network ACL
begin
dbms_network_acl_admin.add_privilege(
, principal => 'APEX_040100'
, is_grant => true
);
commit;
end;
/
begin
, host => '*.cdyne.com'
);
commit;
end;
/
APEX user granted connect
privilege to our demo ACL
Use APEX user for your
APEX version
Weather URL added to ACL
Was not needed for UTL_DBWS
java based, but is needed for
APEX_WEB_SERVICE

declare
directions clob;
begin
directions := apex_web_service.make_rest_request(
p_url => 'http://maps.googleapis.com/maps/api/directions/xml'
, p_http_method => 'GET'
, p_parm_name => apex_util.string_to_table(
'origin:destination:mode:alternatives:units:region:language:sensor'
)
, p_parm_value => apex_util.string_to_table(
'100 Oracle Pkwy, Redwood City, CA:1400 6th Ave, Seattle,
Washington:driving:true:metric:us:en:false'
)
);
dbms_output.put_line(substr(directions,1,4000));
end;
/
APEX_WEB_SERVICE Google Maps
APEX_WEB_SERVICE REST call
Example of our google maps call
Colon separated
name/value pairs

declare
envelope clob;
weather xmltype;
begin
select xmlroot(
xmlelement(
"soap:Envelope"
, xmlattributes(
'http://www.w3.org/2001/XMLSchema-instance' as "xmlns:xsi"
, 'http://www.w3.org/2001/XMLSchema' as "xmlns:xsd"
, 'http://schemas.xmlsoap.org/soap/envelope/' as "xmlns:soap" )
, xmlelement(
"soap:Body"
, xmlelement(
"GetCityWeatherByZIP"
, xmlattributes('http://ws.cdyne.com/WeatherWS/' as "xmlns")
, xmlelement("ZIP", '98101')
) ) )
, version '1.0'
).getclobval()
into envelope
from dual;
weather := apex_web_service.make_request(
p_url => 'http://wsf.cdyne.com/WeatherWS/Weather.asmx'
, p_action => 'http://ws.cdyne.com/WeatherWS/GetCityWeatherByZIP'
, p_version => '1.1'
, p_envelope => envelope );
dbms_output.put_line(weather.getclobval());
end;
/
APEX_WEB_SERVICE SOAP call
Example of our weather call
Unlike UTL_DBWS SOAP envelope is
manually created here and response
will be wrapped in SOAP envelope

• Create wallet on db server and add certificate:
mkdir /home/oracle/admin/wallet
orapki wallet create -wallet /home/oracle/admin/wallet -pwd myW4lletPasswd -auto_login
orapki wallet add -wallet /home/oracle/admin/wallet -trusted_cert -cert
"/mnt/share/savedcertificate.crt" -pwd myW4lletPasswd
• Use wallet in calls like:
directions := apex_web_service.make_rest_request(
p_url => 'https://maps.googleapis.com/maps/api/directions/xml'
, ...
, p_wallet_path => '/home/oracle/admin/wallet'
, p_wallet_pwd => 'myW4lletPasswd'
);
SSL (https) with APEX_WEB_SERVICE

 Several methods available
 Each has different pros and cons
 No single "best" way
 Pick easiest in each circumstance
 Maybe APEX_WEB_SERVICE is where future
development might be from Oracle?
Summary

 Danish geek
 Oracle SQL Evangelist
 Oracle PL/SQL Developer
 Likes to cook
 Reads sci-fi
 Member of
Danish Beer Enthusiasts
Questions ?
http://dspsd.blogspot.com
http://dk.linkedin.com/in/kibeha/
@kibeha
Kim Berg Hansen
http://goo.gl/QuHQ1f
for this presentation

Oracle database - Get external data via HTTP, FTP and Web Services

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

En vedette

En vedette (20)

Similaire à Oracle database - Get external data via HTTP, FTP and Web Services

Similaire à Oracle database - Get external data via HTTP, FTP and Web Services (20)

Plus de Kim Berg Hansen

Plus de Kim Berg Hansen (9)

Dernier

Dernier (20)

Oracle database - Get external data via HTTP, FTP and Web Services