Oracle Text is a facility within the database that provides more advanced indexing & search techniques - including the ability to index documents stored in your database; on your server; or even the web! Now you can incorporate this functionality into your web application using Oracle Application Express. This presentation will demonstrate how easy it is to combine the two, and give you a platform for further expansion and exploration within a very powerful product.

1. SAGE Computing Services
Customised Oracle Training Workshops and Consulting
Oracle Text in Apex
Advanced Indexing Techniques Integrated with
Application Express
Scott Wesley
Systems Consultant & Trainer

2. Agenda
• Introduction
• Architecture
• Fundamentals
• Considerations
• Setting Up
• Samples
• Index Maintenance
• Visualisation
• New Features

3. Larry Lessig?

4. the law is strangling creativity
http://www.ted.com/talks/larry_lessig_says_the_law_is_strangling_creativity.html
http://presentationzen.blogs.com/presentationzen/2005/10/the_lessig_meth.html

5. Identity 2.0 – Dick Hardt
http://identity20.com/media/OSCON2005/

6. who’s the Dick on your site

7. Connor McDonald
http://www.oracledba.co.uk

8. so today’s going to be more like this

9. and this

10. after I show a few pictures

11. who_am_i;

12. http://strategy2c.wordpress.com/2009/01/10/strategy-for-goldfish-funny-illustration-by-frits/

15. balance

23. Why use Oracle Application Express?

24. Why use Oracle Text?

26. What is Oracle Text?

27. Document Collection

29. Catalogue Information

31. Document Classification

33. Architecture

34. Class Description
Datastore How are your documents stored?
Filter How can the documents be converted to plain text?
Lexer What language is being indexed?
Wordlist How should stem and fuzzy queries be expanded?
Storage How should the index data be stored?
Stop List What words or themes are not to be indexed?
Section Group How are documents sections defined?

35. 1) Example

36. CREATE INDEX ctx_name ON my_names(name)
INDEXTYPE IS ctxsys.context
PARAMETERS ('DATASTORE CTXSYS.DEFAULT_DATASTORE');

37. SQL> SELECT SCORE(1), name
2 FROM my_names
3 WHERE CONTAINS(name, 'fuzzy(john,,,weight)', 1) > 0
4 ORDER BY SCORE(1) DESC;
SCORE(1) NAME
---------- ----------------------------------------
100 John
100 John
70 Jon
70 Jon
63 Joan
63 Joan
52 Jong
48 Jona
8 rows selected.

38. 2) Datastore

39. CTXSYS.DEFAULT_DATASTORE

40. BLOB

41. BFiles

42. Pointers to objects on file system

43. URLs

44. Pointers to objects on the intertube

45. User Defined

46. Why would you?

47. 3) Index Type

48. a) CONTEXT

49. Document Collection

50. large document size

51. provides a score

52. asynchronous index & table data

54. CONTAINS

55. b) CTXCAT

56. Catalogue Information

57. smaller documents

58. text fragments

59. multiple attributes

60. set lists

61. similar to typical index paradigm

62. transactional

64. CATSEARCH

65. c) CTXRULE

66. Document Classification

67. routing information

68. displace manual interaction

69. not binary files

70. MATCHES

71. 4) Considerations

72. location of text

73. document format

74. bypassing rows - images

75. character set

76. language

77. fuzzy matching & stemming

78. wildcard query performance

80. stopwords & stopthemes

81. query performance and storage of LOBs

82. mixed queries

83. 5) Setting up

84. GRANT ctxapp TO ausoug;

85. create & delete indexing preferences

86. use Oracle Text PL/SQL supplied packages

87. 1* select grantee, owner, table_name, privilege from dba_tab_privs where table_name =
'CTX_DDL'
SQL> /
GRANTEE OWNER TABLE_NAME PRIVILEGE
-------------------- ------------ ------------------------------ --------------------
CTXAPP CTXSYS CTX_DDL EXECUTE
APEX_040000 CTXSYS CTX_DDL EXECUTE
APEX_030200 CTXSYS CTX_DDL EXECUTE
AUSOUG CTXSYS CTX_DDL EXECUTE
XDB CTXSYS CTX_DDL EXECUTE
5 rows selected.

88. PLS-00201: identifier "string" must be declared

89. CTX PL/SQL Packages
GRANT EXECUTE ON CTXSYS.CTX_CLS TO ausoug;
GRANT EXECUTE ON CTXSYS.CTX_DDL TO ausoug;
GRANT EXECUTE ON CTXSYS.CTX_DOC TO ausoug;
GRANT EXECUTE ON CTXSYS.CTX_OUTPUT TO ausoug;
GRANT EXECUTE ON CTXSYS.CTX_QUERY TO ausoug;
GRANT EXECUTE ON CTXSYS.CTX_REPORT TO ausoug;
GRANT EXECUTE ON CTXSYS.CTX_THES TO ausoug;
GRANT EXECUTE ON CTXSYS.CTX_ULEXER TO ausoug;

90. Using URL Datastore in 11g
CREATE ROLE apex_url_datastore_role;
GRANT apex_url_datastore_role TO APEX_040000
WITH ADMIN OPTION;
GRANT apex_url_datastore_role TO ausoug;
EXEC
ctxsys.ctx_adm.set_parameter
('file_access_role'
,'APEX_URL_DATASTORE_ROLE');

91. Demonstrations
Script Description
Ctx_blobs.sql Import & index a range of documents
Ctx_bfiles.sql Import & index BFILE pointers
Ctx_urls.sql Index & search URL references
Ctx_dict.sql Index & search English dictionary words
Ctx_views.sql Index view SQL text for impact analysis
Ctx_apex_files.sql Duplicate and search Apex file repository
Ctx_apex_backups.sql Hunt through your (automated) Apex app backups
Ctx_names.sql Basic name filter options
Ctx_products.sql Multiple column searches
Ctx_category.sql Attribute based searching
Ctx_classify.sql Classify documents into categories

112. 6) Index maintenance

113. indexing errors

115. resume failed index

116. ALTER INDEX ctx_surname
REBUILD PARAMETERS ('resume memory 10m');

117. recreate index online (11g)

118. EXEC ctx_ddl.recreate_index_online
('ctx_surname', 'replace lexer sw_lexer');

119. rebuilding an index

120. ALTER INDEX ctx_surname
REBUILD PARAMETERS('replace lexer sw_lexer')
ONLINE;

121. ctx_report.index_stats

122. create table ausoug.my_stats (stats clob);
declare
x clob := null;
begin
for r_rec in
(select *
from ctxsys.ctx_indexes
where idx_owner = 'AUSOUG'
and idx_type = 'CONTEXT') loop
ctx_report.index_stats(r_rec.idx_name,x);
insert into ausoug.my_stats values (x);
end loop;
commit;
dbms_lob.freetemporary(x);
end;
/

123. 7) Data Dictionary

128. SQL> select count(*)
2 from all_views
3 where owner = 'CTXSYS';
COUNT(*)
----------
58

129. 8) Common Questions

130. DML operations on a CONTEXT index

131. ctxsys.ctx_user_pending

132. synchronise the index
synchronize

133. EXEC ctx_ddl.sync_index('ctx_surname');

134. dbms_job

135. dbms_scheduler

136. how often?

138. optimise the index

139. can get fragmented

140. inverted index

141. each entry contains list of documents

142. DOG - DOC1 DOC3 DOC5
DOG - DOC7
DOG - DOC9
DOG - DOC11

143. ctx_ddl.optimize_index

144. capacity planning?

145. Object of Interest Num
Rows
Table
Size
Index
size
Dictionary 150k 7 27
Documents 28 34 1.5
Names 27k 1 6
Views 2k 7 2
BFiles 4
Product 1
URL 1

146. more text

147. cleaner data

148. less overhead

149. document format

151. next steps?

152. read Application Developer’s Guide

154. find examples

155. experiment

156. SAGE Computing Services
Customised Oracle Training Workshops and Consulting
Question time
Presentations are available from our website:
http://www.sagecomputing.com.au
enquiries@sagecomputing.com.au
scott.wesley@sagecomputing.com.au
http://triangle-circle-square.blogspot.com