Deze presentatie is gegeven tijdens de KScope conferentie 2012
Spreker: Patrick Barel
Titel Can Collections Speed Up Your PL/SQL?
Onderwerp: Developers Toolbox - Coding
Deze presentatie gaat over het gebruik van Collections in PL/SQL. Hoe werken deze arrays? Hoe gebruikt Oracle deze structuren in één van de belangrijkste verbeteringen op het gebied van prestaties van PL/SQL code in combinatie met uitgevoerde SQL queries? Daarnaast wordt er ingegaan op het gebruik van Table Functions, waarmee je kracht van PL/SQL in SQL kunt gebruiken. In deze sessie leer je om met minimale inspanning een grote verbetering in de performance en onderhoudbaarheid van je PL/SQL code te bereiken.
1. Developers Toolbox - Coding
Can collections speed up your PL/SQL?
Patrick Barel , AMIS, The Netherlands
Wednesday, June 27, 2012
ODTUG KScope 12
San Antonio, Texas, USA
5. Records
Record type definition
,
TYPE record_type IS RECORD ( field_definition ) ;
Field definition
NOT NULL
:=
expression
DEFAULT
field datatype
10. About Associative Arrays
Number of elements is unbounded, practically speaking.
Valid row numbers range from -231+1 to 231-1.
(i.e. -2,147,483,647 to 2,147,483,647)
This range allows you to employ the row number as an intelligent key,
such as the primary key or unique index value, because…
AAs also:
Can be sparse.
Data does not have to be stored in consecutive rows, as is required in
traditional 3GL arrays and VARRAYs.
Can have index values of integers or strings (Oracle9i R2
and above).
assoc_array_example.sql
collection_of_records.sql
count_vs_rowcount.sql
11. Associative Arrays
Name Changes:
7: PL/SQL Tables
8i: Index By Tables
9i: Associative Arrays
Single dimensioned, unbounded, sparse collection of
homogeneous elements
PL/SQL Only
key value
13. Associative Array
declare
type num_tbl is table of number
index by binary_integer;
l_num_tbl num_tbl; key value
l_idx integer;
begin
...
14. Associative Array
declare
type num_tbl is table of number
index by binary_integer;
l_num_tbl num_tbl; key value
l_idx integer;
begin 1 12
l_num_tbl(1):= 12; 54 5
l_num_tbl(54):= 5;
...
15. Associative Array
declare
type num_tbl is table of number
index by binary_integer;
l_num_tbl num_tbl; key value
l_idx integer;
begin 1 12
l_num_tbl(1):= 12; 3 98
l_num_tbl(54):= 5;
l_num_tbl(3):= 98; 5 3
l_num_tbl(5):= l_num_tbl.count;
l_idx:= l_num_tbl.first; 54 5
...
end;
16. Associative Array
declare
type num_tbl is table of number
index by binary_integer;
l_num_tbl num_tbl; key value
l_idx integer;
begin 2
1 12
l_num_tbl(1):= 12; 3 98
l_num_tbl(54):= 5;
l_num_tbl(3):= 98; 5 3
l_num_tbl(5):= l_num_tbl.count;
10
l_idx:= l_num_tbl.first; 54 5
loop
dbms_output.put_line(l_num_tbl(l_idx));
l_idx:= l_num_tbl.next(l_idx);
exit when l_idx is null;
end loop;
key value
l_num_tbl.delete(2,10); 1 12
dbms_output.put_line(l_num_tbl.count);
end; 54 5
17. Associative Array VARCHAR2 Key
declare
type str_tbl is table of varchar2(40)
index by varchar2(40);
l_str_tbl str_tbl; key value
l_idx varchar2(40);
begin
...
18. Associative Array VARCHAR2 Key
declare
type str_tbl is table of varchar2(40)
index by varchar2(40);
l_str_tbl str_tbl; key value
l_idx varchar2(40);
begin one een
l_str_tbl('one'):= 'een'; two twee
l_str_tbl('two'):= 'twee';
...
19. Associative Array VARCHAR2 Key
declare
type str_tbl is table of varchar2(40)
index by varchar2(40);
l_str_tbl str_tbl; key value
l_idx varchar2(40);
begin four vier
l_str_tbl('one'):= 'een'; one een
l_str_tbl('two'):= 'twee';
l_str_tbl('three'):= 'drie'; three drie
l_str_tbl('four'):= 'vier';
l_idx:= l_str_tbl.first; two twee
...
end;
20. Associative Array VARCHAR2 Key
declare
type str_tbl is table of varchar2(40)
index by varchar2(40);
l_str_tbl str_tbl; key value
l_idx varchar2(40);
begin four vier
l_str_tbl('one'):= ‘een'; o
one een
l_str_tbl('two'):= ‘twee';
l_str_tbl('three'):= 'drie'; three drie
l_str_tbl('four'):= 'vier'; tr
l_idx:= l_str_tbl.first; two twee
loop
dbms_output.put_line(l_str_tbl(l_idx));
l_idx:= l_str_tbl.next(l_idx);
key value
exit when l_idx is null;
end loop; four vier
l_str_tbl.delete('o', 'tr');
dbms_output.put_line(l_str_tbl.count); two twee
end;
21. Retrieval
for idx in emp_t.first .. emp_t.last
loop • Dense
... • Count > 0
end loop;
for idx in 1 .. emp_t.count
loop
• Dense
...
end loop;
idx := emp_t.first;
while idx is not null
loop
•…
...
idx := emp_t.next (idx);
end loop;
23. About Varrays
Has a maximum size, associated with its type.
Can adjust the size in Oracle10g R2.
Part of object model, requiring initialization.
Is always dense; you can only remove elements
from the end of a varray.
Can be defined as a schema level type and used
as a relational table column type.
varray_example.sql
24. Varray
Single dimensioned, always bounded, never
sparse collection of homogeneous elements
SQL and PL/SQL
Can be used as column datatype in a table
Stored “in-line” in same table
Retains its ordering key value
Needs to be initialized and extended
26. Using Varray
declare
type ename_vt is varray (10) of varchar2(10);
ename_t ename_vt;
begin Initialization
ename_t := ename_vt();
ename_t.extend (1);
ename_t(1) := 'Spencer';
...
end;
declare
type ename_vt is varray (10) of varchar2(10);
ename_t ename_vt := ename_vt ('Davis');
begin
...
end; Initialization and
Extending
27. How Variable is the Varray?
Pre 10gR2: VARRAY needed to be recreated.
10gr2 and up: ALTER TYPE MODIFY LIMIT
Only to increase the limit
varraylimit.sql
29. About Nested Tables
Name reflects fact that this collection can be
"nested" inside relational table as a column.
Type can be defined at schema level.
No practical, pre-defined limit on a nested table.
Valid row numbers range from 1 to 231-1.
(i.e. 1 to 2,147,483,647)
Part of object model, requiring initialization.
Is always dense initially, but can become sparse
after deletes.
nested_table_example.sql
30. Nested Tables
Single dimensioned, unbounded, sparse collection
of homogeneous elements
SQL and PL/SQL
Can be used as column datatype in a table
Stored “out-of-line” in a separate table
Initially dense, can be sparse
key value
32. Using Nested Tables
declare
type ename_nt is table of varchar2(10);
ename_t ename_nt;
begin Initialization
ename_t := ename_nt();
ename_t.extend (1);
ename_t(1) := 'Spencer';
...
end;
declare
type ename_nt is table of varchar2(10);
ename_t ename_nt := ename_nt ('Davis');
begin
...
end; Initialization and
Extending
33. Differences
Feature Associative Array Nested Table VArray
SQL – PL/SQL PL/SQL only SQL and PL/SQL SQL and PL/SQL
Dense - Sparse Sparse Initially Dense Dense
Can become sparse
Size ‘Unlimited’ ‘Unlimited’ Limited
Order Unordered Unordered Ordered
Usage Any set of data Any set of data Small sets of data
Use in Table No Yes Yes
34. Bulk Processing in PL/SQL
FORALL
Use with inserts, updates and deletes.
Move data from collections to tables.
BULK COLLECT
Use with implicit and explicit queries.
Move data from tables into collections.
In both cases, the "back back" end processing in the
SQL engine is unchanged.
Same transaction and rollback segment management
Same number of individual SQL statements will be
executed.
But BEFORE and AFTER statement-level triggers only
fire once per FORALL INSERT statements.
statement_trigger_and_forall.sql
35. BULK COLLECT for multi-row querying
SELECT * BULK COLLECT INTO collection FROM table;
FETCH cur BULK COLLECT INTO collection;
Fetch one or more rows into a collection.
Collection is always filled sequentially from index value 1.
Query does not raise NO_DATA_FOUND if no rows are
fetched.
Instead, the collection is empty.
Use FETCH with LIMIT to manage memory.
36. BULK COLLECT for multi-row querying
SELECT * BULK COLLECT INTO collection FROM table;
FETCH cur BULK COLLECT INTO collection;
Fetch one or more rows into a collection.
Collection is always filled sequentially from index value 1.
Query does not raise NO_DATA_FOUND if no rows are
fetched.
Instead, the collection is empty.
Use FETCH with LIMIT to manage memory.
37. Limiting retrieval with BULK COLLECT
If you are certain that your table will never have
more than N rows, use a VARRAY (N) to hold the
fetched data.
If that limit is exceeded, Oracle will raise an
error.
If you do not know in advance how many rows
you might retrieve, you should:
Declare an explicit cursor.
Fetch BULK COLLECT with the LIMIT clause.
38. Details on that LIMIT clause
The limit value can be a literal or a variable.
Use a variable for the limit to give you
maximum flexibility.
With very large volumes of data and small
numbers of batch processes, however, a larger
LIMIT could help.
39. Terminating loops containing BULK COLLECT
LOOP
FETCH my_cursor BULK COLLECT INTO l_collection LIMIT 100;
EXIT WHEN my_cursor%NOTFOUND; BAD IDEA
You will need to break the habit of checking
%NOTFOUND right after the fetch.
You might skip processing some of your data.
Instead, do one of the following:
At the end of the loop, check %NOTFOUND.
Right after fetch, exit when collection.COUNT = 0.
At end of loop, exit when collection.COUNT < limit.
bulklimit_stop.sql
40. When to convert to BULK COLLECT
Prior to Oracle10g, you should convert all multiple row
fetch logic, including cursor for loops, to BULK
COLLECTs.
For Oracle10g and above, leave your cursor for loops in
place if they...
contain no DML operations.
seem to be running fast enough.
Explicit BULK COLLECTs will usually run faster than
cursor for loops optimized to Bulk Collect.
41. Use FORALL for multi-row DML operations
PROCEDURE upd_for_dept (...) IS
BEGIN
FORALL indx IN low_value .. high_value
UPDATE employee
SET salary = newsal_in
WHERE employee_id = list_of_emps (indx);
END;
Binding array
Convert loops that contain inserts, updates or deletes to
FORALL statements.
Header looks identical to a numeric FOR loop.
Implicitly declared integer iterator
At least one "bind array" that uses this iterator as its
index value.
42. More on FORALL
Use any type of collection with FORALL.
One DML statement is allowed per FORALL.
Each FORALL is its own "extended" DML statement.
The collection must be indexed by integer.
The binding array must be sequentially filled.
Unless you use the INDICES OF or VALUES OF clause.
SQL%ROWCOUNT returns total number of rows modified
by entire FORALL.
Unreliable when used with LOG ERRORS.
Use the SQL%BULK_ROWCOUNT cursor attribute to
determine how many rows are modified by each
statement.
bulktiming.sql
bulk_rowcount.sql
43. FORALL and collections of records
Prior to 11g, you cannot reference a field of a record in
FORALL.
You must instead break data into separate collections, or...
You can also perform record-level inserts and updates.
In 11g, this restriction is lifted (but it is an undocumented
feature).
http://technology.amis.nl/blog/2367/implementation-restricted-relaxed-in-oracle-11g
11g_field_of_record.sql
44. INDICES OF and VALUES OF
Prior to Oracle10g R2, the binding arrays in a FORALL
statement must be sequentially filled.
Now, however, you can bind sparse collections by using
INDICES OF and VALUES OF in the FORALL header.
PROCEDURE upd_for_dept (...) IS
BEGIN
FORALL indx IN INDICES OF list_of_emps
UPDATE employee
SET salary = newsal_in
WHERE employee_id = list_of_emps (indx);
10g_indices_of*.sql
10g_values_of*.sql
45. Exception handling and FORALL
When an exception occurs in a DML statement....
That statement is rolled back and the FORALL stops.
All (previous) successful statements are not rolled
back.
Use the SAVE EXCEPTIONS clause to tell Oracle to
continue past exceptions, and save the error information
for later.
Then check the contents of the pseudo-collection of
records, SQL%BULK_EXCEPTIONS.
Two fields: ERROR_INDEX and ERROR_CODE
46. Converting old-fashioned code to bulk
slow-by-slow
Change from integrated, row-by-row approach to
a phased approach.
Phase 1: get the data with BULK COLLECT.
Filling those collections
Phase 2: massage collections so they are ready
for DML operations.
Phase 3: push the data to the database with
FORALL.
cfl_to_bulk_0.sql
cfl_to_bulk_5.sql
10g_indices_of.sql
10g_values_of.sql
47. Bulk Processing Conclusions
Most important performance tuning feature in PL/SQL.
Almost always the fastest way to execute multi-row
SQL operations in PL/SQL.
You trade off increased complexity of code for
dramatically faster execution.
But in Oracle Database 10g and above, the compiler
will automatically optimize cursor FOR loops to
BULK COLLECT efficiency.
No need to convert unless the loop contains DML or
you want to maximally optimize your code.
Watch out for the impact on PGA memory!
53. Table Functions
Table functions are functions that produce
a collection of rows
(either a nested table or a varray)
that can be queried like a physical database table.
You use a table function like the name of a
database table, in the FROM clause of a query.
55. Table Functions
Create a function in PL/SQL
Make sure it returns a collection
Query it using the TABLE() operator
Table functions can be pipelined
(return results as they are produced)
Table functions can be paralellized
56. Table Functions
You can use Table Functions when
Calculations cannot (easily) be done in SQL
You want to take advantage of PL/SQL e.g.
caching or package variables
You want to leverage the power of PL/SQL in
SQL
Make your views more dynamic
57. Resources
Online
tahiti.oracle.com
For all documentation online
www.allthingsoracle.com
o http://allthingsoracle.com/collections-in-oracle-pt-1/
o http://allthingsoracle.com/collections-in-oracle-part-2/
o http://allthingsoracle.com/bulk-processing-in-oracle-part-1/
o http://allthingsoracle.com/bulk-processing-in-oracle-part-2/
Books
Oracle PL/SQL Programming
Chapter 12 (collections) and
chapter 21 (bulk processing)
Oracle PL/SQL for DBAs
Chapter 1 (collections and bulk processing)