1. Partition Defragmenter
Yoram Benchetrit
yoram.benchetrit@fr.ibm.com
Mercredi 3 Octobre 2012
User Group Informix France

2. Agenda
A bit of partition background
The problem with fragmented data
How does the feature work
Functionality & considerations
Defragmenter in OAT
Example
User Group Informix France

3. A bit of partition background
What is an extent ?
– Collection of physically contiguous pages on disk
– Extent can hold data page, index page, bitmap
page etc…
What is a partition ?
– Logical collection of all the extents allocated for a
given table or index.
– Space for partition is allocated in units of extents
and is not necessarily contiguous.
Each partition are unique within the system
– oncheck –pt <partnum> User Group Informix France

4. A bit of partition background
Partition logical to physical extent relationship
User Group Informix France

5. A bit of partition background
Before Defragmenting
Chunk 14
Extents for table A
Extents for
other tables
Free pages
User Group Informix France

6. A bit of partition background
After Defragmenting
Chunk 14
Extents for table A
User Group Informix France

7. A bit of partition background
Previous Solutions
– Table unload and reload
Oncheck –me
– Off-line table reorganization
– Non-recoverable
– Unsupported
User Group Informix France
7

8. A bit of partition background
How many extents exist ?
– Run oncheck –pe.
– Run oncheck –pt.
– SQL
database sysmaster;
select unique te_partnum, count(*)
from systabextents
group by 1
having count(*) > NN
order by 1 desc;
• Where NN is a value.
User Group Informix France

9. The problem with fragmented data
What’s the problem with many extents?
– Full partition page.
• Prohibits new inserts.
– Might not be at maximum number of rows.
• Need to either delete older entries, if possible.
– or
• Requires a manual table recreation.
– (FYI: Actually, this can no longer exist in 11.70)
User Group Informix France

10. The problem with fragmented data
Many extents degrades performance.
– Need to skip over non-contiguous disk areas.
– Slower retrieval of data.
– Mapping logical page numbers to physical
addresses is slower.
– Chunk allocations are slower with long free
list pages
– Archive extent-sorting phase is slower
User Group Informix France

11. The problem with fragmented data
What’s the problem with many extents?
– Many tables with many extents feels
overbearing.
• Outputs from oncheck –pe & -pt.
User Group Informix France

12. How does the feature work
Run new SQL API functions.
– sysadmin database.
– execute function [admin|task] ("<defragment",
"<database>:[<owner>.]<table>[#index]")
• or
– execute function [admin|task] ("<defragment
partnum", "<partnum> [, <partnum>, ... ]”);
User Group Informix France

13. How does the feature work
– What goes on behind the scenes.
• Check validity.
• Create new extent.
• Might need just one adding
• Start a loop and process all old extents.
– Read old extent.
– Move all old extent pages to the new extent.
– Delete old extent.
• Loop end.
User Group Informix France

14. How does the feature work
Log the work using five new log types.
– Start defragmentation.
– End defragmentation.
– Add new defragmentation extent.
– Remove old defragmentation extent.
– Move page.
Will result in a surge in logical log usage.
User Group Informix France

15. How does the feature work
Monitoring progress.
– onstat -g defragment.
• Searching for extents.
• Merging extents.
• Defrag_completed.
• Defrag_failed.
User Group Informix France

16. How does the feature work
Recoverable if server shuts down or
crash
– Defragmenter cleaner thread will remove extra
extents after fast recovery
• Rolls back current unit of work (UOW) if needed.
– 08:52:06 Defragmenter cleaner thread now running
– 08:52:06 Defragmenter cleaner thread cleaned:1 partitions
– Will the object be consistent ?
• Yes.
– Choose to run again to complete the job.
User Group Informix France

17. Functionality
Can defragment table online
No locks placed on object
Only allowed one defragmenter running per
dbspace
Replication is supported (HDR, MACH11, ER)
Run via the SQL API interface.
User Group Informix France

18. Considerations
Functionality not allowed
– Functional indexes.
– VTI & VII objects.
– Temporary tables.
– Sort tables.
– Optical blobs.
– System catalog tables
User Group Informix France

19. Considerations
Cannot stop once job submitted
Cannot run simultaneously when table being
– Altered
– Truncated
– Drop
– Compress, Repack, Shrink
Not the same as compress/repack/shrink
Enterprise Replication.
– Table needs to be in CDR alter mode.
• Mode unset automatically at end of job
User Group Informix France

20. Considerations
Operation is logging-intensive
– Each page move is logically logged (DFMVPG)
• 70-80 bytes per page moved
Small units of work to avoid long transaction.
– It may send other transactions over the threshold
New extents created at end of dbspace.
Schedule during low activity.
Perform after a backup.
User Group Informix France

21. Defragmenter in OAT
On the OAT menu, expand
Space Administration > Space Optimization
User Group Informix France
21

22. Defragmenter in OAT
On the Actions menu,
click Optimize Space.
Select the
tables.
User Group Informix France
22

23. Defragmenter in OAT
Select Defragment extents and click OK.
User Group Informix France
23

24. Defragmenter in OAT
Automatic Storage Optimization
– Can automate defragmenting using OAT.
– Enabled via scheduler task “auto_crsd”
• This can also do compressing, repacking, shrinking.
• ‘crsd’ stand for compress, repack, shrink, defragment
– This can also be enabled via SQL
User Group Informix France

25. Defragmenter in OAT
‘auto_crsd’ task details
– In OAT -> Task Scheduler -> Task Details
• UPDATE ph_task SET tk_enable = 'T‘ WHERE tk_name = 'auto_crsd';
User Group Informix France

26. Defragmenter in OAT
‘auto_crsd’ task parameters.
– UPDATE ph_threshold SET value = 'F WHERE name = 'AUTODEFRAG_ENABLED';
– UPDATE ph_threshold SET value = ‘100‘ WHERE name = 'AUTODEFRAG_EXTENTS';
User Group Informix France

27. Example
SQL to create three fragmented tables
– create table tab1 (a serial, b int, c int);
– create table tab2 (a serial, b int, c int);
– create table tab3 (a serial, b int, c int);
– create procedure ins (a int, b int) define c int;
• for c = a to b
• insert into tab1 values (0, 88, 99);
• insert into tab2 values (0, 88, 99);
• insert into tab3 values (0, 88, 99);
• end for;
– end procedure;
– execute procedure ins (1, 1000000);
User Group Informix France

28. Example
oncheck –pe before defragment.
User Group Informix France

29. Example
oncheck –pt before defragment.
– FYI : in 11.70 No longer limited to one partn page.
User Group Informix France

30. Example
SQL API function.
– execute function task ("defragment", "oltr:tab1");
(expression) OK
User Group Informix France

31. Example
oncheck –pe after defragment.
User Group Informix France

32. Example
oncheck –pt after defragment.
User Group Informix France

33. Example
onlog output
User Group Informix France

34. Summary
Informix Defragmenter Capabilities
– Partition Extents Defragmentation
– New System Admin SQL API
– OAT GUI Interface
– Automatic Defragmentation
– Can be completed with compress/repack/shrink
for optimal storage optimization
User Group Informix France

35. Questions ?
User Group Informix France

36. Merci
Yoram Benchetrit
yoram.benchetrit@fr.ibm.com
Mercredi 3 Octobre 2012
User Group Informix France