CS: Introduction to Record Manipulation & Indexing

Record Manipulation &
Indexing
•records/fields
•index placement; index management
•manipulating fixed-length record files
•re-using space in fixed-length files
•varying length records:[VLR] adds; dels; mods;
•free lists for VLR - placement strategies (first, best, worst)
•varying length record maintenance

© Katrin Becker
All Rights Reserved

Records and Indexing

14-Sep-03 1

Records in General
A record is:
• An identifiable, describable data set
• Often contains a sub-structure
• Typically part of a larger structure
This definition also works for: files; fields;
…
© Katrin Becker
All Rights Reserved


14-Sep-03 2

Records and Fields
FILE SYSTEM containing files

FILE containing records

RECORD

FIELD containing elements

containing
fields

© Katrin Becker
All Rights Reserved


14-Sep-03 3

Record Manipulation
• Operations on Records:
–
–
–
–

© Katrin Becker
All Rights Reserved

Searches
Additions
Deletions
Modifications


14-Sep-03 4

Record Manipulation - Search
Sequential Search
• While NOT done:
– Position file pointer
– Read record
– Examine record to see if it’s the one
• Yes DONE
• No CONTINUE
© Katrin Becker
All Rights Reserved


14-Sep-03 5

Other Searches
• What changes?
– Binary search:
• We position the file pointer in a different
fashion (the rest is the same)

– Search with an index
• We apply the search to the index and retrieve
the record only when located in the index

© Katrin Becker
All Rights Reserved


14-Sep-03 6

Record Manipulation –
Addition
New record gets added to the end.

• Insertion into middle of file is impractical.
• If there is an index, then we also perform
an addition to the index (addition to the
end of this list is infeasible – WHY? ).
© Katrin Becker
All Rights Reserved


14-Sep-03 7

Addition with an Index - 1
INDEX

1. New record gets added to the end.

RECORDS
© Katrin Becker
All Rights Reserved


14-Sep-03 8

INDEX

2. Locate place where index entry needs to go

RECORDS
© Katrin Becker
All Rights Reserved


14-Sep-03 9

INDEX

3. Insert New Index entry (it’s a record too)

RECORDS
© Katrin Becker
All Rights Reserved


14-Sep-03 10

Records vs. Index:
Assertions & Questions
• Moving file records is more expensive
than moving index records.
• Should index be IN record file or its
own file? (How do we maintain it? )
• If IN file: should it be at the beginning,
end, middle, distributed?
• What if we are able to hold the index in
memory?
• What if we can’t?
© Katrin Becker
All Rights Reserved


14-Sep-03 11

Record Manipulation - Deletion
• Locate record (Search)
• Mark space as deleted
• Remove index entry? (why or why not)

© Katrin Becker
All Rights Reserved


14-Sep-03 12

Deletion with an index - 1
INDEX

1.

Locate index entry

RECORDS
© Katrin Becker
All Rights Reserved


14-Sep-03 13

INDEX

1.

Locate index entry

2. Locate record

RECORDS
© Katrin Becker
All Rights Reserved


14-Sep-03 14

INDEX

3. Delete (mark) record

RECORDS
© Katrin Becker
All Rights Reserved


14-Sep-03 15

INDEX

4. Delete (mark?) index entry

RECORDS
© Katrin Becker
All Rights Reserved


14-Sep-03 16

Record Manipulation - Modification
•
•
•
•

Locate record
Read record
Modify record
Re-write record (assuming fixed-size
records – what if the record is now a
different size? [see later])

© Katrin Becker
All Rights Reserved


14-Sep-03 17

File Behaviour – 1 start

Record count = 9

© Katrin Becker
All Rights Reserved


14-Sep-03 18

File Behaviour – 2 add record

Record count = 10

© Katrin Becker
All Rights Reserved


14-Sep-03 19

File Behaviour – 3 add record

Record count = 11

© Katrin Becker
All Rights Reserved


14-Sep-03 20

File Behaviour – 4 delete

Record count = 10

© Katrin Becker
All Rights Reserved


14-Sep-03 21


Record count = 9

© Katrin Becker
All Rights Reserved


14-Sep-03 22

File Behaviour – 6 add

Record count = 10

© Katrin Becker
All Rights Reserved


14-Sep-03 23


Record count = 11

© Katrin Becker
All Rights Reserved


14-Sep-03 24


Record count = 12

© Katrin Becker
All Rights Reserved


14-Sep-03 25


Record count = 11

© Katrin Becker
All Rights Reserved


14-Sep-03 26


Record count = 10

And so on…….
© Katrin Becker
All Rights Reserved


14-Sep-03 27

What’s happening to the file?
• File grows – does not shrink (we get
fragmentation)
• We end up covering more ground to do the
same job
•

Q: If we are doing random access, why does it matter?

• The file system has less space to use (the
fragmentation is internal from the
perspective of the file system).
• Worst case = EVERY record access ends
up costing us a seek.
© Katrin Becker
All Rights Reserved


14-Sep-03 28

Re-Using Space in the File [FLR]
• When there is a deletion, locate the
last record in the file, end move to the
free slot
– Costs:
• Additional file access to locate (where will we
remember where the last records is?) and
retrieve last record.
• Records will loose locality faster than if we
simply mark the slot. (Why do we care?)

© Katrin Becker
All Rights Reserved


14-Sep-03 30

Re-Using Space – Way 2
• Make a list of places where records
have been deleted.
• When doing addition, check for empty
‘slot’ before placing new record at end.
Q: What about the index?

• When doing deletion, add location of
deleted record to ‘free-list’

© Katrin Becker
All Rights Reserved


14-Sep-03 31

What does the Free-List look like?
INDEX

All we need is
the location.
Order is
unimportant.

RECORDS
© Katrin Becker
All Rights Reserved


14-Sep-03 32

How to decide which ‘slot’
to re-use?
• In FLR every slot will fit a new record.
• We can just take the first one – FreeList can then be maintained as a stack
(which is easy).
• Do we keep Free-List information in
the file?

© Katrin Becker
All Rights Reserved


14-Sep-03 33

Indexing – What is it?
• Table-of-contents for a file (directory)
• Uses keys
• Byte Offset (BO) vs Relative Record
Number (RRN)

© Katrin Becker
All Rights Reserved


14-Sep-03 34

Primary Key Properties:
•
•
•
•

Unique
Canonical
Data-less
Unchanging

© Katrin Becker
All Rights Reserved


14-Sep-03 35

Indexing – How does it Look?
• Must have:
INDEX

– Key
– Way to locate record

• It is itself a structure containing
‘records’ (each index entry is a
record)
• It may be separate from the main
data or in the same file.
• It may be copied into memory for
manipulation and only updated
infrequently; or the file copy may be
maintained as well.
© Katrin Becker
All Rights Reserved


14-Sep-03 36

Indexing – File Ops?
• Tied to records:
– If records added – new/update index entry
– If record deleted – ‘delete’ index entry
– If record modified – maybe no change to
index; maybe update BO [byte offset]

© Katrin Becker
All Rights Reserved


14-Sep-03 37

Fixed-length vs Varying Length
•
•
•
•
•

VLR provides greater flexibility.
VLR increases maintenance overhead.
VLR decreases wasted space. *
VLR makes index virtually essential.
VLR complicates Free-List
maintenance.

*may simply waste space in a different place or a different way.
© Katrin Becker
All Rights Reserved


14-Sep-03 38

VLR Index
INDEX

• Requires:

– Key
– Byte offset
– Record size? [optional]

RECORDS
© Katrin Becker
All Rights Reserved


14-Sep-03 39

VLR Search Operation
INDEX

•
•

Same as for FLR:

1. Locate key in index
2. Locate record in file

Binary search still possible
on index, but NOT on
records alone.

RECORDS
© Katrin Becker
All Rights Reserved


14-Sep-03 40

VLR Deletion Operation - 1
INDEX

Locate key

RECORDS
© Katrin Becker
All Rights Reserved


14-Sep-03 41

INDEX

Locate record

RECORDS
© Katrin Becker
All Rights Reserved


14-Sep-03 42

INDEX

Delete record

RECORDS
© Katrin Becker
All Rights Reserved


14-Sep-03 43

VLR Addition Operation – 2c
INDEX

We will end up with left-over
unused (and probably
unusable space).
We call this “First-Fit”
(because we are using
the first slot that we find
that fits).

Free-List

New
Record

RECORDS
© Katrin Becker
All Rights Reserved


14-Sep-03 53

VLR Addition Operation – 2g
INDEX

Free-List

1. Insert record.
3. Update Index
Notice the index entry is
sorted differently.
What’s the advantage to
leaving ‘spaces’ in the
index?

2. Delete
FreeList
entry.

New
Record

RECORDS
© Katrin Becker
All Rights Reserved


14-Sep-03 57

Free-Lists
• May want to keep Free-List sorted.
• If the List is short it may not matter.
• Placement Strategies:
– First Fit
– Best Fit
– Worst Fit

• It could be its own list or we could make the
regular index serve double-duty.

© Katrin Becker
All Rights Reserved


14-Sep-03 61

Summary
• Managing space inside the file is our
business.
• We must choose:
– FLR / VLR?
– Index? (what kind?)
– Secondary indices?
– Re-claim free space? How?

© Katrin Becker
All Rights Reserved


14-Sep-03 62

CS: Introduction to Record Manipulation & Indexing

Recommandé

Recommandé

Contenu connexe

Similaire à CS: Introduction to Record Manipulation & Indexing

Similaire à CS: Introduction to Record Manipulation & Indexing (11)

Plus de Katrin Becker

Plus de Katrin Becker (20)

Dernier

Dernier (20)

CS: Introduction to Record Manipulation & Indexing