Cis435 week02

Solving Recurrences and the QuickSort Algorithm 1
Advanced Data Structures &
Algorithm Design
Solving Recurrences and the QuickSort
Algorithm

Solving Recurrences and the QuickSort Algorith2
The Search Problem
 A complement to the sorting problem
discussed last week is the searching
problem:
 Given a set of elements A = {a1, a2, …, an} and a
search parameter s, return the location i of s in A
 If A is an ordered set, then the binary search
algorithm can be used to search A for some value

The Binary Search Algorithm
 The binary search algorithm operates as
follows (where A is the input set, and mid is
the index of the element at the midpoint of A):
 If s = Amid, return mid
 If s < Amid, search the left half of A
 If s > Amid, search the right half of A

The Binary Search Algorithm
int BinarySearch(const DataType A[], int first, int
last, const DataType &s)
{
if ( last < first ) return –1; // not found
int mid = (first + last) / 2;
if ( s == A[mid] ) return mid;
if ( s < A[mid] ) // search left
return BinarySearch(A, first, mid-1, s);
else
return BinarySearch(A, mid+1, last, s);
}
int BinarySearch(const DataType A[], int first, int
last, const DataType &s)
{
if ( last < first ) return –1; // not found
int mid = (first + last) / 2;
if ( s == A[mid] ) return mid;
if ( s < A[mid] ) // search left
return BinarySearch(A, first, mid-1, s);
else
return BinarySearch(A, mid+1, last, s);
}
What is the order of growth of the running time of
BinarySearch?
What is the order of growth of the running time of
BinarySearch?

Analysis of Binary Search
 Each execution requires up to 4 constant
operations and 1 recursive function call that
operates on ½ the array, so:



>+
≤
=
1if)1()2/(
1if)1(
)(
nOnT
nO
nT
 We shall see tonight that this equates to
T(n)=O(log2n)

Solving Recurrences
 Recall our recurrence relation from last week:



>+
=
=
1if)()2/(2
1if)1(
)(
nnOnT
nO
nT
 How do we solve this recurrence to
determine the actual running time?

Solving Recurrences
 There are three methods that we can use:
 The Substitution Method
 We guess a bound, then use mathematical induction to
prove our guess is correct
 The Iteration Method
 Converts the recurrence into a summation, then relies
on techniques for bounding summations
 The Master Method
 Provides bounds for recurrences of the form
T(n)=aT(n/b)+f(n) utilizing a set of rules

The Substitution Method
 Involves guessing the form of the solution,
then using mathematical induction to find the
constants and show that the solution works
 Can only be applied in cases when it is easy
to guess the form of the answer

The Substitution Method: An Example
 We can guess that the solution is of the form
T(n)=O(nlog2n)
 By definition, this means that T(n) has some
upper bound of cnlog2n for some constant c > 0
 We’ll use induction to prove that this bound holds



>+
=
=
1if)2/(2
1if)1(
)(
nnnT
nO
nT

1aslongas-log
log
2loglog
)2/(log
))2/(log]2/[(2)(
:yieldsrecurrencetheintothisngSubstituti
)2/(log)2/()2/(
:isthat,2/forholdsboundthat thisassumingbystartWe
2
2
22
2
2
2
≥≤
+−=
+−=
+≤
+≤
≤
cncn
ncnncn
ncnncn
nncn
nnncnT
nncnT
n

 Now we must show that our solution holds for
the boundary conditions
 This means that we must be able to show that we
can choose the constant c to be large enough so
that the bound holds for all n, including the
boundary conditions
 For example, assume that our only boundary
condition is that T(1)=1
 With our solution, T(1) <= c1log21 = 0, no matter
what c is

 This problem can normally be easily
overcome
 Asymptotic notation only requires us to prove that
T(n)<=cnlog2n for some n >= n0
 We can choose n0 to remove the difficult boundary
condition from consideration
 E.g., choosing n0 to be 2 or 3 removes the difficult
boundary of 1 from consideration

The Iteration Method
 No guesses are required, but significant
algebra is
 The idea is to expand the recurrence and
express it as a summation of terms
dependent only on n

The Iteration Method: An Example
)64/(27)16/(9)4/(3
)))64/(3)16/((3)4/((3
))16/(3)4/((3
)4/(3)(
:followsasititeratecanWe
:)4/(3)(recurrenceheConsider t
nTnnn
nTnnn
nTnn
nTnnT
nnTnT
+++=
+++=
++=
+=
+=
How much iteration is enough?How much iteration is enough?

The Iteration Method: An Example
 The i-th term in the series is 3i
(n/4i
)
 This iteration hits n=1 when (n/4i
)=1 or when
i>log4n
 If we continue the iteration to this point, we
discover a decreasing geometric series:
)(
)(4
2.9)(identity)(
4
3
)1(3...64/2716/94/3)(
0
3log
log
4
4
nO
nOn
nn
nnnnnT
i
i
n
=
+=
Θ+





≤
Θ+++++≤
∑
∞
=

The Iteration Method
 There are two important points to focus on
when utilizing the Iteration Method:
 How many times must the recurrence be iterated
to reach the boundary condition?
 What is the summation of the terms?
 The book describes a technique called
“recurrence trees”, which are helpful in
finding a solution

The Master Method
 Provides a “cookbook” approach to solving
recurrences
 It is used specifically to solve recurrences of the
form T(n)=aT(n/b)+cnk
 Recurrences of this form describe solutions that divide
problems into subproblems of size n/b, where cnk
is the
cost of dividing the problem and combining the results

The Master Method
 To use the master method, you must
memorize three case:





<
=
>
=
k
k
k
k
k
a
ba
ba
ba
nO
nnO
nO
nT
b
if
if
if
)(
)log(
)(
)(
log
 Note that these are simplified versions of the
solutions found in the book

QuickSort
 QuickSort is a typical example of a divide-
and-conquer algorithm:
 Divide – partitions the input around an arbitrary
element
 Conquer – sorts each partition
 Combine – sorting is performed in-place, and so
requires zero work
 The algorithm consists of two functions, the
recursive QuickSort(), which utilizes
Partition() to divide the array

QuickSort
void QuickSort(ArrayType &A, int begin, int end)
{
if ( begin < end )
{
int q = Partition(A, begin, end);
QuickSort(A, begin, q-1);
QuickSort(A, q+1, end);
}
}
void QuickSort(ArrayType &A, int begin, int end)
{
if ( begin < end )
{
int q = Partition(A, begin, end);
QuickSort(A, begin, q-1);
QuickSort(A, q+1, end);
}
}

QuickSort: Partition
int Partition(ArrayType A[], int begin, int end)
{
ArrayType x = A[end];
int i = begin - 1;
for ( int j = begin ; j < end ; ++j )
{
if ( A[j] < x )
{
++i;
swap(A[i], A[j]);
}
}
swap(A[i+1], A[end]);
return i+1;
}
int Partition(ArrayType A[], int begin, int end)
{
ArrayType x = A[end];
int i = begin - 1;
for ( int j = begin ; j < end ; ++j )
{
if ( A[j] < x )
{
++i;
swap(A[i], A[j]);
}
}
swap(A[i+1], A[end]);
return i+1;
}

How Partition Works
9 7 4 5 2 1 6 3
i j
9 7 4 5 2 1 6 3
i j
9 7 4 5 2 1 6 3
i j
97 4 52 1 6 3
i j

How Partition Works
9 7 452 1 63
The final result of the first partitioning:
•The partition element is in it’s final sorted position
•QuickSort now operates on each subarray
independently

Analysis of QuickSort
 In the best case, Partition() divides the array evenly
at every level
 Each problem is divided into two equal subproblems of size
n/2
 Partition() itself is O(n) (it has a single loop that executes
some number of times based on the size of the input)
 The recurrence then is T(n)=2T(n/2)+n
 Using the Master Method:
 a=2, bk
=21
=2, a=bk
 this is Case 2
 T(n)=O(nk
log2n)=O(n1
log2n)=O(nlog2n)

 In the worst case, the size of one partition is
1 and the other is n-1
 Observing that T(1)=O(1), then the worst-case
recurrence is:
 T(n)=T(n-1)+O(n)+O(1)
 T(n)=T(n-1)+O(n)

 When will this worst-
case occur?
 How likely will it
occur?
 Why does it occur?
 How can we
improve it?
)(
)(
)()1()(
:iterationusecanweHere
2
1
1
n
k
k
nnTnT
n
k
n
k
Θ=






Θ=
Θ=
Θ+−=
∑
∑
=
=

Homework
 Reading: 4.1-4.3; 8.1-8.2; 8.4
 Exercises:
 4.3-1, 2, 3

Cis435 week02

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

En vedette

En vedette (9)

Similaire à Cis435 week02

Similaire à Cis435 week02 (20)

Plus de ashish bansal

Plus de ashish bansal (13)

Dernier

Dernier (20)

Cis435 week02