DAA NOTES

1. III.PERFORMANCE ANALYSIS
Performance analysis of an algorithm
depends upon two factors
i amount of memory used
ii amount of compute time consumed on
any CPU.
Formally they are notified as complexities
in terms of:
1.Space Complexity.
2.Time Complexity.

2. Space Complexity of an algorithm is the amount of memory it
needs to run to completion i.e. from start of execution to its
termination. Space need by any algorithm is the sum of
following components:
Fixed Component: This is independent of the characteristics
of the inputs and outputs.
This part includes: Instruction Space, Space of simple
variables, fixed size component variables, and constants
variables.
Variable Component: This consist of the space needed by
component variables whose size is dependent on the particular
problems instances(Inputs/Outputs) being solved, the space
needed by referenced variables and the recursion stack space
is one of the most prominent components. Also this included
the data structure components like Linked list, heap, trees,
graphs etc.
Therefore the total space requirement of any algorithm 'A' can
be provided as

3. Space(A) = Fixed Components(A) + Variable Components(A)
Among both fixed and variable component the variable part is
important to be determined accurately, so that the actual space
requirement can be identified for an algorithm 'A'. To identify
the space complexity of any algorithm following steps can be
followed:
Determine the variables which are instantiated by some default
values.
Determine which instance characteristics should be used to
measure the space requirement and this is will be problem
specific.
Generally the choices are limited to quantities related to the
number and magnitudes of the inputs to and outputs from the
algorithms.
Sometimes more complex measures of the interrelationships
among the data items can used.

4. Example: Space Complexity
Algorithm Sum(number , size) procedure will
produce sum of all numbers provided in 'number' list
{
result=0.0;
for count = 1 to size do will repeat from 1,2,3,4,....size
times
result= result + number[count];
return result;
}
In above example, when calculating the space
complexity we will be looking for both fixed and variable
components. here we have
Fixed components as 'result','count' and 'size' variable
there for total space required is three(3) words.

5. Variable components is characterized as the
value stored in 'size' variable (suppose value
store in variable 'size 'is 'n'). because this will
decide the size of 'number' list and will also
drive the for loop. therefore if the space used
by size is one word then the total space
required by 'number' variable will be 'n'(value
stored in variable 'size').
therefore the space complexity can be written as
Space(Sum) = 3 + n;

6. Time Complexity of an algorithm(basically when
converted to program) is the amount of computer
time it needs to run to completion.
The time taken by a program is the sum of
the compile time and the run/execution time
.The compile time is independent of the
instance(problem specific) characteristics.
following factors effect the time complexity:
Characteristics of compiler used to compile the
program.
Computer Machine on which the program is
executed and physically clocked.
Multiuser execution system.
Number of program steps.

7. Therefore the again the time complexity consist of two
components fixed(factor 1 only) and variable/instance(factor
2,3 & 4), so for any algorithm 'A' it is provided as:
Time(A) = Fixed Time(A) + Instance Time(A)
Here the number of steps is the most prominent instance
characteristics and The number of steps any program statement
is assigned depends on the kind of statement like
comments count as zero steps,
an assignment statement which does not involve any calls to other
algorithm is counted as one step,
for iterative statements we consider the steps count only for the
control part of the statement etc.
Therefore to calculate total number program of program steps we
use following procedure. For this we build a table in which we list
the total number of steps contributed by each statement. This is
often arrived at by first determining the number of steps per
execution of the statement and the frequency of each statement
executed. This procedure is explained using an example.

8. Example: Time Complexity
Statement
Steps
per
execution
Frequenc
y
Total
Steps
Algorithm Sum(number,size)
0 -
0
{ 0 - 0
result=0.0;
1 1 1
for count = 1 to size do
1
size+
1
size + 1
result= result + number[count];
1 size size
return result;
1 1 1
} 0 - 0
Total
2size +
3

9. In above example if you analyze carefully frequency of "for
count = 1 to size do" it is 'size +1' this is because the
statement will be executed one time more die to condition
check for false situation of condition provided in for
statement. Now once the total steps are calculated they
will resemble the instance characteristics in time
complexity of algorithm. Also the repeated compile time of
an algorithm will also be constant every time we compile
the same set of instructions so we can consider this time
as constant 'C'. Therefore the time complexity can be
expressed as: Time(Sum) = C + (2size +3)
So in this way both the Space complexity and Time
complexity can be calculated. Combination of both
complexity comprises the Performance analysis of any
algorithm and can not be used independently. Both these
complexities also helps in defining parameters on basis of
which we optimize algorithms.

10. Processing Speed
In the computer world, frequency is often
used to measure processing speed. For
example, clock speed, measures how many
cycles a processor can complete in one
second. If a computer has a 3.2GHz
processor, it can can complete
3,200,000,000 cycles per second. FLOPS,
which is used to measure floating
point performance, is also a frequency-based
calculation (operations per second). Finally,
computing speed may also be defined
in MIPS, which measures instructions per
second.

11. Complexity of Algorithms
The complexity of an algorithm M is the function f(n)
which gives the running time and/or storage space
requirement of the algorithm in terms of the size ‘n’ of
the input data. Mostly, the storage space required by
an algorithm is simply a multiple of the data size ‘n’.
Complexity shall refer to the running time of the
algorithm.
The function f(n), gives the running time of an
algorithm, depends not only on the size ‘n’ of the input
data but also on the particular data. The complexity
function f(n) for certain cases are:
•Best Case: The minimum possible value of f(n) is
called the best case.
•Average Case : The expected value of f(n).
Worst Case: The maximum value of f(n) for any key
possible input.

12. Asymptotic Notations
The following notations are commonly use notations in
performance analysis and used to characterize the
complexity of an algorithm:
•Big–OH (O) ,
•Big–OMEGA (Ω),
•Big–THETA ( ) and
•Little–OH (o)

13. Big Oh Notation, Ο
The notation Ο(n) is the formal way to express the upper bound of an algorithm's
running time. It measures the worst case time complexity or the longest amount of
time an algorithm can possibly take to complete.
For example, for a function f(n)
Ο(f(n)) = { g(n) : there exists c > 0 and n0 such that f(n) ≤ c.g(n) for all n > n0. }

14. Omega Notation, Ω
The notation Ω(n) is the formal way to express the lower bound
of an algorithm's running time. It measures the best case time
complexity or the best amount of time an algorithm can
possibly take to complete.
For example, for a function f(n)
Ω(f(n)) ≥ { g(n) : there exists c > 0 and n0 such that g(n) ≤ c.f(n)
for all n > n0. }

15. Theta Notation, θ
The notation θ(n) is the formal way to express both the
lower bound and the upper bound of an algorithm's
running time. It is represented as follows −θ(f(n)) = {
g(n) if and only if g(n) = Ο(f(n)) and g(n) = Ω(f(n)) for all
n > n0. }

16. Amortized Analysis is used for algorithms where an
occasional operation is very slow, but most of the other
operations are faster. In Amortized Analysis, we
analyze a sequence of operations and guarantee a
worst case average time which is lower than the worst
case time of a particular expensive operation.
Aggregate Method
The aggregate method is used to find the total cost. If we want to
add a bunch of data, then we need to find the amortized cost by
this formula.
For a sequence of n operations, the cost is −
Let us consider an example of a simple hash table insertions. How
do we decide table size? There is a trade-off between space and
time, if we make hash-table size big, search time becomes fast,
but space required becomes high.

17. The solution to this trade-off problem is to use Dynamic Table (or Arrays).
The idea is to increase size of table whenever it becomes full. Following are
the steps to follow when table becomes full.
1) Allocate memory for a larger table of size, typically twice the old table.
2) Copy the contents of old table to new table.
3) Free the old table.
If the table has space available, we simply insert new item in available space.

18. What is the time complexity of n insertions using the above scheme?
If we use simple analysis, the worst case cost of an insertion is O(n). Therefore,
worst case cost of n inserts is n * O(n) which is O(n2). This analysis gives an
upper bound, but not a tight upper bound for n insertions as all insertions
don’t take Θ(n) time.
So using Amortized Analysis, we could prove
that the Dynamic Table scheme has O(1)
insertion time which is a great result used in
hashing.

19. Randomized Algorithms
An algorithm that uses random numbers to decide what to do next
anywhere in its logic is called Randomized Algorithm.
Classification
Randomized algorithms are classified in two categories.
Las Vegas: These algorithms always produce correct or optimum
result. Time complexity of these algorithms is based on a random
value and time complexity is evaluated as expected value.
Monte Carlo: Produce correct or optimum result with some
probability. These algorithms have deterministic running time and
it is generally easier to find out worst case time complexity.
Applications
Graph algorithms: Minimum spanning trees, shortest
paths, minimum cuts.
Counting and enumeration: Matrix permanent Counting
combinatorial structures.
Parallel and distributed computing: Deadlock avoidance
distributed consensus.
Probabilistic existence proofs: Show that a combinatorial object
arises with non-zero probability among objects drawn from a
suitable probability space.