Data Structure and Algorithm chapter two, This material is for Data Structure and Algorithm course.

Chapter Two:
Algorithmanalysis
1/10/2024 Data Structures and Algorithms -1-

Algorithm analysis
Algorithm analysis refers to the process of determining how much
computing time and storage that algorithms will require.
In other words, it’s a process of predicting the resource
requirement of algorithms in a given environment.
In order to solve a problem, there are many possible algorithms. One
has to be able to choose the best algorithm for the problem at hand
using some scientific method.
To classify some data structures and algorithms as good, we need
precise ways of analyzing them in terms of resource requirement.
1/10/2024 Data Structures and Algorithms 2

The main resources:
• Running Time
• Memory Usage
• Communication Bandwidth
Note: Running time is the most important since computational time is the
most precious resource in most problem domains.
There are two approaches to measure the efficiency of algorithms:
1.Informal Approach
2.Formal Approach

Informal Approach
Empirical vs Theoretical Analysis
Empirical Analysis
• it works based on the total running time of the program. It uses actual
system clock time.
Example:
t1(Initial time before the program starts)
for(int i=0; i<=10; i++)
cout<<i;
t2 (final time after the execution of the program is finished)
Running time taken by the above algorithm
(Total Time) = t2-t1;
1/10/2024
Data Structures and Algorithms
4

Cont…
It is difficult to determine efficiency of algorithms using this approach,
because clock-time can vary based on many factors.
For example:
a) Processor speed of the computer
b) Current processor load
c) Specific data for a particular run of the program
Input size
Input properties
d)Operating System
Multitasking Vs Single tasking
Internal structure

Theoretical Analysis
• Determining the quantity of resources required using mathematical concept
by analyzing an algorithm according to the number of basic operations
(time units) required, rather than according to an absolute amount of time
involved.
Why we use operations in theoretical approach to determine the efficiency
of algorithm because:
The number of operation will not vary under different conditions.
It helps us to have a meaningful measure that permits comparison of
algorithms independent of operating platform.
It helps to determine the complexity of algorithm.

Complexity Analysis
Complexity Analysis is the systematic study of the cost of computation,
measured either in:
Time units
Operations performed
The amount of storage space required.
Two important ways to characterize the effectiveness of an algorithm are
its Space Complexity and Time Complexity.
Time Complexity: Determine the approximate amount of time (number of
operations) required to solve a problem of size n. The limiting behavior of
time complexity as size increases is called the Asymptotic Time
Complexity.
1/10/2024
7

Cont…
Space Complexity: Determine the approximate memory required to
solve a problem of size n.
The limiting behavior of space complexity as size increases is called
the Asymptotic Space Complexity.
Asymptotic Complexity of an algorithm determines the size of
problems that can be solved by the algorithm.
1/10/2024
8

Factors affecting the running time of a program:
CPU type
Memory used
Computer used
• Programming Language C (fastest), C++ (faster), Java (fast) C is
relatively faster than Java, because C is relatively nearer to Machine
language, so, Java takes relatively larger amount of time for
interpreting/translation to machine code.
Algorithm used
Input size
• Note: Important factors for this course are Input size and Algorithm
used.

Analysis Rule
Assignment operation Example:- i=1 (1 time unit)
Single arithmetic operation Example:- x+y (1 time unit)
Input/output operation Example:- cin>>a; or cout<<a; (1 time
unit)
Single Boolean operation Example:- i<=5; (1 time unit)
Function return Example:- return x; (1 time unit)
Function call Example:- add(); (1 time unit)
1/10/2024
10

Example 1
Q1. Write an algorithm and analyze the time complexity for the
problem adding two numbers.
Step 1. accept first number (1 TU)
Step2. accept the second number (1 TU)
Step3. add two number (1 TU)
Step4. print the result (1 TU)
T(n)=4 TU
Steps of algorithms are different based on programmer points of view.
1/10/2024
11

Looping statements
Running time for a loop is equal to the running time for the statements inside the
loop times number of iterations.
Example 1:
int n=5;
For(int i =1; i<=n; i++)
{
cout<<i;
return 0;
}
T(n)=1+(1+n+1+n+n+1)=3n+4
12
Wednesday, January 10, 2024

Examples: 2
{
int n;
int k=0;
cout<<“Enter an integer”;
cin>>n;
for(int i=0;i<n; i++)
}
T(n)=2n+5
13

Examples: 2
{
int i=0;
while(i<n)
{
cout<<i;
i++;
}
int j=1;
while(j<=10)
{
cout<<j; j++;
}
T(n)=3n+34
14

Examples: 3
{
Nested loop
{
int k=0;
for(int i=1 ; i<=n; i++)
for( int j=1; j<=n; j++)
k++;
}
T(n)=3 n2 +4n+3
15

Examples: 3
{
int sum=0;
for(i=1;i<n; i++)
sum=sum+i;
}
T(n)=1+1+n+n-1+n-1(1+1)=3n-1
16

Examples: 4
{
{
int sum=0;
for(i=0;i<n;i++)
for(j=0;j<n;j++)
sum++;
}
T(n)=1+(1+n+1+n)+n(1+n+1+n+n)=3n2+4n+3
17

Formal Approach to Analysis
• In the above examples we have seen that analyzing Loop statements is so complex.
• It can be simplified by using some formal approach in which case we can ignore
initializations, loop controls, and updates.
1. Simple Loops: Formally, for loop can be translated to a summation. The index and
bounds of the summation are the same as the index and bounds of the for loop.
• Suppose we count the number of additions that are done. There is 1
addition per iteration of the loop, hence n additions in total.
for (int i = 1; i <= N; i++) {
sum = sum+i;
}
N
N
i


1
1

2. Nested Loops:
Nested for loops translate into multiple summations, one for each for loop.
Again, count the number of additions. The outer summation is for the outer
for loop.
1/10/2024
19

3. Consecutive Statements:
Add the running times of the separate blocks of your code.
1/10/2024
20

4. Conditionals:
If (test) s1 else s2: Compute the maximum of the running time for s1
and s2.
1/10/2024
21

Categories of Algorithm Analysis
• Algorithms may be examined under different situations to correctly
determine their efficiency for accurate comparison.
Best Case Analysis
Worst Case Analysis
Average Case Analysis
1/10/2024
22

1.Best Case Analysis
Best case analysis assumes the input data are arranged in the
most advantageous order for the algorithm.
It also takes the smallest possible set of inputs and causes
execution of the fewest number of statements.
Moreover, it computes the lower bound of T(n), where T(n) is
the complexity function.
Examples: For sorting algorithm
If the list is already sorted (data are arranged in the required
order).
For searching algorithm If the desired item is located at first
accessed position.
1/10/2024 COSC 709: Natural Language Processing 23

2. Worst Case Analysis
It assumes the input data are arranged in the most disadvantageous order for
the algorithm.
Takes the worst possible set of inputs. Causes execution of the largest number
of statements. Computes the upper bound of T(n) where T(n) is the
complexity function.
Example:
While sorting, if the list is in opposite order.
While searching, if the desired item is located at the last position or is
missing.
Worst case analysis is the most common analysis because, it provides the
upper bound for all input (even for bad ones).
1/10/2024
24

3. Average Case Analysis
It determine the average of the running time overall permutation of input
data.
Takes an average set of inputs.
It also assumes random input size.
It causes average number of executions.
Computes the optimal bound of T(n) where T(n) is the complexity
function.
Sometimes average cases are as bad as worst cases and as good as best
cases.
1/10/2024
25

Order of Magnitude
Order of Magnitude refers to the rate at which the storage or time grows as
a function of problem size.
It is expressed in terms of its relationship to some known functions.
This type of analysis is called Asymptotic analysis.
Asymptotic analysis
• Asymptotic Analysis is concerned with how the running time of an algorithm
increases with the size of the input in the limit, as the size of the input
increases without bound!

Types of notations
There are five notations used to describe a running time
function. These are:
 Big-Oh Notation (O)
 Big-Omega Notation ()
 Theta Notation ()
 Little-o Notation (o)
 Little-Omega Notation ()
Note: The complexity of an algorithm is a numerical function
of the size of the problem (instance or input size).
1/10/2024 COSC 709: Natural Language Processing 27

1. Big-Oh Notation
 Definition: We say f(n)=O(g(n)), if there are positive constants no and c, such that to the right of no, the value
of f(n) always lies on or below c.g(n).
 As n increases f(n) grows no faster than g(n). It’s only concerned with what happens for very large values of n.
It describes the worst case analysis. Gives an upper bound for a function to within a constant factor.
O-Notations are used to represent the amount of time an algorithm takes on the worst possible set of
inputs, “Worst-Case”.

2. Big-Omega (Ω)-Notation (Lower bound)
Definition: We write f(n)= Ω(g(n)) if there are positive constants no and c such that to the right of
no(k) the value of f(n) always lies on or above c.g(n).
As n increases f(n) grows no slower than g(n). It describes the best case analysis. Used to
represent the amount of time the algorithm takes on the smallest possible set of inputs-“Best
case”.
Big-Omega (Ω)-Notation (Lower bound)
1/10/2024
29

3. Theta Notation (θ-Notation) (Optimal bound)
Definition: We say f(n)= θ(g(n)) if there exist positive constants no, c1 and c2 such that to the right of
no, the value of f(n) always lies between c1.g(n) and c2.g(n) inclusive, i.e., c1.g(n)<=f(n)<=c2.g(n), for all
n>=no.
As n increases f(n) grows as fast as g(n). It describes the average case analysis. To represent the
amount of time the algorithm takes on an average set of inputs- “Average case”.
1/10/2024
30

4. Little-oh (small-oh) Notation
Definition: We say f(n)=o(g(n)), if there are positive constants no and
c such that to the right of no, the value of f(n) lies below c.g(n).
As n increases, g(n) grows strictly faster than f(n). It describes the
worst case analysis.
Denotes an upper bound that is not asymptotically tight.
Big O-Notation denotes an upper bound that may or may not be
asymptotically tight.
1/10/2024
31

5. Little-Omega (ω) notation
Definition: We write f(n)=ω(g(n)), if there are positive constants no
and c such that to the right of no, the value of f(n) always lies above
c.g(n).
As n increases f(n) grows strictly faster than g(n).
It describes the best case analysis and denotes a lower bound that is
not asymptotically tight.
Big Ω-Notation denotes a lower bound that may or may not be
asymptotically tight.

Arrangement of common functions by growth rate. List of typical growth rates
1/10/2024
33

Chapter two lesson 2
Sorting and Searching
1/10/2024
34

Simple Sorting and Searching Algorithm
Why do we study sorting and searching algorithms?
These algorithms are the most common and useful tasks operated by
computer system. Computers spend a lot of time searching and sorting.
1. Simple Searching algorithms
Searching:- is a process of finding an element in a list of items or
determining that the item is not in the list.
To keep things simple, we shall deal with a list of numbers. A search
method looks for a key, arrives by parameter.
By convention, the method will return the index of the element
corresponding to the key or, if unsuccessful, the value -1.
1/10/2024
35

1. Simple Searching algorithms
There are two simple searching algorithms:
Sequential Search
Binary Search
Sequential Searching (Linear)
The most natural way of searching an item. Easy to understand and
implement.
Algorithm:
In a linear search, we start with top (beginning) of the list, and compare the
element at top with the key.
If we have a match, the search terminates and the index number is returned.
If not, we go on the next element in the list.
If we reach the end of the list without finding a match, we return -1.

Sequential Searching (Linear)
6 3 11 15 9
Searching for the value 15, linear search examines 6, 3, 11, and 15
Array num list contains:
Benefits:
Easy algorithm to understand
Array can be in any order
Disadvantages:
Inefficient (slow): for array of N elements, examines N/2 elements on average for value in array, N
elements for value not in array.

Implementation:
int LinearSearch(int list[ ], int key);
int main()
{
int list[] = {6, 3, 11, 15, 9};
int k = 15;
int i = LinearSearch(list, k);
if(i == -1)
cout << "the search item is not found" << endl;
else
cout << "The value is found at index position " << i << endl;
return 0; }
int LinearSearch(int list[ ], int key) {
int index = -1;
for(int i=0; i < n; i++) {
if(list[i]==key) {
index=i;
break;
} }
return index;
}
1/10/2024
38

Binary Searching
It assumes the data is sorted it also uses divide and conquer strategy
(approach).
Algorithm:
In a binary search, we look for the key in the middle of the list. If we get
a match, the search is over.
If the key is greater than the element in the middle of the list, we make
the top (upper) half the list to search.
If the key is smaller, we make the bottom (lower) half the list to search.
Repeat the above steps (I,II and III) until one element remains.
If this element matches return the index of the element, else
return -1 index. (-1 shows that the key is not in the list).
1/10/2024
39

Binary Search - Example
L mid R
Data= 59 n=10 index=n-1
1/10/2024
40
5 9 17 23 25 45 59 63 71 89
Data = a[mid]
Data < a [mid]
Data > a[mid]
L R mid
0 9 4
5 9 7
5 6 5
6 6 6
Benefits:
Much more efficient than linear search.
For array of N elements, performs at most log2N comparisons
Disadvantages:
Requires that array elements be sorted

Implementation
int BinarySearch(int list[ ], int key);
int main()
{
int list[] = {5,9,17,23,25,45,59,63,71,89};
int k = 59;
int i = BinarySearch(list, k);
if(i== -1)
cout << "the search item is not found" << endl;
else
cout << "The value is found at index position " << i << endl;
return 0;
}
int BinarySearch(int list[ ], int key)
{
int found=0,index=0;
int L=9, R=0, middle;
do
{
middle=(R + L)/2;
if(key==list[middle])
found=1;
else
{
if(key < list[middle])
R=middle-1;
else L=middle+1;
}
}while(found==0 && R >=L );
if(found==0)
index=-1;
else
index=middle;
return index;
}
1/10/2024
41

Simple Sorting Algorithms
What is sorting?
Sorting: is a process of reordering a list of items in either increasing or
decreasing order. Ordering a list of items is fundamental problem of
computer science. Sorting is the most important operation performed by
computers. Sorting is the first step in more complex algorithms.
Importance of sorting
• To represent data in more readable format.
• Optimize data searching to high level.
The most common sorting algorithms are:
• Bubble Sort
• Selection Sort
• Insertion Sort
1/10/2024
42

Bubble Sort
[NOTE: In each pass, the largest item “bubbles” down the list until it settles in its final position. This is where
bubble sort gets its name.]
Example,
Suppose we have a list of array of 5 elements A[5]={40,50,30,20,10}. We have to sort this array using bubble
sort algorithm.
Complexity Analysis:
• Analysis involves number of comparisons and swaps.
• How many comparisons? 1+2+3+…+(n-1)= O(n2)
• How many swaps? 1+2+3+…+(n-1)= O(n2)
1/10/2024
43

Selection Sort
 Selection sort is an in-place comparison sort algorithm. In this algorithm, we repeatedly select the smallest
remaining element and move it to the end of a growing sorted list. It is one of the simplest sorting algorithm.
 In this algorithm we have to find the minimum value in the list first. Then, Swap it with the value in the first
position.
 After that, Start from the second position and repeat the steps above for remainder of the list.
 Advantage: Simple and easy to implement.
 Disadvantage: Inefficient for larger lists.
1/10/2024
44

Insertion Sort
Insertion sort algorithm somewhat resembles Selection Sort and Bubble sort.
Array is imaginary divided into two parts - sorted one and unsorted one.
At the beginning, sorted part contains first element of the array and unsorted one
contains the rest.
At every step, algorithm takes first element in the unsorted part and inserts it to
the right place of the sorted one.
When unsorted part becomes empty, algorithm stops.
In more detail:
• Consider the first item to be a sorted sublist (of one item)
• Insert the second item into the sorted sublist, shifting the first item as needed to make room
to insert the new addition
• Insert the third item into the sorted sublist (of two items), shifting items as necessary
• Repeat until all values are inserted into their proper positions
1/10/2024
45

Insertion Sort
It is a simple algorithm where a sorted sub list is maintained by entering on element at
a time.
An element which is to be inserted in this sorted sub-list has to find its appropriate
location and then it has to be inserted there. That is the reason why it is named so.
Example

Cont…
It is reasonable to use binary search algorithm to find a proper place for
insertion.
Insertion sort is simply like playing cards: To sort the cards in your hand, you
extract a card, shift the remaining cards and then insert the extracted card in
the correct place.
This process is repeated until all the cards are in the correct sequence. Is
over twice as fast as the bubble sort and is just as easy to implement as the
selection sort.
Advantage: Relatively simple and easy to implement.
Disadvantage: Inefficient for large lists.
1/10/2024
47

1/10/2024
48
Thank You !!!

Data Structure and Algorithm chapter two, This material is for Data Structure and Algorithm course.

Recommandé

Recommandé

Contenu connexe

Similaire à Data Structure and Algorithm chapter two, This material is for Data Structure and Algorithm course.

Similaire à Data Structure and Algorithm chapter two, This material is for Data Structure and Algorithm course. (20)

Dernier

Dernier (20)

Data Structure and Algorithm chapter two, This material is for Data Structure and Algorithm course.