Computer Problem-Solving Efficiency of Algorithms Properties of algorithms Developing an Algorithm Complexity Analysis Big-O Notation Pseudocode Flowcharts

1. Lecture 6
Algorithms and Data
Structures
(CS221)
Abdisalam Issa-Salwe
Taibah University
College of Computer Science & Engineering
Computer Science Department
Outline
• Computer Problem-Solving
• Efficiency of Algorithms
• Properties of algorithms
• Developing an Algorithm
• Complexity Analysis
• Big-O Notation
• Pseudocode
• Flowcharts
2
1

2. Algorithms
An algorithm is a finite set of precise instructions
for performing a computation or for solving a
problem.
A set of instructions for solving a problem or
subproblem in a finite amount of time using a finite
amount of data
The instructions must be unambiguous
3
Algorithm (cont…)
Algorithm is a systematic method of
instructing an agent how to accomplish a
task
Usually expressed in a step-wise
sequential form
May involve alternation, iteration or
recursion
Detail and language may depend on the
agent
4
2

3. Algorithm (cont…)
It is a set of instructions of how to carry out
a process.
A list of the steps that must be followed to
complete the process and to reach the
solution
An algorithm can be represented
diagrammatically in the form of a flow
chart.
5
Divide and Conquer
Problem solving: The act of finding a
solution to a perplexing, distressing,
vexing, or unsettled question
Break up a large problem into smaller
units that we can handle
Applies the concept of abstraction
The divide-and-conquer approach can be
applied over and over again until each
subtask is manageable
6
3

4. Computer Problem-Solving
7
Problem-Solving Phases
8
4

5. Task 1: Making A Cup of Tea!
The following list 1) Take tea bag out of cup
represents the steps Boil the water
needed in order to make 2) Put tea bag in cup
a cup of tea. Put the list in 3) Add milk?
the right order and 4) Pour boiling water in cup
produce an algorithm in 5) Fill kettle with water
POWERPOINT showing 6) Stir
the processes and 7) Ready
decisions you need to
8) Add sugar ?
make.
9) Fetch cup
Answers follow
9
START
The answers…
Fill kettle
1. Fill kettle
Boil the water
2. Boil the water
3. Fetch cup Fetch cup
4. Put tea bag in cup Put tea bag in cup
5. Pour boiling water in cup Pour boiling water in cup
6. Take tea bag out of cup
Take tea bag out of cup
7. Add sugar ?
8. Stir Add sugar ? N
9. Add milk?
10. Stir Stir in sugar
11. Ready Add milk? N
Now add the shapes… Pour in milk Ready
10
5

6. Description of an Algorithm
• Numbered steps with
indentation commonly
used
• Different descriptions
possible for the same
algorithm
• Difference between the
algorithm description
(design) and the
completion of the task
(implementation)
• More than one algorithm
possible for the same
task
11
Efficiency of Algorithms
• What resources are required to
accomplish the task
• How one algorithm compares with other
algorithms
12
6

7. Efficiency and Complexity
• Efficiency
• How much time or space is required
• Measured in terms of common basic
operations
• Complexity
• How efficiency varies with the size of the task
• Expressed in terms of standard functions of n
• e.g.: O(n), O(n2), O(log n), O(n log n)
13
Properties of algorithms
Input from a specified set,
Output from a specified set (solution),
Definiteness of every step in the computation,
Correctness of output for every possible input,
Finiteness of the number of calculation steps,
Effectiveness of each calculation step and
Generality for a class of problems.
14
7

8. Developing an Algorithm
The plan must be suitable in a suitable
form
Two methodologies that currently used
Top-down design
Top-down design
Object-oriented design
15
Top-Down Design
Breaking the problem into a set of
subproblems called modules
Creating a hierarchical structure of
problems and subproblems (modules)
16
8

9. Testing the Algorithm
Desk checking Working through a design at a
desk with a pencil and paper
Walk-through Manual simulation of the design
by the team members, taking sample data
values and simulating the design using the
sample data
Inspection One person (not the designer)
reads the design (handed out in advance) line
by line while the others point out errors
17
Complexity Analysis
Number of CPU cycles it takes to run an algorithm
depends on the computer on which the algorithm is run.
Count of how many instructions are executed depends
on the programming language used to implement the
algorithm and the way the programmer writes the
program.
We want a measure of algorithm efficiency that is
independent of the computer, the programming
language, the programmer, and all the complex details of
the algorithm such as incrementing of loop indices,
setting of pointers, etc.
In general, the running time of an algorithm increases
with the size of the input, and the total running time is
roughly proportional to how many times some basic
operation (such as a comparison instruction) is done.
18
9

10. Complexity Analysis (cont…)
We therefore analyze the algorithm’s efficiency by
determining the number of times some basic operation is
done as a function of the input size. This is called a time
complexity analysis of an algorithm.
The basic operation may be a single instruction or a
group of instructions; in some cases, we may want to
consider more than one basic operation.
The input size may be easy to determine – such as the
size of an array for Sequential or Binary Search -- or it
may be more difficult.
In some algorithms, the basic operation is always done
the same number of times for every instance of size N.
When this is the case, the every-case time complexity of
the algorithm, T(n), is defined as the number of times the
algorithm does the basic operation for an instance of
size n.
19
Worst-Case Time Complexity Analysis
Basic operation: the comparison of an
item in the array with searchkey.
Input size: n, the number of items in the
array.
The basic operation is done at most n
times, which is the case if searchkey is the
last item in the array or if searchkey is not
in the array. Therefore, W(n) = n.
20
10

11. Average-Case Time Complexity Analysis
Basic operation: the comparison of an item in the array
with searchkey.
Input size: n, the number of items in the array.
We first analyze the case in which it is known that
searchkey is in the array, where the items in the array
are all distinct, and where we have no reason to believe
that searchkey is more likely to be in one array slot than
it is to be in another. Based on this information, for 1 ≤ k
≤ n, the probability that searchkey is in the kth array slot
is 1/n. If searchkey is in the kth array slot, the number of
times the basic operation is done to locate searchkey
(and therefore, to exit the loop) is k. This means that the
average time complexity is given by
21
Best-Case Time Complexity Analysis
Basic operation: the comparison of an
item in the array with searchkey.
Input size: n, the number of items in the
array.
Because n ≥ 1, there must be at least one
pass through the loop. If searchkey =
array[0], there will be one pass through the
loop regardless of the size of n. Therefore,
B(n) = 1.
22
11

12. Performance Analysis
Determining an estimate of the time and
memory requirement of the algorithm.
Time estimation is called time complexity
analysis
Memory size estimation is called space
complexity analysis.
Because memory is cheap and abundant, we
rarely do space complexity analysis
Since time is “expensive” , analysis now
defaults to time complexity analysis
23
Big-O Notation
The most common method and notation
for discussing the execution time of
algorithms is "Big O”.
For the alphabetized dictionary the
algorithm requires O(log N) steps.
For the unsorted list the algorithm requires
O(N) steps.
Big O is the asymptotic execution time of
the algorithm.
24
12

13. Big-O Notation (In Practice)
When computing the complexity,
f(n) is the actual time formula
g(n) is the simplified version of f
Since f(n) stands often for time, we use
T(n) instead of f(n)
In practice, the simplification of T(n)
occurs while it is being computed by the
designer
25
Big-O Notation
(Common Complexities)
T(n) = O(1) // constant time
T(n) = O(log n) // logarithmic
T(n) = O(n) // linear
T(n) = O(n2) //quadratic
T(n) = O(n3) //cubic
T(n) = O(nc), c≥ 1 // polynomial
T(n) = O(logc n), c≥ 1 // polylogarithmic
T(n) = O(nlog n)
26
13

14. Other Algorithmic Analysis Tools
Big Omega T(N) is Ω( F(N) ) if there are
positive constants c and N0 such that
T(N) > cF( N )) when N > N0
Big O is similar to less than or equal, an upper
bound.
Big Omega is similar to greater than or equal,
a lower bound.
Big Theta T(N) is θ( F(N) ) if and only if
T(N) is O( F(N) )and T( N ) is Ω( F(N) ).
Big Theta is similar to equals.
27
Applying algorithm
When applying the theory of algorithm analysis, one must
sometimes be aware of the time that it takes to execute the
basic operation, the overhead instructions, and the control
instructions on the actual computer on which the algorithm
is implemented.
“Overhead instructions” includes things such as
initialization instructions before a loop; the number of times
these instructions execute does not increase with input
size.
“Control instructions” means instructions such as
incrementing an index to control a loop; the number of
times these instructions execute increases with input size.
The basic operation, overhead instructions, and control
instructions are all properties of an algorithm and the
implementation of the algorithm; they are not properties of a
problem.
28
14

15. Applying algorithm (cont…)
Algorithms with time complexities such as n and
100n are called linear-time algorithms,
because their time complexities are linear in the
input size n.
Algorithms with time complexities such as n2
and 0.01n2 are called quadratic-time
algorithms, because their time complexities are
quadratic in the input size n.
A fundamental principle: any linear-time
algorithm is eventually more efficient than any
quadratic-time algorithm.
Algorithms can be grouped into orders
according to their eventual behavior.
29
Pseudocode
Uses a mixture of English and formatting
to make the steps in the solution explicit
30
15

16. Pseudocode
Pseudocode is a kind of structured English for
describing algorithms.
It allows the designer to focus on the logic of the
algorithm without being distracted by details of
language syntax.
At the same time, the pseudocode needs to be
complete.
It describe the entire logic of the algorithm so
that implementation becomes a rote mechanical
task of translating line by line into source code.
31
Pseudocode (cont…)
The vocabulary used in the pseudocode
should be the vocabulary of the problem
domain, not of the implementation
domain.
The pseudocode is a narrative for
someone who knows the requirements
(problem domain) and is trying to learn
how the solution is organized
32
16

17. Constructs for flow of control (cont…)
1. SEQUENCE: A linear progression where one
task is performed sequentially after another.
Sequential control is indicated by writing one
action after another, each action on a line by
itself, and all actions aligned with the same
indent. The actions are performed in the
sequence (top to bottom) that they are
written.
2. WHILE: A loop (repetition) with a simple
conditional test at its beginning.
The WHILE construct is used to specify a loop with a
test at the top. The beginning and ending of the loop
are indicated by two keywords WHILE and
ENDWHILE.
33
Constructs for flow of control (cont…)
3. IF-THEN-ELSE: A decision (selection) in which
a choice is made between two alternative
courses of action.
Binary choice on a given Boolean condition is
indicated by the use of four keywords: IF, THEN,
ELSE, and ENDIF.
4. REPEAT-UNTIL: A loop with a simple
conditional test at the bottom.
This loop is similar to the WHILE loop except that the
test is performed at the bottom of the loop instead of
at the top. Two keywords, REPEAT and UNTIL are
used.
34
17

18. Constructs for flow of control (cont…)
5. CASE: A CASE construct indicates a
multiway branch based on conditions that
are mutually exclusive.
Four keywords, CASE, OF, OTHERS, and
ENDCASE, and conditions are used to
indicate the various alternatives.
FOR: This loop is a specialized construct
for iterating a specific number of times,
often called a "counting" loop.
Two keywords, FOR and ENDFOR are used.
35
Constructs for flow of control (cont…)
NESTED CONSTRUCTS: The constructs
can be embedded within each other, and
this is made clear by use of indenting.
Nested constructs should be clearly indented
from their surrounding constructs.
INVOKING SUBPROCEDURES: Using
the CALL keyword.
36
18

19. Flowcharts (cont…)
Step-form and pseudocode program
designs are both text-based, the
statements are written.
Flow charts are a graphical method of
designing programs and once the rules
are learned are very easy to draw.
37
Flowcharts
A well-drawn flow chart is also very easy
to read since it basically uses just two
symbols, two decision constructs. and two
iteration constructs:
the sequence symbol,
the decision symbol,
the decision construct if ... then
the decision construct if ... then ... else
the repetition construct - repeat
the repetition construct - while
38
19

20. Flowcharts (cont…)
39
Flowcharts (cont…)
Every flow chart has a START symbol and a STOP
symbol
The flow of sequence is generally from the top of the
page to the bottom of the page.
This can vary with loops which need to flow back to an entry
point.
Use arrow-heads on connectors where flow direction
may not be obvious.
There is only one flow chart per page
A page should have a page number and a title
A flow chart on one page should not break and jump to
another page
A flow chart should have no more than around 15
symbols (not including START and STOP)
40
20

21. Flowcharts (cont…)
The major symbols are the DECISION (also
known as selection) and the SEQUENCE (or
process) symbols.
The START and STOP symbols are called the
terminals.
The SUBPROCESS symbol is a variation on the
sequence symbol.
Connectors are drawn between the symbols
There must be at least one other sequence
symbol to represent input/output processes
41
Flowcharts (cont…)
Processes can have only one entry point and one exit
point.
Decisions have only one entry point, one TRUE exit
point and one FALSE exit point.
Repeat loop: repeat loop has the process preceding the
decision; a repeat loop will always execute the process
part at least once.
While loop is the reverse of the repeat loop; the
decision comes first, followed by the process.
IF ... THEN construct is also known as the NULL ELSE,
meaning that there is no ELSE part.
IF ... THEN ... ELSE ... construct has a process at each
branch of the decision symbol.
42
21

22. Topic Questions
What is Algorithms and how it relates to
computerized programming?
Can you a problem with two solutions?
which one of the solution you think it is more
efficient?
What is the difference between time
complexity and space complexity?
What is flowchart and how can be used to
solve a problem?
43
References
Michael Main, Data Structures and Other
Objects Using Java (Third Edition)
Abdisalam Issa-Salwe, Taibah University,
Madinah, Saudi Arabia.
44
22