•Common Problems Needs Computers •The Search Problem •Basic Search Algorithms –Algorithms used for searching the contents of an array •Linear or Sequential Search •Binary Search •Comparison Between Linear and Binary Search •Algorithms for solving shortest path problems –Sequential Search Algorithms •Depth-First Search •Breadth First Search –Parallel or distributed Search Algorithms •Parallel Depth-First Search •Parallel Breadth First Search

1. Prepared by:
Eng. Hossam Fadeel
Supervisor:
Dr. Ahmed El-Mahdy
3/18/2022
C
S
E
5
0
3
:
P
a
r
a
l
l
e
l
C
o
m
p
u
t
i
n
g

2. 2

3. Outlines
• Motivation
• Common Problems Needs
Computers
• Basic Search Algorithms
• Algorithms used for
searching the contents of
an array
– Linear or Sequential Search
– Binary Search
– Comparison Between
Linear and Binary Search
• Algorithms for solving
shortest path problems
– Introduction to Graphs
– Sequential Search
Algorithms
• Depth-First Search
• Breadth First Search
– Parallel or distributed
Search Algorithms
• Parallel Searching
Algorithms
• Parallel Depth-First Search
• Parallel Breadth First
Search
3

4. Motivation
• You want to go from Point A to Point B.
– You need to employing some kind of search.
– Example.
• For a direction finder, going from Point A to Point B literally means finding
a path between where you are now and your intended destination.
– Going from Point A to Point B is different for every situation.
4

5. Common Problems Needs Computers
• There are some very common problems that we use
computers to solve:
– Searching through a lot of records for a specific record or
set of records or for solution for unsolved problem
– Sorting, or placing records in a desired order
• There are numerous algorithms to perform searches
and sorts.
5

6. Basic Search Algorithms
• Algorithms used for searching the contents of an
array Like:
– Linear or Sequential Search
– Binary Search
• Algorithms used for solving shortest path problems
Like:
– Breadth-First Search and
– Depth-First Search
6

7. ALGORITHMS USED FOR SEARCHING
THE CONTENTS OF AN ARRAY
7

8. Linear or Sequential Search and Binary
Search
Linear or Sequential Search
 This is a very simple algorithm.
 It uses a loop to sequentially step
through an array, starting with
the first element.
 It compares each element with
the value being searched for (key)
and stops when that value is
found or the end of the array is
reached.
Binary Search
Requires array elements to be in order
1. Divides the array into three
sections:
– middle element
– elements on one side of the
middle element
– elements on the other side
of the middle element
2. If the middle element is the
correct value, done. Otherwise,
go to step 1. using only the half
of the array that may contain the
correct value.
3. Continue steps 1. and 2. until
either the value is found or there
are no more elements to
examine
8

9. Linear or Sequential Search
• Start at the beginning and examine each element in turn.
• It takes linear time in both the worst and average cases.
9
Start Here
Let’s Search for “2”:
8
2
1 3 4 6
5 7 10
9 11 12 14
13
0
64
14
13 25 33 51
43 53 84
72 93 95 97
96
6
Results “Nothing Found”

10. Linear or Sequential Search
• Start at the beginning and examine each element in turn.
• It takes linear time in both the worst and average cases.
10
Start Here
Let’s Search for “72”:
Search Success “Found 72”
8
2
1 3 4 6
5 7 10
9 11 12 14
13
0
64
14
13 25 33 51
43 53 84
72 93 95 97
96
6

11. 8
2
1 3 4 6
5 7 10
9 11 12 14
13
0
64
14
13 25 33 51
43 53 84
72 93 95 97
96
6
Binary Search
11
lo hi
• Binary search. Given value and sorted array a[], find index I
such that a[i] = value, or report that no such index exists.
• Invariant. Algorithm maintains a[lo]  value  a[hi].
• Ex. Binary search for 33.

12. Binary Search
• Binary search. Given value and sorted array a[], find index I
such that a[i] = value, or report that no such index exists.
• Invariant. Algorithm maintains a[lo]  value  a[hi].
• Ex. Binary search for 33.
8
2
1 3 4 6
5 7 10
9 11 12 14
13
0
64
14
13 25 33 51
43 53 84
72 93 95 97
96
6
lo hi
mid
12

13. Binary Search
• Binary search. Given value and sorted array a[], find index I
such that a[i] = value, or report that no such index exists.
• Invariant. Algorithm maintains a[lo]  value  a[hi].
• Ex. Binary search for 33.
8
2
1 3 4 6
5 7 10
9 11 12 14
13
0
64
14
13 25 33 51
43 53 84
72 93 95 97
96
6
lo hi
13
Binary Search Example

14. Binary Search
• Binary search. Given value and sorted array a[], find index I
such that a[i] = value, or report that no such index exists.
• Invariant. Algorithm maintains a[lo]  value  a[hi].
• Ex. Binary search for 33.
8
2
1 3 4 6
5 7 10
9 11 12 14
13
0
64
14
13 25 33 51
43 53 84
72 93 95 97
96
6
lo mid hi
14

15. Binary Search
• Binary search. Given value and sorted array a[], find index I
such that a[i] = value, or report that no such index exists.
• Invariant. Algorithm maintains a[lo]  value  a[hi].
• Ex. Binary search for 33.
8
2
1 3 4 6
5 7 10
9 11 12 14
13
0
64
14
13 25 33 51
43 53 84
72 93 95 97
96
6
lo hi
15

16. Binary Search
• Binary search. Given value and sorted array a[], find index I
such that a[i] = value, or report that no such index exists.
• Invariant. Algorithm maintains a[lo]  value  a[hi].
• Ex. Binary search for 33.
8
2
1 3 4 6
5 7 10
9 11 12 14
13
0
64
14
13 25 33 51
43 53 84
72 93 95 97
96
6
lo hi
mid
16

17. Binary Search
• Binary search. Given value and sorted array a[], find index I
such that a[i] = value, or report that no such index exists.
• Invariant. Algorithm maintains a[lo]  value  a[hi].
• Ex. Binary search for 33.
8
2
1 3 4 6
5 7 10
9 11 12 14
13
0
64
14
13 25 33 51
43 53 84
72 93 95 97
96
6
lo
hi
17

18. Binary Search
• Binary search. Given value and sorted array a[], find index I
such that a[i] = value, or report that no such index exists.
• Invariant. Algorithm maintains a[lo]  value  a[hi].
• Ex. Binary search for 33.
8
2
1 3 4 6
5 7 10
9 11 12 14
13
0
64
14
13 25 33 51
43 53 84
72 93 95 97
96
6
lo
hi
mid
18

19. Binary Search
• Binary search. Given value and sorted array a[], find index I
such that a[i] = value, or report that no such index exists.
• Invariant. Algorithm maintains a[lo]  value  a[hi].
• Ex. Binary search for 33.
8
2
1 3 4 6
5 7 10
9 11 12 14
13
0
64
14
13 25 33 51
43 53 84
72 93 95 97
96
6
lo
hi
mid
19

20. Binary Search
20
Note That:
The search pool must be sorted
For large n, a binary search is much faster

21. Comparison Between Linear and Binary Search
• Linear search is easy to
implement
– Basically, just a for loop
– Has O(n) complexity for n
stored items
– More efficient than binary
search for small n
• Binary search is harder to
implement but more
efficient
– Requires data to be sorted
– Has O(log n) complexity for n
stored items
21
O(logN)
O(N)

22. ALGORITHMS FOR SOLVING
SHORTEST PATH PROBLEMS
22

23. INTRODUCTION TO GRAPHS
23

24. Graph traversal
• Graph traversal is the problem of visiting all the nodes in
a graph in a particular manner, updating and/or checking their
values along the way.
– Tree traversal is a special case of graph traversal.
• Representing a problem as a graph can make a problem much
simpler
– More accurately, it can provide the appropriate tools for solving the
problem
• Examples of applications that involve graph processing:
– Maps. shortest route between two points.
– Web content. Links between pages.
– Computer networks. Nature of the interconnection structure.
24

25. Basics of Graphs
 Graph – mathematical object consisting of a set of:
 V = nodes (vertices, points).
 E = edges (links, arcs) between pairs of nodes.
 Denoted by G = (V, E).
 Captures pairwise relationship between objects.
 Graph size parameters: n = |V|, m = |E|.
25
V = { 1, 2, 3, 4, 5, 6, 7, 8 }
E = { {1,2}, {1,3}, {2,3}, {2,4}, {2,5}, {3,5}, {3,7}, {3,8}, {4,5}, {5,6} }
n = 8
m = 11
cycle C = 1-2-4-5-3-1

26. Undirected graph Directed graph
isolated vertex
adjacent
loop
multiple
edges
Simple graph: an undirected graph without loop or multiple edges
Degree of a vertex: number of edges connected
, for directed graph (indegree, outdegree)
G=(V,E)
Basics of Graphs

27. Basics of Graphs
x y
path: no vertex can be repeated
a-b-c-d-e
trail: no edge can be repeat
a-b-c-d-e-b-d
walk: no restriction
a-b-d-a-b-c
closed if x=y
closed trail: circuit (a-b-c-d-b-e-d-a,
one draw without lifting pen)
closed path: cycle (a-b-c-d-a)
a
b
c
d
e
length: number of edges in
this (path,trail,walk)

28. • An undirected graph is a tree if it is connected and does not contain a
cycle.
• Rooted tree. Given a tree T, choose a root node r and orient each edge
away from r.
• Importance: Models hierarchical structure.
28
a tree the same tree, rooted at 1
v (an inner node)
parent of v
child of v
root r
leaves
Basics of Graphs

29. 29
Graph Representation: Adjacency Matrix
 Adjacency matrix. n-by-n matrix with Auv = 1 if (u, v) is an edge.
 Two representations of each edge (symmetric matrix for undirected graphs; not
for directed graphs).
 Space: proportional to n2.
 Not efficient for sparse graphs (small number of edges compared to the
maximum possible number of edges in the graph).
 Checking if (u, v) is an edge takes (1) time.
 Identifying all edges takes (n2) time.
1 2 3 4 5 6 7 8
1 0 1 1 0 0 0 0 0
2 1 0 1 1 1 0 0 0
3 1 1 0 0 1 0 1 1
4 0 1 0 1 1 0 0 0
5 0 1 1 1 0 1 0 0
6 0 0 0 0 1 0 0 0
7 0 0 1 0 0 0 0 1
8 0 0 1 0 0 0 1 0

30. 30
Graph Representation: Adjacency List
 Adjacency list. Node indexed array of lists.
 Two representations of each edge.
 Space proportional to m + n.
 Checking if (u, v) is an edge takes O(deg(u)) time.
 Identifying all edges takes (m+n) time = linear time for G(V,E).
 Requires O(m+n) space. Good for dealing with sparse graphs.
1 2 3
2
3
4 2 5
5
6
7 3 8
8
1 3 4 5
1 2 5 8
7
2 3 4 6
5
degree = number of neighbors of u
3 7
n = |V|, m = |E|.

31. 31
Graph Traversing
 Given a graph G(V,E), explore every vertex and every edge
 Using adjacency list is more efficient
 Example algorithms:
Depth-first search (DFS) Breadth-first search (BFS)

32. SEQUENTIAL SEARCH ALGORITHMS
32

33. DFS (Depth First Search) and BFS
(Breadth First Search)
• DFS (Depth First Search) and BFS (Breadth First Search) are
search algorithms used for graphs and trees.
• When given an ordered tree or graph, like a (Binary Search
Tree) BST, it’s quite easy to search the data structure to find
the node that you want.
• When given an unordered tree or graph, the BFS and DFS
search algorithms can come to find node you’re looking for.
– The decision to choose one over the other should be based on the
type of data that one is dealing with.
33

34. BFS (Breadth First Search)
• In a breadth first search,
– Start at the root node, and then scan each node in the first level
starting from the leftmost node, moving towards the right.
– Then continue scanning the second level (starting from the left) and
the third level, and so on until scanned all the nodes, or until find the
actual node that searching for.
– When traversing one level, we need some way of knowing which
nodes to traverse once we get to the next level.
– The way this is done is by storing the pointers to a level’s child nodes
while searching that level.
– The pointers are stored in FIFO (first-in-first-out) queue. This, in turn,
means that BFS uses a large amount of memory because we have to
store the pointers.
34

35. BFS (Breadth First Search)
• An example of BFS
– The numbers represent the order in which the nodes are
accessed in a BFS:
35
Example 1: Example 2:

36. BFS (Breadth First Search)
36

37. DFS (Depth First Search)
• In a depth first search,
– Start at the root, and follow one of the branches of the tree as far as
possible until either the node looking for is found or hit a leaf node ( a
node with no children).
– If hit a leaf node, then continue the search at the nearest ancestor
with unexplored children.
• A major advantage of DFS is that its storage requirement is
linear in the depth of the state space being searched.
37

38. DFS (Depth First Search)
• An example of DFS
– The numbers represent the order in which the nodes are
accessed in a DFS:
38
Example 1: Example 2:

39. DFS (Depth First Search)
39

40. Differences between DFS and BFS
• Comparing BFS and DFS,
– the big advantage of DFS is that it has much lower memory
requirements than BFS,
• It’s not necessary to store all of the child pointers at each level.
• Depending on the data and what type of problem are
looking for, either DFS or BFS could be advantageous.
40

41. PARALLEL OR DISTRIBUTED SEARCH
ALGORITHMS
41

42. Parallel or Distributed Search
Algorithms
• Modern computers exploit parallelism on the hardware level.
• Parallel algorithms are designed to solve algorithmic problems
– Using many processing devices (processes, processors, processor cores, nodes,
units) simultaneously.
– Technically easier to build a system of several communicating slower
processors than a single one that is multiple times faster.
– Often require the invention of original and novel algorithms, radically different
from those used to solve the same problems sequentially.
– Compatibility with machine architecture, choice of suitable shared data
structures, and compromise between processing and communication
overhead.
– More than one search process is invoked, partitioning the workload among
processes.
– The most important problem is to minimize the communication (overhead)
between the search processes.
42
Parallelism

43. Parallel Searching Algorithms
43
“Parallel State Space Searching Algorithms” Professor Keyes – APAM 4990
Hart Lambur, Blake Shaw

44. Parallel Searching Algorithms
44
“Parallel State Space Searching Algorithms” Professor Keyes – APAM 4990
Hart Lambur, Blake Shaw

45. Parallel Searching Algorithms
45
“Parallel State Space Searching Algorithms” Professor Keyes – APAM 4990
Hart Lambur, Blake Shaw

46. Parallel Searching Algorithms
46
“Parallel State Space Searching Algorithms” Professor Keyes – APAM 4990
Hart Lambur, Blake Shaw

47. Parallel Depth-First Search
• The critical issue in parallel depth-first search algorithms is the
distribution of the search space among the processors.
47
The unstructured nature of tree search and the imbalance resulting
from static partitioning.

48. Parallel Depth-First Search
• A parallel formulation of DFS based on dynamic load
balancing is as follows:
– Each processor performs DFS on a disjoint part of the search space.
– When a processor finishes searching its part of the search space, it
requests an unsearched part from other processors.
– Whenever a processor finds a goal node, all the processors terminate.
– If the search space is finite and the problem has no solutions, then all
the processors eventually run out of work, and the algorithm
terminates.
48

49. Parallel Depth-First Search
49
A generic scheme for dynamic load balancing

50. Parallel Best-First Search
• In most parallel formulations of BFS, different processors
concurrently expand different nodes from the open list.
– Differ according to the data structures they use to implement the
open list.
– Given p processors, the simplest strategy assigns each processor to
work on one of the current best nodes on the open list.
• This is called the centralized strategy because each processor gets work
from a single global open list.
50

51. Parallel Best-First Search
51
A general schematic for parallel best-first search using a
centralized strategy

52. A Parallel formulation of Best-First
Search
• We parallelize BFS by sharing the work to be done among a number of
processors.
• Each processor searches a disjoint part of the search space in a BFS
fashion.
• When a processor has finished searching its part of the search space, it
tries to get an unsearched part of the search space from the other
processors.
• When a goal node is found, all of the processors quit.
• If the search space is finite and has no solutions, then eventually all the
processors would run out of work. (search terminate)
52

53. Parallel Breadth First Search
53

54. References
• http://www.programmerinterview.com/index.php/data-structures/dfs-vs-bfs/
• http://www.kirupa.com/developer/actionscript/depth_breadth_search.htm
• http://www.cs.duke.edu/~josh/bfsdfs.html
• http://www.math.brown.edu/~banchoff/STG/ma8/papers/emorgan/algorithms.ht
ml
54
There are many references I use it for this
presentation when preparing, but I didn’t
remember most of them.

55. 55