Complexity Theory - Introduction. P and NP. NP-Complete problems. Approximation algorithms. Bin packing, Graph coloring. Traveling salesperson Problem.

1. Analysis and Design of Algorithms
Deepak John
Department Of Computer Applications, SJCET-Pala

2. Types of ProblemsTypes of Problems
 Optimization problem: construct a solution that
maximizes or minimizes some objective functionmaximizes or minimizes some objective function
 Decision problem: A question that has two possible
answers, yes and no.
Examples:
•SHORTEST-PATH (optimization)
Given G, u,v, find a path from u to v with fewest edges.
•PATH (decision)
Given G, u,v, and k, whether exist a path from u to vGiven G, u,v, and k, whether exist a path from u to v
consisting of at most k edges.

3. The class PThe class P
 P is the set of decision problems that can be solved in polynomial
time .if the size of the input to the problem is n then the problemp p p
can be solved in time O(nk) for some constant k.
 Polynomially bounded
an algorithm is said to be polynomilly bounded if its worst case
complexity is bounded by a polynomial function of the input
i (if th i l i l h th t f h i t f isize(if there is a polynomial p such that for each input of size n
the algorithm terminates after at most p(n)steps)
 P: Polynomial algorithms - These include all the sorting P: Polynomial algorithms These include all the sorting
algorithms, with running times on the order of n, lg n,n lg n and n2.

4. Class NPClass NP
 Class NP: Problems that can be solved in a polynomial number
of steps by a nondeterministic polynomial algorithms.of steps by a nondeterministic polynomial algorithms.
 A nondeterministic algorithm: a two-stage procedure that takes
as its input an instance I of a decision problem and does the
following
1. “guessing” stage: An arbitrary string S is generated that can
b h h f l i f h i ibe thought of as a guess at a solution for the given instance
2. “verification” stage: A deterministic algorithm takes both I
and S as its input and check if S is a solution to instance Iand S as its input and check if S is a solution to instance I,
(outputs yes if s is a solution and outputs no or not halt at all
otherwise)

5. NP-CompleateNP Compleate
 is the class of problems that are the hardest problems in NP.
 example: example:
A hamiltonian cycle of an undirected graph is a simple
cycle that contains every vertex
The Relationship between P ,NP and NPC

6. Approximation algorithmsApproximation algorithms
 an algorithm which return solutions that are guaranteed to
be close to an optimal solution.be close to an optimal solution.
 We use performance ratio to measure the quality of an
approximation algorithm.
 We are going to find a Near-Optimal solution for a given
problem.
 We assume two hypothesis :
1. Each potential solution has a positive cost.
2. The problem may be either a maximization or a
minimization problem on the cost.

7. Performance ratio
 For minimization problem, the performance ratio of algorithm A is
defined as a number r such that for any instance I of the problem,
OPT(I) is the value of the optimal solution for instance I and
A(I) is the value of the solution returned by algorithm A on instance IA(I) is the value of the solution returned by algorithm A on instance I.
 For maximization problem, the performance ratio of algorithm A is
defined as a number r such that for any instance I of the problem,
OPT(I)/A(I)
is at most r (r≥1),

8. Bin Packing ProblemBin Packing Problem
 Pack a set of N BINS = {1, 2, …, n} items, each with size ti , i=1,
2,…,n, into identical bins, each with capacity C.
 the bin packing problem is to determine the minimum number of bins
to accommodate all items.
 Fi di th ti l l ti i NP h d bl Finding the optimal solution is a NP-hard problem.
 Minimize the number of bins without violating the capacity constraints

9. Online bin packing
 Items arrive one by one.y
 Each item must be assigned immediately to a bin, without
knowledge of any future items. Reassignment is not allowed.
 There exists no online bin packing algorithm that always finds an
optimal solution.
 Next Fit (NF) First Fit (FF) Best Fit (BF) Next Fit (NF), First Fit (FF), Best Fit (BF)
Offline bin packing
 All n items are known in advance, i.e. before they have to be packed., f y p
 Initially sort the items in decreasing order of size and assign the
largest items first.
 First Fit Decreasing (FFD) ,Best Fit Decreasing (BFD)

10. Consider items 3; 6; 2; 1; 5; 7; 2; 4; 1; 9: with bin size 10
Next FitNext Fit
 Put the current item in the current bin if it fits, otherwise in the next
bin.
 O(n)
 Ex:[3; 6] - [2; 1; 5] -[7; 2]- [4; 1] - [9]
First Fit
 Put the current item into the first bin it fits into.
O( 2) O(n2)
 Ex:[3; 6; 1] - [2; 5; 2; 1] - [7] - [4] - [9]

11. Best Fit
 Assign an arriving item to the bin in which it fits bestg g
 Performance of BF and FF is similar.
 Ex:[3;6;1]-[2;5;2;1]-[9]-[4]-[9]

12. Algorithm First-Fit
1: forAll objects i = 1, 2, . . . , n do
2: forAll bins j = 1, 2, . . . do
3: if Object i fits in bin j then
4: Pack object i in bin j4: Pack object i in bin j.
5: Break the loop and pack the next object.
6: end if
7: end for
8: if Object i did not fit in any available bin then
9 C bi d k bj i9: Create new bin and pack object i.
10: end if
11: end for11: end for

13. Consider items 3; 6; 2; 1; 5; 7; 2; 4; 1; 9: with bin size 10
First Fit DecreasingFirst Fit Decreasing
1. Sort the objects in decreasing order
2. Apply First Fit strategy to this sorted list.2. Apply First Fit strategy to this sorted list.
Ex:[9; 1] -[7; 3] -[6; 4] -[5; 2; 2; 1]
Best Fit Decreasingg
1. Sort the objects in decreasing order
2. Apply Best Fit strategy to this sorted list.

14. Graph Coloring ProblemGraph Coloring Problem
 Graph coloring is an assignment of colors to the vertices of a graph.
no two adjacent vertices have the same color
 Chromatic number: is the smallest number of colors with which it
can be colored.
 S ti l l i d id l i Sequential coloring and widgerson coloring

15. Sequential Graph coloring
1 Gi G (VE) ith ti U th i t {1 2 3 } t1. Given G=(V,E) with n vertices. Use the integers {1,2,3, …, n} to
represent colors. Start by assigning 0 to every vertex. Process the
vertices one at a time.
2. For each vertex, Vi, start by coloring Vi with the color 1.
3. Check the neighbors of Vi to see if any is colored 1. If not then go
t th t t Vi+1to the next vertex, Vi+1.
4. If there is a neighbor colored 1, recolor Vi with color 2, and
repeat the neighbor searchrepeat the neighbor search.
5. Repeat the previous step incrementing the color until we find a
color c that has not been used to color any of Vi’s neighbors.

16. Example for sequential coloring

17. Widgerson coloring
 Recursive Algorithm Recursive Algorithm
 A graph with maximum degree ∆ can be easily colored using ∆ +1
colors.
 Base Case: 2 Colorable Graphs
 Find the sub graph of the Neighborhood of a given vertex,
 recursively color this sub graph.
 At most 3√n colors for an n-colorable graph.

18. Algorithm

19. Travelling Salesman ProblemTravelling Salesman Problem
 Definition: Find a path through a weighted graph which starts and
ends at the same vertex, includes every other vertex exactly once,, y y ,
and minimizes the total cost of edges.
 TSP is classify as NP-complete problem, that means no
l i l l ith t t ithi t blpolynomial algorithm can guarantee to come within countable
times of the shortest tour.
 TSP : Find a tour of minimum cost (distance)f ( )
 Problem: To find a Hamiltonian cycle of minimal cost.

20. Nearest Neighbor
1 Pick a reference vertex to start at1. Pick a reference vertex to start at.
2. Walk to its nearest neighbor (i.e., along the shortest possible
edge).(If there is a tie, break it randomly.)
3. At each stage in your tour, walk to the nearest neighbor that
you have not already visited.
4 h h i i d ll i h i4. When you have visited all vertices, return to the starting vertex.

21. Shortest-Link
 Idea: Start in the middle Idea: Start in the middle.
1. Add the cheapest available edge to your tour.(If there is a tie,
break it randomly.)
2. Repeat until you have a Hamilton circuit.
3. Make sure you add exactly two edges at each vertex.
4. Don't close the circuit until all vertices are in it.