JNTUK III-1 Mech Operations Research Notes

1. 1
UNIT 7
DYNAMIC PROGRAMMING
Introduction
 Dynamic programming is a useful mathematical technique for making a sequence of
interrelated decisions.
 It provides a systematic procedure for determining the optimal combination
of decisions.
 In contrast to linear programming, there does not exist a standard mathematical
formulation of “the” dynamic programming problem.
 Rather, dynamic programming is a general type of approach to problem solving, and the
particular equations used must be developed to fit each situation.
 Therefore, a certain degree of ingenuity and insight into the general structure of dynamic
programming problems is required to recognize when and how a problem can be solved
by dynamic programming procedures.
 These abilities can best be developed by an exposure to a wide variety of dynamic
programming applications and a study of the characteristics that are common to all these
situations.
 The technique is useful in a large number of multi-period business problems, such as
(a) smoothing production employment,
(b) allocating capital funds,
(c) allocating salespeople to marketing areas, and
(d)evaluating investment opportunities.
Definition
 The mathematical technique of optimizing a sequence of inter-related decisions over a
period of time is called dynamic programming.
Differences between Linear Programming (LP) and Dynamic Programming (DP)
(1) First, there is no algorithm (like the simplex method) that can be programmed to solve all
problems. Instead, dynamic programming is a technique that allows a difficult problem to be
broken down into a sequence of easier sub-problems, which are then evaluated by stages.
(2) Second, linear programming is a method that gives single-stage (i.e., one-time period)
solutions. Dynamic programming has the power to determine the optimal solution over a one-
year time horizon by breaking the problem into 12 smaller one-month horizon problems and to
solve each of these optimally. Hence, it uses a multistage approach.
Dynamic Programming uses the backward recursive method for solving the problems

2. 2
Bellman’s Principle of Optimality
 Bellman’s principle of optimality is the principle which guides the solution of a problem
using dynamic programming.
 It states:
Regardless of the decisions taken to enter a particular state in a particular stage, the
remaining decisions made for leaving that stage must constitute an optimal policy”
Steps in Dynamic Programming
1. Divide the original problem into sub-problems called stages.
2. Solve the last stage of the problem for all possible conditions or states.
3. Working backward from that last stage, solve each intermediate stage.
4. Obtain the optimal solution for the original problem by solving all stages sequentially.
Terminologyof Dynamic Programming
1. Stage: a period or a logical sub-problem.
2. State variables: possible beginning situations or conditions of a stage. These have also
been called the input variables.
3. Decision variables: alternatives or possible decisions that exist at each stage.
4. Decision criterion: a statement concerning the objective of the problem.
5. Optimal policy: a set of decision rules, developed as a result of the decision criteria, that
gives optimal decisions for any entering condition at any stage.
6. Transformation: normally, an algebraic statement that reveals the relationship between
stages.
 In most applications, dynamic programming obtains solutions by working backward
from the end of the problem toward the beginning, thus breaking up a large, unwieldy
problem into a series of smaller, more tractable problems

3. 3
Example: Shortest Route Problem
Find the shortest path from City 1 to City 10 in the diagram shown below using the
recursive principle of Dynamic Programming.
Solution
Stage 1
 There is only one path from node (8) to node (10). The distance of (8) – (10) = 6.
 Therefore, label node (8) with 6. Indicate the segment (8) –(10) with a *
 There is only one path from node (9) to node (10). The distance of (9) – (10) = 7.
 Therefore, label node (9) with 7. Indicate the segment (9) –(10) with a *
Stage 2
From node (5)
 There are two paths from (5) to (10) viz., (5)-(8)-(10) and (5)-(9)-(10)
 Distance of path (5)-(8)-(10) = Length of (5)-(8) + Label of (8) = 5 + 6 = 11
 Distance of path (5)-(9)-(10) = Length of (5)-(9) + Label of (9) = 3 + 7 = 10
 Therefore label of (5) = Min {11, 10} = 10. Indicate (5) –(9) with a *
From node (6)
 There are two paths from (6) to (10) viz., (6)-(8)-(10) and (6)-(9)-(10)
 Distance of path (6)-(8)-(10) = Length of (6)-(8) + Label of (8) = 7 + 6 = 13
 Distance of path (6)-(9)-(10) = Length of (6)-(9) + Label of (9) = 8 + 7 = 15
 Therefore label of (6) = Min {13, 15} = 13. Indicate (6) –(8) with a *

4. 4
From node (7)
 There are two paths from (7) to (10) viz., (7)-(8)-(10) and (7)-(9)-(10)
 Distance of path (7)-(8)-(10) = Length of (7)-(8) + Label of (8) = 2 + 6 = 8
 Distance of path (7)-(9)-(10) = Length of (7)-(9) + Label of (9) = 4 + 7 = 11
 Therefore label of (7) = Min {8, 11} = 8. Indicate (7) –(8) with a *
Stage 3
From node (2)
 There are three paths from (2) to (10) viz., (2)-(5), (2)-(6) and (2)-(7)
 Distance of path (2)-(5) = Length of (2)-(5) + Label of (5) = 4 +10 = 14
 Distance of path (2)-(6) = Length of (2)-(6) + Label of (6) = 2 + 13 = 15
 Distance of path (2)-(7) = Length of (2)-(7) + Label of (7) = 5 + 8 = 13
 Therefore label of (2) = Min {14, 15, 13} = 13. Indicate (2) –(7) with a *
From node (3)
 There are three paths from (3) to (10) viz., (3)-(5), (3)-(6) and (3)-(7)
 Distance of path (3)-(5) = Length of (3)-(5) + Label of (5) = 4 +10 = 14
 Distance of path (3)-(6) = Length of (3)-(6) + Label of (6) = 3 + 13 = 16
 Distance of path (3)-(7) = Length of (3)-(7) + Label of (7) = 7 + 8 = 15
 Therefore label of (3) = Min {14, 16, 15} = 14. Indicate (3) –(5) with a *
From node (4)
 There are three paths from (4) to (10) viz., (4)-(5), (4)-(6) and (4)-(7)
 Distance of path (4)-(5) = Length of (4)-(5) + Label of (5) = 2 +10 = 12
 Distance of path (4)-(6) = Length of (4)-(6) + Label of (6) = 1 + 13 = 14
 Distance of path (4)-(7) = Length of (4)-(7) + Label of (7) = 5 + 8 = 13
 Therefore label of (4) = Min {12, 14, 13} = 12. Indicate (4) –(5) with a *
Stage 4
From node (1)
 There are three paths from (1) to (10) viz., (1)-(2), (1)-(3) and (1)-(4)
 Distance of path (1)-(2) = Length of (1)-(2) + Label of (2) = 5 +13 = 18
 Distance of path (1)-(3) = Length of (1)-(3) + Label of (3) = 7 + 14 = 21
 Distance of path (1)-(4) = Length of (1)-(4) + Label of (4) = 3 + 12 = 15
 Therefore label of (1) = Min {18, 21, 15} = 15. Indicate (1) –(4) with a *
 Since the label of (1) is 15, the distance of the shortest path from (1) to (10) = 15.
Starting from (1) and following the paths indicated by *s,
the shortestpath is (1)-(4)-(5)-(9)-(10). Its distance is 15.