SlideShare a Scribd company logo
1 of 39
Markov Chains
Description Sometimes we are interested in how a random variable changes over time. The study of how a random variable evolves over time includes  Stochastic Processes .
What is a Stochastic Process? ,[object Object],[object Object],[object Object]
[object Object],[object Object]
The Gambler’s Ruin Problem ,[object Object],[object Object],[object Object]
What is a Markov Chain? ,[object Object],[object Object],[object Object]
[object Object],[object Object],[object Object],[object Object],[object Object]
[object Object],[object Object],[object Object],[object Object]
[object Object],[object Object]
TPM: The Gambler’s Ruin Problem ,[object Object]
[object Object],[object Object],[object Object]
Question ,[object Object],[object Object]
n -Step Transition Probabilities ,[object Object],[object Object],[object Object]
The Cola Example ,[object Object],[object Object],[object Object],[object Object],[object Object]
The Cola Example ,[object Object],[object Object],[object Object],[object Object],[object Object]
The Cola Example ,[object Object],[object Object]
The Cola Example ,[object Object],[object Object],[object Object],[object Object]
[object Object],[object Object],[object Object]
Limiting probabilities ,[object Object],[object Object],[object Object]
Question Find the equilibrium market shares of two firms whose probability transition matrix is as follows  .5 .5 B .3 .7 A B A
Example ,[object Object],1 2 3 1/2 1/2 1/2 1/2 1
All states communicate with each other Starting from 1, the MC can return to 1 in three steps via two possible routes: Route 1:  1 to 3 to 2 to 1 with probability  .5 ×1   ×.5  =  1/4 Route 2:  1 to 2 to 2 to 1 with probability  .5  ×.5×.5   = 1/8  Hence the required probability is =1/4+1/8 = 3/8 .
Steady-State Probabilities ,[object Object],[object Object]
[object Object],[object Object]
An Example ,[object Object],In the long In the long run, what fraction of the customers purchase the  respective brands?
Solution ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Inventory Example ,[object Object],[object Object],[object Object],[object Object],[object Object]
Inventory Example ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Inventory Example: (One-Step) Transition Matrix ,[object Object],[object Object],[object Object],[object Object]
Inventory Example: Transition Diagram 0 1 2 3
Inventory Example: (One-Step) Transition Matrix
Transition Matrix: Two-Step ,[object Object]
Transition Matrix: Four-Step ,[object Object]
Transition Matrix: Eight-Step ,[object Object]
Steady-State Probabilities ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Steady-State Probabilities: Inventory Example ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Mean First Passage Times ,[object Object],[object Object]
[object Object],[object Object]
[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

More Related Content

What's hot

Stat 2153 Stochastic Process and Markov chain
Stat 2153 Stochastic Process and Markov chainStat 2153 Stochastic Process and Markov chain
Stat 2153 Stochastic Process and Markov chainKhulna University
 
Markov chain intro
Markov chain introMarkov chain intro
Markov chain intro2vikasdubey
 
Markov chain and its Application
Markov chain and its Application Markov chain and its Application
Markov chain and its Application Tilakpoudel2
 
Markov Chain and its Analysis
Markov Chain and its Analysis Markov Chain and its Analysis
Markov Chain and its Analysis ShreyasBirje
 
Introduction to Random Walk
Introduction to Random WalkIntroduction to Random Walk
Introduction to Random WalkShuai Zhang
 
Logistic Regression | Logistic Regression In Python | Machine Learning Algori...
Logistic Regression | Logistic Regression In Python | Machine Learning Algori...Logistic Regression | Logistic Regression In Python | Machine Learning Algori...
Logistic Regression | Logistic Regression In Python | Machine Learning Algori...Simplilearn
 
Monte Carlo Simulation
Monte Carlo SimulationMonte Carlo Simulation
Monte Carlo SimulationAguinaldo Flor
 
4 stochastic processes
4 stochastic processes4 stochastic processes
4 stochastic processesSolo Hermelin
 
Lecture: Monte Carlo Methods
Lecture: Monte Carlo MethodsLecture: Monte Carlo Methods
Lecture: Monte Carlo MethodsFrank Kienle
 
Stochastic modelling and its applications
Stochastic modelling and its applicationsStochastic modelling and its applications
Stochastic modelling and its applicationsKartavya Jain
 
Introduction to Optimization.ppt
Introduction to Optimization.pptIntroduction to Optimization.ppt
Introduction to Optimization.pptMonarjayMalbog1
 
Autoregression
AutoregressionAutoregression
Autoregressionjchristo06
 
Machine Learning lecture4(logistic regression)
Machine Learning lecture4(logistic regression)Machine Learning lecture4(logistic regression)
Machine Learning lecture4(logistic regression)cairo university
 

What's hot (20)

Stat 2153 Stochastic Process and Markov chain
Stat 2153 Stochastic Process and Markov chainStat 2153 Stochastic Process and Markov chain
Stat 2153 Stochastic Process and Markov chain
 
Markov chain
Markov chainMarkov chain
Markov chain
 
Markov chain intro
Markov chain introMarkov chain intro
Markov chain intro
 
Markov chain and its Application
Markov chain and its Application Markov chain and its Application
Markov chain and its Application
 
Markov chain
Markov chainMarkov chain
Markov chain
 
Markor chain presentation
Markor chain presentationMarkor chain presentation
Markor chain presentation
 
Markov Chain and its Analysis
Markov Chain and its Analysis Markov Chain and its Analysis
Markov Chain and its Analysis
 
Introduction to Random Walk
Introduction to Random WalkIntroduction to Random Walk
Introduction to Random Walk
 
Markov chain
Markov chainMarkov chain
Markov chain
 
Logistic Regression | Logistic Regression In Python | Machine Learning Algori...
Logistic Regression | Logistic Regression In Python | Machine Learning Algori...Logistic Regression | Logistic Regression In Python | Machine Learning Algori...
Logistic Regression | Logistic Regression In Python | Machine Learning Algori...
 
1. Random walk.pdf
1. Random walk.pdf1. Random walk.pdf
1. Random walk.pdf
 
Monte Carlo Simulation
Monte Carlo SimulationMonte Carlo Simulation
Monte Carlo Simulation
 
Markov chain-model
Markov chain-modelMarkov chain-model
Markov chain-model
 
4 stochastic processes
4 stochastic processes4 stochastic processes
4 stochastic processes
 
HIDDEN MARKOV MODEL AND ITS APPLICATION
HIDDEN MARKOV MODEL AND ITS APPLICATIONHIDDEN MARKOV MODEL AND ITS APPLICATION
HIDDEN MARKOV MODEL AND ITS APPLICATION
 
Lecture: Monte Carlo Methods
Lecture: Monte Carlo MethodsLecture: Monte Carlo Methods
Lecture: Monte Carlo Methods
 
Stochastic modelling and its applications
Stochastic modelling and its applicationsStochastic modelling and its applications
Stochastic modelling and its applications
 
Introduction to Optimization.ppt
Introduction to Optimization.pptIntroduction to Optimization.ppt
Introduction to Optimization.ppt
 
Autoregression
AutoregressionAutoregression
Autoregression
 
Machine Learning lecture4(logistic regression)
Machine Learning lecture4(logistic regression)Machine Learning lecture4(logistic regression)
Machine Learning lecture4(logistic regression)
 

Viewers also liked

Lesson 4.1 Extreme Values
Lesson 4.1 Extreme ValuesLesson 4.1 Extreme Values
Lesson 4.1 Extreme ValuesSharon Henry
 
3.1 Extreme Values of Functions
3.1 Extreme Values of Functions3.1 Extreme Values of Functions
3.1 Extreme Values of FunctionsSharon Henry
 
Review Of Probability
Review Of ProbabilityReview Of Probability
Review Of ProbabilityMALAKI12003
 
Extreme value distribution to predict maximum precipitation
Extreme value distribution to predict maximum precipitationExtreme value distribution to predict maximum precipitation
Extreme value distribution to predict maximum precipitationJonathan D'Cruz
 
2 discrete markov chain
2 discrete markov chain2 discrete markov chain
2 discrete markov chainWindie Chan
 

Viewers also liked (7)

Markov exercises
Markov exercisesMarkov exercises
Markov exercises
 
The markovchain package use r2016
The markovchain package use r2016The markovchain package use r2016
The markovchain package use r2016
 
Lesson 4.1 Extreme Values
Lesson 4.1 Extreme ValuesLesson 4.1 Extreme Values
Lesson 4.1 Extreme Values
 
3.1 Extreme Values of Functions
3.1 Extreme Values of Functions3.1 Extreme Values of Functions
3.1 Extreme Values of Functions
 
Review Of Probability
Review Of ProbabilityReview Of Probability
Review Of Probability
 
Extreme value distribution to predict maximum precipitation
Extreme value distribution to predict maximum precipitationExtreme value distribution to predict maximum precipitation
Extreme value distribution to predict maximum precipitation
 
2 discrete markov chain
2 discrete markov chain2 discrete markov chain
2 discrete markov chain
 

Similar to Markov chains1

markov chain.ppt
markov chain.pptmarkov chain.ppt
markov chain.pptDWin Myo
 
IE 423 page 1 of 1 •••••••••••••••••••••••••••••••••••••••.docx
IE 423 page 1 of 1 •••••••••••••••••••••••••••••••••••••••.docxIE 423 page 1 of 1 •••••••••••••••••••••••••••••••••••••••.docx
IE 423 page 1 of 1 •••••••••••••••••••••••••••••••••••••••.docxsheronlewthwaite
 
12 Machine Learning Supervised Hidden Markov Chains
12 Machine Learning  Supervised Hidden Markov Chains12 Machine Learning  Supervised Hidden Markov Chains
12 Machine Learning Supervised Hidden Markov ChainsAndres Mendez-Vazquez
 
Hidden Markov Models with applications to speech recognition
Hidden Markov Models with applications to speech recognitionHidden Markov Models with applications to speech recognition
Hidden Markov Models with applications to speech recognitionbutest
 
Hidden Markov Models with applications to speech recognition
Hidden Markov Models with applications to speech recognitionHidden Markov Models with applications to speech recognition
Hidden Markov Models with applications to speech recognitionbutest
 
Martingales Stopping Time Processes
Martingales Stopping Time ProcessesMartingales Stopping Time Processes
Martingales Stopping Time ProcessesIOSR Journals
 
Rfid presentation in internet
Rfid presentation in internetRfid presentation in internet
Rfid presentation in internetAli Azarnia
 
Book chapter-5
Book chapter-5Book chapter-5
Book chapter-5Hung Le
 
2012 mdsp pr06  hmm
2012 mdsp pr06  hmm2012 mdsp pr06  hmm
2012 mdsp pr06  hmmnozomuhamada
 
stochastic-processes-1.pdf
stochastic-processes-1.pdfstochastic-processes-1.pdf
stochastic-processes-1.pdforicho
 
Univariate Financial Time Series Analysis
Univariate Financial Time Series AnalysisUnivariate Financial Time Series Analysis
Univariate Financial Time Series AnalysisAnissa ATMANI
 
Introduction to Bootstrap and elements of Markov Chains
Introduction to Bootstrap and elements of Markov ChainsIntroduction to Bootstrap and elements of Markov Chains
Introduction to Bootstrap and elements of Markov ChainsUniversity of Salerno
 

Similar to Markov chains1 (20)

Stochastic matrices
Stochastic matricesStochastic matrices
Stochastic matrices
 
markov chain.ppt
markov chain.pptmarkov chain.ppt
markov chain.ppt
 
IE 423 page 1 of 1 •••••••••••••••••••••••••••••••••••••••.docx
IE 423 page 1 of 1 •••••••••••••••••••••••••••••••••••••••.docxIE 423 page 1 of 1 •••••••••••••••••••••••••••••••••••••••.docx
IE 423 page 1 of 1 •••••••••••••••••••••••••••••••••••••••.docx
 
12 Machine Learning Supervised Hidden Markov Chains
12 Machine Learning  Supervised Hidden Markov Chains12 Machine Learning  Supervised Hidden Markov Chains
12 Machine Learning Supervised Hidden Markov Chains
 
Hidden Markov Models with applications to speech recognition
Hidden Markov Models with applications to speech recognitionHidden Markov Models with applications to speech recognition
Hidden Markov Models with applications to speech recognition
 
Hidden Markov Models with applications to speech recognition
Hidden Markov Models with applications to speech recognitionHidden Markov Models with applications to speech recognition
Hidden Markov Models with applications to speech recognition
 
Martingales Stopping Time Processes
Martingales Stopping Time ProcessesMartingales Stopping Time Processes
Martingales Stopping Time Processes
 
Stochastic Processes Assignment Help
Stochastic Processes Assignment HelpStochastic Processes Assignment Help
Stochastic Processes Assignment Help
 
By BIRASA FABRICE
By BIRASA FABRICEBy BIRASA FABRICE
By BIRASA FABRICE
 
Rfid presentation in internet
Rfid presentation in internetRfid presentation in internet
Rfid presentation in internet
 
Probable
ProbableProbable
Probable
 
Book chapter-5
Book chapter-5Book chapter-5
Book chapter-5
 
Talk 5
Talk 5Talk 5
Talk 5
 
NLP_KASHK:Markov Models
NLP_KASHK:Markov ModelsNLP_KASHK:Markov Models
NLP_KASHK:Markov Models
 
A STUDY ON MARKOV CHAIN WITH TRANSITION DIAGRAM
A STUDY ON MARKOV CHAIN WITH TRANSITION DIAGRAMA STUDY ON MARKOV CHAIN WITH TRANSITION DIAGRAM
A STUDY ON MARKOV CHAIN WITH TRANSITION DIAGRAM
 
2012 mdsp pr06  hmm
2012 mdsp pr06  hmm2012 mdsp pr06  hmm
2012 mdsp pr06  hmm
 
stochastic-processes-1.pdf
stochastic-processes-1.pdfstochastic-processes-1.pdf
stochastic-processes-1.pdf
 
Univariate Financial Time Series Analysis
Univariate Financial Time Series AnalysisUnivariate Financial Time Series Analysis
Univariate Financial Time Series Analysis
 
intro
introintro
intro
 
Introduction to Bootstrap and elements of Markov Chains
Introduction to Bootstrap and elements of Markov ChainsIntroduction to Bootstrap and elements of Markov Chains
Introduction to Bootstrap and elements of Markov Chains
 

More from Kinshook Chaturvedi (20)

Working and functions_of_rbi[1]
Working and functions_of_rbi[1]Working and functions_of_rbi[1]
Working and functions_of_rbi[1]
 
Role of idfc_in_infrastucture_finance
Role of idfc_in_infrastucture_financeRole of idfc_in_infrastucture_finance
Role of idfc_in_infrastucture_finance
 
Mutual funds
Mutual fundsMutual funds
Mutual funds
 
Iifcl ppt
Iifcl pptIifcl ppt
Iifcl ppt
 
Basel ii norms.ppt
Basel ii norms.pptBasel ii norms.ppt
Basel ii norms.ppt
 
Retail banking pres
Retail banking presRetail banking pres
Retail banking pres
 
Presentation on lic of india
Presentation on lic of indiaPresentation on lic of india
Presentation on lic of india
 
Mfi dfi
Mfi dfiMfi dfi
Mfi dfi
 
Management of np as imt
Management of np as imtManagement of np as imt
Management of np as imt
 
Life insurance in india final raja
Life insurance in india final rajaLife insurance in india final raja
Life insurance in india final raja
 
Financial inclusion
Financial inclusionFinancial inclusion
Financial inclusion
 
Corporate banking v2
Corporate banking v2Corporate banking v2
Corporate banking v2
 
Corporate banking latest
Corporate banking latestCorporate banking latest
Corporate banking latest
 
Financial mgt exercises
Financial mgt exercisesFinancial mgt exercises
Financial mgt exercises
 
Csac10[1].p
Csac10[1].pCsac10[1].p
Csac10[1].p
 
Csac08[1].p
Csac08[1].pCsac08[1].p
Csac08[1].p
 
Csac05[1].p
Csac05[1].pCsac05[1].p
Csac05[1].p
 
Csac14[1].p
Csac14[1].pCsac14[1].p
Csac14[1].p
 
Csac06[1].p
Csac06[1].pCsac06[1].p
Csac06[1].p
 
Xyber001 16 12_09 (2)
Xyber001 16 12_09 (2)Xyber001 16 12_09 (2)
Xyber001 16 12_09 (2)
 

Recently uploaded

Call Girls In Panjim North Goa 9971646499 Genuine Service
Call Girls In Panjim North Goa 9971646499 Genuine ServiceCall Girls In Panjim North Goa 9971646499 Genuine Service
Call Girls In Panjim North Goa 9971646499 Genuine Serviceritikaroy0888
 
👉Chandigarh Call Girls 👉9878799926👉Just Call👉Chandigarh Call Girl In Chandiga...
👉Chandigarh Call Girls 👉9878799926👉Just Call👉Chandigarh Call Girl In Chandiga...👉Chandigarh Call Girls 👉9878799926👉Just Call👉Chandigarh Call Girl In Chandiga...
👉Chandigarh Call Girls 👉9878799926👉Just Call👉Chandigarh Call Girl In Chandiga...rajveerescorts2022
 
Call Girls Jp Nagar Just Call 👗 7737669865 👗 Top Class Call Girl Service Bang...
Call Girls Jp Nagar Just Call 👗 7737669865 👗 Top Class Call Girl Service Bang...Call Girls Jp Nagar Just Call 👗 7737669865 👗 Top Class Call Girl Service Bang...
Call Girls Jp Nagar Just Call 👗 7737669865 👗 Top Class Call Girl Service Bang...amitlee9823
 
Call Girls Service In Old Town Dubai ((0551707352)) Old Town Dubai Call Girl ...
Call Girls Service In Old Town Dubai ((0551707352)) Old Town Dubai Call Girl ...Call Girls Service In Old Town Dubai ((0551707352)) Old Town Dubai Call Girl ...
Call Girls Service In Old Town Dubai ((0551707352)) Old Town Dubai Call Girl ...allensay1
 
Call Girls Electronic City Just Call 👗 7737669865 👗 Top Class Call Girl Servi...
Call Girls Electronic City Just Call 👗 7737669865 👗 Top Class Call Girl Servi...Call Girls Electronic City Just Call 👗 7737669865 👗 Top Class Call Girl Servi...
Call Girls Electronic City Just Call 👗 7737669865 👗 Top Class Call Girl Servi...amitlee9823
 
Russian Call Girls In Gurgaon ❤️8448577510 ⊹Best Escorts Service In 24/7 Delh...
Russian Call Girls In Gurgaon ❤️8448577510 ⊹Best Escorts Service In 24/7 Delh...Russian Call Girls In Gurgaon ❤️8448577510 ⊹Best Escorts Service In 24/7 Delh...
Russian Call Girls In Gurgaon ❤️8448577510 ⊹Best Escorts Service In 24/7 Delh...lizamodels9
 
Call Girls In DLf Gurgaon ➥99902@11544 ( Best price)100% Genuine Escort In 24...
Call Girls In DLf Gurgaon ➥99902@11544 ( Best price)100% Genuine Escort In 24...Call Girls In DLf Gurgaon ➥99902@11544 ( Best price)100% Genuine Escort In 24...
Call Girls In DLf Gurgaon ➥99902@11544 ( Best price)100% Genuine Escort In 24...lizamodels9
 
Quick Doctor In Kuwait +2773`7758`557 Kuwait Doha Qatar Dubai Abu Dhabi Sharj...
Quick Doctor In Kuwait +2773`7758`557 Kuwait Doha Qatar Dubai Abu Dhabi Sharj...Quick Doctor In Kuwait +2773`7758`557 Kuwait Doha Qatar Dubai Abu Dhabi Sharj...
Quick Doctor In Kuwait +2773`7758`557 Kuwait Doha Qatar Dubai Abu Dhabi Sharj...daisycvs
 
RSA Conference Exhibitor List 2024 - Exhibitors Data
RSA Conference Exhibitor List 2024 - Exhibitors DataRSA Conference Exhibitor List 2024 - Exhibitors Data
RSA Conference Exhibitor List 2024 - Exhibitors DataExhibitors Data
 
FULL ENJOY Call Girls In Majnu Ka Tilla, Delhi Contact Us 8377877756
FULL ENJOY Call Girls In Majnu Ka Tilla, Delhi Contact Us 8377877756FULL ENJOY Call Girls In Majnu Ka Tilla, Delhi Contact Us 8377877756
FULL ENJOY Call Girls In Majnu Ka Tilla, Delhi Contact Us 8377877756dollysharma2066
 
Cracking the Cultural Competence Code.pptx
Cracking the Cultural Competence Code.pptxCracking the Cultural Competence Code.pptx
Cracking the Cultural Competence Code.pptxWorkforce Group
 
Call Girls Kengeri Satellite Town Just Call 👗 7737669865 👗 Top Class Call Gir...
Call Girls Kengeri Satellite Town Just Call 👗 7737669865 👗 Top Class Call Gir...Call Girls Kengeri Satellite Town Just Call 👗 7737669865 👗 Top Class Call Gir...
Call Girls Kengeri Satellite Town Just Call 👗 7737669865 👗 Top Class Call Gir...amitlee9823
 
Falcon's Invoice Discounting: Your Path to Prosperity
Falcon's Invoice Discounting: Your Path to ProsperityFalcon's Invoice Discounting: Your Path to Prosperity
Falcon's Invoice Discounting: Your Path to Prosperityhemanthkumar470700
 
Call Girls From Pari Chowk Greater Noida ❤️8448577510 ⊹Best Escorts Service I...
Call Girls From Pari Chowk Greater Noida ❤️8448577510 ⊹Best Escorts Service I...Call Girls From Pari Chowk Greater Noida ❤️8448577510 ⊹Best Escorts Service I...
Call Girls From Pari Chowk Greater Noida ❤️8448577510 ⊹Best Escorts Service I...lizamodels9
 
Insurers' journeys to build a mastery in the IoT usage
Insurers' journeys to build a mastery in the IoT usageInsurers' journeys to build a mastery in the IoT usage
Insurers' journeys to build a mastery in the IoT usageMatteo Carbone
 
Mondelez State of Snacking and Future Trends 2023
Mondelez State of Snacking and Future Trends 2023Mondelez State of Snacking and Future Trends 2023
Mondelez State of Snacking and Future Trends 2023Neil Kimberley
 
Call Girls Ludhiana Just Call 98765-12871 Top Class Call Girl Service Available
Call Girls Ludhiana Just Call 98765-12871 Top Class Call Girl Service AvailableCall Girls Ludhiana Just Call 98765-12871 Top Class Call Girl Service Available
Call Girls Ludhiana Just Call 98765-12871 Top Class Call Girl Service AvailableSeo
 
It will be International Nurses' Day on 12 May
It will be International Nurses' Day on 12 MayIt will be International Nurses' Day on 12 May
It will be International Nurses' Day on 12 MayNZSG
 

Recently uploaded (20)

Call Girls In Panjim North Goa 9971646499 Genuine Service
Call Girls In Panjim North Goa 9971646499 Genuine ServiceCall Girls In Panjim North Goa 9971646499 Genuine Service
Call Girls In Panjim North Goa 9971646499 Genuine Service
 
👉Chandigarh Call Girls 👉9878799926👉Just Call👉Chandigarh Call Girl In Chandiga...
👉Chandigarh Call Girls 👉9878799926👉Just Call👉Chandigarh Call Girl In Chandiga...👉Chandigarh Call Girls 👉9878799926👉Just Call👉Chandigarh Call Girl In Chandiga...
👉Chandigarh Call Girls 👉9878799926👉Just Call👉Chandigarh Call Girl In Chandiga...
 
Call Girls Jp Nagar Just Call 👗 7737669865 👗 Top Class Call Girl Service Bang...
Call Girls Jp Nagar Just Call 👗 7737669865 👗 Top Class Call Girl Service Bang...Call Girls Jp Nagar Just Call 👗 7737669865 👗 Top Class Call Girl Service Bang...
Call Girls Jp Nagar Just Call 👗 7737669865 👗 Top Class Call Girl Service Bang...
 
Forklift Operations: Safety through Cartoons
Forklift Operations: Safety through CartoonsForklift Operations: Safety through Cartoons
Forklift Operations: Safety through Cartoons
 
Call Girls Service In Old Town Dubai ((0551707352)) Old Town Dubai Call Girl ...
Call Girls Service In Old Town Dubai ((0551707352)) Old Town Dubai Call Girl ...Call Girls Service In Old Town Dubai ((0551707352)) Old Town Dubai Call Girl ...
Call Girls Service In Old Town Dubai ((0551707352)) Old Town Dubai Call Girl ...
 
Call Girls Electronic City Just Call 👗 7737669865 👗 Top Class Call Girl Servi...
Call Girls Electronic City Just Call 👗 7737669865 👗 Top Class Call Girl Servi...Call Girls Electronic City Just Call 👗 7737669865 👗 Top Class Call Girl Servi...
Call Girls Electronic City Just Call 👗 7737669865 👗 Top Class Call Girl Servi...
 
Russian Call Girls In Gurgaon ❤️8448577510 ⊹Best Escorts Service In 24/7 Delh...
Russian Call Girls In Gurgaon ❤️8448577510 ⊹Best Escorts Service In 24/7 Delh...Russian Call Girls In Gurgaon ❤️8448577510 ⊹Best Escorts Service In 24/7 Delh...
Russian Call Girls In Gurgaon ❤️8448577510 ⊹Best Escorts Service In 24/7 Delh...
 
Call Girls In DLf Gurgaon ➥99902@11544 ( Best price)100% Genuine Escort In 24...
Call Girls In DLf Gurgaon ➥99902@11544 ( Best price)100% Genuine Escort In 24...Call Girls In DLf Gurgaon ➥99902@11544 ( Best price)100% Genuine Escort In 24...
Call Girls In DLf Gurgaon ➥99902@11544 ( Best price)100% Genuine Escort In 24...
 
Quick Doctor In Kuwait +2773`7758`557 Kuwait Doha Qatar Dubai Abu Dhabi Sharj...
Quick Doctor In Kuwait +2773`7758`557 Kuwait Doha Qatar Dubai Abu Dhabi Sharj...Quick Doctor In Kuwait +2773`7758`557 Kuwait Doha Qatar Dubai Abu Dhabi Sharj...
Quick Doctor In Kuwait +2773`7758`557 Kuwait Doha Qatar Dubai Abu Dhabi Sharj...
 
RSA Conference Exhibitor List 2024 - Exhibitors Data
RSA Conference Exhibitor List 2024 - Exhibitors DataRSA Conference Exhibitor List 2024 - Exhibitors Data
RSA Conference Exhibitor List 2024 - Exhibitors Data
 
FULL ENJOY Call Girls In Majnu Ka Tilla, Delhi Contact Us 8377877756
FULL ENJOY Call Girls In Majnu Ka Tilla, Delhi Contact Us 8377877756FULL ENJOY Call Girls In Majnu Ka Tilla, Delhi Contact Us 8377877756
FULL ENJOY Call Girls In Majnu Ka Tilla, Delhi Contact Us 8377877756
 
Cracking the Cultural Competence Code.pptx
Cracking the Cultural Competence Code.pptxCracking the Cultural Competence Code.pptx
Cracking the Cultural Competence Code.pptx
 
Call Girls Kengeri Satellite Town Just Call 👗 7737669865 👗 Top Class Call Gir...
Call Girls Kengeri Satellite Town Just Call 👗 7737669865 👗 Top Class Call Gir...Call Girls Kengeri Satellite Town Just Call 👗 7737669865 👗 Top Class Call Gir...
Call Girls Kengeri Satellite Town Just Call 👗 7737669865 👗 Top Class Call Gir...
 
Falcon's Invoice Discounting: Your Path to Prosperity
Falcon's Invoice Discounting: Your Path to ProsperityFalcon's Invoice Discounting: Your Path to Prosperity
Falcon's Invoice Discounting: Your Path to Prosperity
 
Call Girls From Pari Chowk Greater Noida ❤️8448577510 ⊹Best Escorts Service I...
Call Girls From Pari Chowk Greater Noida ❤️8448577510 ⊹Best Escorts Service I...Call Girls From Pari Chowk Greater Noida ❤️8448577510 ⊹Best Escorts Service I...
Call Girls From Pari Chowk Greater Noida ❤️8448577510 ⊹Best Escorts Service I...
 
unwanted pregnancy Kit [+918133066128] Abortion Pills IN Dubai UAE Abudhabi
unwanted pregnancy Kit [+918133066128] Abortion Pills IN Dubai UAE Abudhabiunwanted pregnancy Kit [+918133066128] Abortion Pills IN Dubai UAE Abudhabi
unwanted pregnancy Kit [+918133066128] Abortion Pills IN Dubai UAE Abudhabi
 
Insurers' journeys to build a mastery in the IoT usage
Insurers' journeys to build a mastery in the IoT usageInsurers' journeys to build a mastery in the IoT usage
Insurers' journeys to build a mastery in the IoT usage
 
Mondelez State of Snacking and Future Trends 2023
Mondelez State of Snacking and Future Trends 2023Mondelez State of Snacking and Future Trends 2023
Mondelez State of Snacking and Future Trends 2023
 
Call Girls Ludhiana Just Call 98765-12871 Top Class Call Girl Service Available
Call Girls Ludhiana Just Call 98765-12871 Top Class Call Girl Service AvailableCall Girls Ludhiana Just Call 98765-12871 Top Class Call Girl Service Available
Call Girls Ludhiana Just Call 98765-12871 Top Class Call Girl Service Available
 
It will be International Nurses' Day on 12 May
It will be International Nurses' Day on 12 MayIt will be International Nurses' Day on 12 May
It will be International Nurses' Day on 12 May
 

Markov chains1

  • 2. Description Sometimes we are interested in how a random variable changes over time. The study of how a random variable evolves over time includes Stochastic Processes .
  • 3.
  • 4.
  • 5.
  • 6.
  • 7.
  • 8.
  • 9.
  • 10.
  • 11.
  • 12.
  • 13.
  • 14.
  • 15.
  • 16.
  • 17.
  • 18.
  • 19.
  • 20. Question Find the equilibrium market shares of two firms whose probability transition matrix is as follows .5 .5 B .3 .7 A B A
  • 21.
  • 22. All states communicate with each other Starting from 1, the MC can return to 1 in three steps via two possible routes: Route 1: 1 to 3 to 2 to 1 with probability .5 ×1 ×.5 = 1/4 Route 2: 1 to 2 to 2 to 1 with probability .5 ×.5×.5 = 1/8 Hence the required probability is =1/4+1/8 = 3/8 .
  • 23.
  • 24.
  • 25.
  • 26.
  • 27.
  • 28.
  • 29.
  • 30. Inventory Example: Transition Diagram 0 1 2 3
  • 31. Inventory Example: (One-Step) Transition Matrix
  • 32.
  • 33.
  • 34.
  • 35.
  • 36.
  • 37.
  • 38.
  • 39.