Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

SophiaConf 2018 - Q. Nguyen (Amadeus)

85 vues

Publié le

Support de présentation : The end of airline Data revenue management as we know it ?

Publié dans : Technologie
  • Soyez le premier à commenter

  • Soyez le premier à aimer ceci

SophiaConf 2018 - Q. Nguyen (Amadeus)

  1. 1. Strictly CONFIDENTIAL The end of Airline Revenue Management as we know it? (Deep) Reinforcement Learning for Revenue Management Sophia Conf July 2018 Speaker: Quan NGUYEN ©AmadeusITGroupanditsaffiliatesandsubsidiaries
  2. 2. Revenue Management System (RMS) Introduction − RMS: automated system − Goal: maximize flight revenue − All seats − Whole reservation period (1 year) − What the systems does: − Set the right fares at the right time
  3. 3. Motivation – limitations of RMS RMS assumptions − Rely of historical observations – may represent the future or not − Collection of historical observations − The future is similar to the past (e.g. schedules, fares, volume, WTP, arrival rate, etc.) − Rely on a model representation – may be right or wrong − Requires a demand model for volume and WTP − Requires an optimization model (e.g. EMSR or Dynamic Programming) − No exploration – no validation of assumptions − RMS always takes the “optimal” decision − Assumes monopoly – in reality there is almost always competition − Average policies over all competitors states and actions
  4. 4. Strictly CONFIDENTIAL Introduction to Reinforcement Learning ©AmadeusITGroupanditsaffiliatesandsubsidiaries
  5. 5. 5 ©AmadeusITGroupanditsaffiliatesandsubsidiaries Reinforcement Learning Introduction Agent Environment Action 𝑎 𝑡 Reward 𝑟𝑡 State 𝑠𝑡 𝑟𝑡+1 𝑠𝑡+1 𝐸 ෍ 𝑡=1 𝑇 𝛾 𝑡 𝑟𝑡 Objective: maximize long term reward
  6. 6. 6 ©AmadeusITGroupanditsaffiliatesandsubsidiaries Reinforcement Learning Applied to Revenue Management Airline RMS Customers Action 𝑎 𝑡 = 𝑓 Reward 𝑟𝑡 State 𝑠𝑡 𝑟𝑡+1 𝑠𝑡+1 Bookings (x)Empty Full Depar- ture {t,x} {t+1,x} {t+1,x+1} State 𝑠𝑡 No Sale Sale Time(t)
  7. 7. Strictly CONFIDENTIAL QL and DQL - Monopoly ©AmadeusITGroupanditsaffiliatesandsubsidiaries
  8. 8. Performance of QL in Monopoly 70% 80% 90% 100% 110% 0 50 100 150 200 250 RevenueIndex Years! QL RMS optimal=100% QL ~99.3% Base Reinforcement Learning in a Monopoly 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 4 4 4 4 4 4 4 4 4 4 4 4 4 4 5 5 5 5 5 5 5 5 5 5 5 5 6 6 6 6 6 6 6 6 6 6 7 7 7 7 7 7 7 7 7 7 8 8 8 8 8 8 8 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 7 6 4 9 1 6 3 2 0 3 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3 3 3 3 3 3 4 4 4 4 4 4 4 4 4 4 4 4 4 5 5 5 5 5 5 5 5 5 5 5 6 6 6 6 6 6 6 6 6 6 7 7 7 7 7 7 7 7 7 7 8 8 8 8 8 8 8 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 2 2 7 5 8 5 7 7 6 9 5 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3 3 3 3 3 3 4 4 4 4 4 4 4 4 4 4 4 4 5 5 5 5 5 5 5 5 5 5 5 5 6 6 6 6 6 6 6 6 6 6 6 6 7 7 7 7 7 7 7 7 7 7 7 8 8 8 8 8 8 8 8 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 2 2 2 5 10 8 10 2 1 7 0 5 9 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3 3 3 3 4 4 4 4 4 4 4 4 4 4 5 5 5 5 5 5 5 5 5 5 5 6 6 6 6 6 6 6 6 6 6 6 7 7 7 7 7 7 7 7 7 7 7 8 8 8 8 8 8 8 8 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 0 0 1 1 0 0 0 0 0 0 0 1 0 1 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0 0 0 1 1 0 2 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 1 2 0 1 1 0 2 2 3 3 3 10 9 6 4 8 2 5 2 7 3 0 6 6 1 4 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3 3 3 3 3 4 4 4 4 4 4 4 4 4 4 4 4 5 5 5 5 5 5 5 5 5 5 5 5 5 6 6 6 6 6 6 6 6 6 6 6 6 6 7 7 7 7 7 7 7 7 7 7 7 7 7 8 8 8 8 8 8 8 8 8 8 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 1 1 2 0 1 0 0 0 1 0 2 0 0 0 0 0 0 0 0 0 0 0 0 2 1 0 0 1 1 0 0 0 0 0 1 0 0 1 1 0 0 0 0 0 1 0 0 0 0 1 1 1 0 1 0 0 0 0 0 0 1 0 0 1 1 0 1 0 0 0 1 0 1 0 0 0 1 1 2 3 2 2 4 9 8 8 10 4 10 7 3 10 3 6 6 0 8 7 1 1 2 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3 3 3 4 4 4 4 4 4 4 4 4 4 5 5 5 5 5 5 5 5 5 5 6 6 6 6 6 6 6 6 6 6 6 6 7 7 7 7 7 7 7 7 7 7 7 7 7 8 8 8 8 8 8 8 8 8 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 5 1 1 0 2 1 1 1 1 1 4 1 2 1 1 1 0 3 1 0 1 0 1 0 0 1 0 0 2 2 0 0 2 2 0 1 1 0 0 0 1 0 0 0 0 2 3 1 2 0 1 0 2 1 0 0 1 1 1 0 0 2 0 1 1 2 1 0 0 0 2 0 1 1 4 1 1 1 1 0 1 1 1 1 1 0 1 1 2 0 1 1 1 1 1 2 6 6 7 6 3 7 4 1 5 5 1 7 1 4 3 3 3 4 4 4 4 4 4 4 4 4 4 4 5 5 5 5 5 5 5 5 5 5 5 5 6 6 6 6 6 6 6 6 6 6 6 6 6 7 7 7 7 7 7 7 7 7 7 7 7 7 7 8 8 8 8 8 8 8 8 8 8 8 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 5 7 10 7 0 7 1 0 0 1 1 3 2 0 1 3 2 3 4 1 1 2 4 0 3 2 2 1 0 0 1 2 1 0 2 2 1 0 1 0 0 1 1 0 0 2 0 2 0 0 0 2 1 0 2 1 0 0 2 0 2 0 4 0 1 0 1 2 0 1 3 1 0 2 0 2 1 2 3 0 0 2 0 0 1 1 1 0 0 0 1 1 2 1 4 0 1 1 1 1 0 0 1 1 2 2 1 3 0 4 6 2 4 6 6 1 7 5 5 2 7 4 3 4 4 4 4 4 4 4 5 5 5 5 5 5 5 5 5 5 6 6 6 6 6 6 6 6 6 6 6 6 7 7 7 7 7 7 7 7 7 7 7 7 8 8 8 8 8 8 8 8 8 8 8 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 1 5 6 5 5 7 1 3 1 3 3 1 2 0 1 1 0 2 2 1 1 1 2 1 2 1 0 1 1 1 1 1 3 2 0 2 0 2 1 0 1 3 1 0 1 1 0 1 0 1 0 0 2 1 0 1 1 3 1 1 3 1 0 0 2 0 1 0 1 3 0 3 0 2 1 1 0 1 3 0 3 3 1 2 1 1 0 0 2 0 1 1 0 0 2 1 1 1 1 0 0 1 1 1 3 1 1 0 1 1 1 1 1 1 0 1 2 0 5 6 2 4 2 4 2 3 4 0 2 3 1 3 3 3 5 5 5 6 6 6 6 6 6 6 6 6 6 6 6 7 7 7 7 7 7 7 7 7 7 7 7 7 8 8 8 8 8 8 8 8 8 8 8 8 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 2 1 2 4 4 2 5 7 3 1 1 2 2 0 1 1 2 2 1 0 0 0 2 3 2 2 0 2 0 0 2 0 1 1 0 1 0 1 3 1 2 2 3 2 1 2 2 3 0 0 0 1 2 3 1 2 1 1 0 1 2 0 3 1 1 2 1 2 1 1 2 3 1 2 2 1 3 1 3 1 2 1 0 2 0 3 2 1 0 0 0 0 1 1 2 2 0 2 1 1 1 3 0 2 0 1 2 3 1 1 1 0 0 0 2 1 0 1 1 0 2 2 1 0 0 0 2 1 1 2 5 1 6 1 4 3 5 3 2 4 3 2 3 1 3 6 6 6 6 5 2 6 6 7 7 7 7 7 7 7 7 7 7 7 8 8 8 8 8 8 8 8 8 8 8 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 3 1 4 0 2 4 3 2 1 4 1 4 3 0 0 3 5 1 2 2 2 1 1 0 0 1 1 3 1 1 0 0 0 1 0 1 1 2 2 2 2 1 1 2 0 1 1 0 4 0 1 3 1 1 0 1 1 3 1 2 1 0 1 1 0 1 1 0 1 0 4 3 2 0 1 1 2 1 3 1 0 1 1 1 1 0 0 2 2 1 1 2 1 0 0 2 1 0 2 1 0 1 1 0 2 1 1 0 0 0 1 2 2 1 0 1 1 1 0 0 0 1 1 1 1 1 1 2 0 0 1 1 0 1 0 0 0 1 2 4 1 2 3 4 6 2 3 4 3 5 0 1 3 3 5 6 7 0 2 7 1 7 7 8 8 8 8 8 8 8 8 8 8 8 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 0 1 0 3 1 3 1 2 4 1 2 4 3 3 1 2 3 2 4 3 3 1 3 3 2 1 0 1 3 0 0 1 2 0 2 0 1 1 3 1 4 1 2 1 2 1 0 0 1 4 1 0 2 1 1 2 1 1 1 4 0 3 3 0 2 0 2 2 1 1 0 0 1 1 2 1 1 0 3 1 3 1 1 1 0 3 3 1 1 1 0 3 1 1 0 0 1 1 0 3 1 1 0 1 1 1 2 2 0 0 0 2 2 2 0 0 2 1 1 1 1 0 1 0 3 0 2 1 2 1 0 2 1 1 1 1 2 0 1 1 0 0 1 1 0 2 2 2 1 1 2 4 1 3 2 3 3 2 2 3 0 1 7 0 1 6 4 8 1 4 1 8 8 8 8 8 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 2 2 4 5 1 0 3 3 1 4 3 3 4 1 0 0 3 0 0 3 2 1 3 1 3 3 3 1 1 1 2 1 1 3 0 1 2 2 1 2 1 1 0 1 2 2 0 1 2 1 2 1 1 0 2 1 0 1 0 0 0 1 1 0 0 1 1 1 0 1 1 1 1 1 1 0 0 0 1 0 0 0 1 1 2 3 0 1 1 0 1 3 0 1 1 1 1 0 1 1 0 1 0 0 1 1 1 1 0 1 1 0 0 1 1 1 0 2 0 0 0 0 1 0 1 0 0 0 1 1 0 1 0 0 1 1 0 0 1 1 1 1 0 1 1 1 1 1 1 0 1 2 0 0 1 0 1 3 2 1 2 2 2 1 1 1 5 6 6 1 5 4 1 1 1 8 9 1 9 4 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 3 5 5 5 4 3 0 3 1 2 2 4 4 3 3 3 2 0 3 0 3 1 2 2 3 2 2 2 2 1 0 0 1 1 3 1 1 0 1 1 0 0 2 0 0 0 1 1 2 2 1 2 1 1 1 0 1 1 1 0 0 1 4 0 2 1 0 2 1 0 1 0 0 1 1 1 0 0 0 1 1 2 0 1 1 1 1 1 1 2 1 0 1 0 1 0 1 2 0 0 1 1 0 0 0 1 0 0 1 1 0 0 0 1 1 0 0 1 2 0 2 1 0 3 2 2 0 1 1 1 1 2 0 0 0 1 1 0 2 1 0 1 1 1 2 1 1 0 1 1 2 0 1 0 0 0 1 0 1 1 0 0 1 0 1 2 1 2 3 2 2 2 1 2 8 2 3 1 0 5 7 1 5 0 2 9 1 0 3 5 3 1 9 9 9 9 9 9 9 9 6 6 6 6 5 1 0 3 3 2 2 6 0 6 1 0 0 6 3 5 2 2 0 1 0 1 1 1 0 2 4 2 1 0 1 4 0 4 1 0 2 1 1 0 0 1 1 1 0 2 1 0 1 0 1 1 0 1 1 0 1 1 0 0 1 1 1 0 1 1 1 1 0 0 1 3 0 1 0 1 1 1 1 1 0 0 1 1 2 0 1 1 0 1 1 1 1 1 2 0 2 1 0 1 1 0 1 0 1 1 0 1 0 0 1 1 0 1 1 0 1 0 1 1 1 1 0 1 0 1 1 4 0 1 1 2 0 0 0 1 1 0 0 0 1 1 1 1 1 1 1 1 0 2 1 1 3 0 1 2 0 0 0 1 0 1 0 0 0 0 1 0 0 2 0 1 6 3 7 0 0 1 1 7 1 0 8 6 9 8 9 9 8 3 0 3 1 5 6 9 6 6 6 6 1 6 6 5 1 4 6 0 6 3 5 2 6 1 1 2 2 2 3 2 1 1 2 1 1 0 1 0 0 3 2 1 1 1 2 0 1 2 0 2 2 0 1 0 1 0 0 1 1 0 2 0 2 0 0 0 1 0 0 0 1 1 2 0 1 1 1 1 0 1 1 1 1 0 0 0 1 0 0 2 0 0 0 0 0 1 0 3 1 0 0 1 1 0 0 2 1 0 0 0 0 0 2 1 0 1 0 0 1 1 3 0 1 1 1 1 0 1 0 0 0 1 0 1 1 1 0 2 0 1 1 1 1 1 1 0 1 0 1 0 0 2 0 1 0 1 1 1 0 0 1 1 0 1 1 0 2 1 1 0 1 0 1 0 0 0 0 1 0 1 1 0 0 0 1 0 0 0 1 2 0 1 1 1 0 0 5 0 8 4 4 5 3 3 3 0 7 7 7 7 7 7 7 7 5 0 3 7 4 4 2 1 6 7 7 4 5 1 6 4 5 6 6 2 1 2 0 3 1 0 0 0 0 3 3 2 0 1 0 2 1 0 2 1 1 0 2 1 1 0 0 0 0 1 1 1 1 1 1 1 0 0 0 1 1 1 0 1 1 0 1 1 0 1 0 0 0 1 1 1 1 1 0 1 1 0 1 1 1 1 0 1 1 1 2 0 1 0 0 0 0 0 1 1 0 1 1 0 2 1 1 3 2 0 1 1 0 2 0 0 0 2 1 0 1 0 2 0 1 1 0 1 0 0 3 1 1 0 1 0 0 0 0 0 1 0 1 1 1 0 0 0 1 0 1 0 0 0 0 1 2 0 0 0 0 1 2 0 1 0 1 0 0 0 1 0 0 0 0 1 1 0 0 0 0 0 0 0 1 1 2 2 2 2 1 4 7 7 7 7 7 7 7 7 7 7 7 4 7 7 7 2 4 4 0 1 2 6 7 2 7 6 7 5 3 6 7 2 2 1 4 2 1 2 0 1 0 0 0 2 1 2 1 2 2 1 1 1 0 0 1 0 0 1 1 2 0 1 1 0 2 1 1 1 1 0 1 1 1 0 1 0 1 0 1 0 1 1 0 2 1 1 0 1 2 2 1 0 1 1 1 2 0 1 0 1 0 1 2 1 0 1 1 1 0 0 0 1 2 1 0 2 0 0 1 3 1 1 1 1 1 2 0 0 0 0 1 1 1 0 0 0 2 0 1 0 1 2 1 0 1 1 2 0 1 1 2 2 0 0 1 0 1 1 0 2 0 0 1 0 2 1 1 2 1 0 0 0 1 0 0 0 0 0 1 0 1 1 0 0 0 0 1 1 0 0 0 0 0 0 0 0 1 1 0 0 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 0 0 1 2 1 4 2 7 8 6 2 7 4 3 5 3 3 0 3 1 0 3 1 0 1 0 1 1 1 0 1 1 0 0 1 1 0 1 1 1 0 0 1 0 0 1 0 0 0 1 1 0 1 2 1 0 0 0 0 1 1 0 1 0 1 0 0 0 0 0 0 0 1 1 0 0 0 1 0 0 1 2 0 1 2 1 0 1 0 0 1 0 1 0 0 1 0 0 0 0 1 0 0 0 1 1 0 2 1 1 0 1 0 0 1 0 1 0 1 1 0 0 0 0 0 1 0 0 0 1 0 2 1 0 1 0 1 2 0 1 1 1 1 1 0 0 1 1 1 0 0 0 1 1 1 1 1 0 1 0 0 0 1 0 1 0 1 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 2 8 2 4 4 6 2 8 1 1 2 3 1 2 2 2 3 1 0 1 1 1 1 0 2 0 0 1 1 1 1 1 0 1 1 0 1 0 0 1 2 1 0 0 1 3 0 1 1 0 0 0 0 0 1 1 1 1 0 1 1 0 0 0 1 1 0 1 1 1 0 0 0 0 0 1 1 1 0 1 1 0 0 0 1 1 0 1 1 3 0 0 0 1 1 2 1 1 0 0 0 1 1 2 0 0 0 2 1 2 2 0 1 1 1 0 2 1 0 0 1 0 0 0 0 1 0 1 0 0 0 1 1 0 0 0 0 1 0 1 0 1 1 0 0 1 0 0 0 1 0 0 0 1 1 0 1 2 0 0 1 0 1 0 1 0 0 0 0 1 1 1 0 0 0 1 0 0 0 0 BookingsEmpty Full Timetodeparture QL Fares vs. Optimal Fares Toy scenario Fenceless Fare Structure 10 seats 3 price points
  9. 9. 9 Deep Reinforcement Learning Q-Learning each observation Deep Q-Learning Deep Neural Network (3; 100, 100, 100; 1) 𝑥𝑓 𝑡 each observation 𝑄 𝑥, 𝑡; 𝑓 ← 𝛼 𝑓 + 𝑀𝑎𝑥 𝑓′ 𝑄 𝑥′, 𝑡 + 1; 𝑓′ + 1 − 𝛼 𝑄 𝑥, 𝑡; 𝑓 𝐿 𝜃 = 𝑓 + 𝑀𝑎𝑥𝑓′ 𝑄 𝑥′, 𝑡 + 1; 𝑓′; 𝜃 − 𝑄 𝑥, 𝑡; 𝑓; 𝜃 2 𝑥 𝑡 𝑓 𝑄(𝑥, 𝑡; 𝑓; 𝜃) 𝛼
  10. 10. 10 ©AmadeusITGroupanditsaffiliatesandsubsidiaries Deep QL 5-13Y of flight data ~ 99% of opt. rev Normal QL 165kY of flight data ~ 97% of opt. rev Compare performance of QL vs. DQL in Monopoly RMS optimal=100% QL ~ 97.0% 80% 90% 100% 110% 20 30 40 50 60 Revenue Departures (millions) 80% 90% 100% 110% 1000 2000 3000 4000 5000 Revenue Departures RMS optimal=100% DQL ~ 99.3% Scenario Fenceless Fare Structure 200 seats 10 price points 12.000 times faster
  11. 11. Strictly CONFIDENTIAL Competition ©AmadeusITGroupanditsaffiliatesandsubsidiaries
  12. 12. ©AmadeusITGroupanditsaffiliatesandsubsidiaries Simulation set-up - network AL1 AAA BBB AL2 AAA BBB • Fenceless fare structure • 10 price points • CAP=50 • High demand (Volume/Cap=1.5) • Two customer segments: Business and Leisure pax (mix=50/50) • Input frat5 curve • AL1: RMS(DP), DQL • AL2: RMS(DP), DQL
  13. 13. 13 ©AmadeusITGroupanditsaffiliatesandsubsidiaries DQL calibration Initialization with historical data 1000 departure dates - DQL300 departure dates - RMS Poor Quality! Warm-up Benchmarking DQL Initialization
  14. 14. ©AmadeusITGroupanditsaffiliatesandsubsidiaries DQL calibration Initialization with RMS Bid Prices Supervised Learning 1000 departure dates - DQL RMS 𝑩𝑷(𝒙, 𝒕) table 𝑸(𝒙, 𝒕; 𝒇) table DQL Policy ˜ RMS Policy Benchmarking 300 departure dates - RMS Warm-up DQL Initialization
  15. 15. Performance of RMS vs. RMS 99,8 100,1 Revenue index DCP Class Booking Class Mix Price evolution DCP Booking evolution 0 2 4 6 8 10 12 14 1 2 3 4 5 6 7 8 9 10 RMS RMS 0 10 20 30 40 50 0 5 10 15 20 RMS RMS 50 100 150 200 0 5 10 15 20 RMS RMS 70 80 90 100 110 120 0 200 400 600 800 1000 RMS RMS Depdates PaxPax
  16. 16. 0 5 10 15 20 25 1 2 3 4 5 6 7 8 9 10 DQL RMS Performance of DQL 84.32 105.36 Revenue index Class Booking Class Mix Depdates DQL vs. RMS Pax 70 80 90 100 110 120 0 200 400 600 800 1000 DQL RMS Pax DQL vs. DQL Booking Class Mix Revenue index Class 0 2 4 6 8 10 12 14 1 2 3 4 5 6 7 8 9 10 DQL DQL 92.18 Depdates 91.18 70 80 90 100 110 120 0 200 400 600 800 1000 DQL DQL
  17. 17. Performance of RMS vs. RMS; DQL vs. RMS; DQL vs. DQL Revenue wrt baseline Booking Class Mix Price evolution 5,36% -8,32% -15% -10% -5% 0% 5% 10% DQL vs. RMS DQL vs. DQL 50 100 150 200 0 5 10 15 20 RMS vs RMS DQL vs. RMS DQL vs. DQL 0 5 10 15 1 2 3 4 5 6 7 8 9 10 RMS vs RMS 0 5 10 15 1 2 3 4 5 6 7 8 9 10 DQL vs. RMS 0 5 10 15 1 2 3 4 5 6 7 8 9 10 DQL vs. DQL RMS vs. RMS DCP Class Pax
  18. 18. 18 Conclusion − Reinforcement Learning (RL) opens the door to a radical new approach • Model free - no forecasting and no optimization • Leans by direct price testing − Shown that QL and DQL converges to RMS optimal solution in monopoly − Why does DQL vs. DQL underperform? • Hypothesis: Both ALs agree that WTP is increasing in traditional RMS? − Next steps • Apply to networks • Add more information to the state (e.g., competitors and market) *) Nash equilibrium RMS DQL RMS 100; 100 84.2; 105.4 DQL 105.4; 84.2 91,7*; 91.7*

×