Data driven modeling of systemic delay propagation under severe meteorological conditions
1. Complex World | Seminar 2013
Data-driven modeling of
systemic delay propagation
under severe meteorological
conditions
Pablo Fleurquin
José J. Ramasco
Victor M Eguíluz
@ifisc_mallorca
www.facebook.com/ifisc
http://ifisc.uib-csic.es - Mallorca - Spain
2. Outline
Motivation
Air-traffic data
Network & Cluster construction
Data Results
Model definition
Comparison: model – reality
Effect of large scale disruptions on the system
Conclusions
http://ifisc.uib-csic.es
3. Why is it important?
• Total cost of flight delay in US in 2007 was 41B dollars.
• Rich transport dynamics.
• Cascading failure.
4%
Air
Carrier
Delay
30%
25%
Aircraft
Arriving
Late
Security
Delay
0%
National
Aviation
System
Delay
41%
(http://www.transtats.bts.gov/)
Extreme
Weather
(http://www.eurocontrol.int )
http://ifisc.uib-csic.es
4. Database & network
Database:
Network:
• Airline On-Time Performance Data (www.bts.gov)
• Nodes: airports
• Edges: direct flights between airports
• Node attributes: average delay per flight
Ø Schedule & actual departure (arrival) times
Ø Origin & destination airports
Ø Airline id
Ø Tail number
• 2010 flights:
Ø 6,450,129 flights (74 %)
Ø 18 carriers
Ø 305 airports
http://ifisc.uib-csic.es
5. Cluster definition
Clusters:
• Formed by airports in problem
Ø average delay per flight > 29 min
• Must be connected (flight route between them)
• A group of airports connected by flights that
their average delay is higher than 29 minutes
Cluster(A(
size(4(
(
Cluster(B(
size(2(
(
http://ifisc.uib-csic.es
6. Cluster definition
Clusters:
• Formed by airports in problem
Ø average delay per flight > 29 min
• Must be connected (flight route between them)
• A group of airports connected by flights that
their average delay is higher than 29 minutes
Cluster(A(
size(4(
(
Cluster(B(
size(2(
(
http://ifisc.uib-csic.es
7. Largest daily cluster
Clusters:
• Formed by airports in problem
Ø average delay per flight > 29 min
• Must be connected (flight route between them)
• A group of airports connected by flights that
their average delay is higher than 29 minutes
Cluster(A(
size(4(
(
Cluster(B(
size(2(
(
• April 19, 2010
• Average delay per delayed flight:
Ø 16.9 min
http://ifisc.uib-csic.es
8. Largest daily cluster
Clusters:
• Formed by airports in problem
Ø average delay per flight > 29 min
• Must be connected (flight route between them)
• A group of airports connected by flights that
their average delay is higher than 29 minutes
Cluster(A(
size(4(
(
Cluster(B(
size(2(
(
• March 9, 2010
• Average delay per delayed flight:
Ø 25.7 min
http://ifisc.uib-csic.es
9. Largest daily cluster
Clusters:
• Formed by airports in problem
Ø average delay per flight > 29 min
• Must be connected (flight route between them)
• A group of airports connected by flights that
their average delay is higher than 29 minutes
Cluster(A(
size(4(
(
Cluster(B(
size(2(
(
• March 12, 2010
• Average delay per delayed flight:
Ø 53.2 min
http://ifisc.uib-csic.es
10. B
120
0
10
100
P(>size)
A
Largest cluster size
ongestion and consequently delays are propagating through connected airports in
Cluster size
n intra-day time period.
80
60
40
-1
10
-2
10
slope ~ -0.0496
Characteristic Size: 20.1
20
0
0
60
120
180
Day
240
300
360
-3
10 0
20
40
60
80
100
120
Largest cluster size [Airports]
Figure 3.13. (A) Daily size of the largest cluster. (B) Complementary cumulative distribution ofvariety of the largest cluster (log-normal scale).
the size
• Great
• Consecutive days are very different each other.
Taking into account all days of 2010 the largest connected cluster size is exlored as a function of the day (Figure 3.13 A). A strong variability is thus the
http://ifisc.uib-csic.es
12. Cluster composition
B
1
Jaccard Index
Jaccard Index
A
0.8
0.6
0.4
0.2
0
0
60
120
180
Day
240
300
360
0.6
Top 20 (best days)
Top 20 (worst days)
0.4
0.2
0
0
5
10
Ranking
15
• Jaccard Index:
• Great variety
• Consecutive days are very different each other
• For consecutive days not only they differ in the cluster size also the airports comprising the
cluster are different.
http://ifisc.uib-csic.es
20
13. addition to rotational reactionary delay, the need to wait for load, connecting
ssengers and/or crew from another delayed airplane from the same fleet (airline
) may cause, as well, reactionary delay.
Ø Flight rotation (same tail number)
Model definition
Ø Airport Congestion
March 12
100
Ts
Scheduled
departure time
Figure 4.4: Possible connections within flights of the same airline.
Actual arrival time
Flight A
Actual departure time
Flight A
SAAR
Scheduled
arrival time
80
ATL
ORD
DEN
60
40
20
4a
m
6a
m
8a
m
10
am
12
am
2p
m
4p
m
6p
m
8p
m
10
pm
12
pm
2a
m
For each flight at a particularc airport, connections delay that airport are ranfrom
0
Inbound delay
Departure
mly chosen as follows. Firstly, we take a T window prior to the scheduled Example of SAAR for three major airports: Atlanta International A
Figure 4.6.
(ATL), O’Hare
Time Denver
parture time of the flight. Secondly, we distinguish possible connections of the International Airport (ORD) and [EST] International Airport (DEN
Scheduled turn around time
me airline from other flights, that have a scheduled arrival time within the 29T
Subprocesses
§ Schedule Airport Arrival Rate (SAAR)
§ Ts
ndow (Flights B and D in the example of Figure 4.4). Finally, from these possi§ First Arrived First served
e connections we§ Schedule (arrival/departure)
select those with probability ↵ ⇤ flight connectivity factor. The
When aircraft rotation and airport congestion is present the equation
The equation that govern the rotation subprocess is given by:
§ β
ght connectivity factor was defined in 3.1.2 and ↵ is an e↵ective parameter of
by:
ntrol that allowsT j modify the strength of); T j e↵ect) in Ts ] model. For instance, T j (p ) = max[T j (p ); T j (p ) + T j (p ) + T ]
to (pij ) = max[T j (pij this (pij + the
(4.2)
act.a
ij
s
act.d
sch.d
act.a ij
q ij
sch.d ij
= 0 means that there is no connection between flights with di↵erent tail number, act.d
hile ↵ = 1 makes the fraction of connecting flights of the same airlinewhere q means the time spent by the aircraft in the queue waitin
equal to
where j corresponds to destination airport and i to the origin one. The
e fractionØ Flight connectivity (different tail number)
of connecting passengers in the given airport. In the simulations, ↵ is the full model dynamics is govern by a combination of th
served. Finally,
ndexes act.d,act.a and sch.d correspond respectively to Actual Departure, AcInitial Conditions
Scheduled departure
ried according to the case under study and T is always taken to be 180 minutes
subprocesses:
Arrival and Schedule Departure. time
Actual departure time.
§ From the data…
hours).
Sch. arrival
Flight E | Airline X
Flight E | Airline X
j
j
j
T selected. By - j Known à when, Ts ; max[T jand the 0
| Airline X
t us suppose that from the previous example Flight D was randomly act.d (pij ) = max[Tsch.d (pij ); Tq (pij ) + Tact.a (pij ) +where act.a (pi0 j )]], 8i 6=
waiting time
.2 Flight connectivity
is subprocess an airplane is able to fly if and if only their connections have already
departure delay for the first flight of
Actual arrival time.
rived to the airport, if not it has to wait until this condition is satisfied (Figure
the sequence.
Flight D | Airline X
ddition to rotational reactionary delay, the need to only source of connecting
5). It is important to note that flight connectivity is thewait for load, stochasticity
Initial conditions
§ ΔT another delayed airplane from the same4.4 (airline
Departure
engers and/or crewlack of knowledge about the delay flight connections within the
fleet
the model due to a from
real
§ α Actual Departure time of the next flight leg is given § Random initial conditions…
may cause, as case the
hedule. In thiswell, reactionary delay.
by:
Initial condition refers to the initial delay (min)
- Fixed situation of the first flight of an aircraft se
j
j
j
j
Tact.d (pij ) = max[Tsch.d (pij ); Tact.a (pij ) + Ts ; max[Tact.a (pi0 j )]], 8i0meaning when, where% ofthe departure delay of planes
6= i
(4.3) - and initially delayed this flight. Variations
situation can have a great impact on the delay propagation. In other wor
dynamics of delays over the network is highly sensitive to the initial conditi
time. Flight D
We characterized initial conditions by http://ifisc.uib-csic.es for t
the average delay per flight
15. Data/Model comparison
Data and model comparison for March 12
and April 19
Data April 19
Model
40
1
20
am
4p
m
8p
m
12
pm
12
m
8a
m
4a
am
4p
m
8p
m
12
pm
12
m
0
8a
m
0
m
12
am
4p
m
8p
m
12
pm
Airport c
100 March 12
80
60
40
20
0
am
4p
m
8p
m
12
pm
3
2
60
4a
Cluster size
100 March 12
80
D)
m
C)
Connections
Good agreement between model and reality.
Time
http://ifisc.uib-csic.es
12
Time (EST)
0
m
m
12
am
4p
m
8p
m
12
pm
8a
4a
m
0
m
12
am
4p
m
8p
m
12
pm
4a
m
0
20
8a
20
40
8a
1
60
4a
40
100 March 12
80
m
2
60
8a
Cluster size
Data April 19
Model
Plane
4a
3
100 March 12
80
B)
Cluster size
Full model
Cluster size
A)
16. Data/Model comparison
20
0
4
4aa
m
m
8
8aa
m
m
12
12
aam
m
4p
4p
m
m
8p
8p
m
m
12
12
pm
pm
2
1
0
Time (EST)
D)
Time
Airport c
100 March 12
80
60
40
20
am
4p
m
8p
m
12
pm
0
m
http://ifisc.uib-csic.es
12
am
4p
m
8p
m
12
pm
am
4p
m
8p
m
12
pm
0
40
8a 44aam
mm
12 88aam
a1m m
12
2a
m
4p 44apm
mpm
m
8p 88ppm
mm
1
12
12 2ppm
m
pm
0
Data April 19
Model
m
20
3
60
4a
20
m
Airport congestion
80
8a
40
Data April 19
Model
1
0
Cluster size
60
Plane
0
100 March 12
Time (EST)
Cluster size
40
B)
4
4aa
m
m
8
8aa
m
m
12
12
aam
m
4p
4p
m
m
8p
8p
m
m
4a1122pp
m
mm
0
3
1
4
4aa
m
m
8
8aa
m
m
12
12
aam
m
4p
4p
m
m
8p
8p
m
m
12
12
pm
pm
Cluster size
4
4aa
m
m
8
8aa
m
m
12
12
am
am
4p
4p
m
m
8p
8p
m
m
12
12
pm
pm
1
2
60
12
Time (EST)
0
Connections
agreement between model and reality.
8a
m
12
am
4p
m
8p
m
12
pm
m
8a
4a
m
12
am
4p
m
8p
m
12
pm
m
8a
4a
0
40
20
Data April 19
Model
2
60
Time (EST)
m
1
0
3
100 March 12
80
Time (EST)
100 March 12
80
20
0
C)
Cluster size
40
0
Data April 19
Model
Good
2
60
20
0
1
m
3
20
2
12
Airport congestion
100 March 12
80
40
4a
Time (EST)
60
8a
0
4a
m
8a
m
12
am
4p
m
8p
m
12
pm
0
40
April 19
Plane rotation
100 March 12
80
Data April 19
Model D)
Model
m
20
Connections
3
2 Data
4
4aa
m
m
1
60
4a
40
4a
m
8a
m
12
am
4p
m
8p
m
12
pm
Cluster size
2
60
D)
Cluster size
Data April 19
Model
100 March 12
80
3
Time (EST)
4a
3
0
Full model
0
m Cluster size
8a 44
maam
m
12 88aa
m
a1m m
12
2aa
4p m
m
4
4
mppm
m
8p 88pp
m
mm
12 1122ppm
pm m
Plane rotation
100 March 12
80
20
100 March 12
C)
80
Cluster size
B)
1
4
4aa
m
m
8
8aa
m
m
12
12
aam
m
4p
4p
m
m
8p
8p
m
m
12
12
pm
pm
A)
40
B)
4a 88aa
m
mm
12
12
am
8a a m
4
mppm
4
12 88 m
amppm
m
4p1122ppm
mm
8p
m
12
pm Cluster size
Cluster size
A)
Full model
3
100 March 12
Data April 19
Data and model comparison for March 12
Model
80
2
and April 19
60
17. Data/Model comparison
Data and model comparison for March 12
and April 19
Data April 19
Model
40
1
20
am
4p
m
8p
m
12
pm
12
m
8a
m
4a
am
4p
m
8p
m
12
pm
12
m
0
8a
m
0
m
12
am
4p
m
8p
m
12
pm
Airport c
100 March 12
80
60
40
20
0
am
4p
m
8p
m
12
pm
3
2
60
4a
Cluster size
100 March 12
80
D)
m
C)
Connections
Good agreement between model and reality.
Time
http://ifisc.uib-csic.es
12
Time (EST)
0
m
m
12
am
4p
m
8p
m
12
pm
8a
4a
m
0
m
12
am
4p
m
8p
m
12
pm
4a
m
0
20
8a
20
40
8a
1
60
4a
40
100 March 12
80
m
2
60
8a
Cluster size
Data April 19
Model
Plane
4a
3
100 March 12
80
B)
Cluster size
Full model
Cluster size
A)
18. System resilience
• With random initial conditions…
March 12
α = 0.03
α = 0.1
April 19
• Each day is potentially a bad day, if some initial conditions are met.
• Flight connectivity is a key factor for the rise of congestion in the network.
• Sensitivity to initial conditions.
http://ifisc.uib-csic.es
21. External perturbation: variants
Ø What about the declining phase ?
• Baseline + …
• Baseline + …
• Baseline + …
• Connectivity drops to 0
between 7 pm & 9 pm EST.
• Connectivity drops to 0.13
between 7 pm & 9 pm EST.
• Connectivity drops to 0.13
between 6 pm & 10 pm
EST.
Time [EST]
Time [EST]
C)
Data
Baseline model
Variant 3
60
40
20
am
2p
m
4p
m
6p
m
8p
m
10
pm
12
pm
2a
m
4a
m
12
m
am
10
m
8a
m
0
6a
am
2p
m
4p
m
6p
m
8p
m
10
pm
12
pm
2a
m
4a
m
12
am
0
80
4a
20
m
am
2p
m
4p
m
6p
m
8p
m
10
pm
12
pm
2a
m
4a
m
am
12
m
10
m
8a
6a
4a
m
0
40
10
20
Data
Baseline model
Variant 2
60
m
40
B)
8a
60
80
m
Data
Baseline model
Variant 1
6a
A)
4a
80
Cluster size per hour
Variant 3:
Cluster size per hour
Variant 2:
Cluster size per hour
Variant 1:
Time [EST]
• Improve the matching.
• Make sense to interpret cancelation policies as a decrease on the network connectivity.
• Higher sensitivity to time period ΔTα .
http://ifisc.uib-csic.es
22. Effect of the schedule
• For comparison purposes: schedule of October 20.
§ This day showed a low level of congestion: largest cluster size of 2.
Data
Baseline model
Schedule: Oct 20
100
80
60
40
20
am
2p
m
4p
m
6p
m
8p
m
10
pm
12
pm
2a
m
4a
m
am
12
m
10
m
8a
4a
m
0
6a
Cluster size per hour
• Figure: Initial conditions of October 27 run using the schedule of October 20.
Time [EST]
• Schedule of October 27 was not the reason for the unfolding of the delays.
• Real intervention measures on October 27 were a palliative to the delay spreading mechanism.
http://ifisc.uib-csic.es
23. Conclusions
• We defined a way of measuring the network-wide spread of the delays
Ø Strong variability between days and intraday
• We introduced a model able to reproduce the cluster dynamics in the data
Ø Resilience of the system
Ø Non-negligible risk of system instability (systemic delay)
Ø Other transport modes
• Mimic external perturbations to the system.
Ø Perturbations could be model as a decrease in the airport capacity parameter.
Ø Intervention measures modeled as a decrease in the network connectivity.
Articles:
Ø P. Fleurquin, J.J. Ramasco, V.M. Eguiluz, “Systemic delay propagation in the US airport
network”, Scientific Reports 3, 1159 (2013).
Ø P. Fleurquin, J.J. Ramasco, V.M. Eguiluz, “ Data-driven modeling of systemic delay
propagation under severe meteorological conditions”, Tenth USA/Europe Air Traffic
Management Research and Development Seminar 2013.
Ø Spanish patent pending, filed December 14 2012, number P201231942.
Ø P. Fleurquin, J.J. Ramasco, V.M. Eguiluz, “Characterization of delay propagation in the airport
network”, submitted to Proceedings of the 2012 Air Transport Research Society Conference.
http://ifisc.uib-csic.es