1. AABI Town Hall Meeting
2015 Pilot Source Study Results
February 24, 2016
2. Overview of Meeting
• Introduction – Dr. Elizabeth Bjerke
• Background PSS 2010/2012 – Dr. Mary Niemczyk
• Literature Review – Dr. Cody Christensen
• Methodology – Dr. Guy Smith
• Pilot Background Comparisons – Dr. Elizabeth Bjerke
• Post-Law Pilot Outcomes – Dr. Guy Smith
• Part 135 results – Jenna Ludwick
• Pilot Outcomes Comparison – Dr. MaryJo Smith
• Significance of Study for AABI – Dr. Tom Carney
• Future Research – Dr. Paul Craig
• Recommendations/Conclusions – Dr. Guy Smith
2
3. Thank you to our Sponsors
• Alaska Air
• American Airlines
• Delta Air Lines
• Embry-Riddle Aeronautical University
• JetBlue Airways Corporation
• National Business Aviation Association
• United Airlines
• University of North Dakota
• Ypsilon Associates
3
4. Thank you to our Regional Airline
Participants
• Express Jet Airlines
• Horizon Air
• CommutAir
• Envoy
• Endeavor Air
• Piedmont Airlines
• PSA Airlines
• GoJet Airlines
• Compass Airlines
• Trans States Airlines
• Seaborne Airlines
• Great Lakes Aviation
• Republic Airlines
• Shuttle America
• Empire Airlines
• Mesa Airlines
• Air Wisconsin Airlines
• SkyWest Airlines
• Cape Air Airlines
• RAVN Alaska – Corvus
• RAVN Alaska – Hageland
• Silver Airways
4
5. Industry/Academia Collaboration
• Research Efficiencies
through Partnerships
• Sponsorships
assisted in travel
expense
• Universities assisted
in human resource
expense
5
6. Thank you to our Graduate
Student Assistants
• Data Collection:
• Jenna Ludwick – Embry-Riddle
• Nancy Shane – Farmingdale State College (SUNY)
• Crystal Ferguson – Embry-Riddle
• Kiran Chilakalapudi – Arizona State
• Nicolle Huang – Westminster College
• Data Analysis
• Matthew Price – Purdue
• Allen Xie – Purdue
• Ryan Bearden – Middle Tennessee
6
9. Pilot Source Study
Background
• Colgan Air 3407 February 12, 2009
• Are Part 121 First Officers adequately prepared to fly at
the regional airlines?
9
10. Pilot Source Study
Background
• FAA Issued a ANPRM
• New Pilot Certification Requirements for Air
Carrier Operations
• Question 2A: Are aviation/pilot graduates
from accredited aviation university degree
programs likely to have a more solid
academic knowledge base that other pilots
hired for air carrier operations?
• Commenced Pilot Source Study 2010 at
AABI Winter Meeting 2010
10
11. Pilot Source Study 2010
• Research the success of new pilot
indoctrination for FO in Part 121 operations
• What were the characteristics of pilots who were
hired by the US regional airlines between 2005-
2009?
• How did these characteristics relate to their success
in regional airline training programs?
11
12. Pilot Source Study 2010
• Six regional airlines
• American Eagle
• Atlantic Southeast
• Cape Air
• Horizon Air
• Mesa Airlines
• Trans State Airlines
• Target Population
• Pilots who entered
flight training between
2005-2009
• Convenience sample
• 2,156 pilots
12
13. Pilot Source Study 2010
Results
• Pilots who had the following characteristics
experienced fewer extra training events and
non-completions
• Graduated from an AABI-accredited flight program
• Aviation degree
• Advanced flight training in a collegiate program
• CFI
• 501-1,000 total flight hours
13
14. Pilot Source Study 2012
• February, 2012 FAA issues a NPRM that would
require FO in part 121 operations to hold an
ATP certificate and type rating for the aircraft to
be flown
• Also, FAA proposed following pilots with an
aviation degree or military flight experience
with fewer than 1,500 total flight hours to obtain
an ATP certificate with restricted privileges
• A primary purpose was to acquire additional
data for response to the NPRM
14
15. Pilot Source Study 2012
• Same research design as 2010 study but with
seven different regional airlines
• Because the researchers were all from
universities with AABI accredited flight
programs, an independent analysis of the data
was conducted
15
16. Pilot Source Study 2012
• Regional Airlines
• ExpressJet Airlines
• SkyWest Airlines
• Air Wisconsin Airlines
• Atlantic Southeast
Airlines
• Shuttle America
• Chautauqua Airlines
• Republic Airways
• Target population
• Pilots who entered
flight training between
2005-2011
• Convenience sample
• 4,024 pilots
16
17. Pilot Source Study 2012
Results
• Results for the 2012 study are consistent with
those from the 2010
• Pilots who had the following characteristics
experienced fewer extra training events and
non-completions
• Graduated from an AABI-accredited flight program
• Aviation degree
• Advanced flight training in a collegiate program
• CFI
• 1,001-1,500 total flight hours
17
18. PL 111-216/FOQ rule
• August 1, 2010: US Congress – Public Law 111-216
• Airline Safety and Federal Aviation Administration
Extension Act
• July 15, 2013: FAA – FOQ Rule
• Pilot Certification and Qualification Requirements for Air
Carrier Operations
• August 1, 2013: Effective date for the FOQ Rule
• Jan 2015: Pilot Supply Summit, Daytona Beach, FL
Pilot Source Study 2015
18
20. How does a
pilot go from
primary
training to
becoming a
first officer
in a Part
121 air
carrier?
20
Primary
Training
Experience
Airline
Success
21. Primary Training
• Where did they complete their primary training?
• College
• AABI-Accredited Flight Program
• Academies
• Part 61
• Military
• FBOs
21
23. Types of FAA Airline Transport
Pilot Certificates
• Traditional ATP
• 1,500 hrs
• 23 years old
• Restricted ATP (R-ATP)
• 1,250 hrs & 30 credits
• 1,000 hrs & 60 credits
• 750 hrs Military
• 21 years old
23
24. Air Carrier Pilot Training
24
FO
Prequalification/Interview
Initial Training
Operating Experience
Recurrent
Success
26. PSS 2015 – Background
•Pre-Law
• 2010 HR 3371
– Pilot Source Study 2010
• FOQ ARC – NPRM
– Pilot Source Study 2012
•Post-Law
• Pilot Supply Summit, Daytona Beach, FL (Jan
2015)
– Pilot Source Study 2015
• Conceived as a sampling study
• re-visit the airlines in PSS 2010 and 2012
26
27. Pilot Source Study 2015 – Purpose:
What is the effect of PL 111-216
and the FOQ Rule on pilot hiring
and pilot training in US regional
airlines?
27
29. RAA Annual Convention
• BRIEFINGS
• RAA Board Of Directors
• Regional Operations Council
• Flight Training Committee
• Airlines Want To Be Part Of
This Study
• Result: A Population
Study (All US
Regional Airlines)
29
30. 2015 PSS – a Population Study
30
2010/2012 PSS New to 2015 PSS * Part 135 Operators in 2015 PSS
31. Non-Disclosure Agreement (NDA)
• De-identified Data
• Name
• ID Number
• Age
• Gender
• Ethnicity
• No Other Purpose
• Non-disclosure
31
32. Data Collection Protocol
32
Application/
Resume
(Name, etc.)
Training/ IOE/
Recurrent Data
(Name, etc.)
HR/Recruiting Training/Ops
Identified
Dataset
(Names, etc.)
DE-Identified
Dataset
To PSS
Names, etc.
To Airline
To Airline
42. PSS 2015
42
Educational Background
College Degree
(Associate, Bachelor, Graduate)
(80%)
Non-
Aviation
Degree
(27%)
Aviation
Related &
AABI Flight
(23%)
Missing
Data
(2%)
Aviation
Related &
NOT AABI
Flight
(26%)
High
School
(18%)
43. Degree Year Analysis
(PSS 2015 Only)
• Degree Year
• 41% graduated in the
last five years
• 59% graduated more
than five years ago
• 21% graduated more
than 15 years ago
43
46. Military Pilot Training Comparison
46
Significant
Difference
χ2(1) = 381, p < 0.001
Military Pilot Pre-Law Post-Law
No 5,907 97% 5,788 88%
Yes 165 3% 778 12%
50. Multi-Engine Hours
• Mean is declining each year
• 20% of pilots had less than 100 hours of ME time
• Majority of these pilots qualified for R-ATP under
Institutional Authority
50
57. Extra Recurrent Training (8%)
(Mostly AQP Airlines)
57
Number of Extra
Recurrent Training Events Count
One 215
Two 89
Three 37
Four 14
Greater than Four 12
64. 64
COMPLETIONS based on Educational Background – 84% (N = 5,487)
Non-Aviation
Degree – 81%
N = 1,800
No College Degree – 78%
N = 995
48%
56%
65. 65
NO Extra Training Events based on Educational Background – 62% (N =
5,091)
Non-Aviation Degree
57%
N = 1,398
No College Degree OR
College Degree Unavailable
55%
N = 1,140
26%
20%
73. 73
LESS Extra
Recurrent
LESS Extra
IOE
LESS Extra
Training
FEWER Non
Completions
Completions
↓ Instrument
↓ XC
↓ PIC
↓ SIC
↓ ME
↓ Turbine
↓ Dual-Given
↓ Total Time
Piloting Hours
↓ Instrument
↓ XC
↓ PIC
↓ SIC
↓ ME
↓ Turbine
↓ Dual-Given
↓ Total Time
↓ Instrument
↑ XC
↓ PIC
↑ SIC
↑ ME
↑ Turbine
↓ Dual-Given
↑ Total Time
↑ Instrument
↑ XC
↓ PIC
↓ SIC
↑ ME
↓ Turbine
↓ Dual-Given
↑ Total Time
80. Part 135 Data
80
3 part 135 airlines
339 pilot records
Same source variables as
Part 121 Data
Performance Variables
Training Completions
Extra Training Events
Extra Recurrent
IOE Time
90. CFI Certificate
No CFI Certificate
CFI Certificate
90
Significant Outcomes
CFI Certificate - More Extra Training p < .05
CFI Certificate - More Extra Recurrent p < .05
91. Flight Hours
91
Significant Outcomes
≤ 500 Less Extra Training p < .05
> 1500 Less Extra Recurrent p < .05
< 1500
> 1500
Min to 500 500-
1000
1000-1500 1500 to max
92. Other Significant Findings
• Pilots who graduated prior to 2010 had more
extra training events
• Pilots with less than a 3.0 GPA had more extra
training events and recurrent training
• Pilots with higher XC time had less extra
recurrent training
• Pilots with higher PIC time had less extra
recurrent training
92
93. SUMMARY: Background
Variables Effect on Extra Training
Significant Background Variable Effect Size
AABI Flight – Yes .226
GPA - < 3.0 .212
Graduation Year – Prior to 2010 (older) .189
Flight Hours - < 500 .164
Previous Employment – Flight Instructor .146
Highest Degree – High School .146
93
95. Pilot Outcome Comparison
Between
Pre-Law data 2010 & 2012
& Post-Law data 2015
Note: 2015 data was summarized into the
(fewer) categories that were used in the
2010 and 2012 data collection.
95
97. What do the arrows mean?
Red is Negative Effect
Green is Positive Effect
97
98. Outcome Variable: Extra Training Events
Significant Differences
98
Extra Training: Post-Law pilots required more extra training.
Brown-Forsythe test for unequal variances: F(1, 7,528) = 240.52, p < .001
99. Pre-Law/Post-Law Extra Training
Events with College Degree
99
College Degree lost some of its effect on Extra Training between Pre-Law & Post-Law.
N = 11,090, χ2(6) = 37.953, p < .001; Cramer’s V = .041, p < .001
100. Pre-Law/Post-Law Extra Training Events with:
AABI Flight Degree / Aviation Degree
100
AABI Flight Degree and Aviation Degree lost some of its effect on zero Extra Training
between Pre-Law and Post-Law.
AABI Flight Degree and Aviation Degree increased its effect on fewer Extra Training
between Pre-Law and Post-Law.
AABI Flight Degree: N = 9,293, χ2(3) = 165.882, p < .001; Cramer’s V = .134, p < .001
Aviation Degree: N = 9,105, χ2(3) = 242.788, p < .001; Cramer’s V = .163, p < .001
101. Pre-Law/Post-Law Extra Training
Events with CFI Certificate
101
CFI Certificate lost some of its effect zero Extra Training between Pre-Law & Post-Law.
CFI Certificate increased its effect of fewer Extra Training between Pre-Law & Post-Law.
No CFI Certificate had more Extra Training between Pre-Law & Post-Law.
N = 11,186, χ2(3) = 58.93, p < .001; Cramer’s V = .073, p < .001
102. Pre-Law/Post-Law Extra Training
Events with Total Hours
103
Total Hours: The additional Total Hours required by the legislation is not beneficial to
regional airline training. As Total Hours increased, the number of extra training events also
increased – the result is a significant positive correlation.
N = 11,008, F(3, 11,004) = 348.21, p < .001
Spearman’s rho = .256, p < .001
103. Pre-Law & Post-Law
Completed Training
104
Completions: The Post-Law pilots had a 9.87% increase in non-completions.
N = 11,584, χ2(1) = 281.991, p < .001, Φ = -.156, p < .001
104. Pre-Law/Post-Law Completed
Training with College Degree
105
Bachelor’s Degree had fewer non-completions between Pre-Law & Post-Law.
No Degree or Associates Degree had more non-completions between Pre-Law & Post-Law.
Associates Degree had fewer completions between Pre-Law & Post-Law.
N = 11,473, χ2(6) = 82.349, p < .001; Cramer’s V = .060, p < .001
105. Pre-Law/Post-Law Completed Training with
AABI Flight Degree / Aviation Degree
106
AABI Flight Degree and Aviation Degree had fewer non-completions between Pre-Law &
Post-Law. Non-Aviation Degree had more completions between Pre-Law & Post-Law.
No AABI Flight Degree and Non-Aviation Degree had more non-completions between
Pre-Law & Post-Law. AABI Flight and Aviation Degree had fewer completions between
Pre-Law & Post-Law.
AABI Flight Degree: N = 9,691, χ2(3) = 184.064, p < .001; Cramer’s V = .138, p < .001
Aviation Degree: N = 9,492, χ2(3) = 301.753, p < .001; Cramer’s V = .178, p < .001
106. Pre-Law/Post-Law Completed
Training with CFI Certificate
107
Being a CFI increased the effect on having fewer non-completions between Pre-Law &
Post-Law.
Not being a CFI increased the effect on having more non-completions between Pre-Law &
Post-Law. The effects of the CFI Certificate were eliminated for completions!
N = 11,584, χ2(3) = 108.414, p < .001; Cramer’s V = .097, p < .001
107. Pre-Law/Post-Law Completed
Training with Total Hours
109
Total Hours: The additional Total Hours required by the legislation is less beneficial to
regional airline training for Post-Law pilots. As the number of Total Hours increases, so do
the proportion of non-completions – the result is a significant positive correlation.
N = 11,398, F(3, 11,394) = 454.31, p < .001
Spearman’s rho = .228, p < .001
108. Pre-Law/Post-Law Completed
Training with Extra Training Events
110
Significantly more extra-training events were required between the Pre-Law & Post-Law.
Pilots failing to complete training required more Extra Training before they terminated
training.
Pilots completing training had fewer Extra Training than pilots not completing training.
N = 11,055, F(3, 11,051) = 290.19, p < .001
109. Cost for a 9.87%
Decrease in
Completions
111
Additional 9.87% of pilots in Post-Law data = 544 pilots
Average Salary and Benefits (from 7 airlines) $43,081.71
Average pilot terminates in 53 work days: Pro-rated salary for 2.65 months $9,514
Salary cost of the additional Non-Completers $5,179,888
Training Accommodation Costs ($106/day) $4,667,528
Pro-rated airline costs per pilot = $20,345 * 544 pilots $11,077,100
Total estimated cost of the additional non-completions = $20,924,516
Approximate cost/pilot terminating training: $38,464
110. Cost of the 3,071
Extra Training Events
112
Average cost of an additional training event: $1,336
Total estimated cost of the additional extra training events: $3,960,745
111. Total Additional Cost to Regional Airlines
113
Total estimated cost of additional non-completions
$20,924,516
Total estimated cost of additional extra training events
$ 3,960,745
Grand Total = $24,885,263
Approximate cost per terminated pilot
$45,745
113. • So — What does all of this mean:
• For AABI?
• For the Aviation Industry?
115
114. It’s Time to Be Bold!!!
And we now have the data and analyses to support what
most of us here have long believed.
116
115. Probability of Success
• This research clearly suggests that graduates
of AABI-accredited flight programs are more
likely to be successful in air carrier new-hire
training programs and flight-line qualification.
• The study results should be well-publicized to the
Congress, FAA, Collegiate Aviation Programs and
their administrators, and Industry Partners
117
116. Reaching Out
• AABI should aggressively use the research
results from this study to reach out to non-
accredited collegiate flight programs and
encourage them to pursue accreditation.
• There is demonstrated value in AABI Accreditation
• It’s true—AABI Accreditation is “worth it”
118
117. I/E Partnership
• AABI should partner with industry to seek FAA
regulatory relief (in terms of number of flight
hours required for the R-ATP), for graduates of
AABI-accredited flight programs
• Or…could there be “something better”?
119
118. Something Better?
• The PSS results clearly show that “hours” is not
the variable to measure quality of preparation and
experience
Not a single aspect of any of the
three Pilot Source Studies has
shown that "hours" is a reliable
predictor of performance by
pilots
120
119. Something Better?
• It's time for the FAA to revisit the concept that
the FOQ ARC presented – structured training
programs
• AABI graduates performed well because their
degree programs are structured according to
Criteria.
• Criteria that are established through ongoing
Industry/Educator collaboration
121
120. In the Meantime
• AABI should partner with industry to seek FAA
regulatory relief (in terms of number of flight
hours required for the R-ATP), for graduates of
AABI-accredited flight programs:
• To at least the same relief afforded military
pilots for graduates of AABI-Accredited Flight
Programs, based on the results of this
research.
122
121. Preferential Hiring
• AABI should use the research results from this
study to advocate preferential hiring of AABI
flight program graduates by U.S. airlines.
• There is demonstrated value of an AABI-
Accredited education, in terms of training costs
and likelihood for success in initial training and
line qualification
123
122. What Really Counts
• Graduating from an AABI-Accredited Flight
Program was the only background variable that
gave us positive outcomes in all four of our
outcome variables:
• Fewer non-completions
• Less extra training
• Less extra IOE
• Less extra recurrent
124
123. The Business Case
• Airlines should utilize the results of this
research to review their recruiting and hiring
practices, in order to:
• Maximize potential for success of new pilots
• Minimize training costs
• It Just Makes Good Business Sense!!!
125
124. And One More Thought
• The PSS Created a Model for Successful
Research:
• AABI was the administrative manager
• Commissioning the study, soliciting resources, paying the
bills, and providing the venue for publicizing the results
• The aviation industry and several universities
provided the funding
• The regional airlines welcomed the researchers and
provided the data
• PSS researchers did the work; but they were able
to remain autonomous
• We did our work independent of AABI, any airline, or any
institution
126
125. And That’s Pretty Amazing!!
• Thanks to all of you, who helped make it
happen!!
127
138. Conclusions
• The pilot background characteristics are
significantly different between the Pre-Law and
Post-Law dataset
• Age matters (Graduation date is a proxy variable for
age)
• Recent graduates are more successful
• More second-career pilots in the Post-Law dataset
140
139. Conclusions
• After PL 111-216 airlines saw a decline in
successful completions (10%)
• Airlines are spending a substantial amount of
money on training that produces no return on
investment
• $25 million $0.00 ROI
141
140. Conclusions
• Background characteristics of Post-Law pilots
required the airlines to spend more resources:
• Increased training footprint
• Increased IOE time
• Increased training reduces overall revenue
142
141. Conclusions
• None of the three Pilot Source Studies has
shown that "hours" is a reliable predictor of
performance by pilots.
143
142. Recommendations
• Aviation Industry/Academia collaborative
research needs to continue
• Resources need to be identified
• Priorities need to be established
• Venues for dissemination need to be provided
144
143. Recommendations
• For research:
• A standard application process that creates a
standard database
• Maintaining detailed training records
145
144. Recommendations
• Need to recruit more students into AABI-
accredited programs
• Industry recognition of importance
• Resources available for recruitment
• Use the $25 million proactively
146
145. Recommendations
• Be aware of the unintended consequences
concerning flight instructor quality
• Motivation to flight instruct
• Industry perception of the role of a flight instructor
• A great pilot is not always a good instructor
147
146. Recommendations
• Multiengine issues and unintended
consequences of FOQ rule (50 hours multi)
• Need more qualified MEIs
• Currently no incentive for pilots to obtain a MEI rating
• Lack of MEIs is forcing a reduction in producing
multiengine-rated pilots
• Lack of PIC multiengine experience may require
additional hours in transition
148
147. Recommendations
• The Pilot Source Study provided the data to
make a case for AABI graduates to receive a
R-ATP at lower time
149
