VFR-into-IMC: An Analysis of Two Training Protocols on Weather-Related Posttest Scores
Journal of Aviation Technology and Engineering
VFR-into-IMC: An Analysis of Two Training Protocols on Weather-Related Posttest Scores
Julius C. Keller
According to the Aircraft Owners and Pilots Association Air Safety Institute, 264 accidents were identified as continued visual flight rules (VFR) into instrument meteorological conditions (IMC), during the past ten years. Approximately 89% of those VFR-into-IMC accidents were fatal, causing hundreds of deaths. VFR-into-IMC has been a major concern for the general aviation community, prompting focused efforts. Research, data analyses, outreach, training, and education are recommended practices to address risks associated with VFR-into-IMC. Researchers of the current study sought to evaluate the cause and effect relationship between two training protocols and weather-related posttest scores. A pretest-posttest experimental design was utilized at two testing locations. Participants were randomly assigned to one of three groups: a control group, an interactive online training group, or an interactive workshop group. An analysis of covariance was used to determine whether there was a significant difference between mean posttest scores among the experimental groups while controlling for pretest scores. The treatments did not appear to significantly increase posttest scores after controlling for pretest scores, at either experiment location. Though the results of this study did not yield anticipated findings, much was learned and potentially helpful to general aviation researchers seeking to mitigate VFR-into-IMC encounters. Recommendations for future research and practices are discussed.
VFR-into-IMC; general aviation; weather; aeronautical; decision-making; pilot training
According to a National Transportation Safety Board (NTSB) accident report
, on July 11, 2012, a Cirrus
SR20 impacted terrain and the pilot (the sole occupant) was fatally injured. The accident investigation revealed the
noninstrument rated pilot, who had approximately 340 total flight hours, departed without filing a flight plan from Millington,
TN, intending to fly to Pensacola, FL. There was no indication the pilot checked the weather. A person who spoke with the
accident pilot prior to his departure reported the pilot was in a hurry due to adverse weather in the area. Meteorological
information at the time and place of departure indicated variable wind at four knots, six statute miles visibility with mist,
overcast ceiling at 900 feet, temperature 24˚C, dew point 22˚C, and an altimeter setting of 29.99 in. Hg. Infrared satellite
images showed a large area of low stratiform clouds with
tops near 8000 feet along the route of flight. Additionally,
rain showers were reported east of the accident site
According to the accident report, a review of radar return
data indicated the accident airplane flew at a relatively
consistent altitude of about 1000 feet mean sea level (MSL)
for approximately 25 minutes after takeoff
Primary and multifunction display data were not
recovered from the aircraft wreckage, due to excessive damage.
However, enhanced ground proximity warning system data
were retrieved. The data analysis revealed that, during the
last 10 seconds, the rate of descent increased rapidly from
5000 feet per minute to 15,000 feet per minute.
Additionally, the bank angle varied from a 24 degree roll to the
left to a 28 degree roll to the right during the last 10 seconds.
No anomalies regarding the aircraft or pilot were found
during the investigation
The accident pilot was 48 years old and held a private
pilot certificate with an airplane single-engine land rating.
His medical certificate had no limitations. The probable
cause of the accident was attributed to the non-instrument
rated pilot’s decision to continue into instrument
meteorological conditions (IMC), which resulted in spatial
disorientation, loss of control, and subsequent impact into terrain
. This accident summary highlights many
factors associated with continued visual flight rules (VFR)
into IMC. Though it is impossible to conclusively assert the
reasons for the accident, evidence indicates there were
shortcomings in the pilot’s knowledge, skills, and abilities.
According to the Aircraft Owners and Pilots Association
Air Safety Institute (2015), between 2005 and 2015, 264
accidents were identified as continued VFR-into-IMC. Although
the occurrence of this accident type is relatively low when
compared to the approximately 1200 general aviation (GA)
accidents that occur each year, historically VFR-into-IMC
accidents have had an approximate 90% fatality rate, resulting
in hundreds of deaths. Primarily due to the high fatality rate,
VFR-into-IMC has been a major concern for the GA
community, prompting focused efforts. The Federal Aviation
Administration (FAA) is committed to reducing GA’s fatal accident
rate by 10 percent over the next ten-year period (FAA, 2015).
The FAA plan of action for improving safety includes: data
analysis, outreach and education, flight instructor training,
collaboration with industry, and establishing committees to
develop interventions based on research (FAA, 2015).
The study that is the basis for this paper was a
continuation (Phase II) of FAA-sponsored research to
investigate gaps in pilot training and knowledge as a part of
Project 4 (Weather Technology in the Cockpit) of the
Partnership to Enhance General Aviation Safety, Accessibility,
and Sustainability (PEGASAS). PEGASAS is the current
FAA Center of Excellence for General Aviation. During
the Phase I study of PEGASAS Project 4, researchers
distributed a national survey to GA pilots. Gaps in pilot
knowledge that pertain to VFR-into-IMC, identified during
Phase I, included: (1) the inability of pilots to correlate,
interpret, and apply weather information related to
VFRinto-IMC; (2) a perceived gap in skills related to
VFRinto-IMC decision-making; and (3) pilots’ inadequate
retention of weather knowledge
(Carney et al., 2015)
The purpose of the current study was to evaluate the
effectiveness of two VFR-into-IMC training protocols,
through a pretest–posttest experimental design. One training
protocol utilized an interactive online training module, while
the other incorporated a conventional pilot training
workshop. In addition to the two training protocols, a control
group was included as part of the research design. Data were
collected at two locations, as further discussed in the
methodology section. The researchers expected that either one or
both intervention groups would have significantly higher
posttest scores than the control group. This action research
gives the GA community a clearer understanding of the
complexities related to VFR-into-IMC training and risk
mitigation. The next section discusses extant VFR-into-IMC
research and justification for the instructional methods used
in this study.
Extant VFR-into-IMC Research
Typically, aviation incidents and accidents are caused by
multiple factors. Several research studies point to inadequate
training as a factor that may lead to a VFR-into-IMC
encounter. According to
, studies regarding weather and
VFR-into-IMC have found that pilots often misunderstand
weather, and/or do not acquire sufficient training on how to
use weather reports.
Considering that pilots who were involved in continued
VFR-into-IMC received a weather briefing, it points to
the need to focus on weather education as well as
hazardous pilot attitudes. Perhaps pilots are not heeding the
warnings of weather briefers or sources either because
they do not grasp the required knowledge to evaluate
weather reports and factors (or) they may feel as though
they are invulnerable due to having made the flight before,
overconfidence, or other negative attitudes. (p. 22)
Coyne, Baldwin, and Latorella (2008) asserted that pilots
found weather knowledge and the related decision-making
process to be the most challenging during flight training.
Pilots often felt there was a lack of support in this subject
area and were left on their own to gain a deeper
understanding of weather-related information. Additionally, the
flight training environment may have more restrictive weather
minimums, reducing pilots’ ability to experience weather
phenomena. Moreover, different regions of the United States
may limit opportunities to safely gain weather experience. It is
also possible that outside of the flight training environment
the lack of supervision from a flight instructor may increase
the risk of VFR-into-IMC encounters.
Knect and Lenz (2010) contacted and interviewed 100
pilots who submitted VFR-into-IMC reports to the Aviation
Safety Reporting System. Twenty-seven of the pilots were
non-instrumented rated and encountered IMC
inadvertently. After a thorough review of qualitative and
quantitative data, the researchers concluded these pilots had
the least amount of training, aeronautical knowledge, and
experience. Additionally, the non-instrument rated pilots
flew less capable aircraft, in terms of weather technology.
Based on the results, the authors suggested a need for
training in the areas of risk perception and strategies for
avoiding/handling an IMC encounter.
Wiggins and O’Hare (2003)
evaluated pilot time
required to develop alternative plans of action when handling
adverse weather conditions. Results indicated that the less
experience a pilot had, the longer it took to develop an
alternative course of action. Follow-up research indicated
cue-based training could improve time to make
(Wiggins & O’Hare, 2003)
Hunter, Martinussen, Wiggins, and O’Hare (2011)
investigated personal characteristics of pilots who survived
VFR-into-IMC encounters using pilot judgment, hazardous
events, and personal minimums scales. Demographic and
flight experience data were also collected during the study.
Usable data were collected from 364 participants.
Participants were categorized into three groups: 144 who
indicated they experienced flying into weather (both
deliberately and inadvertently), 114 who had conducted a
flight in which weather conditions were a concern, and 106
who reported no flights for which weather was entered or
was a major concern. The two former groups were used for
primary analyses. Notable results from the study led to a
discussion pertinent to pilot training. The authors suggested
an emphasis of educating pilots on the importance of using
a weather briefer, teaching them how to develop
conservative personal minimums, and developing more detailed
Previous research indicated the need for better training
to mitigate the risks of VFR-into-IMC encounters. Two
primary aspects come into play when attempting to enhance
current training practices: (1) selection of topics to include
in the training and (2) consideration of various instructional
delivery platforms. Based on an evaluation of previous
literature, two major topics arise: weather information and
aeronautical decision-making pertaining to VFR-into-IMC.
The next section explores research pertaining to
Justification for Selected Instructional Methods
Researchers of the current study utilized two training
protocols. One was an online module and the second was a
workshop. The two instructional methods were selected
because of promising potential or the need for further
research based on extant literature. A number of researchers
have investigated the utilization of online training
courses, due to their increasingly widespread use.
examined articles from an online database to
determine the effectiveness of web-based training modules
on learning. Forty-seven articles were used for the study.
Fifteen of the articles reported effect sizes when
comparing online courses to traditional classroom presentations.
Results indicated the average effect size was 0.24, meaning
the average posttest score increased 10 percentage points.
However, a limitation to the study was the small sample
size, with large variability. Effect sizes ranged from 20.4
to 1.6. Based on evidence provided by
Wisher and Olson
, computer-based learning may lead to an
improvement in learning effectiveness, when compared to
traditional classroom presentations. The study reports broad
categories of the fields of study, and did not report the
complexity of the topics taught.
Sitzmann, Kraiger, Stewart, and Wisher (2006), in an
examination of research articles, determined that the
effectiveness of online courses depends on the material taught.
Ninety-six research articles produced data from 19,331
trainees enrolled in 168 courses. The subject matter included
psychology, engineering, computer programming, business,
and technical writing. Participants in the studies were
employees and undergraduate and graduate students. Overall results
indicated web-based instruction was six percent more
effective than traditional classroom instruction, when teaching
declarative knowledge. When web-based instruction was
used to supplement face-to-face instruction, results indicated
a higher level of effectiveness in both declarative and
procedural knowledge. However, web-based training was not more
effective at teaching procedural knowledge.
With regard to aviation, few research studies have
examined the effectiveness of interactive online training
modules pertaining to weather. Knecht, Ball, and Lenz (2010)
evaluated video training products pertaining to aviation
weather-related knowledge and flight performance during
deteriorating conditions. Fifty general aviation pilots
participated in the study. Participants were assigned to one
of two groups. The first group watched a 90-minute video
that did not pertain to weather. The other group watched a
90-minute weather training video. Pretests and posttests were
administered. The researchers concluded that the training
videos were not effective, due to the complexity of weather
(Knecht et al., 2010)
. It may also be
possible that the lack of interaction while participants watched
the video contributed to its ineffectiveness. Researchers of the
current study chose to use an interactive online module to
provide additional evidence pertaining to the effectiveness of
this instructional method.
A workshop was the second instructional method used in
this study. Surprisingly, no research studies that evaluated
the effectiveness of VFR-into-IMC workshops have been
reported in the literature. Therefore, the literature from
other fields was examined. According to
and Stock-Ward (1999)
, a workshop is typically a highly
interactive session facilitated by an expert. ‘‘Workshop’’
and ‘‘seminar’’ are words often interchanged. Similar to a
workshop, a seminar can last from a half-day to two days
and is typically facilitated by an expert. The distinguishing
difference is that a workshop is more interactive. There is
communication among workshop participants and
facilitator, while a seminar typically has one-way
communications, from facilitator to participants. Furthermore, a
workshop tends to have interactive assignments embedded
into the program.
Learners are often enticed by face-to-face presentation
of training material and exchanges among participants. In
addition, instructional methods utilized during a workshop
can be wide-ranging. Mixtures of instructional methods
may improve student understanding of a subject, improve
communication, and positively affect different learning
styles. The execution of the appropriate instructional methods
may make the education process more effective
A study conducted by
examined pre- and
post-workshop surveys directed towards faculty members.
The primary purpose of the study was to determine whether
participant attitudes towards teaching practices would
change. Workshop topics included teaching large classes,
assessments, curriculum, supervising post graduates,
problem-based learning, and teaching in higher education.
Instructional methods included interactive components.
Five hundred faculty members responded to the survey.
Results provided evidence that workshops may promote
change in participants, provide encouragement, and increase
confidence in using desired teaching methods.
Occupational health researchers conducted a study to
evaluate interactive fatigue management workshops for
(Ali, Chalder, & Madan, 2014)
. Research survey
items asked participants how confident they were at
discussing, diagnosing, and managing chronic fatigue.
Additionally, participants were asked how satisfied they were
with the workshop. The questionnaires were distributed
prior to, directly afterwards, and four months after attendance.
Seventy-three participants completed all three questionnaires.
Results supported the concept that knowledge may be
improved by interactive workshops. Eighty-nine percent of
participants rated their experience between five and seven on a
seven-point Likert scale
(Ali et al., 2014)
Empirical evidence from multiple fields has shown that
workshops may be effective at changing knowledge, skills,
abilities, and attitudes. Previous research has shown that
well-planned workshops with lectures, reflective thinking,
discussions, and case studies may be conducive for deeper
learning. However, research pertaining to workshops and
complex aviation topics is still lacking. Research pertaining
to online training modules exhibits mixed results,
particularly with complex subject matter. It was the intent of the
researchers in this study to provide additional evidence
related to the effectiveness of online training modules, and
to compare online effectiveness with that of a conventional
workshop. The study sought to yield evidence to assist
researchers in the field to sort through the complexities and
continue to find beneficial mitigation strategies.
This study examined the effectiveness of two training
protocols on posttest scores at two separate locations, the
FAA William J. Hughes Technical Center (WJHTC) near
Atlantic City, NJ (Location 1), and Purdue University
Airport at West Lafayette, IN (Location 2). The researchers
utilized a pretest–posttest experimental design. All
participants completed intake procedures, which included a
briefing, review and signing of consent forms, completing
a pretest, and random assignment to one of three groups
for the research. Participants assigned to the control group
received no weather training intervention prior to flying two
scenarios in a flight training device. Participants assigned to
the interactive online group were asked to independently
complete an online training module, which typically took
an hour to complete. Participants assigned to the workshop
group were asked to engage in an approximately two-hour
workshop facilitated by the principal investigator, who is a
professional pilot, flight instructor, and expert in aviation
Once the interventions (if assigned) were completed,
participants completed a flight training device session,
which lasted approximately one hour.
Carney et al. (2015)
discussed flight training device session
results in separate reports. After completion of the flight
training device sessions, all participants were asked to
complete the posttest. At Location 1, participants completed the
pretest, flight training device sessions, and posttest within the
same day. Participants at Location 2 completed the posttest
within five days of the initial intake (during two separate
sessions), due to scheduling constraints. Additionally, minor
changes to the workshop at Location 2 were made to
streamline the content, based on experience in conducting the
workshop at the WJHTC. There were no changes to the
interactive online module between administrations at the two
locations. Therefore, results will be reported separately. All
data were collected between July 2015 and September 2015.
Population and Sampling
After a thorough examination of accident reports, it was
found the typical VFR-into-IMC accident pilot was a
noninstrument rated pilot with less than 1000 hours of total
(Carney et al., 2015)
. Therefore, researchers targeted
this population for recruitment and utilized convenience
sampling methods. Participants were contacted through a
pilot database, word of mouth, and advertisement (flyers).
Due to sample size goals, participants outside of the desired
pilot profile were allowed to participate in the study.
Location 1 had more variation and deviation from the
desired profile than did Location 2. However, the
distribution of participants at Location 1 was relatively
balanced. Participants fitted the targeted profile more
accurately at Location 2. Demographics and flight
experience information for each location are reported in the
Results section of this report. To account for variation, each
participant was randomly assigned to one of the three
To assess pilot knowledge and skills pertaining to VFR
into-IMC, 24 weather-related questions were derived from
the Private Pilot Airmen Certification Standards (ACS).
According to the FAA (2015), ACS will replace existing
practical test standards to provide enhanced and holistic
testing for certificate applicants. The FAA targeted June
2016 for implementation of the new testing standards.
The 24 questions were a combination of weather-related
knowledge and skills questions. Questions 1–10 were
knowledge questions, while questions 11–24 were skills
questions. All questions were vetted through several rounds
by seven aviation experts who had extensive flight and
academic teaching experience, to increase external validity.
Due to the close time proximity in which the participants
took the pretest and posttest, the posttest consisted of
slightly different questions from the same ACS categories.
The posttest questions were presented in the same order
as the pretest. Pretest and posttest questions are listed
in Appendices A and B of this report. The private pilot
weather-related ACS test questions codes can be viewed in
A post-hoc internal consistency test was conducted for
the pretest and posttest, with all scores combined
respectively. The Cronbach’s alpha value for the pretest was
0.444 (total count 72) while the Cronbach’s alpha value for
the posttest was 0.682 (total count 71). When comparing
these values to a commonly accepted standard, the pretest
had low internal consistency, while the posttest had an
acceptable level of internal consistency.
An interactive online training module and a workshop
were created by the researchers. The topics presented for
each intervention were similar; however, the workshop
included accident case studies to foster discussion. Table 1
lists the interactive online course topics and Table 2 lists
those for the workshop format. It was expected that one or
both interventions would improve posttest scores.
Participant demographic and flight experience data were
collected as part of the experiment. This information
included age, gender, total flight hours, accrued instrument
time, and time flown in the previous six months. The
demographic and flight experience data were sorted and
depicted for each group, and these data are shown in
Table 3. Table 4 shows participant flight experience
information for each group, and Table 5 shows class of airplane
most often flown and type of training received. Forty-eight
participants started and completed the pretest and posttest
(n 5 48) at Location 1. Once again, variation was caused
by the need to include pilots outside of the desired profile
to increase the sample size. This is further discussed in the
An ANCOVA, using SPSS 23H, was run to determine the
effect of two different VFR-into-IMC training protocols
and a control group on posttest scores after adjusting for
pretest scores. There was an approximate linear relationship
between pretest and posttest scores for each
experimental group, as assessed by a visual inspection of a
scatterplot. Homogeneity of regression slopes as the interaction
term was not statistically significant, F(2,42) 5 1.486,
p 5 0.238. Standardized residuals for the interventions and
for the overall model were normally distributed, as assessed
by the Shapiro–Wilk test (p . 0.05).
There was homoscedasticity and homogeneity of
variances, as assessed by visual inspection of a scatterplot and
Levene’s test of homogeneity of variance (p 5 0.177),
respectively. Though Barrlett’s test for variance is more
sensitive, particularly for datasets that depart from
normality, Levene’s test was used in both datasets due to the small
sample size and normal distribution (National Institute of
Standards and Technology, n.d.). No outliers were found
in the data, as assessed by no cases with standardized
residuals greater than ¡3 standard deviations. After
adjustment for pretest scores, there was a statistically significant
difference in posttest scores between the interventions,
F(2,44) 5 4.923, p 5 0.012, gp2 5 0.183. Approximately 41%
of the variance was explained by the pretest. The observed
power for the ANCOVA test was 0.780. Power values
above 0.80 are considered adequate.
Interactive online group
A post-hoc analysis was performed with a Bonferroni
adjustment. The data provided sufficient evidence to
indicate the workshop group posttest scores were
significantly lower than those of the control group (p 5 0.009).
There was no significant difference between the control
group and the online group (p 5 0.340). Lastly, no
significant difference was found between the online group
and workshop group (p 5 0.411). Table 6 shows pretest
statistics, and adjusted and unadjusted posttest values when
using pretest scores as a covariate. Adjusted mean values
were calculated from the multiple regression equation and
are determined by how far above or below average the
group was from the control variable (pretest).
Figures 1–5 represent trends in the pretest and posttest
scores. As seen in Figure 1, the pretest scores between the
experimental groups were virtually the same. Questions 9
and 20 appear to have the lowest frequency of correct
responses for all three groups. After the treatments (if
applicable) were administered, the posttest line graph
appears to be more sporadic. Line graphs were also created
to show the comparison of pretest and posttest score for
each experimental group separately.
As with Location 1, demographics, flight experience,
and category/class of aircraft most often flown are shown in
Tables 7, 8, and 9, respectively. Twenty-three participants
completed the posttests (n 5 23). One participant assigned
to the online group elected to discontinue participation
during the experiment. Therefore, all of the associated data for
that participant were removed for the ANCOVA analysis.
As with data from Location 1, an ANCOVA was run
to determine the effect of two different VFR-into-IMC
training protocols and a control group on posttest scores
after adjusting for pretest scores for data collected at
Location 2. There was an approximate linear relationship
between pretest and posttest scores for each experimental
group, as assessed by visual inspection of a
scatterplot. Homogeneity of regression slopes as the interaction
term was not statistically significant, F(2,17) 5 0.237,
p 5 0.791. Standardized residuals for the interventions
were normally distributed, as assessed by the Shapiro–Wilk
test (p . 0.05).
There was homoscedasticity and homogeneity of
variances, as assessed by visual inspection of a scatterplot
and Levene’s test of homogeneity of variance (p 5 0.112),
respectively. There were no outliers in the data, as assessed
by no cases with standardized residuals greater than ¡3
standard deviations. After adjustment for pretest scores,
there was no statistical difference in posttest scores between
the experimental groups, F(2,19) 5 0.696, p 5 0.511,
gp2 5 0.068. Approximately 10% of the variance was
explained by the pretest. The observed power for the ANCOVA
test was 0.150. Power values above 0.80 are considered
adequate. Table 10 depicts the pretest score statistics, unadjusted
posttest scores, and adjusted posttest scores when using
pretest scores as a covariate.
Figures 6–10 represent trends in the pretest and posttest
scores. As seen in Figure 6, the pretest scores between the
experimental groups were similar. Pretest questions 9, 19,
and 20 seemed problematic for the treatment groups. After
the treatments, if applicable, were administered, the
posttest line graph appears to have more variation. Posttest
questions 9, 14, 16, and 24 had the lowest frequency of
correct responses. Line graphs were also created to show
the comparison of pretest and posttest scores for each
experimental group separately.
Discussion and Conclusions
The purpose of this study was to develop training
modules that would enhance GA pilots’ knowledge and
skills pertaining to ACS weather-related questions. The
treatments did not appear to significantly increase posttest
scores when adjusted for pretest scores at either experiment
location. Though previous research has shown the
effectiveness of interactive online training modules and
(e.g., Silk, Perrault, Ladenson, & Nazione, 2015;
Sitzmann et al., 2006)
, enhancing GA pilot knowledge and
skills in a short amount of time remains complex
et al., 2010)
Neither of the treatment groups yielded statistically
higher mean posttest scores at either location. Moreover,
the adjusted mean posttest score for the workshop group
at Location 1 was significantly lower than that for the
control group. This led the researchers to consider possible
reasons for this result. As stated in the methodology, all
participants completed not only the pretest and posttest,
but also a flight training device session, which lasted
approximately one hour. Therefore, the workshop group
had the longest participation time and spent almost five
hours at the facility. Additionally, there were reports that
the flight training devices at Location 1 were difficult
to control due to technical problems. Given these issues,
it can be reasonable to assume fatigue may have played
Note. SE 5 single engine; ME 5 multi-engine; CFI 5 certified flight instructor.
a factor in the test results. It was also noted that the group
average for pretest scores when combining both locations
was 17.588 or 73%. This finding indicates that overall
pilot weather knowledge may be considered low. The
low overall average provides more evidence that
weather-related aviation knowledge may be lacking among
GA pilots and efforts should be continued to address
When reviewing the flight experience of participants,
there were many that fell outside of the desired profile.
There were commercial pilots, pilots holding an instrument
rating, and pilots who may have had more than 1000 hours
of total flight time among the test cohorts. In addition, the
process researchers used to collect data may have been
problematic. For example, when asking for total time, the
highest option was 301+. It is unclear if participants in this
category had 301 total hours or much more (e.g., 2000 total
hours or 10,000 total hours). Random assignment may
have accounted for the distribution of differences among
Lastly, when examining the mean posttest scores at
Location 1, the difference between the control and workshop
group was 2.674. Though this difference led to a statistically
significant result, it may lack practical significance. The
difference between two and three correct responses may be
nominal. It is recommended that future researchers pay
particular attention to the collection of demographic/flight
experience information and participant fatigue concerns. Technical
issues with flight training devices may occur unexpectedly,
requiring difficult decisions to be made by researchers.
Dividing the pretest, treatments (for those assigned),
flight training device sessions, and posttest into two
sessions at Location 2 helped to mitigate participant fatigue
issues. In addition, there were no known issues with the
flight training device used at Location 2. Additionally, the
flight experience of the participants at Location 2 was much
more uniform, and fitted the desired pilot profile more
accurately. Only one participant indicated having more than
300 total flight hours. However, the results failed to reveal
It is important to note that the two training modules
utilized at each location were not designed to ‘‘coach’’
participants or teach to the test questions. It may be
possible that immersive training could assist in enhancing
knowledge and skills. Immersive training can include
repetitive exposure to topics. For example, computer-based
programs can assist in providing participants with
visual cues, multiple weather reports, and decision-making
opportunities. Utilizing different instructional methods or
a combination of methods may provide deeper learning of
the complex material. Exploring the length of time
participants spend with researchers may also influence the
results. It may be possible that a larger sample size could
have yielded more desirable findings.
Lastly, a pretest and posttest with higher internal
consistency values may assist with measurement.
Unfortunately, due to time constraints of the project, a pilot test
was not conducted. It is suggested that future researchers
conduct a beta test to establish acceptable reliability, since
questions derived from the ACS do not guarantee high
reliability. Furthermore, researchers may use this study to
build upon the research design and understand potential
limitations and successes.
Although this study did not demonstrate significant
differences between the control group and the treatment
groups, VFR pilots should consistently address
VFR-intoIMC concerns during preflight planning and in flight.
While the GA community explores methods to mitigate
VFR-into-IMC events, following recommended practices is
essential. Pilots should self-study VFR-into-IMC material,
which includes preflight planning, both preflight and
inflight decision-making, operational pitfalls, the use of
all available resources, and meteorological conditions that
contribute to low visibility/low ceiling weather events.
Preflight decision-making should include previous, current,
and forecast weather reports. Go/no-go decisions must be
made, based on the capability of the pilot and aircraft.
Pilots should appropriately file flight plans and use inflight
weather services. Recognition of deteriorating conditions
should be based on reports and/or visual cues. Decisions
must be made in a timely manner to avoid illegal or
lessthan-desirable weather conditions. Conservative personal
minima should be practiced and consistent evaluation of
hazardous attitudes should be utilized. Proficiency in
dealing with the aforementioned aspects should be
discussed during pilot flight reviews. Lastly, regulators and
organizations should increase promotion of resources, and
pilots should take full advantage of those resources.
Though this study did not yield the anticipated results,
valuable knowledge was learned to assist researchers in this
area. Future researchers can address the limitations that
existed in this study. Limitations were found primarily in
small sample sizes, low power found in the Location 2 data,
demographic questions, issues with participant fatigue
during the experiment (particularly for the workshop
participants at Location 1), and technical issues with the
simulators at Location 1. Future researchers can attempt to
generate a larger sample size and consider two
experimental groups, instead of three. This may help improve the
power or the ability to detect statistical differences. The
power found in the Location 1 dataset was 0.78 and is
considered close to acceptable, while the power found in
the Location 2 dataset was considerably lower at 0.15.
As previously noted, specific questions in regards to
demographics may leave researchers with less doubt
regarding the extent of pilot experience. Additionally,
restricting pilots outside of the desired profile from
participating may increase demographic similarities, but there may
be a tradeoff with sample size. Random assignment can
account for differences and create a desired distribution
among experimental groups. Researchers of the current
study also acknowledge that participants may have faced
fatigue issues, particularly with the workshop group at
Location 1. Those participants spent approximately five
hours in the research environment. Lastly, unforeseen
technical issues may arise and disrupt the research process. This
limitation, which occurred at Location 1, was challenging
but was addressed appropriately by delaying the study until
the issues were solved. Doing so, however, further
decreased the sample size. Future researchers should have a sound
plan for dealing with technical failures.
Appendix A: Pretest Questions
(PA.I.C.K4l) How will frost on the wings of an airplane affect takeoff performance?
(PA.I.C.K4f) If an unstable air mass is forced upward, what type clouds can be expected?
(PA.I.C.K4f) An almond or lens-shaped cloud which appears stationary, but which may contain winds of 50 knots or more, is referred to as
(PA.I.C.K4e) One weather phenomenon which will always occur when flying across a front is a change in the
(PA.I.C.K4g) Possible mountain wave turbulence could be anticipated when winds of 40 knots or greater blow
(PA.I.C.K4j) One in-flight condition necessary for structural icing to form is
(PA.I.C.K4f) The conditions necessary for the formation of cumulonimbus clouds are a lifting action and
(PA.I.C.K4k) If the temperature/dew point spread is small and decreasing, and the temperature is 62˚F, what type weather is most likely to develop?
(PA.I.C.K4k) Low-level turbulence can occur and icing can become hazardous in which type of fog?
(PA.I.C.K4j) During an IFR cross-country flight you picked up rime icing which you estimate is 1/20 thick on the leading edge of the wings. You are
now below the clouds at 2000 feet AGL and are approaching your destination airport under VFR. Visibility under the clouds is more than
10 miles, winds at the destination airport are 8 knots right down the runway, and the surface temperature is 3 degrees Celsius. You decide to
(PA.I.C.S1) Which of the reporting stations have VFR weather? (Refer to Figure 12.)
(PA.I.C.S1) What are the wind conditions at Wink, Texas (KINK)? (Refer to Figure 12.)
(PA.I.C.S1) The base and tops of the overcast layer reported by a pilot are (refer to Figure 14)
(PA.I.C.S1) If the terrain elevation is 1295 feet MSL, what is the height above ground level of the base of the ceiling? (Refer to Figure 14.)
(PA.I.C.S2) From which primary source should information be obtained regarding expected weather at the estimated time of arrival if your
destination has no Terminal Forecast?
(PA.I.C.S5) Between 1000Z and 1200Z the visibility at KMEM is forecast to be (refer to Figure 15)
(PA.I.C.S5) What is the forecast wind for KMEM from 1600Z until the end of the forecast? (Refer to Figure 15.)
(PA.I.C.S1) What is the outlook for the southern half of Indiana after 0700Z? (Refer to Figure 16.)
(PA.I.C.S1) The Chicago FA forecast section is valid until the twenty-fifth at (refer to Figure 16)
(PA.I.C.S5) What is indicated when a current CONVECTIVE SIGMET forecasts thunderstorms?
(PA.I.C.S4) Which in-flight advisory would contain information on severe icing not associated with thunderstorms?
(PA.I.C.S1) What is the status of the front that extends from Nebraska through the upper peninsula of Michigan? (Refer to Figure 18.)
(PA.I.C.S1) What weather phenomenon is causing IFR conditions in central Oklahoma? (Refer to Figure 18.)
(PA.I.C.S1) What weather is forecast for the Florida area just ahead of the stationary front during the first 12 hours? (Refer to Figure 20.)
Appendix B: Posttest Questions
1 (PA.I.C.K4l) Why is frost considered hazardous to flight?
2 (PA.I.C.K4f) What are characteristics of a moist, unstable air mass?
3 (PA.I.C.K4f) Crests of standing mountain waves may be marked by stationary, lens-shaped clouds known as
4 (PA.I.C.K4e) Steady precipitation preceding a front is an indication of
5 (PA.I.C.K4i) Where does wind shear occur?
6 (PA.I.C.K4j) In which environment is aircraft structural ice most likely to have the highest accumulation rate?
7 (PA.I.C.K4h) What conditions are necessary for the formation of thunderstorms?
8 (PA.I.C.K4k) In which situation is advection fog most likely to form?
9 (PA.I.C.K4k) Low-level turbulence can occur and icing can become hazardous in which type of fog?
10 (PA.I.C.K4j) During an IFR cross-country flight you picked up rime icing which you estimate is 1/20 thick on the leading edge of the wings. You are
now below the clouds at 2000 feet AGL and are approaching your destination airport under VFR. Visibility under the clouds is more than 10 miles,
winds at the destination airport are 8 knots right down the runway, and the surface temperature is 3 degrees Celsius. You decide to
11 (PA.I.C.S1) The wind direction and velocity at KJFK is from (refer to Figure 12)
12 (PA.I.C.S1) The remarks section for KMDW has RAB35 listed. This entry means (refer to Figure 12)
13 (PA.I.C.S1) The wind and temperature at 12,000 feet MSL as reported by a pilot are (refer to Figure 14)
14 (PA.I.C.S1) The intensity of the turbulence reported at a specific altitude is (refer to Figure 14)
15 (PA.I.C.S2) From which primary source should information be obtained regarding expected weather at the estimated time of arrival if your destination
has no Terminal Forecast?
16 (PA.I.C.S5) In the TAF from KOKC, the ‘‘FM (FROM) Group’’ is forecast for the hours from 1600Z to 2200Z with the wind from (refer to Figure 15)
17 (PA.I.C.S5) In the TAF from KOKC, the clear sky becomes (refer to Figure 15)
18 (PA.I.C.S3, S1) What sky condition and visibility are forecast for upper Michigan in the eastern portions after 2300Z? (Refer to Figure 16.)
19 (PA.I.C.S1) What sky condition and type obstructions to vision are forecast for upper Michigan in the western portions from 0200Z until 0500Z?
(Refer to Figure 16.)
20 (PA.I.C.S5) What information is contained in a CONVECTIVE SIGMET?
21 (PA.I.C.S4) Which in-flight advisory would contain information on severe icing not associated with thunderstorms?
22 (PA.I.C.S1) The IFR weather in northern Texas is due to (refer to Figure 18)
23 (PA.I.C.S1) According to the Weather Depiction Chart, the weather for a flight from southern Michigan to north Indiana is ceilings (refer to Figure 18)
24 (PA.I.C.S5) The enclosed shaded area associated with the low-pressure system over northern Utah is forecast to have (refer to Figure 20)
Appendix C: Airmen Certification Standards—Weather Information
Aircraft Owners and Pilots Association . ( 2015 ). Accident Analysis . Retrieved from http://www.aopa.org/asf/ntsb/searchResults.cfm
Ali , S. , Chalder , T. , & Madan , I. ( 2014 ). Evaluating interactive fatigue management workshops for occupational health professionals in the United Kingdom . Safety and Health at Work , 5 ( 4 ), 191 - 197 .
Brooks-Harris , J. E. , & Stock-Ward , S. R. ( 1999 ). Workshops: Designing and facilitating experiential learning . Thousand Oaks , CA: Sage Publications.
Carney , T. , Brown , L., Duncan , J. , Whitehurst , G. , Rantz , W. , Seiler , B. , … Mayes, P. ( 2015 ). Weather technology in the cockpit (WTIC) Project B: Unexpected transition from VFR to IMC. Unpublished WTIC B phase 1 final report .
Coyne , J. T. , Baldwin , C. L. , & Latorella , K. A. ( 2008 ). Pilot weather assessment: Implications for visual flight rules flight into instrument meteorological conditions . International Journal of Aviation Psychology , 18 ( 2 ), 153 - 166 .
Federal Aviation Administration. ( 2015 ). Fact sheet: General aviation safety . Retrieved from https://www.faa.gov/news/fact_sheets/news_ story. cfm?newsId521274
Guskey , T. R. ( 2014 ). Planning professional learning . Learning , 71 ( 8 ), 10 - 16 .
Hunter , D. R. , Martinussen , M. , Wiggins , M. , & O'Hare , D. ( 2011 ). Situational and personal characteristics associated with adverse weather encounters by pilots . Accident Analysis & Prevention , 43 ( 1 ), 176 - 186 .
Ison , D. ( 2014 ). Correlates of continued visual flight rules (VFR) into instrument meteorological conditions (IMC) general aviation accidents . Journal of Aviation/Aerospace Education & Research , 24 ( 1 ), 1 - 26 .
Keller , J. C. ( 2015 ). Unexpected transition from VFR to IMC: An examination of training protocols to mitigate pilot gaps in knowledge and performance (unpublished doctoral dissertation) . Purdue University, West Lafayette, Indiana.
Knecht , W. , Ball , J. , & Lenz , M. ( 2010 ). Effects of video weather training products, web-based preflight weather briefing, and local versus nonlocal pilots on general aviation pilot weather knowledge and flight behavior, phase I. Federal Aviation Administration , DOT/FAA/ AM-10/1, Office of Aerospace Medicine, Washington, DC. Retrieved from http://www.faa.gov/data_research/research/med_humanfacs/ oamtechreports/2010s/media/201001.pdf
Knecht , W. R. , & Lenz , M. ( 2010 ). Causes of general aviation weatherrelated, non-fatal incidents: Analysis using NASA aviation safety reporting system data . Final Report DOT/FAA/AM-10/13.
National Institute of Standards and Technology. (n.d.) . Levene's test for equality of variances . Retrieved from http://www.itl.nist.gov/div898/ handbook/eda/section3/eda35a.htm
National Transportation Safety Board. ( 2012 ). NTSB identification: ERA12FA438 . Retrieved from http://www.ntsb.gov/_layouts/ntsb. aviation/brief2.aspx? ev_id520120711X12055&ntsbno5ERA12FA 438&akey51
Rust , C. ( 1998 ). The impact of educational development workshops on teachers' practice . International Journal for Academic Development , 3 ( 1 ), 72 - 80 .
Silk , K. J. , Perrault , E. K. , Ladenson , S. , & Nazione , S. A. ( 2015 ). The effectiveness of online versus in-person library instruction on finding empirical communication research . The Journal of Academic Librarianship , 41 ( 2 ), 149 - 154 .
Sitzmann , T. , Kraiger , K. , Stewart , D. , & Wisher , R. ( 2006 ). The comparative effectiveness of web-based and classroom instruction: A metaanalysis . Personnel Psychology , 59 ( 3 ), 623 - 664 .
Wiggins , M. , & O'Hare , D. ( 2003 ). Weather-wise: Evaluation of a cuebased training approach for the recognition of deteriorating weather conditions during flight . Human Factors: The Journal of the Human Factors and Ergonomics Society , 45 ( 2 ), 337 - 345 .
Wisher , R. A. , & Olson , T. M. ( 2003 ). The effectiveness of web-based training . Retrieved from http://indianstrategicknowledgeonline.com/ web/Effectiveness%20of% 20Web-based%20Training .pdf