The Anatomy of a Scenario
From the Examiner's Chair
Article No. 8
Good scenarios don't lead applicants to predetermined answers. They give applicants decisions to make—and allow the consequences of those decisions to determine what happens next.
Introduction
Scenario-based training and testing have become an important part of aviation education and practical testing.
But I think we sometimes misunderstand what a scenario actually is.
Too often, what we call a scenario is really a guided discussion.
The instructor or examiner already knows where they want the applicant to end up. A situation is presented, the applicant makes a decision, and if that decision doesn't lead in the intended direction, another question redirects them toward the desired path.
Eventually, the applicant arrives at the answer the instructor wanted all along.
That's not how I think a good scenario should work.
A good scenario needs to be flexible.
It needs to be organic.
The person conducting it may know what knowledge, risk management, or skill needs to be evaluated, but that doesn't mean they should know exactly how the applicant will get there.
The applicant makes a decision.
That decision has consequences.
Those consequences create the next situation.
Then the applicant makes another decision.
Actions have consequences.
That's the anatomy of a scenario.
A Scenario Isn't a Guided Discussion
There's nothing wrong with guided discussion as a teaching technique.
Sometimes an instructor wants a student to discover a particular concept, and carefully chosen questions are an excellent way to get them there.
But that's different from allowing a scenario to develop.
In a true scenario, the applicant has to be allowed to make decisions.
If the decision is reasonable but different from what I would have done, I don't need to redirect the applicant toward my choice.
I need to follow theirs.
That may take the practical test in a completely different direction than I anticipated.
That's okay.
I'm not trying to determine whether the applicant can figure out what I would do.
I'm trying to determine whether the applicant can exercise the privileges they're seeking safely and competently.
The Airman Certification Standards establish what I need to evaluate.
They don't require every applicant to reach those standards through an identical conversation.
"Unable" Isn't the End of the Scenario
Consider an instrument rating applicant who has planned an IFR cross-country.
The flight will take place in IMC, and I've told the applicant that an AIRMET for icing begins approximately 1,000 feet above the minimum en route altitude.
The applicant recognizes the concern and plans the flight at a lower altitude—perhaps right at the MEA.
So far, so good.
Then the scenario changes.
ATC needs the aircraft to climb 1,000 feet for traffic.
"What are you going to do?"
One of the most common responses is:
"I'd tell ATC unable."
Okay.
Now what?
There's still traffic out there.
ATC still has a separation problem.
The applicant's response may have been completely appropriate, but saying "unable" doesn't make the situation disappear.
So we continue.
What happens next depends upon the applicant.
Another applicant may make an entirely different decision, which sends the scenario in another direction.
Sometimes the applicant comes up with a thoughtful solution that resolves the problem safely.
And when that happens, there's no reason for me to invent another obstacle just to keep the scenario alive.
The scenario succeeded because the applicant did.
That's an important distinction.
A scenario isn't supposed to be a trap that becomes progressively more difficult until the applicant finally makes a mistake.
It's supposed to create an opportunity for the applicant to demonstrate judgment.
A Decision Changes the Scenario
IFR cross-country planning provides another good example.
I may ask an instrument applicant to prepare a flight with IMC en route and relatively low ceilings around the destination.
During the practical test, we'll discuss the route, destination, alternate and fuel planning.
At some point I'll ask:
"How much extra fuel do you have?"
The answer is usually reassuring.
Plenty.
Then we move forward in the flight.
You're nearing the destination.
You've lost all communications.
What are you going to do?
Which approach are you going to execute?
I don't need to tell the applicant what I want them to do.
They make the decision.
Then we follow it.
I don't want to describe every possible branch of that scenario because doing so would defeat much of its value. But applicants sometimes discover that the fuel they thought was extra wasn't nearly as extra as they believed.
Their original planning decisions matter.
Their destination matters.
Their alternate matters.
The approach they choose matters.
Their response to the communications failure matters.
Each decision changes what options remain available later.
That's much closer to real flying than answering a series of unrelated questions about fuel requirements, lost communications, approaches and alternate airports.
Real flights don't organize problems neatly by ACS Task.
Neither should every scenario.
Sometimes I Don't Create the Scenario at All
Some of the most valuable scenarios I've encountered during practical tests weren't created by me.
They simply happened.
During one Commercial Pilot multiengine add-on practical test, an applicant was flying a simulated one-engine-inoperative instrument approach.
The applicant forgot to extend the landing gear.
He performed a GUMPS check and announced that the gear was down.
It wasn't.
Later, he performed another GUMPS check.
Again, he announced that the gear was down.
It still wasn't.
At that point, the task was already unsatisfactory.
But I didn't immediately announce that.
The situation wasn't yet unsafe, and I wanted to allow the applicant an opportunity to recognize and manage his own mistake.
As we approached minimums, I knew I would soon have to terminate the approach and instruct him to perform a two-engine go-around.
Then, approximately 200 feet above minimums, he realized what had happened.
The gear wasn't down.
He hurriedly extended it.
The sudden increase in drag increased the descent rate.
He rapidly added power to the operating engine.
The airplane began decelerating.
Now the situation was becoming far more serious.
With substantial asymmetric thrust and rapidly decreasing airspeed, we were approaching an unacceptably low speed near the ground.
The scenario had reached the point where I needed to intervene.
I instructed him to add power on both engines, remove the view-limiting device, stabilize the airplane and land.
After touchdown and clearing the runway, I informed him that the task was unsatisfactory.
The Lesson Wasn't "Remember the Gear"
The debrief afterward was far more valuable than simply telling the applicant he'd forgotten the landing gear.
We talked about what had actually happened.
If this had been a real flight in IMC with an actual engine failure and he simply forgot the landing gear and landed gear-up, the result would likely have been significant damage to the airplane.
His ego and reputation might have taken some damage too.
But he very likely would have survived.
Instead, when he recognized his mistake, the urgency to correct it created a much more dangerous situation.
He was trying to save the airplane from a gear-up landing.
In doing so, he was moving toward a situation in which he might lose control of the airplane.
The lesson wasn't simply:
"Remember to put the gear down."
It was much more important.
Don't create a catastrophe trying to salvage a mistake.
There was another lesson too.
The applicant had completed two GUMPS checks and verbally stated that the gear was down both times.
He knew the checklist.
He said the words.
But he hadn't actually verified the condition of the airplane.
A checklist only works when it causes us to verify reality—not when it becomes a recitation.
Why Let It Continue?
Someone might reasonably ask why I didn't stop the applicant immediately when I recognized that the gear wasn't down.
After all, the task was already unsatisfactory.
Because the practical test isn't only about producing an outcome.
It's also an opportunity to observe how an applicant recognizes and manages changing circumstances.
As long as safety permits, allowing a situation to develop can reveal much more than stopping it at the first error.
Had I immediately said, "The gear isn't down. This task is unsatisfactory," the applicant might have left believing that I hadn't given him enough time to recognize and correct his mistake.
Instead, he discovered the problem himself.
More importantly, he experienced the consequences of how he chose to correct it.
When those consequences began moving the airplane toward an unsafe condition, I intervened.
There's an important boundary here.
Scenario-based testing never means allowing an unsafe situation to continue simply to see what happens.
The examiner has a responsibility to intervene when safety requires it.
But before reaching that point, sometimes the most valuable thing an examiner can do is allow the applicant's decisions to play out.
The Examiner Doesn't Need a Predetermined Answer
A good scenario isn't a puzzle where the examiner secretly knows the answer and the applicant has to discover it.
There may be several acceptable solutions.
There may be one particularly good solution.
There may also be several poor ones.
What matters is what happens after the applicant chooses.
The examiner may know exactly what needs to be evaluated without knowing exactly how the scenario will get there.
That's what makes scenario-based testing powerful.
The applicant isn't demonstrating an ability to predict the examiner's preferred answer.
The applicant is demonstrating an ability to make decisions as a pilot.
For Flight Instructors
Flight instructors can use the same principle during training.
Give students situations that require decisions.
Then resist the temptation to immediately rescue them when their decision differs from yours.
If the student's choice is safe and reasonable, let it develop.
Ask yourself:
What would realistically happen next?
Then let that become the next part of the scenario.
Sometimes the student's solution will work.
Let it.
Don't manufacture another failure simply because the lesson plan expected the scenario to continue.
Other times, a decision will create a new problem.
Allow the student to recognize it and work through it when safety permits.
And occasionally, reality will create a scenario you never planned at all.
Those can become some of the best teaching opportunities you'll ever have.
Scenario-based training isn't about becoming better at inventing stories.
It's about becoming better at allowing students to experience the consequences of their decisions in an environment where those consequences can be safely explored.
Putting It Into Practice
When building or working through a scenario, consider a few questions.
For instructors and examiners:
Am I creating a genuine decision, or am I guiding the applicant toward an answer I've already chosen?
If the applicant makes a reasonable decision different from mine, am I willing to follow it?
What would realistically happen next as a consequence of that decision?
Am I allowing the scenario to develop naturally rather than manufacturing additional problems?
If the applicant successfully resolves the situation, am I willing to let the scenario end?
Do I know when safety requires me to stop allowing the situation to develop?
Does the scenario evaluate something meaningful, or have I simply disguised a trivia question inside a story?
For applicants and pilots:
What decision am I actually being asked to make?
What are the immediate consequences of that decision?
What options will remain afterward?
Am I solving the problem in front of me while inadvertently creating a more dangerous one?
If the situation changes, am I willing to abandon my original plan?
Am I making the decision I believe is best—or trying to guess what the examiner wants me to say?
A useful scenario should make those questions matter.
From the Examiner's Chair
Scenario-based testing isn't about taking ordinary knowledge questions and wrapping them inside a story.
It isn't about guiding an applicant down a predetermined path.
And it certainly isn't about continuing to add problems until the applicant finally fails.
A good scenario creates a realistic environment in which decisions matter.
Sometimes I introduce the next complication.
Sometimes the applicant's previous decision creates it.
And sometimes the airplane, weather, ATC, or circumstances create a scenario neither of us anticipated.
My responsibility isn't to force the applicant back onto the path I imagined.
It's to evaluate what they do with the situation that's actually in front of them.
That's what pilots do in the real world.
They make a decision.
That decision changes the situation.
Then they make another one.
Actions have consequences.
And perhaps that's the most important thing a good scenario can teach.
A good scenario doesn't teach you what decision to make. It teaches you that every decision creates the next one.
— Brad Kramer
FAA Designated Pilot Examiner
About the Author
Brad Kramer is an FAA Designated Pilot Examiner based in Evansville, Indiana. He has over 25 years of aviation experience and has conducted FAA practical tests for more than a decade. His goal is to help applicants and flight instructors better understand the principles behind safe flight operations, sound aeronautical decision-making, and the Airman Certification Standards.
