DogfightDogfight

June 13, 2016

In June, the FAA was set to introduce the airman certification standards, a new testing document that will replace the practical test standards. AOPA chaired the working group that developed the standards, and as part of our special coverage on this important change AOPA Pilot and Flying magazine are working together by concurrently featuring a “Dogfight” between working group member John King and instructor extraordinaire Rod Machado.

The ACS will save lives

Making situational awareness a habit

By John King

He was leaving from a job in his beloved Cessna Cardinal. He took off from a remote airport on a dark night. Light snow was falling. He did not make it home. He had simply exceeded his capabilities as a pilot.

He was an architect, a ski-resort founder, a community leader, and an all-around good person. To Martha and me, he was a friend. He loved his flying and worked hard at it, yet on that night he was not up to the task.

It is the story of many, many pilots. They are extraordinarily competent and leaders in their communities, yet they take off and fly in circumstances they will not survive—sometimes taking people they care about with them—and devastate those they leave behind. It is a tragic scenario that so many of us in aviation have seen far too often. These pilots have met standards, but the aviation training community has failed them.

The airman certification standards (ACS) are the result of concerned members of the aviation community coming together as a working group to solve the problem. The group’s members come from aviation associations, universities, manufacturers, pilot unions, course developers, and training providers—all working together in collaboration and cooperation with those in the FAA in charge of certificating airmen.

The group started with the agreement that knowledge tests needed improvement. Because there were no standards for knowledge tests like there are for the practical test standards, too many questions were tricky, trivial, or obscure. This forced our friend in the Cardinal, like many other folks, to study things that had no bearing on his ability to manage the risks of flight just to pass the test. He could, for instance, do multiple interpolations to calculate an inconsequential difference in takeoff distance, or work backward ADF problems when ADF was essentially obsolete.

He was not, however, required to know how what he was learning related to the application of skills, or how to identify and manage the risks associated with using those skills in real-world flying. The working group recognized that a training system that did not teach pilots the habits of situational awareness and risk management right from the start would continue to fail them.

The ACS changes all that. It provides a standard for the FAA to develop meaningful, relevant test questions. Far more important, the ACS will help pilots understand how knowledge, risk management, and skill work together. Nobody doubts that stick-and-rudder skills are essential, but they are not sufficient. No matter how skilled, all pilots are at risk of exceeding their own or their aircraft’s capabilities.

While the evaluation of an applicant’s risk management required by the ACS is important, the real payoff is in what pilots learn and the habits they develop while preparing to be evaluated. Using the ACS for training and testing will create the habit of thinking systematically about what’s happening now, what bad thing might happen if you don’t do something about it, and what you can do to prevent that. Rather than distracting from skill training, risk-management training puts skill into context so pilots know how to use their skills when it really counts.

Pilots report that when learning is in context, it is more fun, more interesting, and more meaningful—and they learn better and faster. Also, they feel better prepared to do the real-world flying that inspired them to get a pilot certificate in the first place.

Learning risk management doesn’t have to be costly, either. With proper planning, it comes as a side benefit of skill training. Plus, examiners who used the ACS during prototype testing reported that practical tests did not take more time.

Using risk management in flying is, of course, not new. We were all told “watch out for this” or “don’t do that.” With the ACS, though, we will be teaching and testing for the habit more systematically and less subjectively. The PTS told the examiner to evaluate a pilot’s “aeronautical decision making” and to cover a long list of “special emphasis” items. The ACS puts those things in context and provides standards for specific and observable risk-management behaviors, rather than leaving them to the examiner’s imagination.

Countless pilots and their passengers will benefit.

John King is co-founder of King Schools and was a member of the ACS working group.

Show me the data

Longer standards, subjective measures compromise the ACS

By Rod Machado

As a flight instructor for 43 years, my primary ambition is seeing that a qualified person can earn a private pilot certificate at a reasonable cost and with a minimal regulatory burden. Until now, the practical test standards (PTS) supported that enterprise. In my opinion, the new airman certification standards (ACS) do not.

The PTS is a simple document, forged using instructor feedback and insight over a 30-year period. It’s Spartan by design, easy to understand, and practical to use. However, the ACS committee (i.e., the FAA and industry participants) opted to replace it with the ACS. The ACS literally doubles the verbiage of the PTS by adding an assessment of risk-management skills and knowledge-specific elements to the mix. Many pages of the ACS now look like the flip side of a rental car contract.

What does the private pilot ACS accomplish? The ACS committee’s primary claim is that the new standards will reduce aviation accidents at the private pilot level. Unfortunately, there is not one bit of evidence supporting that claim! The committee also claims it will make flight training “less costly” and shorten the FAA practical exam. Really? I can’t imagine how doubling the verbiage in a government document will improve anything, much less aviation safety, and reduce flight training times and costs. Color me a skeptic, but this is similar to believing that adding hand lotion to your airplane’s fuel tank will make your landings smoother, softer, and younger looking. Not only is there no evidence to support these claims, but common sense and experience suggest they are false.

What’s most unsettling is that the ACS tests an applicant’s risk-management ability. According to the FAA’s Risk Management Handbook, “Risk management is unique to each and every individual, since there are no two people exactly alike in skills, knowledge, training, and abilities. An acceptable level of risk to one pilot may not necessarily be the same to another pilot.” Clearly, risk management is a subjective activity, which can only increase the subjectivity of the once relatively objective practical test. Nevertheless, the FAA is asking us to believe that the relative objectivity of the practical test won’t change. Hand lotion, please.

The desire to integrate higher order thinking skills (i.e., risk management) into the private pilot curriculum is noble in its intent—which is to keep private pilots from falling out of the sky. Yes, private pilot applicants can be taught to think like airline transport pilot-rated pilots as long as they are willing to spend years in training and acquire 1,500 hours of flight time. That, of course, means a lot fewer private pilots. Fortunately, private pilots are not falling out of the sky! When one does, it’s not primarily the result of a risk-management failure. The 2005 Human Error and General Aviation Accidents study made this clear. It concluded that, “skill-based errors [not decision errors] were associated with the largest portion of GA accidents (79.2 percent).” This is why one division of the FAA has placed emphasis on reducing loss of control (LOC) accidents. Ironically, the FAA division involved with the ACS believes that higher order (airline-type) thinking skills are more relevant to accident prevention at the private pilot level.

When the original PTS was created, it was developed and vetted over a two-year period, and involved 100 percent of all designated pilot examiners (1,854 DPEs) and 75 percent of FAA aviation safety inspectors. The private pilot ACS, however, was tested over a period of six months and involved eight DPEs in one FAA district office in Florida. The ACS committee stated that questionnaires offered to training participants didn’t reveal any adverse effects in this limited sampling. Then again, only 54 student pilots completed certification under the ACS. How many applicants dropped out of training because of increased training times and expense remains unknown.

Given the professional standing and qualifications of the ACS committee members, there can be no doubt about their good intentions. What is in doubt is the factual evidence supporting claims that the ACS will reduce accidents, while not interfering with the average person’s opportunity to earn a private pilot certificate.

Rod Machado has been a pilot since 1970 and an active flight instructor since 1973.