Now that the new ACS testing standards are in effect, the good news is, it’s a whole new ballgame. This is a good thing, as the old ballgame, that is, the way the FAA administered tests, was, at best, useless, and, at worst, an abusive bureaucratic exercise that cost test applicants untold millions of dollars over the years in educational products designed simply to teach the test. Luckily, in the process of memorizing the answers to the FAA’s terrible test, a little aviation knowledge worked its way in around the edges, as we pilot applicants studied with one goal in mind, and that goal wasn’t aviation knowledge, but simply passing the test.
And what a terrible test it was, and I’ve made no secret about my opinion on the subject for many years. Composed of a collection of useless questions with a few good ones thrown in (perhaps to keep test takers off-balance), the knowledge tests represented outdated subjects (lots of ADF questions and no GPS questions at all until recently), unanswerable questions (certain cross-country computation items were nearly impossible to calculate even when you knew the process for doing so), and questions that asked test takers to calculate solutions in ways that would never, ever happen in an actual airplane (as with the questions on how to determine your position by using geometry and working solutions, presumably as you motored along on or hopefully near the airway).
I’d like to say that the reason for the bad test questions was that the FAA was terrible at designing tests and the questions that made them up, and that part is true. Unfortunately, it’s even worse than that. The impossible-to-answer questions were thrown in there in order for the FAA to manipulate the overall test results. A valid test, as anyone who’s taken intro-level education classes will know, typically has a results curve that’s the classic bell shape or some variation of it. In order to achieve this result profile, the test needs to have some easy, lots of medium and some pretty hard questions. By writing a test this way, you can differentiate between test takers in a fairly fine way. Those of you who remember taking the SAT (or your kids taking it) know that this most traditional of college entrance exams is anything but a crapshoot. To get a really great score, the test taker needs to be really smart and well versed in academic processes. This isn’t how any FAA knowledge test has ever worked (until recently, that is). To mimic the profile of a valid test, the FAA, instead of writing a variety of great questions that ran the gamut from easy to really, really hard, chose to mimic the results of great tests by creating a number of questions that they knew test takers would get wrong, not because they represented high-level problem-solving, but because there were essentially no good answers for them. Only 25 percent will get that kind of question “right.”
This approach gave rise to the test prep industry that helped pilot applicants “study” for the knowledge exams, in many cases, by flat-out memorizing the answers to the questions (which for some reason were in the public domain).
As I’ve freely admitted for many years, I did this on every one of my knowledge tests, and as a result, I aced every one of them. Years ago, I completed my CFI and Fundamentals of Instruction written tests with perfect scores and in so little time it must have been clear I couldn’t even have taken the time to read the questions. By the time I took the test, I recognized the items by their shape on the page.
I didn’t and still don’t feel guilty about the way I prepared for the tests. If they had been valid tests, I would have approached the whole thing differently. But because the tests were scams to begin with, I just responded in kind. They were playing me and I played them right back, with credit given to John and Martha King, who helped me every step of the way.
This isn’t to say I didn’t learn anything about flying. I did, and a lot of it was thanks to John and Martha. In fact, I spent all kinds of extra time learning what really matters to pilots in order to keep us (and our passengers) safe. It would have been nice if the tests had addressed a little of that material, but they didn’t.
Thank goodness that with the adoption of the new ACS, times have officially changed. The answers to the really good test questions are once more being kept secret, so pilots everywhere will have to actually study in order to pass them.
Nothing could make me happier. The whole idea of any kind of worthwhile test is to be an accurate indicator of the applicant’s knowledge. And because the new knowledge tests (they’re coming online one by one, so be patient) are valid ones, it means something when you get a good score on one. The cherry on top is the fact that the ACS’s standards are fair and sensible. They are, finally, based on actual aviation skill, knowledge and judgment.
Now we get to put the time in to get those great scores we all want (something that most of were doing for our own benefit anyway). It’s just a relief that there’s some connection between what the FAA asks us and the reality of our flying world.
Proficiency: What’s In A Word?
The term “proficiency” is all the rage in aviation these days, and it’s one that, while few have thought deeply about it, changes the very assumptions we have about how we tackle that thing we all know is part and parcel of what we do as pilots, that thing called “risk.”
Proficiency isn’t a new term in aviation. It’s been used for decades to talk about something fairly specific: a level of competency that’s a combination of both recency of experience and the level of expertise that pilot currently possesses at performing the underlying skills to a particular set of aviation skills, like landing an airplane at night or flying an instrument approach.
Today we’re using it in a very different way, in a way that suggests that proficiency and safety are joined at the hip, that proficiency, in fact, is the very way that one achieves safety. We don’t even use the word “safety” as we once did. Instead, we discuss it as risk. Even then we’re in the habit of talking about it as risk that has been thoroughly analyzed and understood, its high points sanded down and taught to pilots around the globe, all to lower the chances of a mishap taking place. The point is, the result of that training is what we call “proficiency.” These are all good things, very good things. That’s what “risk mitigation” is, and risk mitigation is what we’re all about, as we’re so far away from “risk elimination” that the very concept is naïve and therefore easily dismissed. Flying a small airplane doesn’t have less risk than driving a car, but the very acceptance that it does not gives us the ability to look closely at and analyze those risk factors in order to cut them down as much as possible. By doing this, we can greatly cut our risk, to the point, I’d argue, that for some well-trained and proficient pilots of small planes the activity is less risky than driving is for some drivers of family transportation.
For ages the term “safety” has been paramount in aviation risk management attempts. The term seems perfect. After all, what do we really want in aviation but for every pilot and every passenger to arrive safely at their destinations every time they take to the air. It’s a term that has taken root in so many aviation institutions it’s hard to keep track—from the largest flight training organization in the world, FlightSafety International, for which the word is a literal part of the brand name, to the nationwide organization of certificated flight instructors, the Society of Aviation Flight Educators, whose very acronym spells out the word SAFE, to the AOPA’s longstanding Air Safety Institute and the very governmental board, the National Transportation Safety Board, whose job it is to analyze transportation failures, including air transport accidents, and come up with ways to keep them from happening again.
It all offers the vague suggestion that we all need to do something that’s somehow, fundamentally less risky than what we’re doing now. Homer Simpson, from the popular prime-time animated comedy The Simpsons, gave just such advice. As the laughably incompetent safety officer at a nuclear power plant, he would tell his charges to “Safen up,” a directive that gave neither useful nor actionable advice.
The term “proficiency,” on the other hand, clearly implies what it is that we need to be doing to cut risk. We need to fly more and to train more pointedly at getting better at flying the kinds of maneuvers and making the kinds of judgments we know put us at the greatest risk, something one might refer to as “directed skill building in areas with high demonstrated risk factors,” that is, if you wanted no one to remember what to call it. “Proficiency” gets all those key concepts across in a single, euphonic word.
I don’t want to imply that proficiency is a new concept in training; far from it. Great training organizations, including FlightSafety, have implemented remarkably well-executed pilot and crew proficiency training programs for decades. The new program for HondaJet training, I’d like to reiterate if you missed it in my original review of the new HondaJet and its surrounding programs, nails proficiency perfectly. That training eschews the arcane details of airplane operation—like pressure values in the hydraulic system—that the pilot simply doesn’t need to know to conduct the flight safely. Instead, the program teaches the pilot what to do when there’s a problem with the hydraulic system. That’s useful, actionable information.
And a pilot who knows how to do just the right thing in that instance is more proficient and has learned one small way of hundreds to cut risk were that particular fault to arise.
What we need to do as an industry is take the underlying concepts behind the word “proficiency”—recency of flight, quality of instruction and purposefulness of instruction—and put those concepts into practice. That’s the way that safety, which we all know is the absence of mishaps, will happen. And that’s what we’re all looking to accomplish.
