Redbird Flight announced at Oshkosh AirVenture 2022 a, well, not a “product” exactly, but a technology it is developing to merge a real-world flight sim and a virtual reality headset that allows the pilot to “see” in 360 degrees. It’s an amalgam of the hardware-based world and at least two others that are created in order to make the experience of being in a sim, even a fixed-motion one, come to life.
The mind, some Madison Avenue publicity wonk once said, is a terrible thing to waste. When it comes to flight training, and this might sound obvious, the mind is a wonderful thing to take advantage of, and Redbird Flight’s latest futuristic flight sim device puts the mind to work and without the sim pilot’s gray matter breaking a sweat.
I swung by Redbird’s exhibit space at Oshkosh the other day and met with old friend and Redbird VP of marketing Josh Harnagel to check out the big news from Redbird, which in case you’re new to aviation, has been one of the most innovative companies in our little neck of the woods for 15 years now. And at most Oshkosh gatherings, the company has some innovation to announce. These announcements have pulled the drape off of such products such as virtual instructors (GIFT); sim-based scenario training (a development for which I take at least partial credit) that is used as a primary tool at EAA’s Pilot Proficiency Center and a thousand other places that provide flight training; pilot performance monitoring, which is the core technology behind the company’s new Redbird Pro; and more.
Like a few of those other technologies, mixed reality—there’s not a product name for it yet—doesn’t try to create a new product within an existing paradigm but, rather, to create a new paradigm altogether.
But what is mixed reality? Imagine virtual reality, in which the user (read: pilot for our purposes) interacts with a completely digitally fabricated world. Everything you see and hear—they’re working on smell, touch and, who knows, maybe taste—is invented for you. It’s impossible, if you think about it very hard, not to let your mind jet off to potential applications, almost all of which are in active development somewhere by somebody.
Then imagine—okay, you don’t have to imagine it—regular reality, you know, that world out there constructed of atoms and molecules that we know and love simply as “reality.” I won’t go down any philosophical rabbit holes on this one except to say, if you start thinking about that very hard, it quickly becomes clear that reality isn’t as brick-and-mortar as we all instinctively know it is. But for the sake of this discussion, let’s just assume it’s as real as real can be.
The problem with virtual reality comes when there’s an operator of some kind who relies on physical controls to do whatever it is that virtual reality is attempting to mimic. In the case of what we do, we’re talking pilots and flight controls/electronics.
The Holy Grail would be to create a virtual reality world that lets in only the controls and operator parts (the physical controls, in this case, a flight sim’s controls and the pilot) while creating an outside world, the airplane and the world outside, including airports and terrain and weather, so the pilot could fly using physical controls while seeing a world around them that’s rendered via virtual reality.
That is what this is.
If you know anything about computer technology, and just about every pilot does, you might think that something like this would require enormous computing power and a really great headset. And you’re right. Even five years ago, a rig like this would have been impossible. For starters, the computers needed to do the processing were prohibitively expensive. On top of that, the headset tech, Harnegel told me, wasn’t available at any cost.
That rig consists of several critical components, including the Varjo XR-3 headset; the Quantum3D image generation system; the flight simulator software, here by Redbird; and the simulator, where a Redbird TD2 desktop non-motion sim with the excellent Alloy rudder pedals/brakes. All of that work together to provide though the headset a clear view of the outside world in spectacular fidelity and detail, the inside of the airplane and the flight controls in front of you and at your feet. It essentially creates a mask through which those controls can be seen in real life, like you’re looking at them as you do every day. So, when you look down to ensure that you’re adjusting the prop control and the not the fuel mixture, you see your actual hand and the actual physical controls on the simulator. Again, the computing power required to mask that real-world stuff as the pilot moves their head from side to side and up and down is enormous.
But would it work? To find out, I sat down at the simulator and donned the headset, which immediately calibrated my eyes—it creates the virtual world based on what you’re looking at, instantly changing the view as you move your eyes to look at the runway ahead of you, scan for traffic or to see how the kids are doing in back—okay, there weren’t any kids back there, yet! Josh, whom I looked at but couldn’t see, as he wasn’t part of the simulation, pointed out that as I looked around the airplane’s pillar, the scenery behind it matched the movement of my head, just like in real life. It’s nothing short of stunning.
Flying the plane—it was a generic model, felt like a Skyhawk—was very much like it is in one of Redbird’s sim without the virtual reality component. That is, it’s pretty realistic, though the simulated plane is touchier than the real one, so at first I overcontrolled! and later overcontrolled on landing. It was an extremely satisfying flying experience. As I turned my head I saw the interior of the plane—it was a Skyhawk!—and behind it, the outside world, including dynamic weather and terrain.
I took off from San Francisco without talking to the tower (still waiting for my letter from the FSDO on that omission) and I climbed out straight ahead until at pattern altitude or thereabouts and proceeded to make some turns, and then some steep banked turns.
The point here is not only to fool your mind but to fool the rest of your body too. There’s no motion involved. It’s a desktop sim, which you can purchase from Redbird, by the way, but my body was, no question about it, feeling motion. I “felt” the G’s as I pulled though the 60-degree banked turn, and I “felt” the negative G’s as I pushed and unloaded those G’s while returning to straight and level flight. There were, of course, no G’s being pulled, other than the normal weight of myself, but my mind, which knows very well what the maneuver feels like, filled in the blanks, effectively providing the missing motion component. Upon landing, I even “felt” the bump on touchdown and the deceleration, neither of which was present anywhere but in my brain.
While I was at the Redbird booth, two different people approached Josh and inquired about buying such a rig. Josh’s deadpan response was that it would cost eighty grand or so, but, he explained, Redbird seriously expects to be able to get it all down to consumer-level pricing at some point.
It’s also talking with the FAA about certificating such a sim, which will certainly be a protracted process, though one that Redbird has some confidence could happen. I hope it does. With full-motion sims being prohibitively expensive for consumers, this technology would give a pilot 90% of the full-motion experience at, eventually, pennies on the dollar.
Not the technology would be confined to consumer applications. It could be put to use as a realistic (and, therefore, effective) supplementary or introductory device for organizations that provide training in $10 million-plus full-motion rigs.
