Synthetic Vision: Picturing The Future

Synthetic vision is the next big thing in business aviation displays. Corporate pilots typically fly into a lot of smaller airports whose approach procedures they may not have committed to heart. So a technology that can let pilots see an offset ILS approach in 3D, superimposed cleanly on the primary flight display (PFD) within a picture of the surrounding terrain, is likely to attract interest. No matter that pilots cannot use synthetic vision systems (SVS) to navigate. The technology is likely to pay for itself as a tactical flight aid by reducing workload and flight errors as pilots negotiate unfamiliar terminal areas and conduct approaches in low-visibility conditions.

The synthetic picture is based on information from onboard terrain, obstacle and airport databases that is often combined with dynamic symbology, such as the flight path vector. Derived from head-up displays (HUDs), the flight path vector symbol indicates where the aircraft is tracking. Adding the symbol not only enhances the realism and three-dimensionality of the display. It boosts the tactical value by enabling the pilot to see at a glance where the aircraft is going in relationship to the terrain. It incorporates heading, wind effects and angle of climb and descent data without requiring the pilot to infer this information from other instrumentation and the flow of the terrain on the display.

Tactical Tool
Synthetic vision technology will move pilots well beyond the integrated and intuitive presentations of navigation maps, flight plans, weather hazards, and the like that fill today’s large-format, multifunction displays and help pilots with strategic flight management. Synthetic vision systems are being implemented on the PFD. SVS imagery, in an attenuated, see-through form, will eventually appear on head-up displays, as well, where the two types of presentations will complement each other. Meanwhile, SVS on the PFD, alone, will make the artificial horizon much more intuitive to the pilot.

Synthetic vision technology has had a long gestation-the concept dates back to the 1950s-and no doubt will continue to evolve, as more systems are fielded and the aviation community becomes more familiar with what the technology can do. Though it is now limited to advisory status, the possibility exists further down the road-with input from imaging sensors and automated vision processing techniques-for synthetic vision to take on a greater role. Numerous ideas are being floated about operational credit for more capable future systems.

Gulfstream is the first original equipment manufacturer (OEM) to announce an SVS system for new production and in-service aircraft. The airframer expects certification of its Synthetic Vision-Primary Flight Display (SV-PFD), based on Honeywell technology, next year. This real-time-updated, virtual depiction of the world outside the cockpit will increase the pilot’s ability to interpret depth, terrain texture, obstacles, runways and approaches, Gulfstream believes. The SV-PFD will be available on Plane
View-equipped G550, G500, G450 and G350 models. The system could allow pilots to make tactical decisions faster and more accurately, increasing the safety of flight, the airframer says.

Gulfstream and Honeywell certainly aren’t alone in the bizjet market, however. Universal Avionics has obtained a supplemental type certificate (STC) for an SVS product on a Part 25 aircraft. The company received its Vision 1 STC in May 2006 on a Bombardier Challenger 601. In the future there are likely to be other announcements from companies such as Innovative Solutions & Support (IS&S) and Rockwell Collins.

Official Support
FAA is "absolutely" supportive of the development of SVS technology as a means of increasing awareness of terrain and improving safety, says John McGraw, manager of the agency’s Flight Technologies and Procedures Division in the Flight Standards Service. FAA’s recent Advisory Circular (AC) 23-26, although aimed at small aircraft, indicates a receptive stance. While it stresses the need for developers to provide SVS in combination with other flight and navigation information, so that the pilot is not tempted to rely on synthetic vision alone to avoid terrain, the AC also calls for high levels of realism. The document emphasizes that synthetic vision systems "are not intended as substitutes for approach charts." But it goes on to say that once developers decide to incorporate 3D terrain, "they should consider displaying other features and information necessary to reproduce a clear, daytime picture that also correlates directly to the approach charts." The AC specifically mentions the inclusion of features such as terrain, obstacles, towers and runway orientation. The document also stresses the importance of the zero pitch line when terrain is depicted on the PFD-terrain above that key reference should indeed be above the aircraft’s altitude. The line, furthermore, should be bold, solid, in high contrast to the background scene, and extended across the entire display.

Not a Panacea
Synthetic vision is not a panacea, however. While it is independent of environmental conditions, it is highly dependent on the accuracy of the aircraft’s navigation solution and databases containing terrain, obstacle and runway information, as well as on the hardware and software employed to generate the images. SVS is not a sensor and cannot "see" objects that aren’t in its databases. Whether the integrity and resolution of the data driving these displays can someday support low-altitude navigation remains to be seen. It is certainly not the case now. Early operations with synthetic vision will probably be restricted to approved instrument approach procedures. "I think, for a safety benefit, we will concentrate our efforts on how we can implement SVS with instrument approach procedures and ground operations," says Les Smith, manager of FAA’s Flight Operations Branch. "That way we can guarantee that the aircraft is kept on a protected path."

While competitor Rockwell Collins has been very visibly developing synthetic vision concepts under NASA, U.S. military and internal funding, Honeywell was quietly creating technology of its own. Gulfstream has emerged as the company’s launch customer for what Honeywell calls the Integrated Primary Flight Display, or IPFD (see August 2005, page 26). The company has opened a certification project with FAA, says Bob Smith, vice president, advanced technology. That means it is in the process of understanding how best to certify the system, he says. "The technology is mature, but the product development, how it integrates into our cabinet and how it actually gets certified-that’s what we’re doing now," Smith says.

Honeywell’s highly realistic depiction of the terrain surrounding an aircraft is based on the company’s enhanced ground proximity warning system (EGPWS) data set, which boasts more than 500 million flight hours of use on a wide range of aircraft. But the company stresses that neither the EGPWS data nor the terrain depiction is meant to be used for navigation-the system is advisory only. So far some 200 pilots have logged more than 600 flight hours, over a period of six years, on the development project.

Honeywell summarizes its system in three words. The company says its integrated PFD is:

• Ambient-it requires no special effort of the pilot to gather the data;
• Natural-it is intuitive, so the pilot doesn’t spend a lot of time interpreting the data; and
• Continuous-it updates itself constantly in real time.

The IPFD is designed to improve situational awareness and reduce workload, not just by providing terrain and airport imagery, but through synergies between the imagery and new ways of presenting traditional symbology. A prototype display representing an offset ILS approach to the airport at Innsbruck, Austria, is a good example. In the illustration the glideslope deviation indicator-depicted as a column of blue dots just inside the altitude tape-indicates the aircraft is coming in a little high on the approach. The pilot can actually "see" the glideslope angle graphically as a horizontal line of dots extending from the center of the glideslope indicator. In the illustration, the green aircraft symbol-the flight path vector telling where the aircraft is tracking-is still slightly above the depicted glideslope angle.

Synergies
The localizer display, represented as series of parallel blue slashes conformal to the terrain, shows the aircraft is deviating slightly to the right of the required lateral course. (The blue aircraft symbol is not exactly centered.) However, the pilot is correcting to the left, as is indicated by the green-colored flight path vector. All of the vertical and lateral deviation information is presented in the pilot’s forward field of view, minimizing interpretation in this critical phase of flight. The combination of the flight path vector, the more intuitive localizer and glideslope symbology, and imagery of the external world is designed to make the tactical situation instantly understandable. "It screams at you what’s going on," says Chad Cundiff, Honeywell’s vice president of crew interface products.

Synthetic vision systems incorporating the flight path vector symbol will warn pilots of dangerous terrain when that symbol appears below the zero pitch line. Honeywell also plans to color such terrain hazards red on the IPFD when the EGPWS’ approximately 60-second "Caution Terrain" alert is triggered. 

This terrain avoidance presentation allows the pilot to visualize the situation in a more intuitive way than is possible with standard top-down or vertical EGPWS views. Using the company’s currently available terrain warning presentations, the pilot still "has to mentally reconstruct the topography to fully characterize the terrain," Smith says. The pilot does not have to extract and synthesize information before knowing how to act, because the pilot is, in a sense, "in the data."

Honeywell also integrates its Runway Awareness and Advisory System (RAAS)-an EGPWS add-on-into its synthetic vision system. This mode of the SVS shows runway and taxiway markings, so that a pilot knows exactly where he is on the airport surface. The RAAS presentation shows the runway designation (25L), the distance remaining (7,000 feet), the runway centerline and the initial 1,000-foot runway marker.

Cundiff emphasizes the tactical nature of Honeywell’s synthetic vision system, contrasting it with the company’s integrated navigation (INAV) display, a strategic flight management tool that is consulted hour by hour rather than minute by minute. INAV provides pages showing views such as the top-down map with the aircraft’s current position and extended flight plan, terrain and traffic overlays, and storm locations in relation to the flight path.

The IPFD system, by contrast, is the display the pilot looks at while flying the aircraft on takeoff, cruise, approach or during a missed approach. "If you can’t see out the window, you’re controlling the aircraft, looking at this display," Cundiff says.

Rockwell Collins
Rockwell Collins has not yet announced a bizjet customer, but with years of SVS research under its belt, it could do so some time soon. Although the company can render terrain with great realism, it prefers to employ a more stylized, "checkerboard" pattern. Collins finds that this presentation of the ground as "a set of tiles" gives pilots better depth perception and better awareness of terrain closure rates, says Tim Rayl, senior director of marketing for business and regional systems. "Tiles and lines interact with your perception" better than realistic shading, he contends. And this approach "causes you not to get overfocused on the pretty picture below," he comments.

Sensor data will be a key part of Collins’ technology, Rayl says, referring to "Synthetic Enhanced Vision," a term the company uses to describe the combination of imaging sensor and synthetic data. Rayl says he expects to see synthetic vision systems such as this "flying, fully approved," in corporate aviation aircraft by the end of the decade.

The enhanced part of SE-Vision uses an infrared (IR) camera to add some real, sensed information to the synthetic world. "In essence, you look through a portal in your PFD," Rayl says. The center is the sensor image and around the periphery is the synthetic image; the company is trying, in a sense, to "fuse" sensor and database "images." The combination of these different data types promises the ability to "present a much more credible, realistic view of the outside world," he says. Within the sensor domain, information from dual-band IR, and possibly millimeter-wave radar, could be combined. The company is looking at weather radar, as well, but more for "running background checks of the data that’s being presented."

Rayl stresses the importance of the HUD, as well. Sensor information, which can be displayed on a HUD-based enhanced vision system, is particularly important in the final phases of flight. "We don’t want pilots flying heads-down into the synthetic world," he says. "We would like to have them in the outside world as much as possible, especially on transitioning to landing." Looking through the glass of a HUD, he says, positions the pilot’s eyes optimally for transitioning from what sensors and databases are telling them to what can be seen in the real world. He says the Collins HUD, aka the Headup Guidance System, will have "some synthetic," as well as enhanced vision elements, and predicts that Collins will combine both head-up and head-down technologies. Synthetic vision on a Collins head-up display could use the company’s nonintrusive "wire frame" terrain rendering technique to highlight approaching ridge lines. As a pilot transitions from the head-down display to the HUD, the "imagery will have a very consistent look and feel," Rayl says.

IS&S
Innovative Solutions & Support is also deeply engaged in SVS research. The company has developed display symbology, such as a flight path marker and distance cueing, for presentation on the PFD, and has received good feedback from pilots, says Shahram Askarpour, vice president of engineering. It plans to introduce the system after completing development of its terrain awareness and warning system (TAWS) database, which it plans to certify at Class A. The company plans to achieve database resolution of 3 arc seconds around airports, Askarpour advises.

IS&S developed and subsequently patented an FAA-approved integrity monitoring system that allows it to use commercial graphics processors for display of primary flight data. The company says it developed this monitoring processor, which is qualified to Level A of both DO-254 and DO-178B, in order to accommodate changing industry requirements and still be able to offer a low-cost and quick time-to-market solution.

The custom processor monitors the output of the commercial graphics processor and compares the generated image data in real time to assure the integrity of the data being presented to the flight crew. The custom processor does so by generating a subset of graphical data for comparison with the video images being presented. The same technique is applied to the SVS by checking the essential information, such as the height of a ridge or the contours of a runway, Askarpour explains. The company believes that this level of data integrity is required for SVS on Part 23 and Part 25 airplanes. IS&S plans to make the SVS system available to bizjet customers in mid 2007. It will initially install the system on its Pilatus PC-12.

The aviation industry inevitably will demand operational credit for the new technology, just as it did with enhanced vision systems. Operators equipped with imaging sensor-based EVS technology on head-up displays are allowed lower minimums on Cat I approaches.

The Right Questions
A presentation at an FAA SVS workshop earlier this year touched on a number of important questions that need to be considered before new ways of operating with SVS can be approved. For example, would SVS be appropriate for Cat I, II or III approaches? Would it be suitable for specific taxiing and takeoff operations with nonstandard ground facilities? Or would it be used in lieu of certain infrastructure requirements? There could be many potential operational uses. But it’s still early days for synthetic vision. "Understanding the intended function within a specific operational context is key to addressing the corresponding operational certification requirements that must be considered before a specific type of operation can be approved," McGraw emphasizes.

"You’d like to get some operational benefits," comments Rayl. "You’d want to be able to have Cat II-type minimums, using the synthetic display, to get to all kinds of airports." But, he adds, "nobody’s willing to commit to that because there’s a long certification road to walk to get that to happen."

Among the areas FAA is investigating is the use of SVS systems in taxi operations. "Depending on the resolution and the accuracy of the database, [SVS] could certainly be a benefit in low-visibility conditions," McGraw says. A synthetic presentation of the taxiways, for example, could reduce runway incursions in low-visibility ground operations or operations with reduced lighting. SVS could assist the pilot in taxiing to the correct spot on the surface of the airport. However, it all depends upon what applicants bring to the table, he stresses. "With a system in place, we can work through these issues to see how much advantage we can take."

It seems unlikely, however, that today’s synthetic vision systems (without the addition of imaging sensors) will be enough to allow visual flight rules (VFR) operations in instrument flight rules (IFR) conditions outside of controlled IFR approach procedures. SVS will probably be part of a bigger picture, including improved onboard navigation systems.

The right questions to ask about synthetic vision-at least for now-may be, not what credit does it promise, but what is it giving pilots that they didn’t have before and whether it makes operations safer.

Cundiff says that Honeywell’s SVS doesn’t provide new data. But it portrays existing data in aircraft terrain/obstacle/airport databases and on the PFD in a more natural way with more intuitive symbology. If this type of SVS technology, as expected, heightens short-term tactical awareness and reduces flight errors, it will be a considerable step forward.