Virtual reality (VR) is not a new concept; it has been used to design and develop new aircraft for some time. What is new, however, is the "virtual cockpit," and it is about to become a reality soon. Leading in its research and development (R&D) is the U.S. Army Aircrew Integrated Systems Program Office at Redstone Arsenal in Huntsville, Ala. The staff there is working feverishly on the Virtual Cockpit Optimization Program (VCOP).
According to Scott Dennis, the program’s lead engineer, the VCOP concept is to "provide the pilot with information such as situational awareness, sensor imagery, flight data and battlefield information in a clear and intuitive manner." The operative word is "intuitive." With VCOP, the military pilot will be able to conduct missions in a safer and more intuitive environment.
Giant Leap Forward
Unquestionably, VCOP represents a giant leap ahead in cockpit design philosophy. It embraces not just one, but five independently developed technologies, including:
A binocular, full-color, high-resolution, high-brightness helmet-mounted display (HMD) that incorporates virtual retinal display (VRD) technology;
Three-dimensional (3-D) audio;
Speech recognition;
Intelligent information management; and
Crew-aided cognitive decision aides.
VCOP’s increased computer processing power and advanced software and display technologies will present information to the pilot in a spherical world. "The VRD makes the VCOP possible," says Dennis. "We focus laser light on a refractive lens, and this creates a retinal scanning device [RSD]." The Army’s requirement for VCOP’s use in its helicopters is 1280-by-1034-pixel resolution. "Avionics systems within the aircraft are never changed," he emphasizes. "They are always redundant to the VCOP system."
The VCOP’s powerful processor and graphics generator gather information from aircraft systems and superimpose it on the pilot’s field of view, providing a see-through or augmented vision capability. The HMD uses head-tracker technology, so the pilot can view different imagery in 3-D space.
"What you see depends on where you look," says Dennis, adding that this greatly minimizes a pilot’s "head down" time and workload. At the same time, he adds, it raises the pilot’s situational and systems awareness.
Three-dimensional audio will augment the 3-D visual system. Audio cues for caution, warning and advisory data, as well as for communications, will sound as though they are localized in the pilot’s HMD. "In addition, by adding voice recognition software, the pilot can control specified tasking in a hands-off manner," says Dennis.
What About Training?
New technology creates a training challenge, which Dennis describes as a "human factors" issue. The Army is working on a training program that is early in development. "However," he says, "we feel it would be fairly easy to train aircrews in the new technology."
Why does the Army have the lead in this ambitious program? "It’s logical for us to put all the various VCOP technologies together because of the complex demands of rotorcraft flight," Dennis answers. The U.S. Army has more helicopters than any other military service. He adds that the VCOP application to helicopters works better at low altitudes and speeds, as well as the 6� of flight dynamics.
The Army’s lead contractor for VCOP is Microvision Inc. of Bethel, Wash., which has partnered with the nearby Boeing Phantom Works. Microvision describes the VCOP concept quite succinctly: it "collects and processes flight data, mission cues and sensor imagery from the aircraft system and then merges and presents the information to the pilot in a graphical, head-up mode on a VRD-based HMD."
Microvision designs and develops light scanning technologies that project images onto the retina of the eye. This allows pilots to see large, full-motion images without the need for a display surface. A safe, low-power beam of light literally "paints" rows of pixels onto the eye. To the pilots wearing the integrated HMD, the image appears to be floating directly in front of them at about an arm’s length away, as if on a large computer or television monitor. VCOP delivers the image with extraordinary brightness, contrast, resolution and color fidelity.
"VCOP will put the pilot in the right place at the right time," says Dennis. "It will virtually eliminate pilot limitations in mission and emergency situations." Pilots will be able to dynamically change any part of the mission "on the fly," he adds.
The aircrew cognitive decision aid is made possible by the Army’s Rotorcraft Pilot Associate (RPA) software, developed by Boeing. RPA automatically performs authorized tasks on behalf of the aircrew and makes recommendations based on tactical situations, techniques and procedures. "RPA is the additional crewmember in the cockpit, keeping the pilots in touch with all mission tasking, as well as the aircraft flight director…all within the HMD," says Dennis.
Jack Clevenger, Microvision’s VCOP program manager, reports, "We are currently in phase two of the program, which ends in mid-2001." The company seeks funding for the program’s next phase: to modify the HMD to include the pilot retinal unit and the display drive, which contains all electronics and photonics.
A simulator-based technology demonstration of VCOP before the Army aircraft systems managers and key ranking officers is scheduled for June 15. "Prior to the June demo, we will seek pilot evaluations," says Clevenger. The program manager and lead engineer hope positive pilot evaluations and positive demos will lead to flight demonstrations.
The VCOP development program’s following phase will be in fiscal years 2002-2003, where a great deal of miniaturization and ruggedization of the display unit is planned. The Army’s utility Black Hawk helicopter will be the demonstration platform.
The platform on which the VCOP might be fielded is not yet determined, though Microvision is utilizing the Army’s new Comanche scout/attack helicopter as a guideline. At this point in the program, it is hard to project end production goals, says Dennis. "We need to get the airframers to see the benefits of the technology, and we need the inputs of the pilots to form a solid design and integration concept." As for the system’s cost, Clevenger says Microvision is working on a design-to-cost model.
According to Dennis, VCOP technology has many applications in both air and ground combat systems. Microvision has been looking into other uses, including air and ground simulators, aviation maintenance repair–even use by air traffic controllers. In other words, VCOP technology, developed by the U.S. Army and Microvision, could be opening the door to a vast, new visual world.
Joint Helmet-Mounted Cueing Systems
Helmet-mounted sights have been operational with the Israeli Air Force for five years. However, the U.S. Air Force and Navy are catching up. The Joint Helmet-Mounted Cueing System (JHMCS) development program has the same bottom-line approach as the Army’s VCOP, providing the pilot with a head-up 3-D visual, audio and tactile capability.
The Air Force’s Aeronautical Systems Division (ASD) is managing the program with a contractor team from Visual Systems International (VSI) of San Jose, Calif. VSI is a joint venture between Kaiser Aerospace, a Rockwell Collins company, and Israel’s Elbit Systems Ltd.
According to Jim Stahl, the VSI program manager, "JHMCS is a monocular cathode ray tube-(CRT-)based system designed for high-performance fighter aircraft and provides the pilot with a display and sight system designed for integration into a fighter jet HMD." While the flat-panel display scan produces an image in pixels, the CRT produces an image in calligraphic display lines.
The fighter pilot gains from the JHMCS a 20� view on the right eye only. Essentially, the information projected onto the HMD visor is the same as on the aircraft’s head-up display (HUD). JHMCS also includes a magnetic head-tracking device. "With this system, the pilot can designate and select his targets and weapons while monitoring flight parameters," Stahl says.
The U.S. fighter aircraft designated to utilize the JHMCS include the F-15, F-18 E/F (integrated by Boeing), F-16 (integrated by General Dynamics) and the F-22. The Joint Program Office (JPO) will acquire the system for both new and older aircraft. Stahl says Denmark has shown interest in retrofitting the JHMCS in its F-16s, and Greece in acquiring the system for its new F-16s.
The JHMCS development program is nearly complete, and production deliveries are to begin soon. The initial buy is for 37 ship sets designated for the new F-18E/F program. More than 2,300 fighter aircraft eventually will be integrated with the new HMD. The first aircraft delivery will take place in November.
Tom Furness: Pioneer
Dr. Tom Furness is considered the father of retinal scan technology. For some 25 years, he championed the development of aircraft cockpits that take into account human perceptual organization. For more than 20 years, he worked in visual display human engineering for the U.S. Air Force’s Aeronautical Systems Division (ASD) at Wright-Patterson Air Force Base, Ohio. Furness currently is director of the University of Washington’s Human Interface Technology Lab (HITL) in Seattle.
"While I was working for the Air Force, we determined we needed a better way to connect pilot performance with aircraft avionics systems," he says. "We began by working to place new technology into pilot headgear." And that included research into intuitive interface and retinal scans with lasers. Furness’ study resulted in a simulated virtual cockpit, called the "Super Cockpit."
When the Cold War ended and military priorities changed, the program virtually died. The U.S. Army and Microvision came to its rescue, and the Super Cockpit lives on.
Looking back, Furness recalls how crude computer processing was 20 years ago. "It was large, heavy and under-powered…[so] many scientists said [retinal scanning] couldn’t be done…but we invented photon stream retinal scanning. We developed the retinal scanning device and in 1993 licensed it to Microvision."
To Furness, VCOP is not new, and in addition to projecting into helmet-mounted displays, it can project "onto aircraft windshields, or nearly anything." Currently, Furness’ research involves the development of affordable virtual interface technologies for such applications as medical imaging, virtual prototyping, prostheses for the handicapped, virtual classrooms and televirtuality.