Brochures

Annual Reports

SimLabs Technical Papers

Real-Time Simulation Documentation

by Nancy S. Dorighi and Boris Rabin
NASA Ames Research Center

Presented for the 2002 Federal Aviation Administration Technology Transfer Conference

Abstract
Improving runway safety at our nation’s airports is one of the highest priorities of the Federal Aviation Administration (FAA). One new and innovative way to evaluate runway safety initiatives for airports is through the use of virtual reality. The National Aeronautics and Space Administration (NASA) FutureFlight Central is a virtual airport tower in which air traffic controllers (ATCs) can immerse themselves in a simulation of their airport. Computer modeling of proposed changes to airport surface configurations allows controllers to "try out" ideas in the safety of virtual reality. In a study conducted in April 2001, NASA studied several alternatives for ways to improve runway safety at Los Angeles International Airport (LAX). This paper reports the results of the LAX Runway Incursion Studies and the progress to date in carrying out the next steps at LAX.

Modeling the LAX Tower
FutureFlight Central is a virtual airport tower that replicates for air traffic controllers, any airport environment as close to reality as possible. From the stairs that emerge through the floor of the tower cab, to the 360-degree out-the-window scene and the surrounding urban area, the simulation of LAX created a credible work environment that helped accurately assess human factor implications of the ideas under study. Three main aspects of the LAX controller’s job were emulated to create this reality: the out-the-window view, the voice communications, and the tower cab interior.

First, the out-the-window scene was created by using the facility’s twelve projectors to display a continuous computer generated image of the three-dimensional (3D) model of LAX (Figure 1).

Figure 1: NASA FutureFlight Central Tower Cab

The 3D model was based on computer aided design (CAD) data provided by the airport engineering department. It was overlaid with realistic details of the terminal area, landscape, and distant cityscape using high-resolution photo textures. (Figure 2).

Figure 2: Comparison of LAX Tower Views (above) with Simulated View (below)

Moving vehicles such as aircraft and service trucks were modeled in 3D, and moved about the airport scene under the control of "pseudo-pilots" who occupied the downstairs stations in FutureFlight. Pseudo-pilots controlled pushback time, taxi route, speed, take-off and landing in coordination with ATC in the virtual tower. Engineers used daytime conditions for the scene, although other times of day are possible. They modeled both clear day visibility and fog.

For voice communications, controllers operated a touch panel that allowed them to select radio frequencies, similar to actual FAA tower voice communication systems. Pseudo-pilots at the other end of the radio responded with standard phraseology for the aircraft under their control. Frequency congestion was a real part of the radio environment in FutureFlight, just like the real world (Figure 3).

Figure 3: LAX Controller Wearing Headset in FutureFlight Central

The tower cab interior provided a third aspect of realism. Controllers used stations accurately oriented to their positions relative to the stairs and airport scene. Flight strips and plug-compatible head set jacks with long base cords, made the virtual tower as familiar as possible. Center-of-the-room modular tables were configured to the rectangular shape in the LAX tower. Controllers used surface radar displays, which synchronized information with the activity going on in the out-the-window scene. Hanging BRITES displayed the ASR-9 radar (Figure 4).

Figure 4: LAX Tower D-BRITE Display in FutureFlight Central

Case Study of LAX Runway Incursion Alternatives

Background
Los Angeles International Airport has the fourth busiest airfield in the nation. Air traffic has grown rapidly over the past ten years. However, the airfield and airspace have the same capacity and configuration they did ten years ago. In both 1998 and 1999, LAX lead the list of the nationås busiest airports for number of runway incursions. Despite numerous changes to pavement markings, operating procedures, taxiway lighting and air traffic control procedures, runway incursions and surface incidents year after year present a major concern at LAX. (Figure 5)

The FAA and Los Angeles World Airports (LAWA), the operator of Los Angeles International Airport, determined that resolving the runway incursion problem requires an approach that will minimize possibility for human error in the cockpit and in the tower.

LAWA, the FAA, United Airlines (UAL), and NASA entered into a joint agreement to use FutureFlight Central (FFC) at NASA Ames Research Center to study changes to the Los Angeles International Airport., A study Steering Group was assembled with participation of LAWA, FAA Tower Control, UAL, FAA Western Pacific Region, National Air Traffic controllers Association (NATCA) and the Air Transport Association (ATA). This group was comprised of people intimately familiar with runway incursion issues specific to LAX.

Figure 5: LAX Runway Incursion Data 1997-2000. Incursions shown in white.^a

^a Source Los Angeles World Airports, Airfield Operations/2nd Quarter 2000, Airports Graphics Group/DAW7/00.

Approach
LAX Runway Incursion Studies at FFC were conducted in two phases. Phase I was a baseline simulation. This phase had two purposes: first, to validate FutureFlight Central as a simulation environment for accuracy in representation of ATC operations at LAX, and secondly, to establish a baseline for comparison with the alternatives simulated in Phase II.

Figure 6. FutureFlight Central Tower Layout

North Complex
Runways 24L and 24R



South Complex
Runways 25L and 25R

Phase I Baseline
FFC tower was configured to replicate layout of the LAX tower (Figure 6). Three operational conditions were used for validation: a VFR Arrival Rush, a VFR Departure Rush, and an IFR Arrival/Departure Rush. Air Traffic Control Specialists from LAX tower operated hour-long simulation exercises over a four-day period. Phase I data collected at FFC included controller workload, aircraft surface movement data, and controller communications. This data was compared to that obtained from the LAX airport. LAX officials, FAA Air Traffic Controllers, and FAA observers judged that the FFC simulation was sufficiently representative of LAX operations that FFC could be used to study the impact of the alternatives proposed in Phase II on operations at LAX.

• Controllers rated their simulation workload as "about the same as LAX."
• Controller rated the realism of the simulation as "about the same as LAX."
• The simulation successfully tasked controllers with the highest sustained traffic arrival and departure rates experienced at LAX.
• Outbound taxi times were accurate within 1-2 minutes of LAX times for aircraft originating in the North and South Complex gates, representing 82% of aircraft in the simulation.
• Runway occupancy times were within three seconds of corresponding LAX times for the inner runways, 24L and 25R. For the outer runways, 24R and 25L, occupancy times were longer than LAX.
• Controller voice communications closely modeled available recordings from the LAX tower. Duration of transmissions was on average 5-8% longer at FFC. Results indicated 10-15% more transmissions per hour at LAX, and the air time distribution (percentage of time controller, pilot or neither were transmitting) was approximately 3% less for both controllers and pilots in FFC.

Phase II Alternatives
The purpose of Phase II was to evaluate "ßair traffic control techniques, pilot procedures, airfield pavement geometry, and traffic management solutions to help eliminate runway incursions at LAX." Alternatives were compared objectively against data collected during Phase I and subjectively by the controllers and observers on the workload, efficiency, and safety criteria.

Each alternative was tested under visual flight conditions using two traffic scenarios: a peak arrival rush and a peak departure rush.

Both the north and south sides of LAX were simulated, with a complement of 22 airlines and an aircraft mix representative of LAX in the summer of 2000, for which NASA obtained actual LAX operational statistics.

To ensure a valid comparison of the data between Phase I and Phase II, alternative scenarios were built from the Baseline scenarios, using the same arrival and departure rates as well as the same mix of aircraft fleet. Following alternatives were proposed for testing:

Alternative #1: Swapping Inboard and Outboard Runway Operations
Aircraft arrive on the inboard runways, and depart on the outboard runways with some landings occurring on the outboards, and some departures occurring on the inboards, depending on traffic demands (Figure 7).

Figure 7: Alternative 1. Swapping Inboards and Outboards Operations

Alternative #2: A Second Local Controller Added to South-side Operations.
Runway 25R is under control of Local 1. 25L is under control of Local 3. Local 3 coordinate crossing of the inboard runway with Local 1 internally and give clearance to pilots. This was to eliminate need for pilots to change frequency from Local 3 to Local 1.

Figure 8: Alternative 2. Two Local Controllers on the South Side

Alternative #3: Utilizing a B16 Extension AA One Way
All aircraft arriving on 25L turn left onto the Alfa taxiway. Aircraft stay on the local frequency until crossing 25L on Uniform, at which time they contact GC-3. A B-16 extension to be utilized to avoid crossing 25R. All Aircraft bound for the North Complex to be sent up Alfa-Alfa. Aircraft bound for the South Complex turn onto Bravo. All 24L/R arrivals bound for the South Complex taxi via the South Route. The Bridge route is not available (Figure 9).

Figure 9: Alternative 3. Utilizing B-16 Extension, AA Is One Way

Alternative #3a: Utilizing a B16 Extension. Bridge Route Open
Controllers discovered in the simulation that Alternative 3 made Taxiway Echo overly congested and could potentially lead to a gridlock (Figure 9). They proposed Alternative 3a to improve traffic flow and workload distribution between Ground Controller 1 and Ground Controller 3. All aircraft arriving on 25L turn left onto the Alfa taxiway. The B-16 extension is utilized to avoid crossing 25R. For aircraft bound for the North Complex taxiing on the B-16 extension, GC-3 has the option of the West Route (Alfa-Alfa) or the North route (Quebec). Aircraft bound for the South Complex turn onto Bravo. Bridge route is open (Figure 10).

Figure 10: Alternative 3a. Utilizing B-16 Extension, Bridge Route Open

Alternative #4: Utilizing a B-16 Extension at ATC Discretion
Again, a B-16 extension will be used, but the controller has discretion over its use, with one basic rule to guide him. For arrivals on 25L, if the controller can issue an instruction to cross 25R without having to issue a hold-short command, he may exit the aircraft to the north (J, K, etc.). If the controller anticipates having to issue a hold-short command, he will exit the aircraft left onto Alfa (Figure 11). Alfa-Alfa will be controlled by GC-2, and the Bridge route will be available. For aircraft bound for the North Complex taxiing on the B-16 extension, GC-3 has the option of the West Route (Alfa-Alfa) or the North Route (Quebec). Traffic sent along the West Route must hold short of Alfa-Alfa and contact GC-2.

Figure 11: Alternative 4. B-16 Extension, ATC Discretion

Alternative #5: Utilizing a B-16 Extension with a 2nd Local Controller on the South-Side
Utilizing a B16 extension under the rules of Alternative 3a, Local 1 controls Runway 25R and Local 3 controls 25L. For the aircraft bound for the North Complex and taxiing on the B-16 extension, GC-1 has the option of the West Route (Alfa-Alfa) or the North Route (Quebec). The Bridge Route is open.

Data Collected

In FutureFlight Central, engineers collect video and audio data, subjective surveys, and surface metrics during the course of each simulation run. Remote cameras in the tower, make it possible to watch and record cross-cab coordination, flight strip passing, facial expression and heads-up vs. heads-down time. This video, synchronized with out-the-window scene and voice communication recordings, can provide data for analyzing potential workload problems. Measurements were takes of individual aircraft taxi times, and combined to assess arrival and departure rate and other measures of airport efficiency. Three types of test data were collected during this study:

• Controller subjective measures
• Airport operations statistical data
• Controller voice communications data

Controller Subjective Measures
ATC participants contribute expert knowledge of the efficiency, safety, communication, coordination, and traffic complexity and manageability of each alternative through responses on questionnaires. Each controller completed a survey immediately following each run of a scenario. Because each controller was randomly reassigned to a different work position during each scenario, their individual differences (response biases, fatigue-related effects, etc.) should have distributed approximately randomly over all of their ratings and not add bias to any single test condition.

The answers to questions used a scale from 1 to 5 where value '3' represents "about the same" as current LAX operations. A rating of 5 means "better than LAX today" and a rating of 1 means "worse than LAX today." For each question the Mean Rating and Standard Deviation was calculated by controller position.

In a final question, controllers could select up to three criteria to indicate the most challenging aspects of each alternative. Every time a controller selected a criterion, it was counted as an "occurrence." The resulting value, "Frequency of Occurrence", indicated how frequently this operational criterion was marked as critical across all positions.

Airport Operations Data
During all six days of Phase II, FFC collected airport operations data in order to compare the Baseline with the alternative scenarios. Collected data enabled calculations of average departure rate, outbound taxi time by origination point, arrival rate, and inbound taxi time by destination point.

Controller Voice Communications Recordings
FFC created digital audio recordings of each simulation run. Voice data was recorded from each controller station on the South side. At each position, the controller's microphone provided an input signal to one channel and the pilot's transmissions received through the headphones were recorded on another channel. In addition the console microphone was recorded on a separate channel. This capability allowed assessment of the controller workload through analysis of their inter-position communication.

Summary of Results

The results were based on analysis of the above data as compared with the baseline for each alternative. The following figure represents the subjective rankings of the alternatives by LAX controllers: rankings included questions on safety and efficiency. These results indicate that LAX controllers regard the B-16 extension under procedures of Alternative 3a as the safest and most efficient.

Figure 12: LAX ATC Rankings of Airport Alternative Configurations^b

For rating critical issues, alternatives 3a, 4, and 5 had the least number of occurrences. There were all procedural variations of the B-16 extension.

Figure 13: Critical Issues of the Alternatives^b

Non-critical issues (Frequency of Occurrence less than 0.3)
Critical issues (Frequency of Occurrence more than 0.3)

^bLos Angeles International Airport Runway Incursion Studies Phase II Alternatives Simulation, NASA FutureFlight Central, FFC-LAX-R002, August 2001

The following diagram combines Controller Subjective Ratings vs. Departure Rate, one of the important indicators of the efficiency of the alternatives. The vertical axis shows the controllers combined subjective ratings for each alternative. The horizontal axis shows the average departure rate per hour during a departure rush. The results were similar for the arrival rush.

Figure 14: Controller Subjective Ratings during Peak Departure Scenarios^c

^cLos Angeles International Airport Runway Incursion Studies Phase II Alternatives Simulation, NASA FutureFlight Central, FFC-LAX-R002, August 2001

Conclusions

• Based on controllerså subjective judgements of safety, the following were the top three alternatives for addressing LAX runway incursions. Significantly, both simulations that included no change to the current airport geometry ranked relatively low for safety
  • B-16: Bridge Open (Alt. 3a)
  • B-16: With 2 Locals (Alt. 5)
  • B-16: ATC Discretion (Alt. 4)
• Based on the subjective judgements of the controllers, alternatives that included a B-16 taxiway extension were more easily managed than the current LAX airport plan.
• Based on controllerås subjective evaluations, all alternatives which include a B-16 taxiway were regarded as more easily managed than the alternatives which included no modification to the airport geometry (e.g. simply swapping the runways use or adding another Local controller to South operations).
• Alternative 1 (Current Plan: Swapping Runways), while offering improved arrival taxi times and requiring less coordination by controllers, was not subjectively judged as safe as other alternatives. It was also regarded as having about the same potential for runway incursions as the current mode of operations.
• Alternative 2 (Current Plan: Two South Locals), resulted in lower departure rate and was judged by controllers as having a higher potential for a runway incursion than current operations, mostly because of the increased coordination required between Locals. «

Discussion

The virtual reality based evaluation of LAX safety alternatives provided a key benefit that would be impossible with any other type of assessment, short of testing changes in a live tower. It enabled real-time operational testing by the very people who would be most impacted by the changes, namely, the controllers. By showing that they could make a variation of the B-16 extension work, without negatively impacting their workload, efficiency, or judgement of safety, a way to reduce runway incursions was validated beyond the theoretical or simulation analysis.

In general, airport planners and designers have failed in the past to adequately utilize the insight air traffic controllers can lend to airport planning. In the virtual reality evaluation in FutureFlight Central, LAX controllers suggested a variation that had not been considered in the study design, namely, to combine the features of the best B-16 Alternative 3a with the additional south side local controller Alternative 2. By giving the controllers the flexibility to fine tune procedures, they optimized a "hybrid" alternative. This alternative not only rated the highest subjectively, but also performed the best under analysis of departure metrics.

Cultivating the input of those affected by decisions can only lead to greater consensus for the ultimate solution. By including airline and air traffic control involvement, LAWA, the FAA and United Airlines built support and momentum toward an eventual solution.

Next Steps for LAX

The B-16 extension that emerged as the most favored alternative, was assessed in FutureFlight Central primarily for controller human factors, and made some assumptions about conformance to implementation regulations. In particular, while runway safety improvements need to be treated with a sense of urgency, LAX also needs to consider how they will integrate into long-term plans and the current operational scheme. Furthermore, Terminal Instrument Procedures (TERPS) regulations and Code of Federal Regulations part 121, Operating Requirements: Domestic, Flag, and Supplemental Operations, must be considered specifying obstacle avoidance clearance distance under instrument flight conditions and engine-out conditions, respectively.

LAX is working through these next steps with implementation issues by submitting an application to the FAA Airspace Branch for review. LAWAås proactive and urgent approach to the runway incursion problem has moved them from number one for two years running at 10 incursions in 1999 to 7 incursions in 2001. The addition of the B-16 extension, assuming it passes FAA scrutiny, will further improve safety at LAX by reducing the conditions under which human errors can be made.

Summary

Thorough testing and refinement by LAX tower controllers in NASAås FutureFlight Central, enabled LAWA, the FAA and United airlines to save time and money in finding ways to improve runway safety. The key benefit of testing in a near real-life operational setting is that safety was maintained while alternatives were evaluated as closely to field implementation as possible. NASA determined that the B-16 extension under combined features of two proposed alternatives, provided the most desirable solution of those tested. It could reduce the possibility of human error by eliminating runway crossings altogether from the south side operations. Arrival taxi time, though negatively impacted, would be offset by improving the efficiency of departure operations. The LAWA team narrowed their options through virtual reality testing to ensure the expense of detailed designs and Environmental Impact Statement (EIS) would be spent on a plan that controllers considered workable and analysis indicated viable. LAX is now proceeding with the implementation phase, which includes conformance to TERPS regulations and other considerations.

References

1. Los Angeles International Airport Runway Incursion Studies Phase I Baseline Simulation, NASA FutureFlight Central, FFC-LAX-R001, May 2001

2. Los Angeles International Airport Runway Incursion Studies Phase II Alternatives Simulation, NASA FutureFlight Central, FFC-LAX-R002, August 2001