The autopilot behavior is perplexing; it responds differently to the same pilot action. Setting the altitude to a value behind the current aircraft altitude results in two different responses: In the first case the aircraft will continue at the current climb-rate and "kill the capture". In the second case the aircraft will capture the newly set altitude. So how can the pilot anticipate what the aircraft will do in each case?
It turns out that there is a hidden condition here, that revolves around the altitude at which the aircraft transitions to the "Capture" mode (7,000 feet in our example): If the newly set altitude is below (e.g., 6,000 feet) the "start capture altitude", then the aircraft will kill the capture. But if the newly set altitude is above (e.g., 8,000 feet) the "start capture altitude", the aircraft will capture the specified altitude.
Do pilots know about this behavior and the condition? No. It’s not in the manual nor is it mentioned in ground school or Initial Operating Experience (IOE) -- it is practically unknown! Now we are not talking about some minor deficiency, but a critical maneuver that sometimes takes place very close to the ground. The pilots’ user-model is inadequate for the task of capturing an altitude after resetting the altitude to a value behind the aircraft. If we go back to our earlier discussion, we have a condition in which the user-model is inadequate for the task. This deficiency corresponds to region 3 in Figure 1. (page 6/7)

The autopilot behavior is perplexing; it responds differently to the same pilot action. Setting the altitude to a value behind the current aircraft altitude results in two different responses: In the first case the aircraft will continue at the current climb-rate and "kill the capture". In the second case the aircraft will capture the newly set altitude. So how can the pilot anticipate what the aircraft will do in each case?
It turns out that there is a hidden condition here, that revolves around the altitude at which the aircraft transitions to the "Capture" mode (7,000 feet in our example): If the newly set altitude is below (e.g., 6,000 feet) the "start capture altitude", then the aircraft will kill the capture. But if the newly set altitude is above (e.g., 8,000 feet) the "start capture altitude", the aircraft will capture the specified altitude.
Do pilots know about this behavior and the condition? No. It’s not in the manual nor is it mentioned in ground school or Initial Operating Experience (IOE) -- it is practically unknown! Now we are not talking about some minor deficiency, but a critical maneuver that sometimes takes place very close to the ground. The pilots’ user-model is inadequate for the task of capturing an altitude after resetting the altitude to a value behind the aircraft. If we go back to our earlier discussion, we have a condition in which the user-model is inadequate for the task. This deficiency corresponds to region 3 in Figure 1. (page 6/7)

Actually, not. The display is also inadequate for the task – and here is why: To resolve what the aircraft will do we need to know the altitude at which the autopilot transitions to capture (e.g., 7,000 feet). But in practice, it is almost impossible to obtain this value with the current display. First, the pilot has no preview of this value and the interface does not display it. Secondly, the transition to the "Capture" mode happens automatically. In order to obtain the altitude at which the autopilot transitions to "Capture", the pilot must "hunt" for the automatic transition, and at that very moment look down to the altitude tape on the interface and catch the aircraft altitude as it rolls by. Thirdly, this altitude value is not retained by the display; once the transition takes place, the value is gone and there is no way to retrieve it. For all practical purpose, it is impossible to reliably obtain this value. The current display is indeed inadequate for the task. (page 7)

Actually, not. The display is also inadequate for the task – and here is why: To resolve what the aircraft will do we need to know the altitude at which the autopilot transitions to capture (e.g., 7,000 feet). But in practice, it is almost impossible to obtain this value with the current display. First, the pilot has no preview of this value and the interface does not display it. Secondly, the transition to the "Capture" mode happens automatically. In order to obtain the altitude at which the autopilot transitions to "Capture", the pilot must "hunt" for the automatic transition, and at that very moment look down to the altitude tape on the interface and catch the aircraft altitude as it rolls by. Thirdly, this altitude value is not retained by the display; once the transition takes place, the value is gone and there is no way to retrieve it. For all practical purpose, it is impossible to reliably obtain this value. The current display is indeed inadequate for the task. (page 7)

"5. CONCLUSIONS ...
The major issues associated with the FMS-related incidents, addressed in this analysis include:
- Raw Data and FMS/Aircraft Status Verification
- FMS Algorithmic 'Behavior'
- Improper Use of the FMC Automation Level
- FMC Programming Demands
- Multiple FMC Page Monitoring Requirements
- Complex ATC Clearances
- Complex FMC/CDU Tasks
- Lack of Adequate Pilot Training
- FMC/MCP Interaction Errors
- Inaccurate Pre-Stored Databases
All of these factors, singly or together, can combine to increase the pilots' workload to the point that they lose their situational awareness and 'get behind the airplane.' In this situation, the pilot who continues to focus on trying to understand what the FMC/CDU is doing is no longer truly involved in flying the airplane, but trying to troubleshoot a computer that happens to be installed in an airplane. The pilots that did best with FMS-related problems, in high workload situations, were those that elected to reduce the level of automation (by turning OFF the selected function) and appeared to recognize that they needed to become actively involved in flying the airplane.
From these reports, it is clear that the current FMSs have not been designed for optimal use under all circumstances, by the flight crew, in the environment where ATC is heavily burdened and expects pilots to remain flexible and responsive to their changing needs of moving traffic. Based on this analysis, it would appear that pilots should not try to use the full features of the FMS under all conditions. Many of the pilots submitting these reports learned that fact, but only after they experienced the incident that initiated the ASRS report. This lends credence to those pilots who argued that the training they received was not adequate to prepare them for using these systems operationally." (page 5.1)

"From these reports, it is clear that the current FMSs have not been designed for optimal use under all circumstances, by the flight crew, in the environment where ATC is heavily burdened and expects pilots to remain flexible and responsive to their changing needs of moving traffic." (page 5.1)

"4.1.2.2 FMS Algorithmic 'Behavior' ...
In many of the reports, an altitude excursion was the result of the FMS not performing as expected, or the flight crew not recognizing that the FMS was not working properly or was misprogrammed. It is likely that these incidents occur because the FMS algorithms are designed to level off the aircraft at the last minute. If the flight crew missed the 900-foot and 300-foot cues that signal approaching the selected altitude, this leveling off is the major cue to the crew that the desired altitude will be acquired. The last-minute nature of the leveling-off process, coupled with missing the altitude alert cues, means that the crew knows a problem has occurred only when the airplane does not [emphasized] level off, at which time it is probably too late to perform any actions that can prevent the altitude deviation. One pilot described the experience this way:
[ASRS Report #125410] ... 'On departure, we were cleared to climb to 12,000 feet, but we had an altitude deviation and climbed to 12,450 before returning to our assigned altitude of 12,000. At 11,000, I called 1,000 to go and then looked back outside to clear for traffic in the turn. I looked back inside and saw that we were at 11,800 climbing at 4,000 feet per minute (fpm). I pushed forward the yoke the same time I said '12,000' ... This aircraft is a popular modern transport with an excellent thrust to weight ratio, glass cockpit, autothrottles, FMC's, the works. With this aircraft's power it has quite a good climb rate and the automated systems fly the aircraft exceptionally well, but they do not climb or descend the aircraft according to the Airman's Information Manual (AIM). It is not at all unusual to approach within 300-400 feet of an altitude at 4,000 fpm. The computer will capture the altitude with about a 1.25 G pull or a .75 G pushover so that the passengers don't really feel it... I feel that if the AIM descent and climb rates were programmed into the computer that would be a better system. That way, high vertical speed in the last 1,000 feet would be the exception and not the rule and much more likely to result in a timely level off instead of an altitude bust. After all, it would take more that [in sic] 30 seconds to overfly/underfly an altitude by the magic 300 feet at 500 fpm as opposed to only slight more that [in sic] 4 seconds it would take at 4,000 fpm.'
This type of algorithm can encourage the occurrence of altitude excursion since it does not leave much room for error compensation." (page 4.7-4.8)

Issue:

automation may use different control strategies than pilots(Issue #122)See Issue details

"Another issue cited in some of the reports was the difficulty that the flight crew had in recognizing programming errors once the data were entered into the FMC/CDU. These pilots maintained that the FMC should be more capable in reviewing and alerting the pilots to entries that appear to be in error or do not logically fit with the rest of the data entered." (page 4.12)

"A common observation by the majority of the pilots submitting these reports was the belief that they did not have enough information about what the FMS was doing to be able to effectively monitor the system." (page 4.5)

6. RECOMMENDATIONS ...
6.1 Design-Related Recommendations
This analysis of the FMS-related reports from the ASRS database has provided a valuable look at the problems crews are having with current FMSs. On the basis of this review, the following recommendations suggest how the Description and Characterization study can be focused to concentrate on those issues that appear to have special importance. ...
3) Feedback sources for each automation level, and for each task, need to be specified. A major concern for many flight crews is the inability to effectively predict and understand what the FMS is doing. Issues of adequate and meaning feedback need to be addressed." (page 6.1)

"4.1.2.4 FMC Programming Demands
Many of the ASRS reports included the complaint that the FMC/CDU is difficult and time consuming to program. This complaint is magnified in the case where, for whatever reason, the FMC rejects the programmer's (pilot not flying) initial attempt. Under these conditions, it is not uncommon for the pilot flying to then get involved as well, at which point no one is flying the airplane. The frequency of these of these comments gives rise to the impression that the design of the current FMC/CDU does not appear to be optimal for the pilot's needs in the operational environment.
[ASRS Report #107738] ... In summary, we missed our crossing restriction due to pilot flying doing pilot not flying duties, that is, extensive CDU reprogramming and not monitoring the flight path. I also didn't monitor the flight path close enough while involved in other duties. We received the clearance from MSP center, but failed to comply. Only one person should be doing heads down FMS work while the other monitors flight path. Very busy time in two person cockpit requires extreme discipline.
[ASRS Report #87750] ... In training they emphasized that one pilot should fly and the other should program the FMC. I understood and believe that, however, most of the experienced pilots I have been flying with since training seemed to do most of their own FMC Management while flying, especially if I was otherwise occupied on the other radio. Following that example, which may work for an experienced large transport pilot but certainly not for one at my level, I fell into the trap they had warned me about! I pushed buttons, but I did not check the response to the input before going on to something else. No one was flying the aircraft. In the future I will initiate all altitude changes on the MCP (wing flight level change) when the other pilot is unable to enter data in the FMC, and will check the basic aircraft instruments for a response to the inputs I make to the complex, multifaceted auto Flight Control system." (page 4.10-4.12)

"The question arises as to why crews are reluctant to use automation levels other than the FMC. In at least one case, the reason was obvious. The captain insisted that the first officer use the FMC because the company's policy was always to use it." (page 4.9)

Issue:

company automation policies and procedures may be inappropriate or inadequate(Issue #166)See Issue details

"4.1.2.5 Multiple FMC Page Monitoring Requirements
The organization of information within the FMC/CDU appears to be an issue for some pilots. Monitoring the overall status and performance of the aircraft includes being aware of fuel status, lateral path, position, vertical path, and so on. To adequately monitor aircraft status by means of the FMC, the crew must review the information that is presented on a number of different pages which are accessed by means of a number of mode and/or line select keys. Extensive monitoring of the FMC/CDU diminishes the crew's ability to monitor the data in the mode control panel at the same time, thus creating the possibility for missing important information about the status of the aircraft.
[ASRS incident report #119836] 'Approach DEN from the east on J80 the captain (pilot flying) asked copilot (pilot not flying) to request FL390 due to building thunderstorms over the Rocky Mountains. I (copilot) put FL390 in the right FMS computer to check aircraft capability for FL390. After entering and executing FL390 in 1 L on FMS, I verified that the altitude window on the mode control panel was at 35,000 feet and that the autothrottles did not add power for the climb. At this point, the mode control panel altitude window was holding the aircraft at current cruise altitude of 35,000 feet. This has been an accepted procedure in this situation. After checking altitude capability in the FMC, I mentioned to the captain that we could make FL390 and would save approximately one percent of fuel with the climb. This whole check took probably less than 20-30 seconds. I then called DEN ATC and was advised to expect FL390 in approximately two minutes due to traffic. Anticipating the higher altitude, I left FL390 in the FMC active cruise page, once again checking to make sure the window read 35,000 feet. I continued to prepare the ACARS position report to be transmitted over DEN. We were approximately three minutes east of DEN. I remember checking the ETA for SLC and entering the fuel over DEN as 22.5. Since I was preparing the position report I changed from the Cruise page in view with the FL390 Cruise active page on it. During the minute or minute and a half of preparing the ACARS position report and waiting for the ATC clearance to FL390 the captain (pilot flying) changed the mode control panel altitude window to 39,000 feet, anticipating the climb. Of course, the FMC not being constrained at 35,000 feet any longer started to slow climb to FL390. The captain also began a passenger announcement to the passengers about DEN and the turbulence, and that we expected to climb to a higher altitude shortly. The center called, 'Maintain FL350.' Without even hesitating, I responded 'Roger maintain 350.' By this time the captain (pilot flying) had already started a push-over. The aircraft had reached an altitude of approximately FL357. After the aircraft was returned to FL350, I checked the mode control panel altitude window and was surprised to see 39,000 feet. We returned to 35,000 feet, our cleared altitude. Within a few minutes, Center cleared to FL390. Crew coordination and lack of communication may have contributed to the altitude excursion and conflict. The mode control panel window is, in my judgment, the last step in the altitude change process, to be changed after the clearance has been received. The autoflight system will not depart the mode control panel altitude, even it the FMC is programmed for a different altitude.' " (page 4.12-4.13)

"A small subset of tasks which are being performed either just before or during the occurrence of an incident appear repeatedly in the ASRS reports reviewed. This suggests that some tasks performed by means of the FMC/CDU are more difficult than others." ... ASRS reports were included to provide examples of each of these tasks ... "Developing and Entering a Crossing Restriction at a Distance From a Fix Along a Radial (126707) 'Cleared to cross 80 miles south of RIC VOR at FL270. We were leveled at FL330. The aircraft has been adapted with a new FMC. This particular restriction was difficult to get accepted into the FMC. It continuously showed down the scratch pad (invalid entry). Nevertheless, the procedure for the entry was correct. ATC called and queried us about it and we initiated the descent with idle power and full speed brakes and 330 knots. ATC asked if we were going to make it. We (I) acknowledge with an "affirmative" and continued with the steep descent. As I was doing so, the winds were showing higher than usual on the FMC Progress page. Upon realizing that the restriction was not going to be met, just when we were going to advise ATC and request vectors so as to meet the crossing restriction, DCA ATC informed us not to make a steep descent because there was not conflicting traffic involved. I understood what he meant by that statement that everything was okay and we did not request vectors, but continued the descent, crossing 80 DME about 1,000 feet high.' Entering a crossing restriction at a distance from a fix is one of the most common types of clearances received. Nonetheless, pilots do appear to have trouble implementing this clearance, as is shown in this example. What is especially interesting about the example is the response of the FMC to the pilot's entered data. When the entered data do not meet the requirements of the FMC, the only feedback received is 'Invalid Entry.' No clues are provided as to the nature of the problem. One would expect that this lack of informative feedback can only contribute to the programmer's frustration. This example also demonstrates a second common occurrence: The programmer's conviction that what he/she programmed in was correct. This conviction is common to many of the ASRS reports ..." (page 4.17-4.18)

Issue:

data entry and programming may be difficult and time consuming(Issue #112)See Issue details

"The data presented in Table 4-1 suggest the same underlying problem: The crew fails to operate the FMS properly and, at the same time, fails to catch the error before an incident occurs." Table 4-1 shows the breakdown of the number of citations for various Flight Crew FMS Actions/Errors. Out of a total of 99 citations, 3 citations [3%] were logic errors which "usually involved the flight crew entering data in a format or form that the FMC would not recognize, or the pilot not understand the underlying limitations of the system when he or she tried to enter the data." (page 4.1-4.2)

Issue:

data entry and programming may be difficult and time consuming(Issue #112)See Issue details

"Of particular interest is the fact that five of the six [83%] holding pattern problems included in Table 4-3 (Phase of Flight) were caused by erroneous information being stored in the database." Table 4-3 Phase of Flight shows that of the 115 citations listed, 6 of them [5%] occurred during the holding pattern phase of flight. (page 4.28)

"6. RECOMMENDATIONS ...
6.1 Design-Related Recommendations
This analysis of the FMS-related reports from the ASRS database has provided a valuable look at the problems crews are having with current FMSs. On the basis of this review, the following recommendations suggest how the Description and Characterization study can be focused to concentrate on those issues that appear to have special importance. ...
6) The number of screens that have to be reviewed in performing some tasks also is an important issue. There is an obvious need to review the overall organization and layout of information across pages, and the means for navigating from one screen to another, in order to determine the contributions of these factors to the complexity of the task." (page 6.1-6.2)

"The referenced reports often dealt with problems such as altitudes not being captured, crossing restrictions not being met, and climb and descent rates being excessive. As Table 4-3 showed, crossing restrictions not met represent 40% of all flight phase categories in these 99 reports. In many of the reports, an altitude excursion was the result of the FMS not performing as expected, or the flight crew not recognizing that the FMS was not working properly or was mis-programmed. It is likely that many of these incidents occur because the FMS algorithms are designed to level off the aircraft at the last minute. ... The last minute nature of the leveling-off process, coupled with missing the altitude alert cues, means that the crew knows a problem has occurred only when the airplane does not level off, at which time it is probably too late to perform any actions that can prevent the altitude deviation." (page 4.7)

"Under ideal conditions, the flight plan programmed into the FMC during preflight will be the flight plan that is actually flown. If this were always the case, virtually all of the errors that occur through FMS use would disappear. One reason as to why flight plans have to be changed, in the air, is Air Traffic Control and today's complex airspace. In areas of high traffic density, ATC clearances issued to a particular flight can be numerous, and in some cases contradictory, making effective use of the FMS difficult due to re-programming requirements, and/or the time needed for the FMS to respond to the new commands. It is also likely that ATC's understanding of the capabilities and limitations of FMS-equipped airplanes may not be what pilots anticipate. High traffic levels with correspondingly high ATC workloads and complex airspace result in very dynamic situations which often require timely and flexible response from the flight crew. ... Advanced cockpit airplanes, however, often engender workload difficulties that are unique as portrayed in the following report. [ASRS incident report #114409] .. 'During climbout from DFW that controller issued a clearance to turn to a heading of 300 degrees, intercept the DFW 274 degree radial, climb to and maintain 16,000 feet, and maintain 250 knots until advised. As the first officer, and pilot not flying, I proceeded to read back the clearance and program the FMS computer for route, speed and altitude. The Captain selected speed intervention of 250 knots and heading to the assigned intercept heading. He also attempted to couple the vertical navigation of the autopilot but this was not accepted so he used flight level change and speed of 250 knots to climb to the assigned altitude of 16,000 feet at 250 knots ... Unfortunately, the autopilot entered an altitude capture mode approaching 10,000 feet instead of continuing to climb to 16,000, In addition, the auto throttle disregarded the 250 kt restriction and continued to accelerate. The controller called to ask our speed and as I looked up from the FMS, I noticed approximately 330 knots... At the time of the incident, the two of us were given an intercept heading, an altitude change, and a speed restriction. In the process of attempting to accomplish the programming for the FMS, listen for ATC, and watch for traffic, the airspeed capture of the auto throttles was overlooked until the speed approached 330 knots.'
In this case, the flight crew was busy dealing with a relatively complex clearance form the ATC which included a speed restriction. ... The use of the FMS in busy airspace in which multiple clearances from ATC are likely, along with multiple aircraft configuration and speed changes, appear to make effective use of the FMS difficult, especially for short-term navigation activities. This difficulty is due to the need for pilots to remain flexible and respond quickly to the needs of ATC. The FMC/CDU, however, apparently is no that east to re-program and is not designed to support short term changes. Although this study did not look at ATC-related problems relative to altitude specifically, many of the ATC-related incidents occurred in the middle altitudes between 10,000 feet and FL240. The complexity of this airspace, and ATC overall, seems to be involving larger portions of a given flight's overall trip. Clearly, the role of ATC should be a major consideration in how the next generation automated systems are designed and operated. (page 4.14-4.16)

"The organization of information within the FMC/CDU appears to be an issue for some pilots. Monitoring the overall status and performance of the aircraft includes being aware of fuel status, lateral path, position, vertical path, and so on. To adequately monitor aircraft status by means of the FMC, the crew must review the information that is presented on a number of different pages which are accessed by means of a number of mode and/or line select keys. Extensive monitoring of the FMC/CDU dimishes the crew's ability to monitor the data in the mode control panel at the same time, thus creating the possibility for missing important information about the status of the aircraft." (page 4.12)

"4.1.2.5 Multiple FMC Page Monitoring Requirements
The organization of information within the FMC/CDU appears to be an issue for some pilots. Monitoring the overall status and performance of the aircraft includes being aware of fuel status, lateral path, position, vertical path, and so on. To adequately monitor aircraft status by means of the FMC, the crew must review the information that is presented on a number of different pages which are accessed by means of a number of mode and/or line select keys. Extensive monitoring of the FMC/CDU diminishes the crew's ability to monitor the data in the mode control panel at the same time, thus creating the possibility for missing important information about the status of the aircraft.
[ASRS incident report #119836] 'Approach DEN from the east on J80 the captain (pilot flying) asked copilot (pilot not flying) to request FL390 due to building thunderstorms over the Rocky Mountains. I (copilot) put FL390 in the right FMS computer to check aircraft capability for FL390. After entering and executing FL390 in 1 L on FMS, I verified that the altitude window on the mode control panel was at 35,000 feet and that the autothrottles did not add power for the climb. At this point, the mode control panel altitude window was holding the aircraft at current cruise altitude of 35,000 feet. This has been an accepted procedure in this situation. After checking altitude capability in the FMC, I mentioned to the captain that we could make FL390 and would save approximately one percent of fuel with the climb. This whole check took probably less than 20-30 seconds. I then called DEN ATC and was advised to expect FL390 in approximately two minutes due to traffic. Anticipating the higher altitude, I left FL390 in the FMC active cruise page, once again checking to make sure the window read 35,000 feet. I continued to prepare the ACARS position report to be transmitted over DEN. We were approximately three minutes east of DEN. I remember checking the ETA for SLC and entering the fuel over DEN as 22.5. Since I was preparing the position report I changed from the Cruise page in view with the FL390 Cruise active page on it. During the minute or minute and a half of preparing the ACARS position report and waiting for the ATC clearance to FL390 the captain (pilot flying) changed the mode control panel altitude window to 39,000 feet, anticipating the climb. Of course, the FMC not being constrained at 35,000 feet any longer started to slow climb to FL390. The captain also began a passenger announcement to the passengers about DEN and the turbulence, and that we expected to climb to a higher altitude shortly. The center called, 'Maintain FL350.' Without even hesitating, I responded 'Roger maintain 350.' By this time the captain (pilot flying) had already started a push-over. The aircraft had reached an altitude of approximately FL357. After the aircraft was returned to FL350, I checked the mode control panel altitude window and was surprised to see 39,000 feet. We returned to 35,000 feet, our cleared altitude. Within a few minutes, Center cleared to FL390. Crew coordination and lack of communication may have contributed to the altitude excursion and conflict. The mode control panel window is, in my judgment, the last step in the altitude change process, to be changed after the clearance has been received. The autoflight system will not depart the mode control panel altitude, even it the FMC is programmed for a different altitude.'
This example provides a feel for the number of information sources the crew must monitor. ... Monitoring a number of pages through the FMC/CDU can contribute to substantial cognitive workload in that the pilot must remember what page is appropriate for finding the desired information and how to access that page, either through mode select or line select keys." (page 4.12-4.14)

In ASRS incident report #123705 "'... My point is that I have almost 3,000 hours in the airplane and I am very knowledgeable in its operation, but pilots cannot rely on the computers to fly the aircraft.'
The reported lack of trust in the FMS that arose from this incident was mirrored in many of the other reports reviewed for this study. Although not cited specifically, it was clear that many of the pilots submitting these reports were, and still are, receptive to the additional sophistication and efficiency represented by the FMS, but have quickly become mistrustful when they experienced errors, irrespective of the cause." (page 4.5)

"Although none of the reports dealt with training directly, many cited training as a factor in the incident's occurence (see Table 4-2). Many of these pilots reported that they did not have a good understanding the of [in sic] underlying logic and limitations of the FMS, and seemed to become easily confused and overloaded in high workload situations, when they continued to try and program the FMS. From the perspective offered by these reports, it appears that current pilot training does not accurately reflect real world needs in using the FMS relative to ATC requirements and the resulting high workload." Table 4-2 lists three Associated Incident Events and Precursors, one of which being "Training/flight crew proficiency related errors/performance problems". Out of the total of 99 citations, 12 (12%) of them were Training/flight crew proficiency related errors/performance problems.(page 4.23)

"This study investigated human error reduction effect of a special type of warning - a warning that does not instruct participants how to do a task and does not involve catastrophic consequences if ignored. Cautions should be exercised in interpreting the results of this study. A practical implication of this study is that warning instructions that do not include knowledge of how to perform the task have only limited effect in human error reduction they may only reduce human errors for HKC [high knowledge content] participants in performing high inference tasks. Whenever possible, do include knowledge of how to perform a task into warnings. Theoretically, this study supports Lehto’s (1991) and Lehto and Salvendy’s (1995) theoretical analysis of when and why warnings can help reduce human error and that is when warnings can help participants transit from skill- or rule-based performance level to knowledge-based level." (page 155)

"SAMPLE NARRATIVES FROM ASRS REPORTS
The following ... ASRS narratives were chosen to illustrate the range of problems identified in the initial survey. ...
[ASRS incident report # 148853] ... On an en-route descent into Dayton our clearance was direct RID VOR, direct DAYTON with a descent to 11,000 ft. The controller gave us a new clearance to cross 10 miles west of RID at 10,000 ft. The captain, being less experienced in using the flight management computer than I, wanted me to show him how to program the descent for the new restrictions. We put the restrictions in the magic box, and for some reason, almost certainly something we did improperly, the machine wanted to make the restriction 10 miles east of RID. By the time we caught the error in the midst of doing checklists and the usual cockpit duties we were too late to make the restriction. Nothing was said and there was no conflict. ...
In flight training on the operation of complex systems such as the autoflight and FMSs is going to happen, but it should be done during the low-workload cruise phase of flight. Unfortunately, as the report of this incident points out, the clearances that require complex reprogramming usually occur during the already busy climb and descent phases of flight." (page 11)

"SAMPLE NARRATIVES FROM ASRS REPORTS
The following ... ASRS narratives were chosen to illustrate the range of problems identified in the initial survey. ... [ASRS incident report # 134179] ... 'Taking off from ORD in a [large transport] with a light load and maximum takeoff power (engine anti-ice on). The first officer, just out of training, was flying the leg while I handled communications. Each of us had only done one previous leg in a [large transport]. (I have several hundred hours in [wide body transport].) The combination of cold weather, maximum power, and a nearly empty aircraft caused the airspeed to increase extremely rapidly after liftoff. The first officer was reluctant to raise the nose to the extreme angle required to maintain 250 (in this case, probably better than 25 [degrees]). When I saw the airspeed zipping through 270, I warned him to slow down and he disconnected the autothrottles, manually retarding power and raising the nose just as the flight director went to altitude capture (between 3,500 and 4,000 ft.). Attempting to level at 5,000 ft. , we overshot by 200-300 ft. (still fast), when we were cleared to 14,000 ft. (I don't really know whether we actually broke 5,300 ft. Before being cleared up.) I punched flight-level change, but the autothrottles refused to engage initially. In the confusion over exactly what was wrong, we both were slow to respond to several heading changes, which understandable annoyed the controller.
Nothing really serious here, except the same old story. Both of us were engrossed in trying to figure out why this computerized marvel was doing what it was, rather than turning everything off and manually flying (which we finally did) until we could sort things out. This is a common tendency in this type of cockpit, but our familiarity with the super high performance of the LGT [large transport] was a contributing factor. It really is a handful to takeoff and level at a low altitude and seems to require an almost immediate power reduction to maintain a reasonable nose attitude at low weights.' ...
In addition the problems caused by the crew's lack of familiarity with high performance of their twin-engine aircraft., [in sic] this incident illustrated two other problems. First, like the crew attempting to set of the hold at waypoint BUCKS 9incident [in sic] No. 1 [ASRS incident report #144196]), this crew mentions being distracted by attempting to determine why the autoflight system was not performing as they expected. Second, as reported in prior examples, this crew appears to have difficulty translating the departure clearance from ATC language to the language of the autoflight and FMS." (page 14-15)

"SAMPLE NARRATIVES FROM ASRS REPORTS
The following ... ASRS narratives were chosen to illustrate the range of problems identified in the initial survey. ...
[ASRS incident report # 148853] ... On an en-route descent into Dayton our clearance was direct RID VOR, direct DAYTON with a descent to 11,000 ft. The controller gave us a new clearance to cross 10 miles west of RID at 10,000 ft. The captain, being less experienced in using the flight management computer than I, wanted me to show him how to program the descent for the new restrictions. We put the restrictions in the magic box, and for some reason, almost certainly something we did improperly, the machine wanted to make the restriction 10 miles east of RID. By the time we caught the error in the midst of doing checklists and the usual cockpit duties we were too late to make the restriction. Nothing was said and there was no conflict. ...
This narrative [ASRS incident report #148853] also illustrates another factor that was observed in several reports: in-flight training, in which one pilot attempts to instruct the other pilot on the use of automatic equipment, can distract both pilots from the primary task of maintaining altitude awareness." (page 11)

"SAMPLE NARRATIVES FROM ASRS REPORTS
The following ... ASRS narratives were chosen to illustrate the range of problems identified in the initial survey. ... 1. Crew Distracted by Programming the Flight Management System
The following report from the first officer of a glass cockpit airliner describes an altitude deviation that occurred during the descent phase of flight as both pilots attempted to program a holding pattern into the flight-management system (FMS). ...
[ASRS incident report # 144196] ... Descending from higher flight levels to 15,000 ft. on Center clearance had anticipated and received clearance to hold at BUCKS due to anticipated weather delay. Captain flying on autopilot using LNAV [lateral navigation] on company stored route everything routine. Captain is a check airman on this aircraft. I am 5-year airline pilot, but only 3 months experience on [glass] equipment. Captain pulled up 'hold' page on FMC [flight-management computer] and began to enter data. We were currently descending on rate of descent command to 15,000 ft. altitude as assigned. I'm reading holding data to captain as he's entering data via keypad. Both eyes off of primary flight instruments but in altitude capture mode (VNAV) [vertical navigation] and LNAV so both pilots are anticipating automatic level off at 15,000 ft. altitude. Captain apparently entered hold at PPOS [present position] instead of at BUCKS in error, and aircraft begins left turn unexpectedly. I, in confusion, not knowing he entered PPOS by mistake, stated, 'This isn't right,' and saw captain disengage autopilot to stop turn while he goes back to FMC page to find out where we should be, got to VOR/ILS [very high-frequency omnidirectional radio range/instrument landing system] mode and picked up en route chart to tune in present position. Aircraft descended below 15,000 ft.. Horn went off at approximately 14,600 ft. Captain called out 'Out of 16 for 15.' I said no (looking up) and said we were at 14,500 ft., only cleared to 15,000 ft., that was a low altitude alert not '1,000 ft. to go' horn. Immediately pulled back up. No ATC [air traffic control] communication took place by either ATC or us about event. Within 30 seconds, ATC gave us new frequency and cleared us to 13,000 ft. No conflict or discussion or awareness of event was stated by ATC. Below 15,000 ft. for about 20 seconds.
Cause is obvious - both pilots' attention diverted from aircraft flight path. I suggest hold-page (not used often) be made more user friendly. Have captain tell first officer to watch flight progress while he is correcting other problems. I personally feel that although the FMS is a great tool, but shouldn't be used all the time, especially below FL180. It takes too much pilot attention, especially when newly assigned to the aircraft. Just because the technology exists doesn't mean it should be used. Better pilot training on FMC/pilot problem areas like this should be provided. Also, loss of a flight engineer's eyes is not in the interest of aviation safety.
... This incident report illustrates several problems that can occur in the altitude-change task. The crew apparently correctly received the clearance, set the altitude alert, and set up the autopilot to descend and capture the cleared altitude of 15,000 ft. If the crew had done nothing more, the aircraft was set up to level off automatically at the new altitude. Both pilots were distracted from their routine task of monitoring the altitude change by the nonroutine task of programming the flight management computer (FMC) for the upcoming holding pattern." (page 8-9)

"We will see in other incidents that there is often a mismatch between the pilots' intentions and the interface that is provided for communicating those intentions to the autopilot and flight management system." (page 10)

Table 2 summarizes the "descriptive factors assigned to altitude-deviation reports" from traditional cockpits and glass cockpits. In the traditional cockpit, 16 out of 50 (32%) reports suggested that complacency was a factor in the incident and in the glass cockpit, 24 out of 50 (48%) reports suggested that complacency was a factor in the incident.(page 7)

Table 2 summarizes the "descriptive factors assigned to altitude-deviation reports" from traditional cockpits and glass cockpits. In the traditional cockpit, 10 out of 50 (20%) reports suggested that training was a factor in the incident and in the glass cockpit, 34 out of 50 (68%) reports suggested that training was a factor in the incident.(page 7)