INPO 09-011 INTRODUCTION This document provides insight into the success factors and behaviors of results-oriented performance improvement (PI) systems. It is intended to help leaders better describe individual roles in performance improvement in a way that is specific, understandable, and relevant to daily work activities. Recognizing that an organization's culture of performance improvement is born of underlying values that are shared and then embraced to create desired behaviors and visible results, an industry task force created the following vision statement for performance improvement: Every individualleader, manager, supervisor, and workerembraces a desire for continuous improvement. Each person goes home at the end of the workday proud of having prevented or corrected a problem, and making the workplace better. A task force representing a broad cross-section of the nuclear power industry participated in the development of this document. Task force members contributed their guidance and expertise to keep the product simple and sensible. The task force first explored whether another performance improvement document is needed, given that key aspects of PI processes have been addressed in INPO Performance Objectives and Criteria, three INPO principles documents, two INPO guidelines, an INPO good practice document, and other industry documents described in Appendix D. The task force concluded that a document was needed to fill a gap in communicating a clearer vision of behaviors that make performance improvement a core business value. Key values and behaviors from previously issued documents are reinforced and are supplemented with an emphasis on today's challenges. Leadership teams can use this document as a tool to achieve better organizational alignment around a vision of performance improvementsomething that can be used to rally the entire workforce.

i

INPO 09-011 The document INPO 05-005, Guidelines for Performance Improvement at Nuclear Power Stations, remains intact as the overall model of performance improvement processes and practices. The model, shown in Figure 1, describes three main areas of performance improvement: identifying performance gaps through performance monitoring, analyzing problems and identifying solutions, and implementing solutions to close the gaps.

Figure 1 This document supplements INPO 05-005 by refining the central elements of leadership and oversight, knowledge and skills, and culture excerpted in Figure 2 below. These central elements are necessary if an integrated PI process is to become a core business value. In addition to INPO 05-005, the task force drew heavily on the concepts described in INPO Principles for a Strong Nuclear Safety Culture to help understand the cultural aspects of performance improvement.

Figure 2 The efforts of the task force resulted in the development of eight key success factors that were further grouped into four central themes. These themes describe the desired outcomes of performance improvement excellence, and they form the outline of this document. The four desired outcomes are as follows:

ii

INPO 09-011

The picture of excellence is well known. Problems are prevented and mistakes are avoided. Performance gaps are thoroughly analyzed and efficiently solved. Performance improvement is ingrained as a core business practice.

Appendix A contains a chart that lists typical behaviors that are illustrative of PI activities being adopted as a core business practice. As many INPO members did with the Performance Improvement model shown in Figure 1, INPO members are encouraged to adopt or tailor the typical PI behaviors chart and to communicate it through posters. This document describes behaviors on three levels of a typical organizational hierarchy: the individual contributor or working level, the supervisor and manager level, and the senior management level. The senior management level most often refers to those in executive management and boards that control budgets; establish the organizational structure; establish policies, visions, and standards; and set the overall tone for the corporation and station. Appendix B includes performance metrics associated with the key success factors described in this document. They are offered as a way to improve the focus on outcomes and results of PI activities. They can supplement the activity-based metrics that most members are currently using. INPO members should consider whether adopting the performance metrics in Appendix B will enhance their existing set of metrics. Appendix C contains discussion topics to help managers assess how well their organizations implement the fundamental performance improvement success factors. Managers are encouraged to reflect on these questions and to use them to stimulate discussion in a variety of management forums. They can also be used as the basis for more formal leadership selfassessment. Appendix D contains key insights from previously issued documents that provide context and background material for leaders to use in communicating a clearer vision of behaviors that make performance improvement a core business value. This material may be useful in management training and orientation programs.

INPO 09-011 A. DESIRED OUTCOME: THE PICTURE OF EXCELLENCE IS WELL KNOWN Success Factor 1: Workers recognize where they've been and where they are, and they are focused on the future. Behaviors and Practices Leaders paint a detailed picture of current performance for the organization by using metrics that focus on desired results, with goals that are benchmarked against industry best practices. Managers use performance monitoring techniques, including trending, to identify current vulnerabilities. Leaders use an appropriate mix of self-assessment, benchmarking, operating experience, and training to ensure plant performance is accurately understood and communicated. This mix is adjusted periodically, as dictated by circumstances. Leaders set increasingly challenging performance goals to counter complacency that sometimes results from satisfaction with current performance. Individuals are engaged in designing and implementing improvement initiatives and in solving problems. Individuals view current performance with an historical perspective.

B. DESIRED OUTCOME: PROBLEMS ARE PREVENTED AND MISTAKES ARE AVOIDED Success Factor 2: People find little problems before they become big ones. Behaviors and Practices Problem prevention and detection behaviors are well ingrained; consequently, fewer corrective actions are needed. Managers create a culture of problem prevention by establishing ways for individuals to share with their coworkers examples of problems they have prevented. Managers and others involved in daily corrective action program screening use a cognitive trending approach to identify and act on repetitive issues before they become problems. Leaders encourage the identification of opportunities for improvement by discussing and rewarding good catches and notifications of near misses. This contributes to building a low-threshold reporting culture.

1

INPO 09-011

Managers, supervisors, and individuals value field observations because they generate feedback that sharpens work habits and provide opportunities to detect and prevent problems. Leading indicators of performance are used to prompt intervention before little problems become consequential. For example, when the rate at which problems are being self-identified changes, managers seek to understand why, and they develop appropriate responses.

Success Factor 3: Workers avoid repeating mistakes that have caused consequential events. Behaviors and Practices Individuals are trained in and thoroughly understand the events basis for error prevention tools. As a result, they consistently apply these tools. Leaders help individuals make the connection between operating experience and their work practices. This helps individuals develop a historical perspective of industry and station events, and it fosters personal commitment to recurrence prevention. The use of operating experience is well integrated into key station processes, including work management, decision-making, and training. Through experience and training, managers develop a practical understanding of error precursors and latent organizational weaknesses. This practical knowledge helps managers predict and intervene so that common pitfalls to organizational problems are avoided. Senior managers use leading indicators of performance decline to prompt their intervention before small problems become consequential.

C. DESIRED OUTCOME: PERFORMANCE GAPS ARE THOROUGHLY ANALYZED AND EFFICIENTLY SOLVED Success Factor 4: The effort spent analyzing problems is commensurate with their importance. Behaviors and Practices Managers are skilled at understanding the significance of events and precursors. Important near-miss events that reveal weaknesses in defense-in-depth are treatedfor learning purposesas if the final barrier had failed.

2

INPO 09-011 When problems are being evaluated, the right analysis tools are used, commensurate with the risk and consequences of the problem. If important problems recur, the depth of investigation is increased to understand the reason(s) for the recurrence. Investigations are assigned to personnel because of their expertise, not their availability. A graded approach is used to allocate PI resources. The result is that the organization is &quot;fixing today's problems today,&quot; and not unnecessarily adding to the investigation and corrective action backlog. Discrete problems are resolved quickly.

Success Factor 5: That which needs to be changed is changed--and no more. Behaviors and Practices Leaders provide tools and resources to manage a low-threshold, high-volume reporting culture efficiently. Managers ensure corrective actions are targeted, timely, and measurable and are effective in solving the identified problems. Training expertise is used to identify and differentiate between training and nontraining solutions. Individuals contribute their knowledge and experience to increase the probability that planned solutions will be effective. Corrective actions are prioritized and managed well, contributing to timely implementation. Change management processes are used, as appropriate, for some corrective actions. The Pareto principle1 is acknowledged as one way to solve problems incrementally and avoid overly broad solutions that sometimes come about through well-intentioned but overzealous approaches.

Success Factor 6: Managers know how improvement activities are progressing. Behaviors and Practices Managers influence culture through both routine and unscheduled observations of daily work activities.

1

The Pareto principle is also known as the 80-20 rule or the law of the vital few. It states that for many problems, roughly 80 percent of the effects come from 20 percent of the causes.

3

INPO 09-011 Managers place their eyes on important problems in the plant and witness key evolutions while they reinforce standards and fundamental practices. Leaders ensure that sufficient resources and training have been provided and that appropriate change management tools are being used to achieve change goals. Managers conduct focused self-assessments when important change initiatives require early and ongoing progress checks. During these checks, managers confirm and explain worker understanding of the reasons for changes. Managers establish effectiveness reviews for actions that are designed to prevent problem recurrence. These are more than problem report number countsthey confirm that behaviors, work practices, and barriers have been institutionalized.

D. DESIRED OUTCOME: PERFORMANCE IMPROVEMENT IS INGRAINED AS A CORE BUSINESS PRACTICE Success Factor 7: Performance improvement is integrated into management systems. Behaviors and Practices Leaders and managers integrate PI activities fully within management systems that define station values and core business (for example, management review meetings, plant health committees, plan-of-the-day meetings, personnel performance reviews, business plans, and budget meetings). Leaders and managers make sure that PI tools are efficient and are useful in managing and prioritizing daily work activities. Processes are integrated to eliminate barriers, frustration, and duplication of effort, as well as to gain user confidence. These wellintegrated processes build teamwork across departmental boundaries. Leaders and managers work together such that performance gaps are identified and investigated using all appropriate PI tools. They keep planned solutions and desired results visible to the workforce, and they ensure individuals know how their work contributes to eliminating performance gaps. Leaders and managers regularly monitor the progress of actions taken to address known performance gaps, and they make course adjustments as needed. They provide a means to ensure that updated practices are anchored in the organization and that improvement is sustained.

4

INPO 09-011 Success Factor 8: Performance improvement is demonstrated through meaningful results. Behaviors and Practices Leaders regularly assess overall plant performance to validate the effectiveness and efficiency of PI activities. Leaders ensure that effectiveness measures for PI activities are outcome-based and results-oriented. Actions such as training and organizational changes, although sometimes necessary, are not mistaken for results. Leaders foster a culture of accountability for results through coaching and mentoring. Managers prioritize improvement activities, commensurate with risk, so that adverse consequences are preempted. Managers use rewards and positive reinforcement to build shared organizational values and other intangible benefits that derive from PI activities, such as improved work-life balance.

5

INPO 09-011

This page is intentionally left blank.

6

INPO 09-011 Appendix A Typical Behaviors Representative of Performance Improvement as Core Business Desired Outcome The Picture of Excellence Is Well Known Problems Are Prevented and Mistakes are Avoided Individuals

Understand current performance with an historical perspective (know where station performance has come from, where it is now, and your role in advancing it toward excellence) Report problem trends to supervisors

Managers &amp; Supervisors

Know how our work groups and station compare to industry excellence, and communicate any gaps widely Periodically lead self-assessment teams Make it a habit to personally look at problems in the field Know how work is really being done Demonstrate through daily routines how we value a low threshold problem reporting culture Reward good catches Welcomeand respond promptly toreports of near misses Model prevention and detection behaviors so that the need for corrective actions is minimized Promote a culture of prevention by providing a way for individuals to share examples of how they have avoided problems Help subordinates make the connection between operating experience lessons learned and their daily work practices Monitor corrective action timeliness and backlogs to achieve a culture of &quot;fixing today's problems today&quot; While screening daily problem reports, identify opportunities to learn more aboutand fixorganizational problems Choose the right level of problem investigation to match the actual/potential consequences of events Raise the level of cause investigation when a problem thought to be solved happens again Give investigation teams the right expertise and oversight Challenge improvement initiative goals to pass the SMART test (specific, measurable, achievable, realistic, timely) Verify the effectiveness of significant corrective actions Make sure periodic trend reviews do more than provide a status reportthey need to add value by being action oriented Follow through on self-assessment and benchmarking actions Observe and provide feedback to workers on how well they are putting into practice work habits taught in training Know precisely how change is progressing by monitoring change initiatives Remove bottlenecks and expedite the handling of problem reports Improve the integration of PI activities Focus on the right performance metrics, and avoid overfocusing on any single metric Ensure corrective action turnaround time is short enough to avoid adverse consequences from identified problems

Senior Managers

Help people at all levels of the organization understand their roles in achieving a continuous improvement culture Set increasingly challenging performance goals to avoid complacency Use self-assessments and benchmarking strategically, to attain business and safety outcomes Keep performance gaps visible to the workforce Promote a vision of problem prevention and detection through rewards and recognition Use leading indicators to prompt intervention before little problems become big ones Establish a culture that recognizes and overcomes the difficulty of using and internalizing operating experience Assess overall plant performance regularly to determine how resources spent on performance improvement activities add value Select the near-miss events that will be treatedfor learning purposesas if the last barrier had failed Lead efforts to resolve organizational contributors to events Kick off training sessions when training has been identified as a solution to an organizational problem Reward problem analyzers as well as finders and fixers Help the organization learn from the subtle, underlying causes of performance trends, including overseeing the right level of causal analysis Ensure a methodical process is used to determine whether organizational or individual accountability underlies problems Confirm that corrective action or management review committees are adding value efficiently Build and sustain improvement momentum by communicating tangible accomplishments of improvement initiatives

Question unexpected conditions Report near misses Adopt the &quot;if in doubt, fill it out&quot; philosophy for reporting problems at a low threshold Use knowledge of major station and industry events to internalize work practices that prevent them from recurring Solicit feedback on performance from peers and supervisors Fix minor problems quickly, and document them for trending Participate actively in event investigations, and help ensure resulting corrective actions will really fix the problem Be persistent and remind managers if problems we've identified are not being fixed in a timely manner When assigned a corrective action, ensure the desired result is understood and the action is fully completed before closure Hold managers accountable for supporting initiatives that come from self-assessments and benchmarking After attending training that was designed to correct performance shortfalls, put this learning into practice on the job

Performance Gaps Are Thoroughly Analyzed and Efficiently Solved

Performance Improvement Is Ingrained as Core Business

Identify and communicate to managers examples of performance improvement processes that do not work well together (selfassessments, benchmarking, corrective action program, training, operating experience) Point out when due date priorities, established by routine PI processes, are inconsistent with real needs Notify managers of low-value performance improvement activities that could be eliminated or changed Embrace performance improvement activities as part of your daily work habits

Model behaviors that demonstrate PI is a core business value, and overtly demonstrate support of PI Provide efficient PI tools that are helpful in managing daily work Confirm that improvements, once attained, are anchored in the organization's culture Be an advocate for PI activities during budget meetings Reinforce the use of an integrated mix of self-assessments, benchmarking, training, and operating experience to understand and solve problems

A-1

INPO 09-011

This page is intentionally left blank.

INPO 09-011 Appendix B Associated Performance Metrics Outcome The Picture of Excellence Is Well Known Success Factors 1. Workers recognize where they've been and where they are, and they are focused on the future 3. Workers avoid repeating mistakes that have caused consequential events 4. The effort spent analyzing problems is commensurate with their importance 5. That which needs to be changed is changedand no more 2. People find little problems before they become big ones 6. Managers know how improvement activities are progressing Performance Improvement Is Ingrained as Core Business 7. Performance improvement is integrated into management systems 8. Performance improvement is demonstrated through meaningful results Metric Management Review Meeting trends Number of adverse trends and performance gaps closed INPO performance indicator index trend Self-identification percentage Site engagement ratio Good-catch trend Number of identified latent organizational weaknesses Active error rate Percentage of problem reports closed to immediate actions Investigation percentage Quality of cause evaluations Number of repeat events Corrective action to problem report ratio Cycle time, by significance category Learning opportunities and impact from self-assessments and benchmarking Open long-term condition reports Equipment reliability index Operational focus aggregate index Corrective action program health index Preventable equipment failures

Problems Are Prevented and Mistakes Are Avoided

Performance Gaps Are Thoroughly Analyzed and Efficiently Solved

Note on activity-based metrics: The intent of this table is to describe outcome and results-based metrics linked to the excellence in performance improvement success factors. It is recognized that activity-based metrics, such as number of management observations of training, numbers of supervisor field observations, and numbers of new problem reports generated, also provide useful information to managers and they should not be discounted. Some activity-based metrics can be early warning signs or leading indicators of decline. B-1

INPO 09-011

Metric Definitions Note: The stations and utilities identified in parentheses following some metrics have recommended/used those metrics. active error rate: This represents human performance errors per 10,000 person hours worked. corrective action program health index: This measures six key corrective action program (CAP) trend areas to assess management engagement in the CAP process. The six measures are ratio of corrective actions to condition reports, corrective actions older than six months, on-time root cause analyses, root cause and higher-tier apparent cause analysis quality, number of trend condition reports, condition report inventory. (Cooper Nuclear Station) corrective action to problem report ratio: This is the ratio of the number of corrective actions to problem reports. The goal is 1.5. A ratio less than 1.25 or higher than 1.75 could reflect not enough or too many actions being taken. (Cooper Nuclear Station) cycle time: This is the average number of days (over the last year) between the time of corrective action initiation and the time of effective implementation of the solution. This is computed by significance category, with the most important one associated with corrective actions to prevent recurrence (CAPR). (Duke Power Company) good-catch trend: Good catches are defined as identified degraded conditions, error precursors, or latent organizational weaknesses that will ensure a preemptive action is taken prior to a consequential result. (NextEra Energy Seabrook) INPO PI index trend: INPO has formulated a performance indicator index for use in tracking overall plant performance. The index is calculated using a weighted combination of WANO and INPO performance indicators and has a value between 0 and 100. Stations with higher indexes generally have better performance in areas monitored by these indicators. While the value of this index at any specific time might provide little discernment of historical station performance, the trend of this index over many years is a useful representation of where overall station performance has been and how it compares to industrywide performance. investigation percentage: This represents the ratio of condition reports sent for further investigation (that is, root cause, apparent cause, equipment apparent cause, common cause, or work group evaluation) divided by the total number of condition reports generated for the month. A high ratio implies a deeper understanding of issues. (Exelon) learning opportunities and impact from self-assessments and benchmarking: In addition to monitoring the number of opportunities for improvement identified during self-assessments and benchmarking, this metric checks for the timeliness and effectiveness of actions born from these activities. (Quad Cities Station)

B-2

INPO 09-011 management review meeting trends: A comprehensive set of sitewide performance indicators is typically scrutinized during monthly collegial reviews. An overall view of improvement momentum can be gained by comparing the total number of metrics that are improving, remaining steady, and declining from month to month. number of adverse trends and performance gaps closed: This is the number of adverse trends closed or the number of gaps identified through the Integrated Performance Assessment (IPA) process or the Performance Improvement Integration Matrix (PIIM) that were closed. (Duke Power Company for IPA, Exelon for PIIM, adverse trends for others) number of identified latent organizational weaknesses: A code is applied to new problem reports to identify and resolve latent organizational weaknesses that can aggravate relatively minor events if not corrected. (Pilgrim Nuclear Power Station) number of repeat events: This metric applies to events that reached the root cause analysis level within a three-year period. It is based on the number of subsequent, independent events that resulted from the same basic cause of the original event. open long-term condition reports: This distinguishes between work that can be implemented on line and work that requires a unique system configuration or refueling outage. These items are discussed monthly with the senior leadership team to confirm the appropriateness of the corrective action and its schedule. (FirstEnergy Nuclear Operating Company) operational focus aggregate index: This is an aggregate index based on the following: the number of operator workarounds, operator burdens, combined control room corrective maintenance work orders, combined control room elective maintenance work orders, degraded control room annunciators, control panel deficiency tags older than 90 days, active fire protection impairments, corrective maintenance inventory, elective maintenance inventory, temporary modifications, and operational decisions with compensatory actions. (Callaway Nuclear Plant) percentage of problem reports closed to immediate actions: This percentage demonstrates the value of a low-threshold, high-volume reporting culture in which low-level problems are documented for trending purposes and do not consume unnecessary resources. It is a percentage of problem reports that are closed at initial screening, with no further investment of resources. (FPL Group, Inc.) preventable equipment failures: This is the number of equipment failures that performance monitoring activities should have prevented. Engineering managers and all discipline supervisors make this determination, because it can be subjective. The point is to identify opportunities to improve performance monitoring capabilities. The identification of gaps in this area has led to improved tools for the engineers, clarified expectations for the priority of performance monitoring activities, the revised frequency of monitoring of some attributes, and so forth. (NextEra Energy Seabrook)

B-3

INPO 09-011 quality of cause evaluations: A healthy corrective action program produces quality evaluations of undesirable events and proposes corrective actions that address the causes, cover the extent of condition, and are comprehensive to prevent recurrence. This metric is associated with root cause and higher-tier apparent cause evaluations only. (Entergy) self-identification percentage: This measures the effectiveness of self-assessment processes by determining which issues outside agencies find that line department assessments did not. It is a calculation of the percentage of problems entered in the corrective action program that are identified by the station organization and NOT identified by regulator, INPO, or other outside agencies or that are self-revealed through events (95% is the goal). (Cooper Nuclear Station) (Note: Variability exists in the industry as to whether quality assurance findings are considered to be self-identified.) site engagement ratio: This is the number of people who write condition reports divided by the number of people on site, over a three-month period. (Exelon)

B-4

INPO 09-011 Appendix C Management Discussion Points The following discussion points are provided to help managers and individuals assess how well they implement the fundamental performance improvement success factors. Success Factor 1: Workers recognize where they've been and where they are, and they are focused on the future. The following questions are provided to stimulate discussion and explore our methods for ensuring we do not become satisfied with current performance. Do we understand how station performance stacks up to the rest of the industry? Can we improve the way this is communicated? Which of our performance metrics are really results-oriented, as opposed to number counts? How do we get more results-oriented metrics that tie directly to our safety and business goals? What are the key metrics that provide us with early warning signs of complacency or decline? How do we prepare and implement an effective intervention strategy? Does the station routinely look for gaps to excellence when deriving areas for improvement during self-assessments and benchmarking? Is proactive trending used to identify emerging trends before they become adverse conditions? Is training used to address performance gaps based on vulnerabilities found during trending? Do workers often make recommendations for improvement? How many are acted on?

Success Factor 2: People find little problems before they become big ones. The following points are provided to examine our prevention and detection culture. How do we encourage and reward prevention and detection? Do we acknowledge good catches? How have we demonstrated these values and set examples? Do we provide communication platforms for individuals to share with their coworkers examples of how they predicted and avoided error-likely situations?

C-1

INPO 09-011 How do we make adjustments to PI activities when resource conflicts arise? When PI efforts are postponed, do we recover in ways that sustain the value of PI over time? Are early warning signals being noticed and correctly interpreted, including signals from self-assessments and trending? Do we behave in ways that promote open discussions in meetings, or do we just assume people will feel free to talk candidly? Do we begin dialogues with our view, or do we allow the participants to offer independent evaluations and candid information? If some individuals remain reticent in discussions, do we ask why? Are we comfortable with the quantity and positive impact of time spent in the plant? Do workers clearly understand that a part of their job function is to identify potential problems and recommend solutions? How is that expectation communicated? Are equipment issues evaluated to determine which should have, or could have, been proactively identified and prevented through performance monitoring activities? Do we identify the high risk activities and then assign resources to ensure work planning, contingency planning, and execution are successful? Do we proactively consider the impacts of changes on other programs, processes, systems, and components?

Success Factor 3: Workers avoid repeating mistakes that have caused consequential events. The following points are provided to explore our methods for establishing an error-tolerant work environment. How do we set the example and demonstrate our desire for people to have a questioning attitude? Do we consistently reward these behaviors? Do we promote those with questioning attitudes to positions of higher responsibility? Are we communicating well to the next generation of workers our personal experiences that we have internalized by living through watershed events? Are there additional avenues we can use to help individuals make the connection between operating experience lessons learned and relevance to their daily work practices? When we encourage workers to avoid complacency regarding chances for error and negative consequences, how do we know if this is influencing their work practices?

C-2

INPO 09-011

Do we investigate the use of simple engineering solutions often enough to eliminate error-likely situations? Do we use work observations as coaching opportunities to ensure that human performance tools are being used in accordance with management expectations?

Success Factor 4: The effort spent analyzing problems is commensurate with their importance. The following points are provided to examine the effectiveness of the graded approach to problem analysis and correction. What methods do we use to ensure organizational contributors to events are identified and corrected? Is the management team aligned on what an organizational contributor is? How do we ensure the people we assign to root cause investigation teams have this same understanding? Are investigation teams composed of people with the right skills? Are the results trusted and seen as credible by managers? Are there barriers that inhibit information flow and interpretations? If so, how can we eliminate these? Do we monitor the reporting habits of work groups and shifts to detect unexplained differences? Do investigations go beyond the &quot;first story&quot; straightforward, easily understood facts to the &quot;second story&quot; details of resources, constraints, and incentives? Are investigation teams well supported with the appropriate management oversight and engagement? Is the CAP inventory backlog periodically assessed to reduce the number of longstanding issues? Do analyses of near-miss events treat the potential consequences as real to help personnel understand and address lessons learned? When key performance metrics cross thresholds of undesired performance (often color-coded red in dashboard window displays), do we tend to leap to solutions before we perform thorough analyses to uncover the deeper underlying causes? Do we recognize the challenge in uncovering the learning from the metrics that are subjective (such as the quality of management field observations), and do we provide sufficient guidance and coaching for the analysts?

C-3

INPO 09-011 Success Factor 5: That which needs to be changed is changed--and no more. The following points are provided to explore how well we remain focused on achieving effective corrective actions. How confident are we that corrective actions will be effective and lasting? What can we do to improve this confidence? Do we evaluate the need for formal change management plans for significant corrective actions? Do we consistently establish effectiveness measures at the time we create a corrective action, so that we can provide a healthy challenge to whether the actions will achieve the goal? What is the management response when problem investigations result in an unwarranted number of corrective or enhancement actions that do not relate to the causal factors? How is this information communicated to us? How is the learning from these situations distributed to other cause analysts? Do we check for possible unintended outcomes of proposed corrective actions and other changes that result from event reviews? Is the management team appropriately involved with proposed solutions to organizational problems? How do we ensure managers have the right expertise to understand human performance shaping factors and organizational contributors to events? Have they been trained to understand the basics of human behavior, including the detection of error precursors and error-likely situations? What effects do management bonus and incentive plans have on reinforcing the importance of performance improvement? Do we consistently use an organizational culpability model to determine organizational versus individual accountabilities for events? Do we have a culture that overuses the &quot;and no more&quot; aspect of this success factor?

Success Factor 6: Managers know how improvement activities are progressing. The following questions are provided to stimulate discussion among managers regarding their depth of knowledge on how work is really done in the plant. What are our expectations for how a manager should interact with workers on a daily basis? How have we communicated these expectations to managers?

C-4

INPO 09-011 How well does the leadership team understand the implementation details of major site processes such as corrective action, work management, training, and human performance? Are expectations realistic? Are managers and individual contributors being asked to do too much? When new or revised expectations for worker behaviors are implemented, how do we plan, execute, and persevere with monitoring and feedback to ensure these behaviors are anchored and sustained? Where does this fit into employee performance evaluations and compensation plans? How robust is our method for providing broad-based feedback to the organization on the ability to change and improve? Do we periodically compare the results of management field observations (including unscheduled observations) and condition report trend information to confirm consistency of the performance picture painted by each? Are reasons for differences explored?

Success Factor 7: Performance improvement is integrated into management systems. The following points are provided to examine the integration of performance improvement in management systems. Do daily work activities include PI? Are PI tools considered for use in a cohesive, integrated way such that resources are used strategically? Do we link important improvement initiatives to individual performance goals and incentives? Do we have realistic plans that integrate appropriate PI tools in the right sequence to close identified gaps? Does the plan identify all stakeholders and their contributions? Are required resources considered and provided, not only for plan implementation, but for sustaining the practices over time? Do we sponsor career paths of individuals who are known for their candor in team analyses and discussion (in other words, we do not just praise individuals for good catches, but we actually promote them because of their effective participation in performance improvement activities)? How are individuals made aware of the important improvement initiatives, including progress and what is expected of them to support improved performance? How do we integrate human performance and other business initiatives into performance improvement systems? C-5

INPO 09-011

Is process efficiency sought when performance improvement tools are applied and activities are performed? When alternative solutions with wide cost variations are presented, how do we ensure candid evaluation of the alternatives? Do we employ challenge meetings?

Success Factor 8: Performance improvement is demonstrated through meaningful results. The following points are provided to examine the relationship between our core business values and performance improvement activities. How clear is the picture of current station performance compared to our vision statement? Do we have a good understanding of significant gaps? How are independent groups, such as INPO, the NRC, and independent oversight boards, used to validate those gaps and identify additional gaps? Do we recognize and prevent the possibility that aggregate performance indexes, such as the operational focus index, could mask unacceptable conditions reflected by lower-tier feeder metrics? Do we recognize and foster intangible benefits by sharing success stories and frequently praising individuals who have made significant contributions to eliminate performance gaps? How do we measure the effectiveness and efficiency of our PI activities? Do our measures focus on outcomes and the identification of important gaps, or do they only measure activity? When PI activities identify gaps, how do we establish the appropriate corrective action turnaround time to avoid adverse consequences of problems? How do we coach and mentor managers and supervisors to achieve a culture of accountability? What role do performance improvement activities play in budget meetings? Are PI activities that improve nuclear safety given the sameor moreconsideration as those that improve production/reliability? Does our leader training and development process use coaching and mentoring to help build a culture of accountability for results?

C-6

INPO 09-011 Appendix D Insightful References and the Evolution of Performance Improvement Excellence As part of the research and development of this document, key insights from previously issued documents were collected to provide context and background material for leaders to use in communicating a clear vision of behaviors that make performance improvement a core business value. This appendix, together with the management discussion points provided in Appendix D, should be used to stimulate discussion and possible self-assessment topics. It can also be used in management orientation and training programs. Success Factor 1: Workers recognize where they've been and where they are, and they are focused on the future. Prime the workforce to see the need for and the goal of improvement actions. Personnel must understand what went before and how that led to the current situation, both favorable and unfavorable. The current state must be understood, so that personnel see the wisdom of sustaining what is good but recognize that improvements can be beneficial. Jim Collins describes how continued improvement, however incremental, together with the observable results creates powerful incentives to foster more improvement.2 Communicate tangible accomplishments, and show how these fit into the overall improvement plan. People respond enthusiastically to the buildup of momentum. INPO 01-005, Indicators of Changing Performance Several common traits were noted among organizations that use performance indicators to detect early signs of changing performance. For example, managers use a broad set of detailed indicators monitoring many aspects of plant, organizational, and personnel performance. Their performance monitoring programs are flexible, and individual performance indicators are added, deleted, or modified as needed. Managers take aggressive action to investigate and address the causes of declining performance. &quot;Results-level&quot; indicators reflect overall plant performance in areas such as plant safety, worker safety, and plant reliability. These indicators are typically &quot;lagging,&quot; since they reflect results of past performance. &quot;Performance-level&quot; indicators reflect human and equipment performance that contribute to current results. In conjunction with the process indicators, &quot;performance-level&quot; indicators provide insights into changing performance before plant operation is impacted. &quot;Process-level&quot; indicators allow monitoring of administrative programs designed to control work activities. Early detection of developing process weaknesses allows corrective actions before weaknesses affect overall performance. &quot;Fundamental-level&quot; indicators address underlying factors that may influence future performance.

2

Collins, Jim. 2001. Good to Great: Why Some Companies Make the Leap ... and Others Don't. New York: Collins Business.

D-1

INPO 09-011

INPO 07-007, Performance Assessment and Trending Performance assessment involves periodically analyzing the issues contained in a variety of sources, such as corrective action data, benchmarking and self-assessment results, observation data from station personnel and external groups, performance indicator information, and lower-tier issue systems. This activity finds performance gaps, creates or adjusts action plans, and develops organizational alignment and understanding of priority issues. Managers and other station personnel who are involved in day-to-day CAP implementation recognize trends because they maintain a mental awareness of current and previous department and station issues and &quot;connect the dots&quot; to identify similar or repetitive problems. This is cognitive trending. When a potential or adverse trend is cognitively identified, it is analyzed and fixed. Cognitive trending is not relied on exclusively, because the fallibility of human memory can impact its accuracy. Trends are also identified by individuals who periodically perform more formal, in-depth analysis of CAP data as an input into performance assessment meetings. Formal trending methods rely on the capability to electronically acquire and manipulate CAP data before subject-matter experts review it. Formal trending involves specialized tasks that are most effectively performed or facilitated by a trend analyst who possesses knowledge of the applicable station computer databases and is familiar with analysis tools. INPO 97-003, Safety Focus During Changing Times - Recognizing Indications of Declining Plant Performance Achieving and maintaining excellence in plant operations involves constant vigilance for signs of declining performance. Because perfection is not achievable, even the best organizations will have some signs of declining performance. As a result, detecting changes in existing and new signs can be difficult. PI Task Force Meeting Notes An important metric is the operational focus index. If this metric is going in the wrong direction, then the management team has taken its eye off the ball. Another important metric is the active error rate, because it is behavior- and resultsoriented. Most performance indicators are rearward-looking; but nevertheless, many have predictive value in pointing toward future performance. Management's challenge is to understand when and how to use these metrics to prompt effective interventions.

D-2

INPO 09-011 Principles for Enhancing Professionalism of Nuclear Personnel Management sets goals that encourage continual improvement in performance and avoid a sense of self-satisfaction or complacency. Principles for a Strong Nuclear Safety Culture The pitfall of focusing on a narrow set of performance indicators is recognized. The organization is alert to detect and respond to indicators that may signal declining performance. High standards are used for comparison. A mix of self-assessment and independent oversight reflects an integrated and balanced approach. Success Factor 2: People find little problems before they become big ones. Achieving Significant Improvement: D.C. Cook Nuclear Plant Becomes an Industry Leader. The Nuclear Professional, 2008-1: 2-7 To guide performance while addressing station challenges, managers instituted the core values of: Prevention, Detection, Correction Safety Guiding Principles Attributes of a Self-Improving Culture and Learning Organization Event-Free Human Performance High-Risk Process

Nuclear Regulatory Commission Advisory Committee on Reactor Safeguards 503rd Meeting, June 12, 2003 This meeting was devoted to discussions of safety culture. The following exchange between ACRS member Dana Powers and Nuclear Energy Institute's Vice President Chuck Dugger reflects the difficulty of putting into practice the seemingly easy task of demonstrating a questioning attitude: Powers: The problem I have with a questioning attitude is that...if I am an employee of an organization that aspires to a questioning attitudeand I amthat it is simply a trap for me, that if something bad happens to me that bumper sticker can be right, you know? That the management will come back and say, &quot;Well, you didn't have a questioning attitude.&quot; On the other hand, if I stop doing things because I start asking ever and ever deeper questions, the management comes back and slaps me around the head and says, &quot;Well, you're not very productive.&quot;

D-3

INPO 09-011 Dugger: You know, questioning attitude is not a tool that is something that we easily understand. As a young reactor operator or building operator..., it was not something that just came easily to me to question why we did things one way or another or why the materiel condition existed the way it was. It was something that I had to be trained in, and it was the training that I got through observation of management that helped me understand what a questioning attitude was. And it was through many training sessions and workshops such as this. Principles for Effective Self-Assessment and Corrective Action Programs Station employees recognize that minor problems are often precursors to more significant events, and they identify undesirable behaviors and deficient processes from these minor problems. There is no single &quot;best choice&quot; solution to various self-evaluation and corrective action decision options. Ultimately, the mix of station self-assessment and independent oversight must reflect an integrated and balanced approach. Management decisions regarding problem reporting systems, reporting criteria, and the extent of evaluation or investigation that problems receive must also strike a balance between evaluation thoroughness, corrective action timeliness, and resource allocation considerations. The most effective balance depends largely on how the self-assessment and corrective action programs fit into management's continuous improvement strategy and the station's level of performance. Management should periodically review this balance and adjust the programs as needed. Principles for Enhancing Professionalism of Nuclear Personnel Management maintains an atmosphere of open communication such that problems are brought to its attention undiluted. Principles for a Strong Nuclear Safety Culture Personnel can raise nuclear safety concerns without fear of retribution and have confidence their concerns will be addressed. An organization fosters a high level of trust through timely and accurate communication, including a free flow of information in which issues are raised and addressed. Employees are informed of steps taken in response to their concerns. Success Factor 3: Workers avoid repeating mistakes that have caused consequential events. Principles for a Strong Nuclear Safety Culture Operating experience is highly valued, and the capacity to learn from experience is well developed. Individuals are well informed of the underlying lessons learned from

D-4

INPO 09-011 significant industry and station events, and they are committed to not repeating these mistakes. Excellence in Human Performance People recognize they are fallible, and even the best make mistakes. Error-likely situations are predictable, manageable, and preventable. Individual behavior is influenced by organizational processes and values. People achieve high levels of performance based largely on the encouragement and reinforcement received from leaders, peers, and subordinates. Events can be avoided by understanding the reasons mistakes occur and applying the lessons learned from past events. Drawing on industry or plant operating experience, workers use prejob briefings to identify the most likely errors, necessary defenses, and potential problems. They cautiously consider factors that could lead to complacency regarding the chance for error and potential negative consequences. Error-likely situations are anticipated. Individuals actively monitor and challenge each other's actions and thought processes. They focus their attention on the task at hand, and they take time to think about the task and to ensure their attention is appropriately focused according to the safety significance of the task. Workers know to use the right error prevention tool in the right application at the right time to get results. Success Factor 4: The effort spent analyzing problems is commensurate with their importance. Good-to-great companies need to create a climate where truth is heard and the brutal facts are confronted (Collins, Jim. 2001. Good to Great: Why Some Companies Make the Leap ... and Others Don't. New York: Collins Business.). This is accomplished through the following: Dissect and examine problems critically, without laying blame. This helps extract the truth. Search for understanding and learning, rather than assigning blame for problems. Establish mechanisms--red flags--to draw attention to problems. The key is to turn ordinary information into information that cannot be ignored.

Excellence in Human Performance Earlier attempts by the industry to improve human performance focused on results and individual behavior at the worker level, a characteristic response to human error that prevails today in many organizations. However, organization and management influences on human behavior are equally important but are often overlooked or underestimated. Experience shows that most causes of human performance problems

D-5

INPO 09-011 exist in the work environment, indicating weaknesses in organization and management. This by no means relieves individuals of their responsibility to work safely and reliably. However, to optimize successful performance at the job site, appropriate individual and leader behaviors must occur in concert with appropriate organizational values and processes. All must work in unison. Leaders avoid creating the appearance of a &quot;last barrier blame&quot; culture. Event analysis efforts consistently include interrogating for flawed or missing upstream defenses. INPO 05-003, Performance Objectives and Criteria A consistent and deliberate approach to problem reporting, analysis, and resolution is used to improve performance. Strong line manager ownership of and involvement in the corrective action program contributes to success in problem identification, prioritization, analysis, and timely resolution. Problems and suggestions for improvement are prioritized, analyzed, and addressed commensurate with their importance. Managers are promptly made aware of significant plant problems and follow up to ensure timely resolution. INPO 05-005, Guidelines for Performance Improvement at Nuclear Power Stations Identified root causes are not superficial. They go beyond obvious problem causes to identify the fundamental cause(s) of why the act or failure resulted in a consequential event. Root cause analyses determine actions to prevent recurrence of the event or problem. They achieve this by either preventing recurrence of the root cause(s) or by erecting sufficient barriers to prevent a recurrence of the root cause from becoming consequential. Root cause analyses additionally identify other actions to address selected, high-importance contributing causes that may provide additional defense-indepth. Root cause and selected apparent cause analyses identify organizational contributors to events. These include failed barriers, such as the use of previous industry or internal operating experience; flawed decision-making; deficient processes or procedures; and cultural concerns. Principles for Effective Self-Assessment and Corrective Action Programs Problems are evaluated, commensurate with their significance. Problem analysis, using tools or combinations of tools such as root or apparent cause analysis, job-task analysis, common cause analysis, event investigations, human performance error investigations, and process analysis, uncovers the underlying causes of problems or adverse trends, commensurate with their significance.

D-6

INPO 09-011 Principles for a Strong Nuclear Safety Culture Expertise in root cause analysis is applied effectively to identify and correct the fundamental causes of events. Schein, Edgar H. 2004. Organizational Culture and Leadership. San Francisco: Jossey-Bass Members [of an organization] must hold the shared assumption that learning is a good thing worth investing in and that learning to learn is itself a skill to be mastered. Success Factor 5: That which needs to be changed is changed--and no more. ACAD 02-001, The Objectives and Criteria for Accreditation of Training in the Nuclear Power Industry Methods to determine the effectiveness of training are considered prior to the development of the training. Carrol, John S. &quot;Knowledge Management in High-Hazard Industries.&quot; In Accident Precursor Analysis and Management: Reducing Technological Risk Through Diligence, edited by James R. Phimister, Vicki M. Bier, Howard C. Kunreuther, National Academy of Engineering. Washington, D.C.: National Academies Press, 2004. In our research on incident investigation teams in nuclear power plants, we assumed that teams that used root cause analysis to make deeper investigations of precursor events would generate more knowledge and that organizations would implement more effective changes that would improve performance. We discovered, however, that the investigation teams and the managers to whom they reported had very different ideas about what constituted a good investigation and a good report. The teams wanted to find the causes of precursors, to dig deeply and identify failed defenses. The managers wanted actionable recommendations that would fix problems and reestablish control. Managers seeking efficiency delegated participation on the team and waited to respond to a draft report rather than taking the time to work directly with the team. As a result, the hand-off from team to manager was often ineffective. Reports were sometimes modified or negotiated to obtain manager &quot;sign off,&quot; and recommendations were sometimes watered down or folded into other activities, or even refused, on the basis of cost or other practicalities. Managers often thought investigation teams were unrealistic, whereas the teams thought managers were defensive.

D-7

INPO 09-011 INPO 05-005, Guidelines for Performance Improvement at Nuclear Power Stations Significant events do not recur. Analysts are proficient in identifying root and contributing causes and in developing corrective actions that preclude recurrence. Leaders use effectiveness reviews to provide specific, focused oversight and follow-up of important improvement actions. They review and question the basis for effectiveness review conclusions, ensuring the focus remains on results achieved and not just actions completed. Experiences from effectiveness reviews are used to adjust future problem resolution actions. Leaders provide direction, oversight, coaching, and mentoring of root cause teams to help define the boundaries of their charterparticularly when scope expansion is vulnerable to well-intended but overzealous extent-of-condition or extent-of-cause reviews. INPO 06-003, Human Performance Reference Manual Performance analysis principles recognize that leaders participate in corrective actions in two main ways: they shape the work environment, and they influence worker behaviors. The work environment includes organizational and cultural conditions that affect individual behavior (for example, distractions, awkward equipment layout, complex procedures, at-risk norms and values, or cavalier work group attitudes toward various hazards). Conditions at the job site that can provoke error are not mysterious and obscure. They are noticeable, if people look for them. Error precursors are, by definition, prerequisite conditions for error and, therefore, exist before the error occurs. Job-site conditions can be changed proactively to minimize the chance of error. This is more likely if people possess an intolerant attitude toward error traps. Principles for Effective Self-Assessment and Corrective Action Programs Evaluations of lower-significance problems focus on correcting the immediate (or apparent) cause and may not address the root cause. For very simple problems, the cause may be obvious and does not need more rigorous analysis to determine corrective actions. For other lower-significance problems, corrective actions may correct the immediate situation but not prevent recurrence. If similar problems occur, trending can identify the commonalties and trend analysis can identify the root (or common) causes. Success Factor 6: Managers know how improvement activities are progressing. Principles for Enhancing Professionalism of Nuclear Personnel Management ensures that work is performed in accordance with established plans, schedules, and procedures to achieve maximum clarity of direction, quality of performance, and management credibility. Appropriate levels of management are actively involved in the day-to-day activities of the plant, including routine operations,

D-8

INPO 09-011 testing, and outages. Leaders use specific and timely feedback methods that enable individuals to change their behaviors. When managers and supervisors coach personnel to attain desired behavior, they must monitor performance, measure against goals and standards, and assess and correct the causes for performance problems to eliminate repeat errors. This coaching for improvement function also extends to management observers who observe anomalous behavior, teamwork breakdowns, or a relaxation of standards. Principles for a Strong Nuclear Safety Culture Managers regularly communicate to the workforce important decisions and their bases, as a way of building trust and reinforcing a healthy safety culture. Worker understanding is periodically checked. SOER 02-4, Rev. 1, Reactor Pressure Vessel Head Degradation at Davis-Besse Nuclear Power Station Based on good historical performance, senior managers assumed that problems were being identified and corrected at the lowest levels in the organization. Station managers were unaware of the actual condition of the RPV head or that conditions related to boric acid accumulation and corrosion had worsened over the past several years. They were not cognizant of the results of RPV head and containment inspections that would have identified the degrading conditions. They did not recognize the extent of other conditions created by boric acid leakage, such as corroded electrical conduit, piping supports, structural steel, and grating in the containment building. In effect, the notion of &quot;management eyes on the problem&quot; was undervalued. INPO 06-003, Human Performance Reference Manual Change management is a methodical process that enables managers to establish the direction of change, align people and resources, and implement the selected modifications throughout the organization. Regardless of the scope of the change, it should be managed.... Experience has shown that change fails most often when implemented without the following: · · · · · · · a clear vision a plan an obvious or self-revealing value added (especially for those most affected) consulting the people affected (without their participation) sufficiently considering the new values, attitudes, and beliefs needed positive reinforcement of new behaviors by supervision and management (emphasis added) patience and perseverance

D-9

INPO 09-011 Effective change management reduces the potential of error by managers when they change things. Without a structured approach to planning and implementing change, the error potential of managers and the support staff is higher. Success Factor 7: Performance improvement is integrated into management systems. Excellence in Human Performance Managers seek to simplify work processes. Processes are designed and maintained simple and easy to use. Individuals cannot repeatedly overcome inefficient or illogical processes. Additionally, managers see that individual performance is not burdened by ineffective coordination among work groups, unrealistic time demands, inaccurate procedures, or distractions. They closely monitor interfaces between functional departments, such as process handoffs, to verify effectiveness and efficiency. Leaders focus attention and energy on preventing recurrence of organizational weaknesses that breed error-provoking conditions and weaken defenses. Leaders can determine organizational weaknesses from broader trends of data derived from plant events, self-assessments, and voluntary reports. Industry Task Force Meeting Minutes The benefit of a fully integrated system of performance improvement is that major performance gaps are identified, these gaps are made highly visible to the leadership team, planned solutions are kept visible, workers are engaged in the solution, a vehicle exists for monitoring progress and check/adjust activities along the way to the final solution, and a means is provided to ensure improvement is sustained (anchored in the organization). INPO 05-005, Guidelines for Performance Improvement at Nuclear Power Stations Managers emphasize and demonstrate the importance of achieving consistency of purpose, strategy and direction, and integration of improvement priorities. They develop and articulate an overall improvement vision and/or strategy and, as small gains are achieved, tie these gains to the vision/strategy to build commitment to help sustain performance improvement. Senior site managers emphasize that problem reporting and resolution using the corrective action program is &quot;a core business value&quot; for the station and unequivocally support, endorse, and expect strong line management ownership of the program. Leadership Fundamentals to Achieve and Sustain Excellent Station Performance Management systems that define the culture, values, and core business practices, such as annual performance reviews, incentive plans, communication plans, business plans, and

D-10

INPO 09-011 other ways managers reward and sanction performance, need to be included within the picture of a fully integrated PI system. Leaders ensure management processes, including business planning, performance management, rewards and recognition, incentive compensation, succession planning, and strategic communication, are aligned toward closing gaps to excellence. Success Factor 8: Performance improvement is demonstrated through meaningful results. Good-to-great companies do not focus on what to do to become great; they focus equally on what not to do and what to stop doing.3 Avoid proclaiming major goals early in the improvement process. Instead, ensure that each step in each action is understood. Demonstrate how both tangible and intangible results will improve plant and personnel performance. As momentum toward improvement builds, people will be encouraged and will believe that major goals can be achieved. Daniels, Aubrey. 1999. Bringing Out the Best in People, How to Apply the Astonishing Power of Positive Reinforcement, McGraw-Hill Managing solely by results is a very risky practice. (Shortcuts etc. may be involved.) It is not enough to know that something is working. We need to know why it works. Instead, pinpoint the specific results you want first, and then identify the active behaviors necessary to produce those results. Active behaviors can be distinguished from inactive behaviors by the &quot;dead-man's test,&quot; which is, &quot;If a dead man can do it, it isn't a behavior, and you shouldn't waste your time trying to produce it.&quot; Examples include zero defects, days without a lost-time accident (dead men never have accidents and never produce defective parts). To effectively apply positive reinforcement, you need to pinpoint behaviors that are active, measurable, observable, reliable, and under the control of the performer. Kirkpatrick, D. L. 1994. Evaluating Training Programs: The Four Levels, San Franscisco, CA: Berrett-Koehler What is quality training? How do you measure it? How do you improve it? The four levels of training evaluation can help you make sound training investment decisions, they can help the training community ensure that courses are working, they can help operations departments identify barriers that are preventing skills from being applied, and they can help your company be confident that the hours your employees spend in training and the dollars that it invests in its people are time and money well spent. Level 1 evaluations involve student reaction. Level 2 evaluations involve learning. Level 3 evaluations involve behaviors. Level 4 evaluations involve results. Lewis, Michael. 2003. Moneyball: The Art of Winning an Unfair Game, W.W. Norton &amp; Co.

3

Collins, Jim. 2001. Good to Great: Why Some Companies Make the Leap ... and Others Don't. New York: Collins Business.

D-11

INPO 09-011

Baseball strategies were often wrongheaded and baseball players were systematically misunderstood. The more you examined the old measurement devices, the less apt they seemed. There was a system of perverse incentives for anyone who trotted out onto a baseball field. The fetish made of &quot;runs batted in&quot; was another good example of the general madness. RBI had come to be treated as an individual achievementfree agents were paid for their reputation as RBI machines when clearly they were not. Big league players routinely swung at pitches they shouldn't to raise their RBI count. Why did they get so much credit for this? To knock runners in, runners needed to be on base when you came to bat. There was a huge element of luck in even having the opportunity, and what wasn't luck was partly the achievement of others.

INPO issues 92-002,Guidelines for the Organization and Administration of Nuclear Power Stations, which contains industry standards for collection and analysis of performance indicator data and for determination of root causes and corrective actions for station events and undesirable performance trends

INPO issues Good Practice 07-007, Performance Assessment and Trending, General Practices for Analyzing and Understanding Performance

D-13

INPO 09-011 Evolution of Performance Improvement Excellence 1970s: 10CRF50 Appendix B, QA Criteria, Section XVI, Corrective Action: conditions adverse to quality ... promptly identified and corrected ... significant conditions adverse to quality cause determined and corrective action taken to preclude repetition ... documented and reported to appropriate levels of management March 1989: INPO issues Principles for Enhancing Professionalism of Nuclear Personnel, which reinforces the following aspects of performance improvement: Management is proactive and responsive. Solutions to problems are pursued with the objective of correcting root causes and improving performance. Solving problems is viewed as an opportunity for making productive improvements. July 1992: INPO issues 92-002, Guidelines for the Organization and Administration of Nuclear Power Stations. Chapter 2, Monitoring and Assessment, contains industry standards for the collection and analysis of key performance indicator data and for the determination of root causes and corrective actions for station events and undesirable performance trends, with particular emphasis on problems or causal factors that have generic implications. March 1997: INPO issues 97-003, Safety Focus During Changing Times ­ Recognizing Indications of Declining Plant Performance. This document communicates ideas for enhancing six areas important in the detection of declining performance, one of which was &quot;learning organization&quot; (self-evaluation and corrective action effectiveness). June 1997: INPO issues AP-903, Performance Improvement Process Description. The PI process described therein was intended to provide a means to ensure business results are achieved through effective monitoring of performance, identification of specific actions to improve less-than-expected performance, and implementation of change to improve performance. It included incorporation of operating experience and analysis of adverse conditions to prevent recurrence. This process applies to overall plant performance, process performance, human performance, and materiel condition. September 1997: When INPO revised Performance Objectives and Criteria for Operating Nuclear Electric Generating Stations (INPO 97-002), a specific performance objective with supporting criteria for self-assessment and corrective action activities was included for the first time. 1998: INPO Themes (Warning Flags) from Extended Shutdowns, includes a theme that plant event significance is unrecognized or underplayed and reaction to events is not aggressive, in addition to which organizational causes of events are not being explored. Also, a theme of isolationism was identified as a result of few interactions with other utilities, INPO, and other industry groups and benchmarking seldom being done or limited to &quot;tourism&quot; without implementation; therefore, a plant could lag the industry and not realize it. December 1999: INPO issues Principles for Effective Self-Assessment and Corrective Action Programs and requests that its members conduct a gap analysis. An industry task force

D-14

INPO 09-011 developed these principles with assistance from INPO and input from many U.S. nuclear utilities and a special committee of senior utility executives. In highly effective organizations, the need for improvement is driven from within rather than by external factors or influences. An INPO review of the utility gap analyses submitted by its members noted most weaknesses were in the areas of training programs, prioritization and timeliness, feedback to employees, trending, and performance indicators on monitoring and assessing. April 2000: After a six-month pilot program, NRC establishes a new Reactor Oversight Process (ROP). The ROP is based on risk-informed &quot;cornerstones of safe operation&quot; and three &quot;crosscutting&quot; elements, so named because they affect and are therefore part of each of the cornerstones. One of these cross-cutting elements is Problem Identification and Resolution that addresses how utility corrective action programs are finding and fixing problems. An issue that emerged from the pilots that had not yet been resolved to the satisfaction of the stakeholder was the treatment of cross-cutting issues in the ROP. July 2000: INPO plant evaluation teams begin evaluating station processes against the Principles for Effective Self-Assessment and Corrective Action Programs. Because of the importance INPO placed on this, the reviews were headed by team managers. Programmatic AFIs were written, even if they had no strong performance bases. After one evaluation cycle, a review of AFIs identified common shortfalls to be shallow problem resolution, weak department self-assessments, and ineffective corrective actionsrepeat events, adverse trends not identified or used, management observation programs not effective. December 2001: INPO 01-005, Indicators of Changing Performance, is issued. INPO benchmarking identified a set of performance indicators potentially useful in detecting early signs of changing performance. Common traits were also noted among organizations that appear to use performance indicators effectively. For example, managers use a broad set of indicators to monitor many aspects of plant, organizational, and personnel performance. Performance monitoring programs are flexible, and individual performance indicators are added, deleted, or modified as needed to monitor areas with identified performance weaknesses or changes to programs and processes. Additionally, managers take aggressive action to investigate and address the causes of declining performance at the individual indicator level April 2002: The NRC continues to refine the ROP. Among the changes made are revisions to the problem identification and resolution (PI&amp;R) inspection procedure. The frequency of inspecting licensees' PI&amp;R processes was extended from an annual to a biennial team inspectionand this had a tradeoff of more frequent, issue-specific PI&amp;R inspections. One issue the NRC considered as needing additional study was guidance for whether licensees should conduct root cause evaluations for ROP performance indicators that cross thresholds. The NRC also performed an assessment of plants that reached the degraded cornerstone column of the action matrix and concluded that weaknesses in the cross-cutting area of PI&amp;R contributed to all five facilities reaching the degraded cornerstone column of the action matrix. Additional guidance was developed for addressing substantive cross-cutting issues in the midcycle or annual assessment letters.

D-15

INPO 09-011 2003: The NRC revises guidance to clarify what constitutes a substantive cross-cutting issue and includes an option of requiring formal response from licensees. 2004: The NRC revises guidance to better define the threshold for a substantive cross-cutting issue and subsequent agency actions. However, a common theme from stakeholder surveys and the 2005 Regulatory Information Conference is that the agency needs better guidance for substantive cross-cutting issues. The staff commits to continue to improve this guidance. Anticipated improvements include providing screening criteria for the minimum threshold for assigning a cross-cutting element to a finding, better definitions of the trending bins, and revisiting exit criteria for substantive cross-cutting issues. November 2004: Personnel from INPO's assistance cornerstone developed Template for Significantly Improving Nuclear Plant Performance, A Framework of Steps and a Sequence of Implementation Based on Industry Experience. The main elements were compiled through a review of the successful performance turnarounds accomplished by several nuclear utilities over the past decade. Input from executives involved with turnaround strategies, preparing plans, and executing initiatives is included in the document. The framework for the steps and the sequence of implementation was derived from industry experience. The template general structure (process for change) is modeled after John P. Kotter's book Leading Change. November 2004: INPO issues Principles for a Strong Nuclear Safety Culture, which reinforces the following elements of performance improvement: Organizational learning is embraced. The capacity to learn from experience is well developed. Self-assessments, corrective actions, and benchmarking are used to stimulate learning and improve performance. Expertise in root cause analysis is applied effectively to identify and correct the fundamental causes of events. Processes are established to identify and resolve latent organizational weaknesses that can aggravate relatively minor events if not corrected. Employees have confidence that issues with nuclear safety implications are prioritized, tracked, and resolved in a timely manner.

June 2005: The NRC Commissioners directed the staff to (1) make further efforts to clarify the guidance on substantive cross-cutting issues, and (2) continue to emphasize the importance of implementing a good licensee corrective action program effectively. The NRC staff developed an approach to enhance the treatment of cross-cutting areas in the ROP and in supplemental procedures to more fully address safety culture. Inspector training was enhanced through Web-based read-and-sign training on substantive cross-cutting issues and on documenting findings in inspection reports. August 2005: INPO issues 05-005, Guidelines for Performance Improvement at Nuclear Power Stations. It was developed in response to many requests from INPO members for a clearer

D-16

INPO 09-011 understanding of the performance improvement function. It describes an underlying concept that high-performing stations seek to continually improve by identifying and closing important performance gaps. It focuses on an integrated approach and defines key characteristics of individual processes and activities. The target audience for this document is line managers. Effective line management involvement and ownership are essential to success in performance improvement activities. Fall 2006: The NRC creates a new &quot;Substantive Cross Cutting Issues Summary&quot; Web page that provides a comprehensive list of plants that have an open, substantive cross-cutting issue. September 2007: INPO issues Leadership Fundamentals to Achieve and Sustain Excellent Station Performance. This document encourages leaders to establish clearly defined owners, roles, and responsibilities to accomplish key station processes, including corrective action, work management, training, and self-assessment. The document also recommends that leaders provide critical oversight and promote continuous improvement of the training, corrective action, selfassessment, and human performance processes and programs. December 2007: INPO issues Good Practice 07-007, Performance Assessment and Trending. This document emphasizes individual behaviors that support finding performance gaps. Such behaviors include managers who demonstrate the importance of finding gaps by discerning the connectivity of seemingly unrelated performance issues using cognitive trending; managers who demonstrate the importance of analyzing problems to a depth commensurate with the significance of the performance gap; and managers who demonstrate the importance of fixing problems correctly the first time, with a low tolerance for problem recurrence. Early 2008: Based on analysis of 2007 INPO areas for improvement, a plant evaluation focus area was established for 2008 to address cause analyses that do not identify organizational causes and contributors, and this continues to be a major contributor to repeat station events. These cause analyses problems are due to shallow analysis depth and an acceptance of weak cause evaluations by responsible managers/sponsors. Weak analysis translates into corrective actions that do not fix the problem. June 2008: INPO convenes a special meeting of industry experts to determine if an industry initiative is needed to significantly improve the use of performance improvement tools. The group concluded that the pace of improving safety and reliability is limited by leaders not reinforcing and supporting behaviors related to finding and fixing performance gaps. Also, the use of PI tools and activities is not fully integrated into other management systems and industry guidance or fully accepted as a core business value. August 2008: Results of the special PI working group were presented to the INPO Executive Advisory Group, which endorsed the concept of the INPO business plan action item for 2009 to more clearly define the fundamentals of performance improvement excellence and develop additional tools. August 2008: The NEI issues in draft form NEI 08-02, Problem Identification and Resolution for New Nuclear Plants During Construction. This document was developed by the New Plant

D-17

INPO 09-011 Problem Identification and Resolution (PI&amp;R) Task Force and selected members of the New Plant Quality Assurance Task Force. These professionals, experts on construction practices and PI&amp;R, drew upon practical lessons learned during the application of PI&amp;R and provided insights to the document. November 2008: During the INPO 2008 CEO Conference, factors that influence sustained high performance were communicated based on an in-depth analysis of high-performing stations. One key factor was that managers use the corrective action program (CAP) to shape organizational performance. Specific elements of the CAP include the following: low threshold for reporting management engagement in sorting and prioritizing problems identification of organizational aspects of more significant issues firm accountability and follow-up to resolve issues