Understand limitations of physician self-assessment of competence and the important role for external feedback and guidance.

Case & Commentary: Part 1

A 50-year-old man was diagnosed with chronic hepatitis C (viral load of 2,500,000 IU/mL) by his internist, who also happened to be an oncologist. This physician was comfortable with chemotherapeutic agents and decided to treat his patient's hepatitis C virus (HCV) without referring him to a hepatologist. He saw this patient in the hospital's outpatient oncology unit. The physician started treatment with weekly injections of pegylated interferon (which is also used for some cancers) and daily oral ribavirin.

While this case raises a number of important issues regarding scope of practice, licensure, credentialing, and board certification, the primary issue is the ability of this physician to recognize his limitations, especially given his decision to venture outside his primary area of practice. Robust research data now exist showing that physicians, like all people, do not self-assess their knowledge and skills accurately when done in isolation (1,2), and that effective self-assessment requires incorporating external data in what Eva and Regehr label "self-directed assessment seeking."(3) Yet, even when physicians receive external feedback regarding their performance, a recent study found that processing and incorporating the data and feedback are complex and require time, skill, and often, assistance from others.(4) Learning how best to facilitate practice-based improvement is an important area of research.

In this case, we would hope the physician would seek answers to a series of questions before providing treatment: What are the current guidelines and recommendations for the treatment of hepatitis C? Are pre-treatment tests and evaluations needed to safely provide a potentially toxic therapy? What is my experience with these drugs? And so on. Gruppen and Frohna provide a useful framework for decision-making when caring for patients outside one's usual scope of practice.(5) Their framework recognizes that clinical reasoning is a complex process that starts when the physician develops a problem representation. This critical step occurs when the physician performs an accurate and appropriately targeted medical interview and physical examination that is tightly integrated with the physician's working memory (the "hard drive") to determine what is happening with the patient. Before initiating action, the next step, called evaluation, requires physicians to ask what gaps might exist in their understanding of the patient's condition, including treatment.(5,6) This critical step is often skipped, especially by experienced physicians, who may be overly confident in their knowledge base.(7-8) This particular case is a set-up for error—a physician experienced in another discipline that uses toxic, high-risk medications believes he or she has the knowledge and skill to care for a condition outside his or her primary scope of practice, largely because there is some overlap between the medications used in this specialty and those potentially required for this patient.

Where do credentialing and board certification fit in these situations? (See Table.) Board certification was originally created to help define new disciplines in medicine and set standards of excellence in practice. Although technically a voluntary process, board certification is now required for employment and credentialing by most hospitals and health plans. Originally, physicians who were certified at the completion of their training were given a lifetime certificate, but all boards now offer only time-limited certificates. In internal medicine, certification became time-limited in 1990, and over the past 10 years the American Board of Internal Medicine (ABIM), along with the other 23 specialty boards under the auspices of the American Board of Medical Specialties (ABMS), instituted maintenance-of-certification (MOC) programs that require physicians to self-assess their knowledge, evaluate their performance in practice, and pass an examination every 10 years.(9) Whether they possess a time-limited certificate or not, physicians can enroll in the MOC program at any time and take advantage of the self-assessment tools to address gaps in their knowledge and practice performance. Relevant to this particular situation, the ABIM MOC program has a Web-based tool called practice improvement modules (PIMs), one of which is devoted to caring for patients with hepatitis C.(10)

Currently, licensure does not specifically address this kind of situation. The approach to licensure in the United States is highly fragmented among nearly 70 state medical boards. There is no specialty-specific licensure; the license authorizes the practice of medicine in general. Currently, renewal of a medical license mostly involves just passing a criminal background check and accruing some amount of continuing medical education (CME) credits that varies from state to state. This legal framework relies on physicians and health care systems to self-regulate scope of practice via credentialing and local monitoring processes. However, similar to recent developments in board certification, the Federation of State Medical Boards (FSMB) is embarking on the development of maintenance-of-licensure (MOL) programs that seek to improve the assessment of practicing physicians.(11) This approach could be "tailored" to a physician's scope of practice—if this physician decides to provide hepatitis C care, MOL assessments could be targeted at the quality of care for these patients. This would represent a profound change, especially if a physician's individual scope of practice was regulated through licensure, because it would potentially allow physicians to move into areas not originally part of their discipline-specific training (in this case, oncology).

While not developing assessments outside the boundaries of their disciplines, certification boards are experimenting with new pathways in MOC that allow for more focus within a discipline. For example, ABIM has begun a small experiment designed for general internists who focus their practice in the hospital setting (called the focused practice in hospital medicine program).(12) In addition to providing the self-directed assessment tools, certification boards are exploring other pathways within the MOC program for individuals who may either wish to focus their practice over time or develop additional proficiencies in a specific medical area.

However, the question remains whether this physician should be the principal physician providing front-line therapy for hepatitis C patients. There are no easy answers here. For example, the physician might be located in an underserved area with too few hepatologists and/or gastroenterologists to see all hepatitis C patients. Furthermore, new discoveries and technologies are always emerging and transforming patient care. Even new treatments and technologies within a discipline will continue to press physicians to acquire new knowledge, skills, and attitudes, making a certification acquired 5–10 years previously potentially misleading if the physician has not kept up his or her skills. Regardless of the structured programs that may exist to help physicians either focus or expand their scope of practice, they will never be perfect or encompass all situations. Therefore, the critical step is the need for physicians to recognize when they are moving outside their scope of practice and appropriately initiate mindful practice and reflection. The physician who does this asks not, "Can I care for this patient?" but rather, "Should I?" If the answer to the latter question is yes, then, "What gaps do I currently have that I will need to address? Who can I consult to help me acquire new knowledge and skills? How will I know if the treatment I am providing is effective and safe?"(3,13)

Case & Commentary: Part 2

Prior to therapy, the physician did not refer the patient for a liver biopsy nor did he obtain a hepatitis C genotype or baseline complete blood count or thyroid stimulating hormone. After 6 weeks of therapy, the HCV viral load was unchanged, and the patient now exhibited pancytopenia. The physician responded by decreasing the interferon dose to every 2 weeks but continued the ribavirin at full dose. He didn't obtain a viral load at 12 weeks. After 9 months of treatment, a viral load was repeated and was again found to be more than 2,000,000 IU/mL. Treatment was continued without change.

Three months later, the insurance company denied the hospital's bill, and the medical director for case management was asked to prepare an appeal letter. In drafting the letter, the medical director realized that this patient's care didn't comport with HCV treatment guidelines. The improper care included: (i) no liver biopsy to see if the patient met the criteria for treatment; (ii) no genotype performed to help guide length of therapy; (iii) no baseline laboratory tests to see if the patient's pancytopenia was preexisting or due to the medications; and (iv) no 12-week viral load to assess response. Moreover, when the patient was found to have pancytopenia, interferon was decreased to every other week rather than reducing the ribavirin dose or continuing the interferon at half-strength weekly dosing, as per guidelines. When the viral load at 9 months showed no response, treatment was continued, exposing the patient to a toxic medication with little possibility of benefit.

Concerned for the patient's safety and well-being, the medical director instructed the clinic staff to discontinue the interferon and insisted that the physician refer the patient to a hepatologist.

The follow-up data nicely, but soberly, highlight this physician's numerous gaps in competency in treating hepatitis C and his failure to recognize these gaps until a medical director intervened. This case reinforces the critical importance of ongoing assessment of physicians. As emphasized above, physicians should be actively seeking assessment from external sources.(3,14) This is the only way to address the blind spots we all have. This physician, and his patient, would have benefitted greatly from external input and assessment. From my perspective, the response of this medical director is absolutely on target. I believe the professional organizations, from medical practices to hospital to medical societies and certification boards, must set the expectation that when a physician wanders outside his or her defined or normal scope of practice, it is a professional imperative that the physician seeks external guidance and input.

Turning back to certification, a fair question is whether current certification processes truly make a difference and identify important gaps in physician competence. The short answer is that certification processes do identify physicians who provide poorer levels of care and can help them improve the quality of care they provide.(15-17) In addition to setting expectations, certification boards, along with medical societies and others, can facilitate improvement by providing assessment methods and tools to help physicians discover gaps in competencies and then provide guidance on how to best address and close those gaps.(15,18) As cognitive neuroscientist Itiel Dror recently highlighted, assessment and learning should be tightly intertwined, and it is illogical to think of either as a separate activity.(19)

All the certification boards recognize the need to improve both initial certification and MOC programs.(9) The MOC programs will become more continuous over the next several years, allowing physicians to perform more frequent targeted "biopsies" of competence. Many tools already exist within the MOC programs that allow physicians to assess their performance and learn new material within the MOC program. The boards continue to explore more efficient and effective modes of self-assessment, especially around scope-of-practice issues, such as the focused practice of hospital medicine program.(12) Finally, ABMS is actively collaborating with FSMB to align the various elements of MOC and MOL when appropriate. Credentialing bodies, licensing boards, and certification boards will need to take the scope-of-practice issues head-on and develop new policies that guide when and how physicians can engage in new and different clinical care activities. Such policies will be difficult to develop in a country where individualism is so highly prized (20) and autonomy is a deeply entrenched value among physicians.(21) Nonetheless, setting clear expectations will go a long way in changing our current approach to this issue and has real potential to prevent patient harm.

Everyone involved in the care of patients must be willing to report to appropriate entities when they believe a physician is impaired or may be operating outside his or her scope of practice and potentially endangering patients. Impeding this change in culture is the lack of robust, systematic assessment approaches, fear of retribution, and the sense of futility that nothing can or will be done.(22) This is when licensing boards, certification boards, and medical societies can assist by creating and providing good assessment tools and methods and by working collaboratively to help physicians who need remediation or wish to make a change in their practice.(22) While the physician in this case may very well have had the best of intentions, his decision to provide care outside his scope led to patient harm. All of us often fail to recognize our limitations and need systematic processes of assessment to identify and fill gaps in competency.

Take-Home Points

All physicians must learn to question their own abilities and recognize when they may be moving outside their range of competence.

Colleagues must speak up when they see a peer providing potentially dangerous care, especially if that care appears to be out of scope of normal practice.

All physicians need to routinely and regularly engage in self-directed assessment-seeking activities.

Faculty Disclosure:Dr. Holmboe has declared that neither he, nor any immediate member of his family, has a financial arrangement or other relationship with the manufacturers of any commercial products discussed in this continuing medical education activity. In addition, the commentary does not include information regarding investigational or off-label use of pharmaceutical products or medical devices.

Table

Table. Regulatory Processes in Physician Practice.

Regulatory Process

Description

Credentialing

Usually a local process that determines the privileges and scope of practice for the physician at a hospital or health plan. A physician most often requests privileges to a credentialing committee that makes the final determination.

Licensure

This is a state-level process in the United States and is a mandatory requirement for medical practice. Requirements for initial licensure and renewal of licensure vary from state to state. Licensure in the United States is not specialty specific.

Certification

Certification is technically a voluntary process that requires completion of an Accreditation Council for Graduate Medical Education–approved training program and passage of a secure examination. Certification is a specialty-specific process.