If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Send correspondence and reprint requests to Dr. William Cooper, M.D., M.P.H., Vanderbilt University School of Medicine, 2135 Blakemore Ave., Nashville, TN 37212.
Center for Patient and Professional Advocacy, Vanderbilt University Medical Center, Nashville, TNDepartment of Pediatrics, Vanderbilt University School of Medicine, Nashville, TN
Center for Patient and Professional Advocacy, Vanderbilt University Medical Center, Nashville, TNDepartment of Medicine, Vanderbilt University School of Medicine, Nashville, TN
Center for Patient and Professional Advocacy, Vanderbilt University Medical Center, Nashville, TNDepartment of Pediatrics, Vanderbilt University School of Medicine, Nashville, TN
Center for Patient and Professional Advocacy, Vanderbilt University Medical Center, Nashville, TNDepartment of Pediatrics, Vanderbilt University School of Medicine, Nashville, TN
This study was designed to examine whether words contained in unsolicited patient complaints differentiated physicians with and without neurocognitive disorders.
•
Cases were more likely to have at least one word describing cognitive impairment (73%) than comparison physicians matched for age/sex (8%) and site/complaint (18%).
•
Patients may provide an important source of information for healthcare organizations interested in identifying professionals with evidence of cognitive impairment, though further research is needed.
•
This article contributes to our understanding of how the patient's voice can help to identify physicians with potential impairment that might warrant further evaluation.
Objectives
Determine whether words contained in unsolicited patient complaints differentiate physicians with and without neurocognitive disorders (NCD).
Methods
We conducted a nested case–control study using data from 144 healthcare organizations that participate in the Patient Advocacy Reporting System program. Cases (physicians with probable or possible NCD) and two comparison groups of 60 physicians each (matched for age/sex and site/number of unsolicited patient complaints) were identified from 33,814 physicians practicing at study sites. We compared the frequency of words in patient complaints related to an NCD diagnostic domain between cases and our two comparison groups.
Results
Individual words were all statistically more likely to appear in patient complaints for cases (73% of cases had at least one such word) compared to age/sex matched (8%, p < 0.001 using Pearson's χ2 test, χ2 = 30.21, df = 1) and site/complaint matched comparisons (18%, p < 0.001 using Pearson's χ2 test, χ2 = 17.51, df = 1). Cases were significantly more likely to have at least one complaint with any word describing NCD than the two comparison groups combined (conditional logistic model adjusted odds ratio 20.0 [95% confidence interval 4.9–81.7]).
Conclusions
Analysis of words in unsolicited patient complaints found that descriptions of interactions with physicians with NCD were significantly more likely to include words from one of the diagnostic domains for NCD than were two different comparison groups. Further research is needed to understand whether patients might provide information for healthcare organizations interested in identifying professionals with evidence of cognitive impairment.
Physician. Shortages and Projections: The 2017 Update: The Complexities of Physician Supply and Demand: Projections from 2014 to 2025: at 48 Exhibit 32 [online].
Patients are uniquely positioned to observe physician behaviors and performance and may share their observations in the form of unsolicited patient complaints (UPCs).
Previous research has shown that physicians associated with large numbers of UPCs relative to their peers are at increased risk for malpractice claims and adverse surgical outcomes.
long-term coworkers may or may not recognize subtle signs of cognitive impairment. Patients, however, who are newly assigned to the physician or interact only in sporadic encounters may notice behaviors that lead them to raise concerns about a physician's performance and voice their concerns in a UPC.
While there is no expectation that patients would diagnose physicians with NCD, the objective of this study was to test the hypothesis that patients might recognize worrisome, eccentric, or other challenging physician behavior, and that their UPCs about that physician would include words potentially describing signs and symptoms of NCD (e.g., “He seemed forgetful.”). We conducted a nested case–control study that included cases of physicians with probable or possible NCD and two comparison groups randomly selected from a cohort of 33,814 physicians practicing in healthcare organizations that utilized the national Patient Advocacy Reporting System (PARS®) program database.
Methods
Study Design
We performed a nested case–control study comparing cases of physicians with probable or possible NCD with two comparison groups selected from among a cohort of 33,814 physicians practicing in 144 U.S. hospitals, healthcare systems, and medical groups that participated in the Vanderbilt PARS program from January 1, 2014, to December 31, 2016.
Data Sources
The PARS program includes a patient complaint reporting database used previously to study associations between performance and professionalism concerns, malpractice claims risk, and adverse surgical outcomes.
PARS data used for this study encompassed coded UPCs attributed to cohort physicians as recorded by each healthcare organization's patient relations staff. All complaints were uploaded securely to the PARS database within the Vanderbilt Center for Patient and Professional Advocacy (CPPA).
CPPA-trained PARS coders reviewed and coded reports, including whether a physician was named in the report, with previously reported inter-rater reliabilities of 73%–100%.
Physicians (N = 33,814) credentialed with an organization participating in the PARS program during the study period were included in the cohort. The population was a dynamic cohort, because physicians entered when they met eligibility requirements (active credentials at a participating PARS site during the study period) and departed the study cohort at the earliest of: 1) the end of the study period; 2) the physician's departure from the organization; or 3) diagnosis of NCD.
Cases: Physicians with Evidence of NCD
Evidence of NCD was defined as having a deficit in recent memory, executive functioning, social cognition, global functioning, or visuospatial functioning during the study period.
Potential cases were identified by communicating with a locally designated leader for the site's PARS program (e.g., chief medical officer, vice president for medical affairs, chief of staff, risk manager, or other individuals involved in medical staff and employment actions related to physicians). As part of CPPA's operations to support the organization's efforts to promote professional accountability, site-based leader contacts shared their awareness of 12 physicians referred for evaluation based on concerns related to possible NCD during the time the organization was participating in the PARS program in accordance with the terms of the confidentiality agreement with each site. In three cases, feedback to the respective healthcare organizations about their physician's patient complaints was the first signal that raised concerns of probable or possible NCD.
We developed our case definition based on recommendations from coauthors with expertise in the fields of geriatrics (TMJ), psychiatry (JJF, ARF), neuropsychology (MLJ), and internal medicine (WM). The coauthor experts, blinded to case status, received brief de-identified synopses of the specialty, age, and gender of potential cases and a randomly selected sample of non-cases along with the text of the physician's UPCs (Supplemental Digital Content 1). After independent review, the experts were asked to rate on a 10-point scale the likelihood of each physician being cognitively impaired. They were also asked whether they would recommend further evaluation or action based on the case description. After these responses were recorded, organizational actions related to the physician were described (e.g., mandatory referral for cognitive screening, practice restrictions) and the experts were asked again to rate the likelihood of impairment and their recommendations. The group then convened by teleconference to review each case and determine their shared opinions of the likelihood of NCD using a consensus model. Because we did not have biomarkers or autopsy evidence to confirm the NCD pathophysiological process,
The diagnosis of dementia due to Alzheimer's disease: recommendations from the National Institute on Aging-Alzheimer's Association workgroups on diagnostic guidelines for Alzheimer's disease.
our case definition focused on evidence that would be available to the organization's leadership for each potential case when they had to consider a physician with possible cognitive impairment, including 1) probable NCD (formal testing confirmed impairment or institutional action was taken due to concern for NCD) or 2) possible NCD (experts concurred that formal cognitive evaluation was warranted, but the institution had not yet conducted an evaluation). The date that the physician met the definition was considered the index date (t1) for each case. All cases were required to have at least four years (1,461 days) of PARS data prior to the index date to ensure sufficient data to support identification.
Comparison Groups
For each case, two comparison groups were randomly selected from the study population of eligible physicians at PARS sites during the study period. Physicians in the first comparison group (four selected per case) were actively credentialed at a PARS intervention site on the index date for their matched case and the previous four years (1,461 days) and were matched to cases according to age (+/− 5 years) and sex. For privacy and confidentiality, cases with age >85 years were matched within 10 years because of the small number of practicing physicians of this age. Physicians in the second group (also four selected per case) were matched on site and the number of UPCs received in the prior four years. Physicians selected for the comparison group became ineligible to be included as comparisons for other case subjects but could become cases at a later date.
Patient Complaints
For each case and comparison physicians, all UPCs from the PARS program were reviewed from the period four years prior to the index date through the index date. Five cognitive domains from the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition, in which deficits might be identified were considered a priori: recent memory, executive function, social cognition, global function, and visuospatial function.
UPCs related to global function included patients' perceptions that the physician seemed to be having trouble performing tasks, questioning whether the physician's age was interfering with function, or expressing loss of trust in the physician. Words mapping to each domain were selected from a de-identified sample of patient complaints and assigned to a domain by the group of expert case reviewers prior to case classification (Table 1).
TABLE 1Words Contained in Patient Complaints About Physicians With Probable or Possible Neurocognitive Disorders
Manual review of all complaint report narratives was performed independently by two physicians (STC, BPK) who were blinded to case status in order to ensure that the selected word was referencing the physician or the physician's behavior (e.g., “confused” in a complaint could refer to the physician's confusion, or the patient might have complained about a confusing instruction or document). Physician reviewers were trained to reliably code complaints using previously published PARS coding approaches
and completed review on training cases until 100% agreement on training cases was obtained. Disagreements on word attributions to the study physicians, which occurred in fewer than 5% of selected word reviews, were resolved through discussion between the two reviewers and the first author.
Covariates
Covariates related to the physician and the organization were collected from the PARS database and publicly available data, such as state licensure verification databases, and included factors shown previously to be associated with physician cognitive impairment.
Covariates included physician demographic characteristics (age, sex), physician training and professional characteristics (graduation from a non-U.S. medical school, lack of board certification), and practice characteristics (surgical vs. non-surgical, primary site of practice in academic, community, or non–hospital-based setting).
Analysis
We compared physician characteristics in cases and comparison physicians using Pearson's χ2 test or Wilcoxon rank sum test. We compared the presence of individual words (e.g., words associated with memory) and word groupings potentially related to NCD for cases versus both comparison groups using Pearson's χ2 test. For these analyses, subjects in the comparison groups were treated as independent despite multiple physicians being matched to a single subject. For the nested case–control analysis, we used conditional logistic-regression analysis to assess the relationship between prior complaints and the risk of NCD, accounting for matching factors and adjusting for potential confounding factors, including surgical versus nonsurgical practice, practice setting, graduation from a non-U.S. medical school, and lack of board certification. Results did not differ when cases were compared to comparison groups separately and pooled. Thus, final multiple regression analyses were pooled.
Secondary analyses assessed the relationship between prior complaints and risk for NCD based on subsets of the case definition. There were no missing data. We used R statistical software (version 3.2.3, R Foundation for Statistical Computing, Vienna, Austria) to perform all analyses.
Human Subjects
Linking of data was conducted by a computer programmer who did not participate in the study analyses. Following linkage, all identifiers were removed and analyses were performed on these de-identified data sets. The study was reviewed by the Vanderbilt University Medical Center Institutional Review Board (IRB #150624), which determined that the study qualified as non-human subjects since all analyses were conducted on de-identified data sets and no individuals could be identified from the data (45 CFR 46.102 [f] [2]).
Results
The physicians with evidence of NCD (cases, N = 15) were predominantly male and had a mean age (+/− standard deviation) of 72.5 + /− 9.1 years (Table 2) (age range 59–90 years); 7 (47%) were greater than 75 years of age. Six (40%) practiced in a surgical specialty; 5 (33%) were graduates of non-U.S. medical schools; 3 (20%) lacked board certification; and 12 (80%) practiced in academic settings. The age- and sex-matched comparison physicians were comparable in terms of the proportion in surgical specialties and those who practiced in academic settings. Age- and sex-matched comparisons were significantly less likely to lack board certification (Table 3). Site- and complaint-matched comparisons were significantly younger than cases but were comparable in terms of sex, practice in a surgical specialty, and organizational setting (Table 3). Site- and complaint-matched comparisons were somewhat less likely to lack board certification than cases, but the difference was not statistically significant.
TABLE 2Characteristics of Cases (Physicians With Neurocognitive Disorders) Compared to Age/Sex-Matched and Site/Complaint-Matched Comparison Groups
TABLE 3Words Included in Patient Complaints for Cases (Physicians With Neurocognitive Disorders) Compared to Age/Sex-Matched and Site/Complaint-Matched Comparison Groups
Variable
Cases (N = 15)
Age and Sex Matched Comparisons (N = 60)
Site and Complaint Matched Comparisons (N = 60)
N patient complaints received in 4 years (mean +/− standard deviation)
Cases had significantly greater numbers of overall UPCs in the previous 4 years than age- and sex-matched comparisons (Table 3). Eleven (73%) had at least one word in a complaint that described potential NCD. In contrast, only 8% (p < 0.001 using Pearson's χ2 test, χ2 = 30.21, df = 1) of age- and sex-matched comparisons and 18% (p < 0.001 using Pearson's χ2 test, χ2 = 17.51, df = 1) of site- and complaint-matched comparisons were associated with at least one complaint describing potential NCD. The proportion of cases who had complaints with words from each domain of NCD (memory, executive functioning, social cognition, global function, and visuospatial impairment) all significantly differed between cases and both comparison groups (Table 3).
Individual words, including “forget,” “remember,” “memory,” “confuse(d),” “inappropriate,” “seemed,” “age,” “trust,” and words suggesting visuospatial impairment (e.g., “couldn't use,” “difficulty using”) were all statistically more likely to appear in UPCs for cases compared to both comparison groups (Figure 1). The proportion of physicians whose UPCs included the words “old” and “odd” did not differ between cases and either comparison group. The words “strange” and “uncomfortable” were more likely to appear in UPCs for cases when compared to age- and sex-matched comparisons but not for site- and complaint-matched comparisons. The word “retire” was significantly more likely to appear in UPCs for cases compared to site- and complaint-matched comparisons but not for age- and sex- matched comparisons.
FIGURE 1All P Values are calculated using Pearson's chi-square test with 1 degree of freedom.
In multiple regression analyses controlling for specialty, organizational setting, board certification, and graduation from a non-U.S. medical school, cases were 20 times more likely to have at least one complaint containing any word describing NCD than the two comparison groups combined (Figure 2). Cases had significantly more cognitive impairment-related words than the combined comparison groups in models for the NCD domains describing impaired memory, distrust, and inappropriate behavior. Results stratified by the probable versus possible case definition were comparable in terms of the proportion of individuals with any word describing potential impairment (Supplemental Digital Content 2). Similarly, sensitivity analyses that excluded the three cases initially identified by patient complaints produced nearly identical results to the primary analyses (Supplemental Digital Content 2).
FIGURE 2Adjusted odds ratios for cases having complaints with words describing neurocognitive disorders compared to comparison physicians.
Patients and families who share observations about their healthcare experiences in the form of UPCs provide opportunities for healthcare organizations to identify clinicians with increased risk for adverse outcomes and malpractice claims.
This nested case–control study analyzed UPCs from a cohort of 33,814 physicians who practiced in healthcare organizations throughout the United States. Analysis of words used in UPCs, as opposed to the number of UPCs, found that physicians with probable or possible NCD were 20 times more likely to be described in terms that map to one of the domains for NCD than were two different comparison groups (matched on age/sex and site/number of UPCs).
Differences in UPCs between cases and comparison physicians were observed for having any word describing potential cognitive impairment, as well as each domain of NCD. Some words (e.g., “old” and “odd”) were not more likely to appear in UPCs for cases than for either comparison group, and the word “retire” was not more likely to appear in cases versus age- and sex-matched comparisons. These findings suggest that patients are not necessarily making judgments based on the physician's appearance or age, but rather on other behaviors that suggest that the physician is not functioning as the patient might expect.
This study differs in several ways from previous studies that described characteristics of physicians identified with possible NCD.
the current study compares cases to age- and sex- and site- and complaint-matched comparison physicians.
The findings suggest a methodology for determining which older physicians might warrant additional assessments without imposing testing burdens on those who do not. In a previous study, physicians in their 60s and 70s were found to receive fewer patient complaints than all younger age groups in at least one specialty.
In the current study, 93% of the cases were male and had an average age of 72.5 years; almost half were over 75 years of age. The predominance of cases among male physicians is not surprising given the low numbers of female medical school graduates at the time that the average case physician would have graduated.
Given the increase in the proportion of female medical graduates since, future cases will likely reflect this population shift over time. Such results suggest that the high risk group would include those older physicians who experience an unusually high number of complaints, a substantial increase in numbers of complaints relative to their baseline, and/or whose complaints include words consistent with potential impairment in domains related to NCD.
This study has several limitations, including the small number of cases, which limits statistical power and the likelihood that the results reflect a true effect. However, the magnitude of those differences was similar across two different comparison groups assessed separately and in pooled analyses. Cases were identified by leaders at institutions that participate in the PARS program, which includes many large health systems and multispecialty practices, with very few solo practice sites; this potentially limits the generalizability of the findings to physicians who practice in similar settings. Cases were included in the study if they had formal testing, if the organization took formal action due to concerns for impairment (i.e., in the event a physician refused mandated screening), or if the case description led experts in neurology, psychiatry, geriatrics, and neuropsychology to have concerns about cognitive impairment. Thus, misclassification of cases may have occurred.
While organizations may find addressing individuals who have subtle or more obvious signs of cognitive impairment to be challenging,
our purpose was to consider whether words included in UPCs might be utilized to identify who those individuals are in the first place, without imposing unnecessary burdens on well-functioning physicians. Words in complaints that reflect impairment domains suggest the need for further testing rather than a definitive diagnosis.
The findings of this study suggest that patients may provide an important source of information for healthcare organizations interested in supporting professionals who may require accommodation or withdrawal from practice, and at the same time maintain a competent workforce and optimize patient safety and outcomes. A frequent finding in some forms of NCD is lack of insight, including the phenomenon of anosognosia,
which highlights the need to use all available sources of information to identify individuals with potential impairment. Their professional colleagues may not be in a position to directly observe interactions with patients, or they may hesitate to report someone whom they have admired and worked with for a long time. Patients, on the other hand, observe the physician's performance in a more private setting and are focused on their self-interest; thus, they may have fewer perceptual or reporting barriers. Of note, in some cases, the patient reported that the staff working alongside physicians later identified as cases had in fact recognized the performance challenges (in one example, a nurse stated to the patient that “Dr. X has good days and bad days”). Thus, while not an exclusive source of data, UPCs may provide an additional tool for signaling potential cognitive impairment, though further research is needed to understand what role UPCs might play as a surveillance tool. Like information from any surveillance tool, UPCs suggesting potential impairment should be considered in the context of all available information about the physician to guide whether further evaluation and support may be needed.
Dr. Cooper and Mr. Domenico had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Cooper, Martinez, Domenico, Johnson, Karrass, Hickson. Acquisition, analysis, or interpretation of data: Cooper, Martinez, Domenico, Callahan, Kirkby, Finlayson, Foster, Johnson, Longo, Merrill, Jacobs, Pichert, Catron, Moore, Webb, Karrass, Hickson. Drafting of the manuscript: Cooper, Moore. Critical revision of the manuscript for important intellectual content: Cooper, Martinez, Domenico, Callahan, Kirkby, Finlayson, Foster, Johnson, Longo, Merrill, Jacobs, Pichert, Catron, Moore, Webb, Karrass, Hickson. Statistical analysis: Cooper, Domenico. Administrative, technical, or material support: Cooper, Karrass, Catron, Hickson, Johnson. Study supervision: Cooper, Hickson.
This study was funded by Vanderbilt University Medical Center. The funder had no role in the design and conduct of the study, interpretation of the data, or preparation of the manuscript. There are no disclosures to report.
Appendix. Supplementary material
The following is the supplementary data to this article:
Supplemental Digital Content 2. Sensitivity Analyses.
References
Association of American Medical Colleges
Physician. Shortages and Projections: The 2017 Update: The Complexities of Physician Supply and Demand: Projections from 2014 to 2025: at 48 Exhibit 32 [online].
The diagnosis of dementia due to Alzheimer's disease: recommendations from the National Institute on Aging-Alzheimer's Association workgroups on diagnostic guidelines for Alzheimer's disease.
30299-9&id=amgp1037-fig-0001.jpg){kind=link}
30299-9&id=amgp1037-fig-0002.jpg){kind=link}