QuestionWhat are the psychometric properties of a patient-reported outcome instrument to assess eye-related symptoms and their effect on functioning and well-being following laser in situ keratomileusis (LASIK)?

FindingsThe Patient-Reported Outcomes With LASIK (PROWL) studies (PROWL-1 and PROWL-2) were prospective observational studies evaluating a total of 574 patients undergoing LASIK surgery for myopia, hyperopia, or astigmatism. The study results support the reliability and validity of these patient-reported outcome measures for use in evaluating LASIK surgery.

MeaningThese studies yield support for use of the PROWL survey in future studies to assess the effects of LASIK surgery on functioning and well-being.

ObjectiveTo develop PRO measures to assess satisfaction, eye-related symptoms, and their effect on functioning and well-being following LASIK based on patient and expert input.

Design, Setting, and ParticipantsThe Patient-Reported Outcomes With LASIK (PROWL) studies were prospective observational studies of patients undergoing LASIK surgery for myopia, hyperopia, or astigmatism. PROWL-1 was a single-center study of active-duty US Navy personnel and PROWL-2 was a 5-center study of civilians. PROWL-1 enrolled 262 active-duty service personnel and PROWL-2 enrolled 312 civilians 21 years or older who spoke English; 241 individuals in PROWL-1 and 280 in PROWL-2 completed a baseline questionnaire before surgery. The analytic sample included those also completing 1 or more follow-up questionnaires: 240 (99.6%) of those in PROWL-1 and 271 (94.4%) of those in PROWL-2. Questionnaires were self-administered through the internet preoperatively and at 1 and 3 months postoperatively in both studies and at 6 months postoperatively in PROWL-1. PROWL-1 began in August 2011 and was completed May 30, 2014; PROWL-2 began in July 2012 and was completed June 27, 2014. Data were analyzed from June 28, 2014, to October 24, 2016.

Main Outcomes and MeasuresScales assessing visual symptoms (double images, glare, halos, and starbursts), dry eye symptoms, satisfaction with vision, and satisfaction with LASIK surgery. Items from the National Eye Institute (NEI) Refractive Error Quality of Life Instrument (NEI-RQL-42), NEI Visual Function Questionnaire (NEI-VFQ), and the Ocular Surface Disease Index (OSDI) were included. All scales are scored on a 0 to 100 possible range. Construct validity and responsiveness to change were evaluated (comparing scores before and after surgery).

ResultsThe median age of the 240-person PROWL-1 analytic sample was 27 years (range, 21-52 years); 49 were women (20.4%). The median age of the 271-person PROWL-2 analytic sample was 30 years (range, 21-57 years); 147 were women (54.2%). Internal consistency reliabilities for the 4 visual symptom scales ranged from 0.96 to 0.98 in PROWL-1 and from 0.95 to 0.97 in PROWL-2. The median (interquartile range) test-retest intraclass correlation was 0.69 (0.57-0.79) and 0.76 (0.68-0.84) in PROWL-1 and PROWL-2, respectively. Product-moment correlations of satisfaction with surgery with visual symptom scales at follow-up evaluations ranged from r = 0.24 to r = 0.49. Measures improved from baseline to follow-up, with effect sizes of 0.14 to 1.98, but scores on the NEI-RQL-42 glare scale worsened at the 1-month follow-up. Hours of work did not change significantly from baseline to 1-month follow-up, with the mean number (mean [SD] difference) in PROWL-1 of 41.7 vs 40.9 hours (−0.8 [18.7]) and in PROWL-2 of 38.8 vs 38.2 hours (−0.6 [17.1]).

Conclusions and RelevanceThe results of these studies support the reliability and validity of visual symptom scales to evaluate the effects of LASIK surgery in future studies.

Introduction

Laser in situ keratomileusis (LASIK) is a refractive corneal surgical procedure for reducing or eliminating myopia, hyperopia, and astigmatism.1,2 The US Food and Drug Administration evaluates the safety and effectiveness of LASIK devices before approving them for use in the US market.3 Dry eye symptoms, dissatisfaction with visual outcomes, and burdensome visual symptoms have been reported following LASIK.4,5 Given the high volume of surgery performed and patient concerns voiced at a public meeting, it is important to evaluate the effect of surgically induced changes on patient-reported outcomes (PROs) of LASIK: eye-related symptoms, daily functioning and well-being (health-related quality of life [HRQOL]), and patient satisfaction.6-8 A better understanding of PROs after LASIK may allow eye care professionals and patients to make more educated decisions. Thus, a reliable and valid PRO instrument is needed to evaluate the effects of this intervention.9

The Patient-Reported Outcomes With LASIK (PROWL) project team developed a questionnaire to evaluate PROs for persons undergoing LASIK for myopia, hyperopia, or astigmatism, incorporating existing HRQOL measures and developing new items. This article summarizes the psychometric performance of the instrument in 2 studies of participants undergoing LASIK surgery.

Methods

Study Design

We conducted a literature search and obtained input from an expert panel of ophthalmologists, optometrists, psychometricians, and clinical researchers. Draft items were revised based on input from patients who underwent LASIK and another group of experts. In addition, 9 individuals contemplating LASIK surgery and 9 who had LASIK surgery performed between 6 months and 2 years ago participated in interviews in Los Angeles, California, and Washington, DC. Four of the post-LASIK participants were satisfied and 4 were dissatisfied with the results of LASIK surgery (1 unknown). These interviews were used to assess item redundancy, content coverage, recall period, instructions, and format, and to revise items for the PROWL studies.

The PROWL-1 study protocol was approved by the Naval Medical Center San Diego Institutional Review Board (IRB) in compliance with all applicable federal regulations governing the protection of human participants. The PROWL-2 study was conducted under the US Food and Drug Administration Research Involving Human Subjects Committee, a central IRB for 3 sites (20/20 Institute, Indianapolis, Indiana; Durrrie Vision, Overland Park, Kansas; and Vance Thompson Vision, Sioux Falls, South Dakota), and university IRBs for 2 sites (The Johns Hopkins University, Baltimore, Maryland; and Stanford University, Stanford, California). All participants provided written as well as oral informed consent. There was no financial compensation in PROWL-1; PROWL-2 participants received compensation. The PROWL studies are registered at Clinicaltrials.gov (NCT0152629 and NCT01655420 for PROWL-1 and PROWL-2, respectively). Both studies were compliant with the Health Insurance Portability and Accountability Act and complied with the tenets of the Declaration of Helsinki.10

Quiz Ref IDPROWL-1 and PROWL-2 were prospective observational studies. Both enrolled patients planning to undergo LASIK surgery for myopia, hyperopia, or astigmatism. Investigators screened, enrolled, and treated study participants with LASIK surgery and followed up participants postoperatively according to their usual clinical practice. PROWL-1 was a single-center study of active-duty Navy military personnel; PROWL-2 was a 5-center study of civilians. Participants completed the questionnaires preoperatively and at 1 and 3 months after surgery using a secure website accessed from any computer. PROWL-1 had an additional 6-month postoperative visit with questionnaire administration. PROWL-1 began in August 2011 and was completed May 30, 2014; PROWL-2 began in July 2012 and was completed June 27, 2014. Data were analyzed from June 28, 2014, to October 24, 2016.

Participant Selection

Participants were 21 years or older, spoke and read English fluently, had not previously received any form of refractive surgery, and were determined to be good candidates for LASIK based on the surgeon’s assessment of medical and ophthalmic health, cognitive function, and physical and social limitations. A random subsample completed the questionnaire twice before surgery within a 3- to 14-day interval.

Prowl-1

PROWL-1 enrolled active-duty service naval personnel with a target refraction of bilateral emmetropia or slight hyperopia (+0.25 diopters [D]). Each participant had access to internet service and received their commanding officer’s permission to participate.

Prowl-2

PROWL-2 enrolled patients with a targeted refraction of bilateral emmetropia. To obtain a sample that reflected the refractive range of the LASIK-treated population, each site oversampled participants with higher refractive errors, including hyperopic participants with a spherical equivalent of equal to or greater than +1.50 D and myopic participants with a spherical equivalent more myopic than −7.00 D. Three strata were created based on spherical equivalent: more hyperopic than +1.50 D, between +1.50 D and −7.00 D, and more myopic than −7.00 D. Site enrollment was capped at 45 individuals with spherical equivalents between −7.00 D and +1.50 D until either 18 participants had enrolled in the other 2 strata or 105 days of enrollment had passed. After 105 days, enrollment was opened without restriction by strata.

Clinical Procedures and Measures

Centers performed the LASIK procedure and postoperative care based on the clinical judgment of the surgeons. The protocol did not dictate the care of participants, but it specified the collection of medical, ophthalmic, and surgical history; visual acuity, slitlamp, and posterior segment ocular examination findings; surgical factors; and postoperative clinical assessments.

Questionnaire

Quiz Ref IDThe questionnaire included new items to assess satisfaction with current vision (1 item), satisfaction with LASIK surgery (8 items), and the existence, bothersomeness, and effect on usual activities in the past 7 days of 4 key visual symptoms using polytomous response options: double images (8 items), glare (8 items), halos (8 items), and starbursts (8 items). We used a written definition of the symptom and images to illustrate symptom severity levels.

The questionnaire also included items from the National Eye Institute (NEI) Refractive Error Quality of Life Instrument with 42 items (NEI-RQL-42),11,12 the NEI Visual Function Questionnaire with 25 items (NEI-VFQ-25),13,14 and the Ocular Surface Disease Index (OSDI).15 All scales are scored on a 0 to 100 possible range. NEI-RQL-42 scales, NEI-VFQ driving scale, visual symptoms scales, and satisfaction with vision are scored such that a higher score is better; OSDI is scored such that a higher score is worse. Potential threats to the validity of the HRQOL measures were evaluated by assessing optimism (Life Orientation Test-Revised),16 health proneness (Brien Holden Vision Institute Quality of Life Scale for Myopia),17 anxiety and depressive symptoms (Patient Health Questionnaire-4),18 socially desirable response set,19 and expectations about spectacle use and vision after surgery (6 items). The questions and rationale for their selection are available in eMethods and eTables 1-11 in the Supplement.

The PROWL-1 preoperative (baseline) version of the questionnaire included 161 items and the postoperative version contained 129 items. The PROWL-2 baseline questionnaire had 154 items and the postoperative questionnaire included 112 items.

We estimated descriptive statistics and internal consistency reliability (coefficient α) for multi-item scales. The subsample with repeated questionnaire administrations within 14 days was used to estimate test-retest correlations (product-moment and intraclass). Reliability coefficients of 0.70 or above were considered adequate for group-level comparisons.22

We evaluated construct validity, using product-moment correlations of vision symptom scores with the satisfaction with LASIK surgery scale (hypothesizing statistically significant, positive associations) and expectations about spectacle use and vision after surgery, optimism, health proneness, depression and anxiety symptoms, and socially desirable responses (hypothesizing nonsignificant correlations). Responsiveness to change was assessed by computing change scores, effect size (change in scores divided by SD at baseline), and 2-tailed, paired t tests of the significance of change. We hypothesized that changes over time would be statistically significant and represent improvement in HRQOL (except for possibly worsening glare and dry eye symptoms). Results were considered statistically significant if the probability was P ≤ .05. Because the number of significant effects is inflated by change, we interpreted our results with this in mind.

Results

Participant Characteristics

Table 1 summarizes baseline characteristics of enrolled participants (262 in PROWL-1 and 312 in PROWL-2) and the analytic sample (240 [99.6% response rate] in PROWL-1 and 271 [94.4% response rate] in PROWL-2 among those eligible for the study). The most common reason participants were not included in the analytic sample was not having LASIK surgery (eFigure in the Supplement).

The median age of the 240 PROWL-1 respondents was 27 years (range, 21-52 years). Most participants were non-Hispanic white (54.6%), with 20.0% Hispanic, 9.2% non-Hispanic black, 9.2% non-Hispanic Asian, and 6.3% who self-identified as being other race/ethnicity; 20.4% were women. PROWL-2 did not include participants in the analytic sample if they did not complete the preoperative questionnaire (n = 5) or did not complete a postoperative questionnaire (n = 16). The median age of the 271 PROWL-2 respondents was 30 years (range, 21-57 years). Most participants were non-Hispanic white (76.4%), with 3.7% Hispanic, 1.8% non-Hispanic black, 12.2% non-Hispanic Asian, and 5.5% who self-identified as other; 54.2% were women.

Questionnaire Completion Time

The median (interquartile range) self-reported time for questionnaire completion at baseline was 25.00 (20.00-35.00) and 20.00 (15.00-25.00) minutes, respectively, in PROWL-1 and PROWL-2; the median (range) internet-recorded time was 22.42 (17.17-28.69) and 17.91 (14.63-22.51) minutes, respectively, in PROWL-1 and PROWL-2. The product-moment correlation between self-reported and recorded time was 0.49 in PROWL-1 and 0.58 in PROWL-2. Most study participants reported that completing the questionnaire by computer was easier than it would have been by paper (54.4% in PROWL-1 and 69.4% in PROWL-2). The mean (SD) number of days between test and retest was 10.74 (3.94) days (range, 5-23 days) in 50 PROWL-1 participants and 6.34 (3.04) days (range, 4-20 days) in 68 PROWL-2 participants.

The 2 largest correlations among scales at baseline (eTable 3 in the Supplement) were between the NEI-RQL-42 far vision and the NEI-VFQ-25 driving scales (PROWL-1, r = 0.81; PROWL-2, r = 0.85) and between the visual symptoms scales of starbursts and halos (r = 0.63 and r = 0.70, respectively).

Correlations between baseline expectations of spectacle use and vision with satisfaction with surgery at 1 and 3 months in both studies and at 6 months in PROWL-1 were not significant. The correlation of baseline health proneness with satisfaction with surgery at 3 months was statistically significant but small in magnitude (r = 0.14, P = .04 in PROWL-1 and r = 0.13, P = .04 in PROWL-2). The Patient Health Questionnaire-4 depressive/anxiety score at baseline was significantly associated with satisfaction with surgery at 6 months in PROWL-2, but was small in magnitude (r = −0.19, P = .004). The correlations of satisfaction with surgery with socially desirable response set were small in PROWL-1 (r = 0.13, P = .47 at 1 month and r = 0.15, P = .29 at 3 months) and not significant in PROWL-2 (r = 0.08, P = .17 at 1 month and r = 0.05, P = .46 at 3 months).

Change From Baseline to Follow-up

Changes from baseline to 1 month (Table 3), 3 months (Table 4), and 6 months (Table 5) postoperatively are reported. As hypothesized, most of the measures improved significantly in both studies from baseline to the 1-month follow-up, but hours worked, OSDI environment scale, and 2 visual symptoms (starbursts and halos) did not change in either study and the new visual symptoms glare scale did not change in PROWL-1. The NEI-RQL-42 glare scale score significantly worsened from baseline to 1 month postoperatively in both studies.

At the 3-month follow-up, every measure improved significantly except that there was no change from baseline for the NEI-RQL-42 glare scale and hours worked in both studies and for the OSDI environment scale in PROWL-1. At the 6-month follow-up (PROWL-1), all measures improved significantly from baseline with 2 exceptions: hours worked and the OSDI environment scale.

The magnitude of change in PROWL-1 ranged from 0.14 baseline SD (OSDI symptom scale) to 1.86 SD (satisfaction with vision) at 1 month, 0.17 SD (halos symptom scale) to 1.95 SD (satisfaction with vision) at 3 months, and 0.20 SD (NEI-RQL-42 glare scale) to 1.98 SD (satisfaction with vision) at 6 months. Magnitude of change in PROWL-2 ranged from 0.14 SD (NEI-RQL-42 clarity of vision) to 1.25 SD (satisfaction with vision) at 1 month and 0.19 SD (halos scale) to 1.34 SD (satisfaction with vision) at 3 months. Repeated-measures analyses that included baseline and all follow-up data in a single model produced results similar to those reported here.

Discussion

Quiz Ref IDThis study provides support for the reliability and validity of the HRQOL instrument administered in the PROWL studies for patients undergoing LASIK surgery. Content validity was enhanced by input from patients who underwent LASIK and clinicians during development of the new items.23 Item-scale correlations indicated that the items were almost always more highly correlated with the scale they were intended to represent than with other scales. Reliability estimates exceeded the 0.70 threshold for adequate reliability for most of the measures. In addition, correlations among the scales indicated that they yield distinct information about HRQOL. Moreover, the HRQOL measures were significantly positively correlated with patient satisfaction with LASIK surgery, consistent with a previous study.24 The measures were uncorrelated or weakly associated with socially desirable response set, health proneness (coping), optimism, depressive and/or anxiety symptoms, and expectations of spectacle use and vision.

Quiz Ref IDResponsiveness to change was supported by improvements in almost every HRQOL measure from baseline to follow-up. The NEI-RQL-42 glare scale showed increases in glare at 1 month in both PROWL-1 and PROWL-2 and reduction in glare at 6 months in PROWL-1. Differences from baseline for all other follow-up times were not found. A previous evaluation of 185 patients before and 4 months after surgical correction of myopic or hyperopic refractive error found increases in glare among myopes.12 Unlike the NEI-RQL-42, the new glare items included a definition that focused solely on glare, as well as an image showing graded severity. We found significant improvements (less glare) on the new visual symptoms glare scale at 1 month in PROWL-2, 3 months in PROWL-1 and PROWL-2, and 6 months in PROWL-1.

Limitations and Strengths

Inferences about the prevalence of the symptoms are limited by oversampling of high myopes and hyperopes. In addition, whether the changes observed persist beyond the time intervals studied is unknown. Furthermore, PROWL studies were limited to English-language participants. Administering the questionnaire to people whose English differs from US English or whose primary language is not English would require translation and assessment of the linguistic and cultural equivalence relative to the original English-language questionnaire. Moreover, associations of the PROs with visual acuity were small or not significant and using acuity as a clinical and/or was not supported. Finally, caution in drawing conclusions is warranted because of the multiple statistical tests performed.

Despite these limitations, most eligible patients were included in the analysis (99.6% in PROWL-1 and 95% in PROWL-2), which is notably better than most surveys.25 The results of PROWL-1 and PROWL-2 were consistent. The new visual symptom items provide a potentially valuable approach that couples images with definitions and facilitates reports about the impact of surgery. Eight questions for each symptom were used to assess frequency, bother, and difficulty doing usual activities when and when not wearing vision correction. Estimates of the minimally important difference of mean scores are reported in eTable 9 in the Supplement. Supplementary Rasch-model analyses were consistent with the reported results (eTables 10 and 11 in the Supplement).

The web-based administration of questionnaires, which allowed for questionnaire completion outside of the clinical visit, facilitated the conduct of the clinical studies by removing 1 level of data entry and potentially provided the participants more anonymity in their responses. Most study participants reported that completing the questions by computer was easier than it would have been to do so by paper.26

Conclusions

The PROWL studies provided support for the reliability and validity of most scales included in the questionnaire for evaluating the effect of LASIK surgery. The newly created measures to assess satisfaction with LASIK surgery and double images, glare, halos, and starbursts supplement the existing OSDI symptoms scale, NEI-RQL-42 scales (clarity of vision, near vision, far vision, diurnal vision, activity limitation, and worry), and the NEI-VFQ-25 driving scale. This collection of measures can be used to help provide estimates of the prevalence of symptoms, functioning, and well-being in future studies evaluating LASIK devices. We recommend use of the new visual symptoms scales, the satisfaction with LASIK surgery scale, and the satisfaction with vision item in future studies. These newly created scales in the PROWL questionnaire could be used to measure important visual symptoms that occur following LASIK.

Conflict of Interest Disclosures: All authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest and none were reported.

Funding/Support: The Department of Health and Human Services (DHHS) and the Department of Defense provided funding for this study.

Role of the Funder/Sponsor: The DHHS and the Department of Defense funded and participated in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Disclaimer: The mention of commercial products, their sources, or their use in connection with material reported herein is not to be construed as either an actual or implied endorsement of such products by the DHHS. The views expressed in this article are those of the authors and do not necessarily reflect the official policy or position of the Department of the Navy, Department of Defense, or the US government. Drs Tarver, Hilmantel, Hofmeister, Ferris, and Eydelman are employees of the US government or are military service members and prepared this work as part of their official duties. Title 17, USC, §105 provides that copyright protection under this title is not available for any work of the US government. Title 17, USC, §101 defines a US government work as a work prepared by a military service member or employee of the US government as part of that person’s official duties. Dr Ferris is the Viewpoint editor of JAMA Ophthalmology but was not involved in the editorial review or the decision to accept the manuscript for publication.