Assessing fellow competence in esophagogastroduodenoscopy (EGD): A pilot study

Goals: To objectively assess when gastroenterology fellows achieve technical competency to perform upper endoscopy independently Background: Trainees are expected to achieve competence in esophagogastroduodenoscopy (EGD) by completion of their fellowship. The procedure experience needed to achieve competence in upper endoscopy is not known and has not been studied. Study: We performed a prospective pilot study examining consecutive EGDs performed by gastroenterology fellows from July 2014 through January 2015 at an academic medical center. Using a brief post-procedure questionnaire, fellows were graded by their attendings and themselves on the amount of verbal coaching and technical assistance required to complete each EGD. Diagnostic and interventional portions of each case were graded separately. Fellows were graded on a 1-5 scale with a score of 1 equivalent to needing 100% assistance and a score of 5 equivalent to requiring no assistance. The primary outcome was independent EGD completion rate, measured as a percentage of the last 20 EGDs performed. Secondary outcomes were attending assessments and fellow self-assessments of the level of attending assistance required to complete each EGD. Results: A total of 496 EGDs performed by 12 gastroenterology fellows were analyzed. First (n=5), second (n=4), and third year (n=3) GI fellows had performed a median of 26, 337, and 592 EGDs, respectively, prior to the start of the study period. After performing 95 EGDs, fellows transiently achieved an independent completion rate of >80% for every 20 procedures performed. After completing 159 EGDs, the independent completion rate did not fall below 80%. The independent completion rate among 2nd and 3rd year fellows never fell below 95%. The mean attending assessment and fellow self-assessment scores at 50, 100, and 150 procedures were 3.2 and 2.80, 4.0 and 3.5, and 4.11 and 3.7, respectively. Mean attending assessment scores for 2nd and 3rd year fellows were 4.9 and 5, respectively at the start of the study period. Conclusion: Skills in diagnostic EGD improve over the course of the first year in GI training. In this small pilot study, technical competence in diagnostic EGD appeared to occur between the 160th and 340th EGD performed during training. Correspondence to: Vilas R. Patwardhan, Department of Medicine, Division of Gastroenterology, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, Massachusetts, USA; Tel: 617-632-1070; Fax: 617-632-1065; E-mail: vpatward@bidmc.harvard.edu


Introduction
Guidelines for procedure exposure and training during gastroenterology (GI) fellowship have been set forth by the Gastroenterology Core Curriculum (GCC) and American Society for Gastrointestinal Endoscopy (ASGE) [1]. The GCC is guided by the four major American gastrointestinal societies including the ASGE, American Gastroenterological Association (AGA), American Association for the Study of Liver Diseases (AASLD), and the American College of Gastroenterology (ACG), and is aimed to guide best practices in gastroenterology training. The GCC describes the threshold numbers of standard endoscopic procedures that fellows should meet by the time of fellowship graduation [2]. While studies have examined the procedural experience required for fellows to achieve technical competency in procedures such as colonoscopy and endoscopic retrograde cholangiopancreatography (ERCP) [3,4], studies on the procedure experience needed to attain competence in esophagogastroduodenoscopy (EGD) are lacking.
Therefore, we performed a prospective pilot study of EGDs performed by GI fellows to determine the procedural experience required to attain technical competence in upper endoscopy.

Methods
We performed a single center, prospective pilot study examining esophagogastroduodenoscopies (EGDs) performed by gastroenterology fellows at an academic medical center, Beth Israel Deaconess Medical Center (BIDMC), in Boston, Massachusetts. All EGDs performed by a general gastroenterology (GI) fellow from July 2014 through January 2015 were included. All general GI fellows (n=12) were asked to participate. Fellow EGDs performed while rotating through the advanced endoscopy unit were excluded from analysis. Advanced fellows were excluded as well. Consent was indicated by willingness of the fellow to fill out the procedure assessment form and submit it for inclusion in our study. At any point in time fellows were able to withdraw from the study.
To collect data on independent procedure completion, as well as fellow, attending, and nursing assessments, all fellows, attendings, and nurses were asked to fill out a brief questionnaire following each upper endoscopy performed ( Figure 1). The questionnaire included the following information: 1) Whether the procedure was completed without attending assistance 2) If the procedure was not completed independently, which aspects required assistance 3) Fellow self assessment 4) Nursing comfort assessment 5) Attending assessment 6) Whether an intervention was performed 7) Whether intervention(s) required attending assistance. Attending assistance was defined as the attending physically handling the scope at any point during the procedure. Verbal coaching was permitted and graded on a Likert scale from 1-5 as follows: 1= procedure could not be completed without attending assistance 2= verbal coaching required for >50% of the procedure 3=verbal coaching required for <50% but >25% of the procedure 4= verbal coaching required for less than 25% of the procedure 5=no verbal coaching required. All procedures completed independently required a score of >1 in the fellow and attending assessments. The scores for fellow self-assessment and Fellows EGD Study -(Fill out and staple to a copy of the EGD report) ** indicates field to be filled out by attending physician attending assessment reflected only the diagnostic portion of the EGD. Interventions were graded separately as indicated in the questionnaire ( Figure 1). Additional information was gathered from the endoscopy report and medical record and included the following information: location of the procedure (i.e. inpatient vs. outpatient), patient age, gender, indication for procedure, type of sedation, procedure count of the fellow in training, and year of training. Procedure counts were obtained from the electronic medical record endoscopy software (G-Care, gMed Florida), which tabulates trainee's procedure count continuously.
Patient comfort was assessed by the nurse using a Likert score from 1-5 (5 being most comfortable and 1 being most uncomfortable as follows: 1= Patient comfortable for <20% of procedure 2= Patient comfortable for <50% of procedure 3=Patient comfortable for >50% but less than 80% of procedure 4=Patient comfortable for >80% of the procedure, but not the entire procedure 5=Patient comfortable during the entire procedure). Moderate sedation was administered using midazolam and fentanyl and cetacaine spray. Cases with monitored anesthesia care (MAC) employed propofol for sedation. Since sedative medications used for endoscopy have amnestic properties i.e. midazolam and propofol, comfort measured by those witnessing the procedure was felt to be more reflective of patient comfort than that recollected by the patient following administration of procedural sedation. The standard of practice at our institution is to use a 180 o Olympus high-definition adult gastroscope except in rare cases when an adult therapeutic gastroscope or pediatric gastroscope was required. This study was approved by the BIDMC Institution Review Board.

Statistical analysis
The primary outcome was independent EGD completion rate, measured as a percentage of the last 20 EGDs performed. All EGDs were analyzed. However, independent completion was determined based solely on the ability to perform the diagnostic portion of the EGD even in the cases where interventions were performed. Secondary outcomes were attending assessments and fellow self-assessments of the level of attending assistance required to complete each EGD. Since the independent completion rate was close to 100% for fellows in year 2 and 3 of training, statistical analysis was restricted to fellows in their first year of training. Descriptive statistics are displayed as mean + standard deviation, median [range], or as otherwise indicated.

Demographics
See Table 1 for patient demographics and fellow characteristics. Twelve gastroenterology fellows were eligible and included in this study. In total, 496 fellow EGDs were performed between July 31, 2014 and January 12, 2015. At the start of the study, the median number of EGDs performed during the entire fellowship to date by first year trainees (FY1) was 26 (range 20-51), 337 (range 314-393) for second year trainees (FY2), and 592 (range 542-700) for third year trainees (FY3). At the end of the study period, the median number of procedures performed in total during fellowship was 161 (range 95-176) for first years, 358 (range 344-428) for second years and 809 (range 631-827) for third year fellows. FY1 trainees performed 254 EGDs, FY2 107, and FY3 135 during the study period. The most common indication for upper endoscopy was bleeding (n=135; 27%) followed by abdominal pain (n=106; 21%). The following interventions were performed: Biopsy (n=200), Endoclip (n=4), BICAP electrocautery (n=3), esophageal band ligation (n=15), and Argon Plasma Coagulation (n=4). MAC anesthesia was used for 32% of the procedures and 37% of the procedures were performed on inpatients.

Independent fellow EGD completion rates
Rates of independent fellow completion (verbal coaching allowed) varied significantly by fellow year, with the most significant learning curve occurring in the fellow year 1 (Figure 2). All fellows consistently achieved greater than 95% independent completion by the start of their second year of fellowship. In the first year of training, fellows first achieved an independent completion rate of >80% by their 95 th EGD. However, they did not consistently achieve greater than 80% independent completion rate by their 160 th . Fellows had over 90% independent completion after their 170 th EGD.
Given the limited number of interventions performed, the study was not felt to be sufficiently powered to analyze the relationship between procedural training and upper GI interventions. Similarly, analysis of the effect of inpatient procedures and MAC anesthesia was restricted to FY1 procedures based on the nearly 100% independent completion rates among FY2s and FY3s and attending and fellows assessments starting off near 5. Among the procedures performed by FY1s, there was no significant difference in independent completion rates based on procedure location, with 67% of inpatient procedures and 75% of outpatient EGDs being completed independently (P=0.52). Similarly, there was no difference in independent completion rates by type of sedation used. The independent completion rate was 67% in cases where MAC was employed and 73% in those employing moderate sedation (P=0.58).

Discussion
In this pilot study of GI fellow technical competency to perform independent EGD, we found that the benchmark of 130 EGDs set forth by the GCC may underestimate the procedural experience required before fellows are ready to enter independent practice. In our study, fellows required over 170 procedures to perform independent EGD in greater than 90% of cases. However, while this benchmark of 90% independent procedure completion has been used in prior studies of fellow cecal intubation in colonoscopy, it is likely not adequate for independent GI practice [5][6].
Additionally, independent procedure completion alone does not take into account the importance of attending coaching that may be pivotal to a fellow completing a procedure independently. This was evident in our cohort, in which fellows who had completed 170 EGDs, and achieved >90% independent completion, had a mean selfassessment score of 4. This suggests that even in those procedures completed independently, verbal coaching may have been needed for up to 25% of each procedure. When accounting for verbal coaching, it was not until the start of second year of fellowship, after fellows had completed a median of 337 EGDs, that no verbal coaching was required.
A recent survey of graduating third year gastroenterology fellows found that while 98% of respondents met the required number of diagnostic EGDs put forth by the GCC (130), fewer numbers of respondents had met the interventional goals for treatment of variceal and non-variceal hemorrhage and esophageal dilation (75%, 40%, and 43%, respectively) [7]. These data are consistent with the findings of our study, in which 26 interventions were completed for 135 EGDs performed for GI bleeding. When combined with the fact that fellows may have more limited opportunities to perform interventional endoscopy (i.e. treatment of peptic ulcer disease bleeding, upper GI AVMs, and esophageal varices) in training, the current benchmark of performing 130 EGDs over the course of GI fellowship training may be inadequate. Previous study in colonoscopy have similarly shown that greater procedural experience is needed to attain technical proficiency in interventions such as polypectomy over and above that needed for cecal intubation [8].
To our knowledge, this type of real world experience in fellow EGD training has not been reported previously nor has the role of verbal coaching in endoscopy training. As demonstrated in this pilot study, assessing a fellow's need for verbal coaching during training may be as important an indicator of progress as a trainee's ability to technically complete procedures independently. Fellow self-assessments, though similar in scoring to the attending assessments in this study, may add further knowledge to deciphering when fellows themselves feel competent to perform endoscopy independently. Self-assessments have been used in surgical training to guide the benchmarks for flexible endoscopy training and have also been used to make GI fellows' progress transparent relative to their peers [9,10]. Optimizing the use of fellow self-assessments in GI endoscopy training is an area for future research.
Our results should be interpreted in the context of the study design. This was a single center pilot study, without prior publications in the literature in GI EGD fellowship training. Hence, our study may not have been powered to detect the role of covariables such as procedure location, procedure indications, and type of sedation used on trainee learning and success. Additionally, all fellows in our center became technically proficient in EGD by the start of their second year, thus reducing the number of fellows actively "learning" EGD during the study period. Similarly, since this study was conducted at a single center, we cannot account for differences in programmatic training between institutions that may affect trainee procedural competency. We should also note, that during the study period, a relatively few number of interventions were performed, thus limiting analysis of competency in endoscopic interventions. However, the procedure exposure in the study period does appear to mirror graduating trainee survey responses from across the US. Future studies of EGD training should focus not only diagnostic, but also interventional aspects of upper endoscopy. Lastly, although this study focuses on technical competency, recently published guidelines fellow procedural competency focus not only on technical, but also cognitive, and integrative competencies [11,12], which are central to independent gastroenterology practice.

Conclusion
In this pilot study of GI fellowship EGD training, skills in diagnostic EGD improved over the first year in GI training. Fellows achieved technical competency in EGD between the 160 th and 340 th EGD performed. Larger longitudinal studies are needed to clarify the  . First year fellow attending assessments and fellow self-assessments of EGD performance.The solid line represents the fellow self-assessment. The dashed line represents the attending assessment.