Development and Application of a Novel and Efficient Skills Assessment Tool: A Pilot Initiative to Measure Vasectomy Competency on a Smartphone

Objective: To develop and evaluate a mobile phone-based skills assessment tool that measures procedural competency of urology residents learning to perform a common, non-robotic urology procedure as a means of tracking current skillset and improvement over time. Methods: The assessment tool was a Qualtrics survey accessed via a smartphone link that breaks down a vasectomy into 6 critical steps. Level of competency was measured on a scale of ‘1–novice’ to ‘5–expert.’ Nine residents from Post graduate year (PGY)-1 to PGY-5 were evaluated by one instructor after completing a vasectomy (86 single-side cases recorded over a 6-month period). We compared individual trainees to each other, analyzed performance (improvement) over time, and evaluated competency against cohort and program averages. Results: As an example, a single resident (‘Resident 2,’ N = 11 cases) was compared to cohort (PGY, M = 7.5/resident) and program (all residents, M = 7.4/resident). Results indicate similar skillfulness across Step 1 (puncturing and isolation of vas and hand positioning; P > 0.1), but marginally lower competency on Step 2 (opening of vasal sheath to expose/isolate vas; vs. cohort: P = 0.076, vs. residents: P = 0.082). Significantly lower competency on Steps 3–6 (all P < 0.04) suggests targeted teaching could improve cautery technique, fascial interposition, hemostasis, and positioning of stumps. Conclusion: Our mobile-based skills assessment is a low cost, novel, and efficient assessment that would support current Accreditation Council for Graduate Medical Education (ACGME) goals to increase competency-based residency training. This tool is easily created and accessed, provides real-time feedback to learners, and can be used for individual and group assessment at a single timepoint or longitudinally.