Research Study: Measuring the Value of AI in FRCS Exam Preparation
Evaluating AI-generated vs human-authored question bank content
About This Study
This research study evaluates whether AI can reliably generate FRCS-style questions for plastic surgery education, and whether human editorial oversight can elevate AI-generated content to human-level quality.
Principal Investigator: Omar Asmar, ST6, Mersey and West Lancashire NHS Trust
We compare 300 single-best-answer questions from different sources.
Study Objectives
- Compare quality ratings and error prevalence across four question sources, with particular focus on whether AI-assisted authoring can match pure human performance
- Quantify the magnitude of quality improvement achieved through human review of AI drafts
- Assess surgeons' ability to identify AI authorship, providing insight into detectability of stylistic patterns
What You'll Do as a Participant
Practising UK-based plastic surgeons (ST3 to Consultant level) answer questions, provide quality ratings (Good or bad), flag suspected errors, and indicate whether they suspect a question was AI-generated.
Your responses are anonymized, and all flagged errors undergo formal adjudication against standard plastic surgery references to confirm true errors versus false reports.
Contribute to surgical education
Help determine the appropriate role for AI in question bank production and the level of editorial oversight required for safe deployment.
Eligibility
Participants must be practising UK-based plastic surgeons ranging from ST3 trainee to Consultant level. No exclusion criteria are applied beyond provision of informed consent. During sign-up, you'll indicate your training grade and FRCS (Plast) certification status to enable subgroup analysis while maintaining anonymity.
Data Protection
IP addresses are not stored to protect participant privacy. User identifiers are pseudonymized. Data are stored in a secure database. No patient data are involved.