Objective To evaluate the effectiveness of a natural language processing (NLP)-driven quantitative assessment and feedback mode applied during orthopedic teaching rounds in enhancing clinical reasoning skills among resident physicians. Methods A prospective, non-randomized controlled study was conducted with 60 orthopedic residents assigned to either the experimental group (n=30) or the control group (n=30). During the 3-month intervention period, the experimental group received personalized quantitative reports generated based on NLP technology analysis of their ward rounds speeches every week, and received one-on-one precise feedback from the supervising physician. The control group, on the other hand, received conventional, subjective impression-based oral feedback. The total score change value of the Mini-CEX was used as the primary outcome indicator, while theoretical knowledge tests, self-confidence questionnaires, and 360-degree evaluations were employed as secondary outcome indicators to comprehensively evaluate the intervention effect. Results At study completion, the experimental group showed a significantly greater mean increase in Mini-CEX score (1.73±0.72) compared with the control group [(0.70±0.61), P<0.001]. Improvements in self-confidence and 360-degree evaluation scores of the experimental group were also significantly higher than those of the control group (both P<0.05). Correlation analysis revealed that the key indicators extracted by the NLP model, such as “frequency of logical connectives”, have a significant positive correlation with the “clinical judgment” sub-score of Mini-CEX. (r=0.68, P<0.001). Conclusion The NLP-based quantitative assessment and feedback mode for teaching rounds effectively enhances the comprehensive clinical thinking ability and self-confidence of orthopedic residents. By making implicit cognitive processes explicit and measurable, this mode achieves objective assessment and precise intervention in clinical teaching, offering valuable empirical evidence and technical approaches for the optimization of the resident training system.
近年来,以自然语言处理(natural language processing,NLP)为代表的人工智能技术取得了长足进步,其在深度分析与挖掘非结构化文本数据方面的能力,为破解上述教学难题提供了新的可能[13-16]。有研究表明,语言是思维的外在表征,通过分析个体在特定场景下的语言学特征,可以有效推断其思维过程的质量[17-18]。将NLP技术引入教学查房场景,通过对查房过程中的师生对话进行量化分析,有望将模糊、隐性的教学过程转化为可度量、可分析的数据,从而为建立客观、多维度的住院医师能力评价体系,以及实施数据驱动的精准教学反馈开辟了全新路径[19-21]。
ChenF, LiH, LiangX, et al. Use of the Delphi method in the construction of the temperature chain management scheme for da vinci robot-assisted urological surgical patients[J]. Therapeutic Hypothermia and Temperature Management, 2025, 15(4): 174-183.
[6]
JohnsonB, BennettC, CarrierJ, et al. Introducing multidisciplinary ward rounds in Malawi: A best practice implementation project[J]. JBI Evidence Implementation, 2025, 23(S1): S35-S43.
[7]
KatesJ, BrownC, CampolietoJ, et al. Health professions students’ reflections about principles of interprofessional collaboration after shadowing interprofessional palliative care rounds[J]. The American Journal of Hospice & Palliative Care, 2025, 42(11): 1102-1108.
[8]
MastalerzK A, JordanS R, ConnorsS C. A qualitative study of patient and interprofessional healthcare team member experiences of bedside interdisciplinary rounds at a VA: Language, teamwork, and trust[J]. Journal of General Internal Medicine, 2025, 40(3): 538-546.
[9]
MoroF, ChiariniV, ScquizzatoT, et al. Variables for reporting studies on extended-focused assessment with sonography for trauma (E-FAST): An international Delphi consensus study[J]. Injury, 2025, 56(1): 111931.
[10]
QuadeA, JarrettE S, RizviR, et al. Impact of a novel virtual rounding queue software on nurse and family presence for rounds in the neonatal intensive care unit: A pilot study[J]. Academic Pediatrics, 2025, 25(2): 102584.
[11]
SharmaS, HuangR S, MalikA, et al. Insights from North American radiology grand rounds: Determining patterns of gender bias in professional introductions[J]. Current Problems in Diagnostic Radiology, 2025, 54(1): 81-86.
[12]
SongA, SafdiehJ E, RobbinsM S. Group text messaging as a residency teaching tool in outpatient neurology and headache: A mixed-methods observational study[J]. Headache, 2025, 65(4): 539-544.
[13]
ConwayA, LiJ, RadM G, et al. Automating sedation state assessments using natural language processing[J]. Journal of Nursing Scholarship, 2025, 57(1): 17-27.
[14]
DalalT C, LiangL B, SilvaA M, et al. Speech based natural language profile before, during and after the onset of psychosis: A cluster analysis[J]. Acta Psychiatrica Scandinavica, 2025, 151(3): 332-347.
[15]
DemilieM M, AschaleA A, SalauA O, et al. Speaker-based language identification for Ethio-Semitic languages using CRNN and hybrid features[J]. Network: Computation in Neural Systems, 2025, 36(4): 1313-1335.
[16]
DoughertyR F, ClarkeP, AtliM, et al. Psilocybin therapy for treatment resistant depression: Prediction of clinical outcome by natural language processing[J]. Psychopharmacology, 2025, 242(7): 1553-1561.
[17]
GoldbergS B, TananaM, StewartS H, et al. Automating the assessment of multicultural orientation through machine learning and natural language processing[J]. Psychotherapy, 2025, 62(3): 292-300.
[18]
GrahamL A, IllarmoS S, WrenS M, et al. Use of natural language processing method to identify regional anesthesia from clinical notes[J]. Regional Anesthesia and Pain Medicine, 2025, 50(3): 271-275.
[19]
KaźmierczakI, JakubowskaA, PietraszkiewiczA, et al. Natural language sentiment as an indicator of depression and anxiety symptoms: A longitudinal mixed methods study[J]. Cognition & Emotion, 2025, 39(7): 1693-1702.
[20]
LiZ B, ZhaoH Y, ZhuG H, et al. Classification method of traditional Chinese medicine compound decoction duration based on multi-dimensional feature weighted fusion[J]. Computer Methods in Biomechanics and Biomedical Engineering, 2025, 28(6): 867-881.
[21]
RobinsonF E, GrimmD, HorningD, et al. An automated content-based measure of closed loop communication among critical care air transport teams[J]. Military Psychology, 2025, 37(2): 168-174.
[22]
SabanM, LutskiM, ZuckerI, et al. Identifying diabetes related-complications in a real-world free-text electronic medical records in Hebrew using natural language processing techniques[J]. Journal of Diabetes Science and Technology, 2025, 19(4): 999-1007.
[23]
WollekA, HaitzerP, SedlmeyrT, et al.Language model-based labeling of German thoracic radiology reports[J]. RöFo - Fortschritte Auf Dem Gebiet der Röntgenstrahlen und der Bildgebenden Verfahren, 2025, 197(1): 55-64.
[24]
ZaidatB, LahotiY S, YuA, et al. Artificially intelligent billing in spine surgery: An analysis of a large language model[J]. Global Spine Journal, 2025, 15(2): 1113-1120.
[25]
BaH, XuL, GuY, et al. Comparative study of problem-based learning and traditional teaching methods on medical students’ outcomes in pediatrics clerkships[J]. Advances in Medical Education and Practice, 2025, 16: 615-624.
[26]
JiaF, WangM, LüQ, et al. Application of the BOPPPS-CBL model in emergency chest pain management teaching for resident physicians: A randomized comparison[J]. BMC Medical Education, 2025, 25(1): 378.
[27]
LiuC, RenM, LuoC E, et al. Application of mini-CEX evaluation combined with segmented teaching approach in the training of surgical clinical practice skills for resident physicians[J]. Journal of Surgical Education, 2025, 82(5): 103497.
[28]
LuoF L, CaiJ, MaH X, et al. Application of a training program system centered on job competency in the standardized training of new nurses[J]. BMC Nursing, 2025, 24(1): 358.
[29]
O’SullivanO E, DaykanY, RotemR. Advancing obstetrics and gynecology training: The shift toward outcome and based education, a narrative review of literature[J]. International Journal of Gynaecology and Obstetrics, 2025, 171(3): 1156-1161.
[30]
RenS S, ZhanH, FernandoA, et al. From shadowing to active learning: Exploring the impact of supervised teaching clinics on gynecology education[J]. Frontiers in Medicine, 2024, 11: 1498393.
[31]
SuiX X, CuiZ, LiQ X. Application of scenario simulation combined Mini-CEX teaching model in gynecology and obstetrics resident teaching: A case study of paclitaxel and cisplatin use[J]. Pakistan Journal of Pharmaceutical Sciences, 2025, 38(1): 157-164.
[32]
UrbinaJ, MonksS M. Validating assessment tools in simulation[M]. Treasure Island (FL): StatPearls Publishing, 2025.
[33]
XuT, CuiL P, LiC. Effect evaluation of the 7E teaching model based on real clinical cases in nursing students’ clinical rounds: A quasi-experimental study[J]. Nurse Education in Practice, 2025, 84: 104316.
[34]
YaoY, YangG, LiX, et al. An assessment of clinical reception training using standard patient and dental simulator in prosthodontic dentistry for dental undergraduates: A historical control trial[J]. BMC Medical Education, 2024, 24(1): 1537.