Exploring the potential utility of AI large language models for medical ethics: an expert panel evaluation of GPT-4
Integrating large language models (LLMs) like GPT-4 into medical ethics is a novel concept, and understanding the effectiveness of these models in aiding ethicists with decision-making can have significant implications for the healthcare sector. Thus, the objective of this study was to evaluate the...
| Auteurs: | ; ; ; ; ; ; ; ; ; ; |
|---|---|
| Type de support: | Électronique Article |
| Langue: | Anglais |
| Vérifier la disponibilité: | HBZ Gateway |
| Interlibrary Loan: | Interlibrary Loan for the Fachinformationsdienste (Specialized Information Services in Germany) |
| Publié: |
2024
|
| Dans: |
Journal of medical ethics
Année: 2024, Volume: 50, Numéro: 2, Pages: 90-96 |
| Accès en ligne: |
Volltext (lizenzpflichtig) Volltext (lizenzpflichtig) |
MARC
| LEADER | 00000naa a22000002c 4500 | ||
|---|---|---|---|
| 001 | 1918775532 | ||
| 003 | DE-627 | ||
| 005 | 20250228102319.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 250228s2024 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.1136/jme-2023-109549 |2 doi | |
| 035 | |a (DE-627)1918775532 | ||
| 035 | |a (DE-599)KXP1918775532 | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rda | ||
| 041 | |a eng | ||
| 084 | |a 1 |2 ssgn | ||
| 100 | 1 | |a Balas, Michael |e VerfasserIn |0 (orcid)0000-0002-5948-0331 |4 aut | |
| 245 | 1 | 0 | |a Exploring the potential utility of AI large language models for medical ethics: an expert panel evaluation of GPT-4 |
| 264 | 1 | |c 2024 | |
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a Computermedien |b c |2 rdamedia | ||
| 338 | |a Online-Ressource |b cr |2 rdacarrier | ||
| 520 | |a Integrating large language models (LLMs) like GPT-4 into medical ethics is a novel concept, and understanding the effectiveness of these models in aiding ethicists with decision-making can have significant implications for the healthcare sector. Thus, the objective of this study was to evaluate the performance of GPT-4 in responding to complex medical ethical vignettes and to gauge its utility and limitations for aiding medical ethicists. Using a mixed-methods, cross-sectional survey approach, a panel of six ethicists assessed LLM-generated responses to eight ethical vignettes. The main outcomes measured were relevance, reasoning, depth, technical and non-technical clarity, as well as acceptability of GPT-4’s responses. The readability of the responses was also assessed. Of the six metrics evaluating the effectiveness of GPT-4’s responses, the overall mean score was 4.1/5. GPT-4 was rated highest in providing technical (4.7/5) and non-technical clarity (4.4/5), whereas the lowest rated metrics were depth (3.8/5) and acceptability (3.8/5). There was poor-to-moderate inter-rater reliability characterised by an intraclass coefficient of 0.54 (95% CI: 0.30 to 0.71). Based on panellist feedback, GPT-4 was able to identify and articulate key ethical issues but struggled to appreciate the nuanced aspects of ethical dilemmas and misapplied certain moral principles. This study reveals limitations in the ability of GPT-4 to appreciate the depth and nuanced acceptability of real-world ethical dilemmas, particularly those that require a thorough understanding of relational complexities and context-specific values. Ongoing evaluation of LLM capabilities within medical ethics remains paramount, and further refinement is needed before it can be used effectively in clinical settings. | ||
| 700 | 1 | |a Wadden, Jordan Joseph |e VerfasserIn |0 (orcid)0000-0002-3636-8574 |4 aut | |
| 700 | 1 | |a Hébert, Philip C. |e VerfasserIn |4 aut | |
| 700 | 1 | |a Mathison, Eric |e VerfasserIn |4 aut | |
| 700 | 1 | |a Warren, Marika D. |e VerfasserIn |4 aut | |
| 700 | 1 | |a Seavilleklein, Victoria |e VerfasserIn |4 aut | |
| 700 | 1 | |a Wyzynski, Daniel |e VerfasserIn |4 aut | |
| 700 | 1 | |a Callahan, Alison |e VerfasserIn |4 aut | |
| 700 | 1 | |a Crawford, Sean A. |e VerfasserIn |4 aut | |
| 700 | 1 | |a Arjmand, Parnian |e VerfasserIn |4 aut | |
| 700 | 1 | |a Ing, Edsel B. |e VerfasserIn |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t Journal of medical ethics |d London : BMJ Publ., 1975 |g 50(2024), 2, Seite 90-96 |h Online-Ressource |w (DE-627)323607802 |w (DE-600)2026397-1 |w (DE-576)260773972 |x 1473-4257 |7 nnas |
| 773 | 1 | 8 | |g volume:50 |g year:2024 |g number:2 |g pages:90-96 |
| 856 | 4 | 0 | |u https://doi.org/10.1136/jme-2023-109549 |x Resolving-System |z lizenzpflichtig |3 Volltext |
| 856 | 4 | 0 | |u https://jme.bmj.com/content/50/2/90 |x Verlag |z lizenzpflichtig |3 Volltext |
| 951 | |a AR | ||
| ELC | |a 1 | ||
| ITA | |a 1 |t 1 | ||
| LOK | |0 000 xxxxxcx a22 zn 4500 | ||
| LOK | |0 001 4675088252 | ||
| LOK | |0 003 DE-627 | ||
| LOK | |0 004 1918775532 | ||
| LOK | |0 005 20250228102319 | ||
| LOK | |0 008 250228||||||||||||||||ger||||||| | ||
| LOK | |0 040 |a DE-Tue135 |c DE-627 |d DE-Tue135 | ||
| LOK | |0 092 |o n | ||
| LOK | |0 852 |a DE-Tue135 | ||
| LOK | |0 852 1 |9 00 | ||
| LOK | |0 935 |a ixzs |a ixzo |a ixrk | ||
| OAS | |a 1 |b inherited from superior work | ||
| ORI | |a SA-MARC-ixtheoa001.raw | ||
