Bridging the auditability gap in consumer medical AI: design and public deployment of a Georgian-language, evidence-based symptom triage system (SheniEkimi) with 150 guideline-anchored scenarios and 29 validated clinical decision rules

Abstract

Background The rapid adoption of consumer-facing generative artificial intelligence (AI) chatbots for self-directed medical advice has exposed users to systematic risks including fabricated citations (“hallucinations,” reported at 28–47% across medical queries), output instability between sessions, and an absence of auditable clinical reasoning. Georgian-speaking populations face compounded risk because commercial AI interfaces offer poor-quality localisation and because no evidence-based, Georgian-language symptom triage tool previously existed. We report the development and public deployment of SheniEkimi, a non-commercial, evidence-anchored alternative. Objectives To design, implement, and publicly deploy a Georgian-language, bilingual (Georgian/English), auditable symptom triage system anchored exclusively in named international clinical guidelines and validated clinical decision rules (CDRs), with a transparent methodology reproducible by independent teams. Methods We constructed a 16-body-system framework adapted from WHO IMAI/IMCI and ICD-11, and selected 150 clinical scenarios meeting four inclusion criteria: assessable without in-hospital diagnostics, backed by an evidence-based triage threshold from an international guideline or externally-validated CDR, mappable to ICD-11, and common in primary-care contexts. Sources were appraised using an AGREE II–informed rubric. Twenty-nine CDRs meeting four quality criteria (peer-reviewed derivation; external validation; explicit thresholds; primary-care applicability) were implemented with full multi-question weighted scoring. The tool was implemented as a single-file WordPress plugin with Apple-Health-inspired interface, Georgian-primary bilingual architecture, runtime language toggling, and full offline capability. No identifiable personal data is collected or transmitted. Results Of 150 scenarios, 69 (46.0%) map to emergency output (112 dispatch), 75 (50.0%) to same-day primary care, and 6 (4.0%) to home self-care with explicit safety-netting. Cardiovascular (n=18), respiratory (n=13), neurological (n=13), gastrointestinal (n=13), and paediatric (n=12) domains together account for 45.3% of all scenarios, reflecting disease-burden weighting and density of validated CDRs. Sources span 39 international guidelines from WHO, NICE, AHA/ACC, ESC, BTS/SIGN, IDSA, ADA, AAP, AGS, RCOG, EAU, GOLD, and specialty consortia, plus 29 externally-validated CDRs. Deterministic validation confirms that every CDR’s scoring chain produces valid triage output across its full range. Full source code, methodology, and scenario registry are published under CC BY 4.0. Conclusions SheniEkimi is the first published Georgian-language symptom triage system anchored in peer-reviewed international evidence, and demonstrates that a volunteer-led public-health organisation can produce a transparent, auditable alternative to opaque proprietary symptom-checkers and to generative-AI medical-advice interfaces. The methodology is reproducible; the implementation is open; the operating model precludes commercial drift. The framework is offered as a template for other regional or minority-language digital health initiatives facing the same gap. Keywords clinical decision support; symptom triage; evidence-based medicine; digital health; artificial intelligence; AI hallucination; mHealth; Georgia; primary care; AGREE II; ICD-11; health equity.

Ähnliche Arbeiten