CLARA – Conversational LLM Application for Real-Time Admissions – A Pilot Study Leveraging Advanced LLMs for Multilingual Patient Admission Using the OpenAI Realtime API

Aim and Research Question(s)

This thesis aimed to develop and evaluate CLARA, a real-time, multilingual conversational AI system for hospital admissions that uses voice input.

• How can CLARA handle incomplete input, corrections, and multilinguality using the OpenAI Realtime API?
• How accurately does CLARA capture and verify key patient data (name, DOB, insurance number, symptoms)?
• How do users perceive its usability and efficiency in simulated multilingual admissions?

Background

Emergency departments are facing rising patient volumes and growing linguistic diversity, which complicates the timely and accurate admission of patients [1]. Traditional form-based approaches are often inflexible and unable to handle spontaneous or multilingual patient input. However, recent advances in conversational AI and large language models (LLMs) enable natural, speech-based interactions that can streamline healthcare workflows, such as triage and data collection [2]. OpenAI’s Realtime API integrates real-time transcription, language understanding, and speech synthesis into a single pipeline, offering new opportunities for multilingual patient engagement [3]. However, challenges such as hallucinations, privacy concerns and system reliability still hinder widespread clinical adoption [4].

Methods

CLARA was built using the OpenAI Realtime API, which enables real-time speech interaction and structured data capture. The system was tested in a pilot study with 30 participants divided into seven language groups. Each group completed five role-played admission sessions using pre-defined patient personas. The evaluation used interaction logs and the MAUQ usability questionnaire.

Results and Discussion

• Accuracy: The name and symptoms fields are over 95% accurate in most languages. The DOB and insurance numbers are slightly less accurate due to strict format validation.
• Latency: The median system response time is ~1,273 ms, which supports real-time dialogue.
• Clarifications: Most corrections were required for names in Spanish and Bosnian, while symptoms generally required no correction.
• Usability: High scores for ease of use and interface satisfaction: Lower usefulness due to the need for an active internet connection. CLARA cannot be used offline.

The results demonstrate high reliability and usability. However, real clinical testing and GDPR-compliant hosting are future challenges.

Conclusion

CLARA demonstrates the technical feasibility and user acceptance of real-time, voice-based, multilingual AI systems for hospital admissions. This pilot study confirms that systems like CLARA can accurately and intuitively operate in simulated conditions, paving the way for clinical pilot trials.

References

[1] Ärzteblatt Redaktion. Notaufnahmen: Immer mehr Patienten. Dtsch Ärztebl. 2011. www.aerzteblatt.de/archiv/109145 [2] Wang D, Zhang S. LLMs in Medical & Healthcare Fields. Artif Intell Rev. 2024;57(11):299. doi:10.1007/s10462-024-10599-w [3] OpenAI. Realtime API Documentation. 2024. platform.openai.com/docs/guides/realtime [4] Ray PP. LLMs in Healthcare: Opportunities & Risks. Brief Bioinform. 2024;25(3):bbae214. doi:10.1093/bib/bbae214