A Systematic Review of Transformer-Based Language Models for Clinical Decision Support in EMR-Based Systems: Implications for Epilepsy Care in Low-Resource Settings
Keywords:
Transformer-Based Language Models; Large Language Models; Clinical Question-Answering; Clinical Decision Support Systems; Electronic Medical RecordsAbstract
Clinical decision-making in low- and middle-income country (LMIC) healthcare settings is often constrained by the inability to efficiently synthesize longitudinal patient data stored within Electronic Medical Record (EMR) systems. While traditional clinical decision support systems (CDSS) rely on rule-based alerts, they lack the contextual reasoning required to interpret complex clinical notes for chronic conditions like epilepsy. Recent advances in transformer-based language models, including BERT-family models, and generative large language models (LLMs) offer promising approaches for clinician-initiated medical question-answering (QA). This study presents a systematic review evaluating the architectures, applications, and EMR integration of transformer- and LLM-based QA and related clinical decision-support systems, with implications for epilepsy management in resource-limited environments. A structured search of primary biomedical and engineering academic databases was conducted to identify peer reviewed studies published up to 23 June 2026 related to CDSS, EMR-based decision support, and LLM applications in healthcare. The synthesis indicates that transformer-based and generative language models demonstrate promising performance in controlled clinical QA evaluations, but evidence for their practical deployment as clinician-facing tools integrated with active EMR databases remains limited. Evidence is also limited concerning the use of these approaches to interpret complex, unstructured timelines required for longitudinal chronic disease management in LMIC infrastructure. LLM-based clinical QA represents a viable entry point for responsible machine learning in resource-constrained settings, but future research must shift from benchmark validation to assessing real-world workflow compatibility and EMR integration constraints.
References
Agweyu, A., Mwaniki, P., Musau, W., Korom, R., Isaaka, L., Wanyama, C., et al. (2026). Safety of a large language model-based clinical decision support system in African primary healthcare. Nature Health. https://doi.org/10.1038/s44360-026-00082-5
Ahn S, Park J, Hur S, Jung KY, Lee JH, Choi SW, Son MH, Kang MJ, Kim YJ, Park H, Cha WC, Yoo J. (2025). Impact of an Evidence-Based Large Language Model (LLM) Diagnostic Decision Support System: A Randomised Controlled Trial. Studies in Health Technology and Informatics, 329, 273–277. https://doi.org/10.3233/SHTI250844.
Fleming, S. L., Lozano, A., Haberkorn, W. J., Jindal, J. A., Reis, E., Thapa, R., et al. (2024). MedAlign: A clinician-generated dataset for instruction following with electronic medical records. Proceedings of the AAAI Conference on Artificial Intelligence, 38(20), 22021–22030. https://doi.org/10.1609/aaai.v38i20.30205.
Griot, M., Vanderdonckt, J., & Yuksel, D. (2025). Implementation of large language models in electronic health records. PLOS Digital Health, 4(12), e0001141. https://doi.org/10.1371/journal.pdig.0001141.
Guo, J., Gao, Y., Liu, Z., Rao, Y., Hu, J., & Wang, H. (2025). EagleCare: Augmenting ICU monitoring with LLMs and knowledge graph reasoning. 2025 IEEE International Conference on Knowledge Graph (ICKG), 130–137. https://doi.org/10.1109/ICKG66886.2025.00024.
He, S., Zhang, Y., & Li, J. (2026). Personalized diabetes treatment support using large language models fine-tuned on electronic health records: Development and evaluation study. JMIR Formative Research, 10, e71541. https://doi.org/10.2196/71541.
Jani, K., Mattay, G., Narra, V., Shetty, A., Stancil, G., Bierhals, A., & Tadavarthi, Y. (2026). Zero-shot thoracic oncologic history generation for radiologists using retrieval-augmented large language model pipeline. Radiology, 319(3), e251581. https://doi.org/10.1148/radiol.251581.
Jeon, S., & Kim, H.-G. (2025). A comparative evaluation of chain-of-thought-based prompt engineering techniques for medical question answering. Computers in Biology and Medicine, 196, 110614. https://doi.org/10.1016/j.compbiomed.2025.110614.
Klang, E., Apakama, D., Abbott, E. E., Vaid, A., Lampert, J., Sakhuja, A., Freeman, R., Charney, A. W., Reich, D., et al. (2024). A strategy for cost-effective large language model use at health system-scale. npj Digital Medicine, 7, 320. https://doi.org/10.1038/s41746-024-01315-1.
Li, Y., Lv, M., Cheng, L., Xie, Y., Gu, Q., He, H., Chen, Z., Fang, D., & Zhou, X. (2026). Explainable and evidence-linked recommendations for spine surgery via a retrieval-augmented LLM agent. European Journal of Radiology, 197, 112734. https://doi.org/10.1016/j.ejrad.2026.112734.
Lu, K.-H., Mehdinia, S., Man, K., Wong, C. W., Mao, A., & Eftekhari, Z. (2025). Enhancing oncology-specific question answering with large language models through fine-tuned embeddings with synthetic data. JCO Clinical Cancer Informatics, 9, e2500011. https://doi.org/10.1200/CCI-25-00011.
Marin F, Castaño J, Segalini A, Botta C, Benitez S, Otero C, Luna D. (2025). Designing a Healthcare Co-Pilot with Generative AI. Studies in Health Technology and Informatics, 329, 618–622. https://doi.org/10.3233/SHTI250914
Mondal, A., Naskar, A., Roy Choudhury, B., Chakraborty, S., Biswas, T., Sinha, S., & Roy, S. (2025). Evaluating the performance and safety of large language models in generating type 2 diabetes mellitus management plans: A comparative study with physicians using real patient records. Cureus, 17(3), e80737. https://doi.org/10.7759/cureus.80737.
Moon, S., He, H., & Fan, J. W. (2022). Effects of information masking in the task-specific finetuning of a transformers-based clinical question-answering framework. 20Question IEEE 10th International Conference on Healthcare Informatics (ICHI), 36–41. https://doi.org/10.1109/ICHI54592.2022.00017.
Muralidharan, A., A. S., A. A. R., Raj, A. B., & Manmadhan, S. (2025). MedSyn: AI-driven medical history summarization using EHR data. 2025 Advanced Computing and Communication Technologies for High Performance Applications (ACCTHPA), 1–7. https://doi.org/10.1109/ACCTHPA65749.2025.11168697.
Patil, S. S., Rathore, A. S., & Ramteke, M. (2024). QLoRA-based fine-tuning of LLMs on multiple medical reasoning tasks to enhance their comprehension of clinical notes. 2024 11th International Conference on Soft Computing & Machine Intelligence (ISCMI). https://doi.org/10.1109/ISCMI63661.2024.10851618.
Pattnayak, P., Patel, H., Agarwal, A., Panda, S., Kumar, B., & Kumar, T. (2025). Clinical QA 2.0: Multi-task learning for answer extraction and categorization. 2025 IEEE International Conference on Electro Information Technology (eIT), 443–449. https://doi.org/10.1109/eIT64391.2025.11103631.
Thio, S., Lewis, M., Denaxas, S., & Dobson, R. J. B. (2026). Unlocking electronic health records: A hybrid graph RAG approach to safe clinical AI for patient QA. Frontiers in Digital Health, 8, 1780700. https://doi.org/10.3389/fdgth.2026.1780700.
Wang, J.-F., Chang, C.-C., Chiang, T.-M., Yeh, T.-C., Cheng, E., Lee, Y.-T., & Chen, H.-I. (2025). An innovative X-RAG technique combined with GPT-4o for summarizing medical information from EHR and EMR to assist doctors in clinical decision-making effectively and efficiently. Health Informatics Journal, 31(3), 14604582251381233. https://doi.org/10.1177/14604582251381233.
Wang, J.-F., Wei, M.-J., Chiang, T.-M., Yeh, T.-C., Cheng, E., Lee, Y.-T., & Chen, H.-I. (2025). Leveraging electronic health record data and up-to-date clinical guidelines for high-accuracy clinical diabetes drug and dosage recommendation. Methods of Information in Medicine, 64(1–02), 1–11. https://doi.org/10.1055/a-2707-2862. (PubMed)
Wen, A., Elwazir, M. Y., Moon, S., & Fan, J. W. (2020). Adapting and evaluating a deep learning language model for clinical why-question answering. JAMIA Open, 3(1), 16–20. https://doi.org/10.1093/jamiaopen/ooz072.
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hróbjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo-Wilson, E., McDonald, S., McGuinness, L. A., Stewart, L. A., Thomas, J., Tricco, A. C., Welch, V. A., Whiting, P., & Moher, D. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ, 372, n71. https://doi.org/10.1136/bmj.n71.
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., Payne, P., Seneviratne, M., Gamble, P., Kelly, C., Babiker, A., Schärli, N., Chowdhery, A., Mansfield, P., Demner-Fushman, D., Agüera y Arcas, B., Webster, D., Corrado, G. S., Matias, Y., Chou, K., Gottweis, J., Tomasev, N., Liu, Y., Rajkomar, A., Barral, J., Semturs, C., Karthikesalingam, A., & Natarajan, V. (2023). Large language models encode clinical knowledge. Nature, 620(7972), 172–180. https://doi.org/10.1038/s41586-023-06291-2.
Singhal, K., Tu, T., Montgomery, J., et al. (2023). Towards expert-level clinical language understanding with Med-PaLM 2. arXiv preprint arXiv:2305.09617.
Sutton, R. T., Pincock, D., Baumgart, D. C., Sadowski, D. C., Fedorak, R. N., & Kroeker, K. I. (2020). An overview of clinical decision support systems: Benefits, risks, and strategies for success. npj Digital Medicine, 3, 17. https://doi.org/10.1038/s41746-020-0221-y.
