Varun Ranganathan, MCOptom

Clinical Optometrist

An OCULAR Interface Exclusive

Synopsis:

In this blog, we explore the transformative impact of multimodal AI in the field of eye care. By integrating data from diverse sources—such as retinal imaging, patient histories, genetic information, and wearable devices—multimodal AI is revolutionising the way we diagnose, treat, and manage eye diseases. We explore how this cutting-edge technology enables early detection of conditions, facilitates personalised treatment plans, and enhances patient monitoring through continuous data analysis.

Keywords: Multimodal AI, Generative AI, AI and drug trials, ocular image analysis, Ocular biosensors.

Introduction

The use of Artificial Intelligence (AI) in eye care is not limited to AI’s ability to pre-detect eye conditions before being clinically apparent. We have barely tapped into AI’s potential to reveal the secrets of pathophysiology of certain eye conditions and deepen our understanding of how our visual system functions.

As we have explained in our previous blogs, Generative AI, and Natural Language Processing (NLP) [GANs link here] are being incorporated into eye care practices [link here], these advancements go beyond just diagnosing eye conditions with AI.

Multimodal AI is the newest ‘kid in the block’ in the field of Generative AI and it can generate data with images, texts, speech, and videos or in other words different modalities as opposed to a single modality.¹

How Multimodal AI works

These systems are trained to gather and combine data from different modalities (as explained above). The data is consolidated and represented, and the system makes predictions into specific output categories.² This improves the overall understanding of the concept, enhances accuracy due to unified representation of data, boosts the AI’s learning by leveraging data from different modalities. Let us now discuss how these AI models can be incorporated in eye care.

Uses in Eye care

Using the AI’s image description ability, a fundus, or Optical Coherence Tomography (OCT) image, the AI can be instructed to describe the image by pointing out various pathological landmarks and important biomarkers.³ This will be hugely beneficial to eye care professionals (ECP) by improving accuracy in diagnosing eye conditions, especially in busy environments.

History taking is considered an integral part of a medical examination and during an eye test, a comprehensive history can lead to better differential diagnosis, allowing the ECP to carry out selective tests. With the AI’s multimodal ability, data can be collected from patient’s symptoms, general health, genetic makeup, and a concise differential diagnosis can be piled, saving valuable time.

Personalised treatment plans including prescription of drugs can also be achieved using AI’s multimodal approach, considering fundus or OCT image analysis, patient’s lifestyle, possibilities of adverse drug reaction (ADR) and other important ocular findings.⁶

Ocular Biosensors is a promising option for monitoring ocular health continuously and non-invasively through embedded chips in wearable devices [biosensor blog link]. These chips are seamlessly connected with mobile applications to provide continuous feedback of ocular parameters. Multimodal AI can play a very crucial part here by collecting data about patients’ lifestyle, general health status, ethnicity and genetic makeup making real-time and prompt diagnosis, which in many cases can be lifesaving.

Conclusion

Humanity has stepped into an era of AI and its uses in every field are gradually becoming limitless. Though we are yet to use the full power of AI, future concerns about the explainability of AI to make ethical decisions have been raised. AI is developed by humans, so our supervision and intervention will be necessary in most scenarios.

References

1. Multimodal AI Models: Understanding Thei https://builtin.com/articles/multimodal-ai r Complexity – Addepto.
2. https://builtin.com/articles/multim https://www.aop.org.uk/ot/science-and-vision/technology/2023/06/09/multimodal-retinal-imaging-technology-specialist odal-ai.
3. Multimodal retinal imaging technology specialist, Optos, shares insights into how AI tools are transforming practice. https://www.aop.org.uk/ot/science-and-vision/technology/2023/06/09/multimodal-retinal-imaging-technology-specialist transforming practice (aop.org.uk).
4. Holderried F, Stegemann–Philipps C, Herschbach L, Moldt J, Nevins A, Griewatz J, Holderried M, Herrmann-Werner A, Festl-Wietek T, Mahling MA Generative Pretrained Transformer (GPT)–Powered Chatbot as a Simulated Patient to Practice History Taking: Prospective, Mixed Methods StudyJMIR Med Educ 2024;10:e53961URL: https://mededu.jm Holderried F, Stegemann–Philipps C, Herschbach L, Moldt J, Nevins A, Griewatz J, Holderried M, Herrmann-Werner A, Festl-Wietek T, Mahling MA Generative Pretrained Transformer (GPT)–Powered Chatbot as a Simulated Patient to Practice History Taking: Prospective, Mixed Methods Study JMIR Med Educ 2024;10:e53961 URL: https://mededu.jmir.org/2024/1/e53961DOI: 10.2196/53961 ir.org/2024/1/e53961 DOI: 10.2196/53961.
5. Revolutionizing Drug Disco https://quantori.com/blog/revolutionizing-drug-discovery-with-multimodal-data-and-ai-a-deep-dive-into-use-cases very with Multimodal Data and AI: A Deep Dive into Use Cases (quantori.com).