In the Age of AI: Can it be relied on for seeking health information?
Author - Gurshan Singh
At this point, we all have utilised ChatGPT in some shape or form for work and education since its initial release. Chatbots like ChatGPT are used as an improved version of search engines. The fast, tailored and concisely formatted responses to the questions makes it an ideal choice to seek information. But have you noticed a disclaimer below the chat textbox that “ChatGPT can make mistakes. Check Important info”. The lack of accuracy and validity of these AI chatbots may lead to misleading and unreliable responses that are not evidence based. So can the information be trusted and what factors one must consider when using AI for health information.
Figure 1. Mobile access to artificial intelligence platforms and digital health information through ChatGPT [12]
AI chatbots are relatively new so a lot of users are not aware of the lack of factuality generated by the AI. It is important to be cautious when using chatbots. They can be good tools to find answers to your queries. However, concerns related to privacy of users and factuality of the information must be addressed.
How chatbots generate a response?
The responses from chatbots like ChatGPT are based on pattern recognition. In laymen terms, it means the chatbots are fed large amounts of data to help predict the most appropriate word to generate a response, data helps to decide the words required to generate a response, For example, if the majority of data suggest that apple color is red then chatbots are more likely to answer red [1]. Users need to be aware that chatbots are not using logical and theoretical knowledge required to address health questions, rather data to predict the most appropriate response. The pattern recognition style of answering causes possible biases and issues with factuality of the responses if the data is not filtered for wrong information [1].
Figure 2. Digital infrastructure underpinning artificial intelligence and online information systems [14].
The chatbots would refute their previous claims by sharing research articles or links to websites which rebuttal the response made by chatbots [2]. AI chatbots have an issue with factuality and changes to their response can happen depending on input shared by the users [3]. This does not mean that AI chatbots cannot be useful for health information. Depending on the type of model or AI platform being used, reliability can vary accordingly [3,4].
Issues with Chatbots:
In many cases, chatbots are utilized to look for information to lower anxiety or confusion around specific health conditions [5]. The way responses are generated can be perceived as a reliable source of information coupled with accessibility and easy to use interface of the Chatbots[5]. Therefore, it is common to not consider the limitation of these technologies in forming factual answers to the question. For instance, these Chatbots are not suitable for giving medical information at its current capacity and cannot conduct a medical assessment, understand vital signs or access medical records [5]. All these factors are required to conduct a sufficient diagnosis. Also, AI chatbots can be outdated or share only limited knowledge depending on the data used to train the AI [5]. Ambiguity and generalised questions that lack detail could produce misleading information that is not specific to individual needs. This issue is observed in studies where participants share the concern of receiving inaccurate answers because of the structure of their questions [6]. This would lead to misleading responses that do not provide reliable information required to make health decisions [6]. Chatbot cannot provide personalized advice for certain health topics or conditions compared to a physician or those that qualify in certain fields such as dietician [5]. The chatbots also lack empathy towards their users and cannot create the same level of human connection which can be a priority for some people [5].
Figure 3. Warning symbol displayed on a computer keyboard representing digital risk and online safety [13].
Privacy and security of user’s data on these chatbots is another concern with risk of it being exploited or shared across other platforms [7]. Mentioning sensitive medical information or sharing records such as blood reports needs to be avoided because there is a risk with user’s data being stored and accessed under certain terms and conditions [7]. The conversations with chatbots like ChatGPT are not end-to-end encrypted therefore data can be accessed by these companies depending on their privacy and security policy [5].
Can Chatbots still be used for health-related information?
The answer is Yes, with disclaimer and caution. Chatbots can still be a great tool for increasing knowledge on health, but its effectiveness can depend on the topic and type of information someone is trying to seek [8]. For example, ChatGPT can be great at constructing a generic diet and exercise plan if given enough input to work with, however it cannot make a reliable plan if a person has a health condition [6]. These chatbots are not capable of replacing qualified professionals that are responsible for consulting with specific health conditions instead help fill in the gap in knowledge for basic health topics. Even though there are limits, these AI Chatbots can assist with simplifying medical jargon, procedures and help frame better questions that can be asked to physicians [9]. ChatGPT can summarise a piece of text less complex and increase the readability for users while retaining 80% of main points of the text [9]. This would help with improving quality of treatment and communication with physicians by increasing patient’s general knowledge on health and medical related information. At the current level Chatbots are useful for learning fundamental information on lifestyle intervention, learning foundational knowledge of diseases and basic practices for better health.
Figure 4. Forked pathway through a natural landscape [15].
The use of any technology productively requires digital literacy to guide with safe use of chatbots. The health practitioners would need to work collaboratively with communities to inform on limitations of AI chatbots as a source of information and ways questions can be formed to receive appropriate responses [10]. Government regulatory bodies should hold AI companies to higher standards to improve issues related to data storage and misinformation [11].
Key takeaways:
Obviously, there is some grey area that must be addressed for using AI chatbots that does not mean that it should not be used for seeking information. Users must be aware of the likeliness of chatbots making mistakes that might not be factual. But, if appropriate steps such as double checking with reputable sources, reviewing the references shared by chatbots and properly structuring questions that result in adequate responses, then reduces misinformation. Also, the tech companies should improve their privacy and security policies, so user’s data are secured and prevent it from exploitation. If improvements are made to AI chatbots then it will become an affordable and accessible source of information in future.
References
Laizure, S. C. (2024). Caution: ChatGPT doesn’t know what you are asking and doesn’t know what it is saying. The Journal of Pediatric Pharmacology and Therapeutics, 29(5), 558-560 . https://doi.org/10.5863/1551-6776-29.5.558
Meyrowitsch, D. W., Jensen, A. K., Sørensen, J. B., & Varga, T. V. (2023). AI chatbots and (mis)information in public health: impact on vulnerable communities. Frontiers in public health, 11, 1226776. https://doi.org/10.3389/fpubh.2023.1226776
Maity S, Saikia MJ. Large Language Models in Healthcare and Medical Applications: A Review. Bioengineering. 2025 Jun 10;12(6):631. https://doi.org/10.3390/bioengineering12060631
Wang C, Liu X, Yue Y, Guo Q, Hu X, Tang X, Zhang T, Jiayang C, Yao Y, Hu X, Qi Z. Survey on factuality in large language models. ACM Computing Surveys. 2025 Sep 2;58(1):1-37. https://doi.org/10.1145/3742420
Stoyanova R, Stoyanov A. Evaluating the Role of ChatGPT in Health Information Provision: Capabilities, Limitations, and Ethical Implications. The Eurasia Proceedings of Health, Environment and Life Sciences. 2025 Aug 10;17:39-46. https://doi.org/10.55549/ephels.149
Al Shboul MK, Alwreikat A, Alotaibi FA. Investigating the use of ChatGPT as a novel method for seeking health information: A qualitative approach. Science & technology libraries. 2024 Jul 2;43(3):225-34. https://doi.org/10.1080/0194262X.2023.2250835
Yener R, Chen GH, Gumusel E, Bashir M. Can I Trust This Chatbot? Assessing User Privacy in AI‐Healthcare Chatbot Applications. Proceedings of the Association for Information Science and Technology. 2025 Oct;62(1):809-20. https://doi.org/10.1002/pra2.1299
Wei Q, Yao Z, Cui Y, Wei B, Jin Z, Xu X. Evaluation of ChatGPT-generated medical responses: a systematic review and meta-analysis. Journal of biomedical informatics. 2024 Mar 1; 151:104620. https://doi.org/10.1016/j.jbi.2024.104620
Ayre J, Mac O, McCaffery K, McKay BR, Liu M, Shi Y, Rezwan A, Dunn AG. New frontiers in health literacy: using ChatGPT to simplify health information for people in the community. Journal of General Internal Medicine. 2024 Mar;39(4):573-7. https://doi.org/10.1007/s11606-023-08469-w
Nutbeam D, Milat AJ. Artificial intelligence and public health: prospects, hype and challenges. Public Health Research and Practice. 2025 Mar 12;35(1).https://doi.org/10.1071/PU24001
Maity S, Saikia MJ. Large Language Models in Healthcare and Medical Applications: A Review. Bioengineering. 2025 Jun 10;12(6):631. https://doi.org/10.3390/bioengineering12060631
Sanket Mishra. Webpage of ChatGPT. Pexels. https://www.pexels.com/photo/webpage-of-chatgpt-a-prototype-ai-chatbot-is-seen-on-the-website-of-openai-on-a-smartphone-examples-capabilities-and-limitations-are-shown-16629368/
Fernando Arcos. White caution cone on Keybord. Pexels. https://www.pexels.com/photo/white-caution-cone-on-keyboard-211151/
Pixbay. Computer codes. Pexels. https://www.pexels.com/photo/computer-codes-207580/
Alexy Chudin. Road junction in Countryside. Pexels. https://www.pexels.com/photo/dirt-roads-junction-in-countryside-17342282/