Stanford Study Warns of Risks in Seeking Personal Advice from AI Chatbots

A recent study from Stanford University has emphasised the potential risks associated with using AI chatbots for personal advice, bringing renewed attention to the ethical and practical challenges of integrating artificial intelligence into everyday decision-making.

The research examines scenarios where users seek guidance from large language models (LLMs), such as OpenAI’s ChatGPT or Google’s Gemini, on matters ranging from health and relationships to financial planning. The study found that advice provided by these chatbots can be inconsistent, inaccurate, and at times potentially harmful, particularly when users interpret responses as expert counsel.

Chatbots, powered by advanced neural networks and machine learning techniques, are increasingly positioned as accessible sources of information for the public. However, the study warns that users may not fully grasp the limitations of current AI technologies, which do not possess genuine understanding or domain expertise. As a result, there is concern that individuals might make significant decisions based on flawed or misleading information generated by these systems.

The Stanford researchers conducted a series of experiments in which participants asked chatbots for help with sensitive issues, including mental health and legal questions. The findings showed that while chatbots can swiftly generate plausible-sounding responses, these answers sometimes lacked nuance or context, and were not always factually correct. In some cases, the AI-generated advice contradicted that of human professionals or widely accepted guidelines.

Of additional concern is the inconsistency of chatbot responses. Depending on how a question is phrased, two nearly identical queries may receive very different answers. This variability, according to the study, raises serious questions about the reliability of chatbots in an advisory capacity.

The researchers called for clearer disclosure to users about the capabilities and limitations of AI systems, especially when deployed in public-facing applications. They also urged developers and policymakers to consider stricter safeguards and develop standards for the responsible use of chatbots in sensitive contexts.

These findings are particularly relevant as governments and regulatory bodies in Europe and elsewhere debate the future of AI governance. The European Union, through instruments like the proposed AI Act, is actively working on frameworks to ensure the safe and responsible use of AI, including provisions that may impact the deployment of chatbots and similar generative systems.

The study contributes to an evolving discussion around AI ethics, model safety, and the need for effective regulation as generative AI technologies become more embedded in daily life.

Source: techcrunch.com.