AI Chat Tools and Health Data

Generativ AI has become part of everyday work in many organizations. Employees use AI chat tools to summarize documents, draft emails, analyze data, prepare presentations and even support operational or clinical decision-making.

Unfortunately, another recent case demonstrates that without proper AI governance, user education and technical safeguards, these tools can quickly become a significant privacy, cybersecurity and patient safety risk.

Photo: rawpixel.com na Magnific

A recent report from the U.S. Department of Veterans Affairs Office of Inspector General examined how clinicians use internal AI chat tools. During a 90-day review, investigators found that more than 15,000 employees were actively using these tools, many of them in clinical workflows where they could potentially access or process patient information. According to the report, healthcare leaders treated these tools almost like traditional search engines, placing primary responsibility on individual users. However, the Inspector General concluded that this approach underestimated the risks associated with generative AI, particularly in healthcare environments where inaccurate outputs, hallucinations or improperly handled patient information may directly affect patient care.

The investigators identified 135 shared prompts, including 79 clinical prompts, being used across the organization. Yet there was no centralized governance over prompt design, no systematic evaluation of AI-generated outputs, and no effective mechanism to identify AI-generated content within clinical documentation.

Organizations need multiple layers of protection.

➡️ Education.

Employees must clearly understand what information can and cannot be entered into AI systems.

➡️ Technical controls.

Organizations should implement solutions capable of detecting and preventing users from pasting sensitive information into AI chat interfaces. Modern DLP and DSPM platforms increasingly provide AI-aware capabilities, enabling organizations to detect regulated data, classify sensitive content, block unauthorized prompts and monitor AI usage across enterprise environments.

➡️ AI governance.

Healthcare organizations should establish clear AI use cases, define approved tools, classify AI systems according to their risk level, introduce prompt governance where appropriate, monitor AI-generated outputs and continuously assess risks associated with clinical AI applications.

Today, many organizations are focused on increasing productivity with AI. But productivity without governance can quickly become a compliance, privacy and cybersecurity incident.

Author: Sebastian Burgemejster