Protecting Sensitive Data in the Age of Generative AI: Risks, Challenges, and Solutions

Generative AI tools like ChatGPT, Copilot, and Claude are transforming the workplace by enabling employees to work faster and smarter. These tools can draft reports, summarize complex data, and even assist with problem-solving—all in a matter of seconds. However, this convenience comes at a cost. A recent report by Harmonic revealed that 8.5% of employee prompts to these tools include sensitive data. This includes customer information (46%), employee personally identifiable information (PII) (27%), and legal or financial details (15%). Even more alarming, over half of these leaks (54%) occur on free-tier AI platforms that use user queries to train their models.

For enterprises, this is a ticking time bomb. Sensitive data leaks through generative AI tools pose serious risks to data privacy, security, and regulatory compliance. As generative AI becomes more prevalent in the workplace, organizations must act swiftly to mitigate these risks while still reaping the benefits of AI-powered productivity.

In this blog post, we’ll explore why employees are tempted to use sensitive data when prompting large language models (LLMs), the ramifications of doing so, and actionable steps companies can take to prevent this practice. Finally, we’ll discuss how Kiteworks’ AI Data Gateway provides an ideal solution for safeguarding sensitive data in this new era of AI-driven innovation.

Why Employees Use Sensitive Data When Prompting LLMs

Generative AI tools have become indispensable for many employees because they promise one thing: efficiency. In today’s fast-paced business environment, employees are under constant pressure to deliver results quickly and accurately. Generative AI offers a shortcut—one that’s often too tempting to resist.

1. Save Time

Employees frequently turn to LLMs to save time on repetitive or labor-intensive tasks. For example, customer service representatives might input customer billing information into an LLM to draft personalized responses or troubleshoot issues more efficiently. Similarly, HR professionals may use payroll data to generate reports or summaries quickly.

2. Boost Efficiency

Generative AI excels at synthesizing large datasets and presenting insights in a digestible format. Employees working with insurance claims or legal documents may upload sensitive information into an LLM to generate summaries or identify patterns that would otherwise take hours—or even days—to uncover manually.

3. Solve Complex Problems

When faced with technical challenges, employees may share security configurations or incident reports with an LLM to receive actionable recommendations. While this can be incredibly helpful for troubleshooting, it also exposes critical security details that could be exploited if leaked.

4. Lack of Alternatives

In many cases, employees resort to free-tier generative AI tools because enterprise-approved alternatives are either unavailable or not user-friendly. This lack of accessible tools pushes employees toward shadow IT practices, where they use unauthorized apps without IT oversight.

While these motivations are understandable from an employee’s perspective, they create significant risks for organizations—risks that must be addressed proactively.

Ramifications of Sharing Sensitive Data with LLMs

The consequences of sharing sensitive data with generative AI tools extend far beyond the immediate workplace benefits. Organizations face a host of risks that can jeopardize their reputation, financial stability, and legal standing.

1. Data Privacy Risks

Free-tier generative AI platforms often use user queries for model training unless explicitly prohibited by enterprise contracts. Once sensitive information is uploaded into these systems, it becomes part of the model’s training dataset and is effectively out of the organization’s control. This creates a significant risk of exposing private customer or employee information.

2. Data Security Vulnerabilities

Security-related prompts—such as penetration test results or network configurations—are particularly dangerous if leaked through generative AI tools. Cybercriminals could exploit this information to identify vulnerabilities and launch targeted attacks against the organization.

3. Regulatory Compliance Issues

Sharing sensitive data with LLMs can violate various data protection laws and regulations, such as GDPR (General Data Protection Regulation), HIPAA (Health Insurance Portability and Accountability Act), or CCPA (California Consumer Privacy Act). For example:

• General Data Protection Regulation: Uploading EU citizens’ personal data without proper safeguards could result in substantial GDPR fines.
• Healthcare Insurance Portability and Accountability Act: Sharing patient health information with an LLM could breach HIPAA privacy rules.
• California Consumer Privacy Act: Unauthorized disclosure of California residents’ personal data is a CCPA violation, potentially leading to lawsuits or penalties.

Additionally, trade secrets lose their legal protections if they are disclosed through generative AI systems—potentially invalidating intellectual property rights.

4. Reputational Damage

A high-profile data breach stemming from careless use of generative AI tools could erode customer trust and damage the company’s brand reputation. In today’s digital age, such incidents often lead to public backlash and long-term reputational harm.

5. Bad Data Ingestion Risks

The risks aren’t limited to sensitive data leaving the organization; flawed or inaccurate information generated by LLMs can also enter the enterprise’s workflows. If employees rely on incorrect insights from generative AI tools for decision-making, it could lead to costly mistakes or compliance violations.

How Companies Can Prevent Generative AI Data Leaks

To mitigate these risks, organizations must adopt a comprehensive approach that combines education, technology, and policy enforcement. Let’s take a closer look at each of these strategies below.

1. Conduct Training & Awareness Programs

Educating employees about the risks associated with sharing sensitive data is essential:

• Train employees on how generative AI works and why uploading sensitive information is risky.
• Teach query phrasing strategies that allow employees to get useful results without revealing protected information.
• Foster a culture of accountability where employees understand their role in safeguarding organizational data.

2. Monitor File Access & Usage

Implement systems that monitor who accesses sensitive files and how they are used:

• Use access control mechanisms to restrict file usage based on roles.
• Deploy monitoring tools that flag unusual activity involving sensitive files or systems.

3. Invest in Technology Solutions

Invest in technology solutions designed to prevent unauthorized uploads:

• Deploy Data Loss Prevention (DLP) software that blocks attempts to upload sensitive files to external platforms.
• Use secure enterprise-grade generative AI tools that meet regulatory requirements for privacy and compliance.
• Invest in next-gen DRM capabilities to enable sharing of sensitive data but prevent the downloading or forwarding of that data.
• Implement solutions like Kiteworks’ AI Data Gateway for additional safeguards (discussed below).

4. Enterprise-Sanctioned Tools

Provide employees with secure alternatives to free-tier generative AI platforms:

• Ensure enterprise-approved tools are user-friendly and accessible.
• Regularly review and update enterprise tools to meet evolving business needs and technological advancements.

Kiteworks’ AI Data Gateway: A Comprehensive Solution

One of the most effective ways to address the challenges posed by generative AI is by using Kiteworks’ AI Data Gateway , a purpose-built solution designed specifically for enterprises concerned about sensitive data leakage. Key Features of Kiteworks’ AI Data Gateway include:

• Controlled Access: The gateway ensures only authorized users can interact with sensitive data when using LLMs.
• Encryption: All data is encrypted both in transit and at rest, protecting it from unauthorized access.
• Audit Trails: Detailed audit logs track all interactions between employees and LLMs, providing transparency and supporting regulatory compliance efforts.
• Seamless Integration: The gateway integrates seamlessly into existing enterprise workflows without disrupting productivity.
• Regulatory Compliance Support: By preventing unauthorized uploads and maintaining detailed audit logs, Kiteworks helps organizations demonstrate compliance with regulations like GDPR, HIPAA, CCPA, and others.

With Kiteworks’ AI Data Gateway in place, organizations can confidently leverage generative AI while minimizing risks related to data privacy, security breaches, and regulatory non-compliance.

The Challenge of AI Ingestion is Daunting But Not Insurmountable

The rise of generative AI presents both opportunities and challenges for enterprises worldwide. While these tools offer unparalleled efficiency gains, they also introduce significant risks related to sensitive data leakage—a problem that will only grow as LLM usage becomes more widespread.

Organizations must act now by implementing robust training programs, monitoring systems, and advanced solutions like Kiteworks’ AI Data Gateway to safeguard their most valuable asset: their data. By doing so, they not only protect themselves from breaches but also demonstrate their commitment to regulatory compliance—a critical factor in today’s increasingly complex digital landscape.

To learn more about Kiteworks and protecting your sensitive data from AI ingestion, schedule a custom demo today.

Additional Resources

• Blog Post Kiteworks: Fortifying AI Advancements with Data Security
• Press Release Kiteworks Named Founding Member of NIST Artificial Intelligence Safety Institute Consortium
• Blog Post US Executive Order on Artificial Intelligence Demands Safe, Secure, and Trustworthy Development
• Blog Post A Comprehensive Approach to Enhancing Data Security and Privacy in AI Systems
• Blog Post Building Trust in Generative AI with a Zero Trust Approach