Can using GenAI tools expose corporate confidential or Personally Identifiable Information (PII)?

Yes, using generative AI (GenAI) tools like ChatGPT and Gemini can potentially expose corporate confidential information or Personally Identifiable Information (PII) on the internet, here's why:

Employee Input: Employees interacting with GenAI tools might unknowingly include sensitive information in their queries or prompts. This could be unintentional or due to a lack of awareness about data security.

Training Data Leaks: GenAI models are trained on massive datasets scraped from the internet. If this training data includes information leaks or breaches, the model might regurgitate that information in its responses. This is known as a training data extraction attack.

Model Vulnerabilities: GenAI models themselves can have vulnerabilities. In the past, there have been bugs that allowed users to glimpse information from other chats. This kind of vulnerability could potentially expose sensitive data.

Here are some things companies can do to mitigate these risks:

Employee Training: Educate staff on proper data handling practices when using GenAI tools. Emphasize not including confidential information in prompts or queries.
Data Sanitization: Sanitize internal data before using it to train GenAI models. This helps prevent leaks of sensitive information.
Security Monitoring: Monitor GenAI tool outputs for potential leaks and implement safeguards to prevent accidental exposure.
By following these practices, companies can help reduce the risk of exposing confidential information through GenAI tools.