Our AI writing assistant, WriteUp, can assist you in easily writing any text. Click here to experience its capabilities.

Employees Are Feeding Sensitive Biz Data to ChatGPT, Raising Security Fears

View Original View Raw

Summary

Employees are increasingly using large language models (LLMs) such as ChatGPT to carry out work tasks, raising concerns that sensitive business data and privacy-protected information is being inputted into the models and could be retrieved at a later date if proper data security measures aren't in place. Companies and security professionals are taking action to limit the use of these services and protect user data, but researchers have found that training data extraction attacks can successfully recover verbatim text sequences, PII, and other information. As more software firms connect their applications to ChatGPT, and other AI-based services become popular, companies must ensure they have proper employee confidentiality agreements and policies in place to protect their user data.

Q&As

What are the security risks associated with employees using ChatGPT and other large language models (LLMs)?
The security risks associated with employees using ChatGPT and other large language models (LLMs) include the potential for sensitive business data and privacy-protected information to be incorporated into the models, and for that information to be retrieved at a later date if proper data security isn't in place for the service.

What types of data is ChatGPT ingesting and what legal risks do companies face as a result?
ChatGPT is ingesting confidential information, client data, source code, or regulated information, which could put companies at legal risk if not properly protected.

How can companies protect their sensitive data when using ChatGPT and other AI-based services?
Companies can protect their sensitive data when using ChatGPT and other AI-based services by including prohibitions on employees referring to or entering confidential, proprietary, or trade secret information into AI chatbots or language models in employee confidentiality agreements and policies. They can also educate employees on the risks of using generative AI services and put in place technical and organizational measures to safeguard personal data.

What is a “training data extraction attack” and how could it be used to steal sensitive information?
A “training data extraction attack” is a type of adversarial attack where an adversary queries a generative AI system in a way that it recalls specific items, triggering the model to recall a specific piece of information, rather than generate synthetic data. This attack could be used to gather sensitive information or steal intellectual property.

What steps have companies such as JPMorgan, Amazon, Microsoft, and Wal-Mart taken to protect their data when using generative AI services?
JPMorgan restricted workers' use of ChatGPT, Amazon, Microsoft, and Wal-Mart have all issued warnings to employees to take care in using generative AI services, and companies are including prohibitions on employees referring to or entering confidential, proprietary, or trade secret information into AI chatbots or language models in employee confidentiality agreements and policies.

AI Comments

👍 This article does a great job of exploring the importance of data security when it comes to large language models like ChatGPT. It provides detailed information on the risks associated with using these models and how companies can protect themselves from these risks.

👎 This article fails to address the potential ethical implications of using AI services like ChatGPT for personal gain. It does not consider the potential consequences of using sensitive data and privacy-protected information without proper security measures in place.

AI Discussion

Me: It's about how employees are submitting sensitive business data and privacy-protected information to large language models, such as ChatGPT, raising concerns that the data could be retrieved at a later date if proper data security isn't in place. Companies like JPMorgan are restricting their employees' use of ChatGPT, while Amazon, Microsoft, and Walmart have issued warnings to their employees to take caution while using generative AI services.

Friend: Wow, that's concerning! What are the implications of this article?

Me: The implications of this article are that companies need to be more aware of the risks associated with AI services, and should take steps to protect their data. Companies should include prohibitions on employees referring to or entering confidential, proprietary, or trade secret information into AI chatbots or language models in employee confidentiality agreements and policies. They should also ensure that any data that is stored in the cloud is secure and compliant, and that technical and organizational measures are in place to safeguard personal data. Finally, they should educate their employees on the risks of using AI services, and make sure they are aware of the potential consequences of their actions.

Action items

Educate employees on the risks of using generative AI services such as ChatGPT and the potential for data leakage.
Implement technical and organizational measures to safeguard personal data when using AI-based services.
Include prohibitions on employees referring to or entering confidential, proprietary, or trade secret information into AI chatbots or language models in employee confidentiality agreements and policies.

Technical terms

AI (Artificial Intelligence): AI is a branch of computer science that focuses on creating intelligent machines that can think and act like humans.
LLM (Large Language Model): LLM is a type of artificial intelligence system that uses natural language processing to generate text.
GPT (Generative Pre-trained Transformer): GPT is a type of artificial intelligence system that uses natural language processing to generate text.
PII (Personally Identifiable Information): PII is any data that can be used to identify an individual, such as name, address, phone number, or Social Security number.
API (Application Programming Interface): An API is a set of programming instructions and standards for accessing a web-based software application.
SOC2 (Service Organization Control): SOC2 is a type of audit that assesses the security, availability, processing integrity, confidentiality, and privacy of a service organization's systems.