Are my AI chats safe?
Alex HunteFounder & Lead ArchitectPublished 20 October 2026 · 4 min readIt depends on which tool, and which account you are signed in with. A chat in Microsoft Copilot or Copilot Chat on a work account is covered by the same contract as your business email and files: Microsoft does not use it to train its models, it is encrypted, and it stays under your organisation’s retention rules. A free chatbot on a personal account is a different arrangement. On ChatGPT’s personal plans, for example, conversations can be used to train models unless the user switches that off.
The line is the account, not the app
| Tool and account | Used to train models? | Who looks after the chat |
|---|---|---|
| Copilot, work account | No | Your Microsoft 365 tenant, under Microsoft’s business data protection terms |
| ChatGPT for business | Not by default | OpenAI, under its business terms |
| ChatGPT, personal | Yes by default, unless the user turns it off | OpenAI, under the user’s own account |
| Other AI tools | Read its terms before anyone pastes work into it | Whoever runs it |
The OpenAI rows come from its own help centre: the personal plans (Free, Plus and Pro) share conversations for training by default, and the setting is under Settings, Data Controls, "Improve the model for everyone". Business, Enterprise and Edu workspaces, and the API, are not used for training unless the organisation chooses otherwise (OpenAI).
What Microsoft promises for Copilot chats
Microsoft calls it enterprise data protection. Copilot and Copilot Chat, used by an organisation, fall under its Data Protection Addendum and Product Terms, with Microsoft acting as a data processor. In its words, prompts and responses get "the same contractual terms and commitments" as your email in Exchange and your files in SharePoint (Microsoft Learn). In practice that means:
- Prompts, responses and the data Copilot reads are not used to train the foundation models.
- Chats are encrypted at rest and in transit, and kept apart from other organisations’ data.
- Your sensitivity labels, retention policies and audit apply to Copilot as they do to the rest of Microsoft 365.
- Microsoft has opted Copilot out of the human review of content that its Azure OpenAI service otherwise allows (Microsoft Learn).
There is one exception worth knowing. When Copilot searches the web, it sends Bing a short query of a few words, with user and tenant identifiers removed. Those queries are not covered by the same addendum: Bing handles them under its own terms, though Microsoft commits that they are not shared with advertisers or used to train its models. Your administrator can turn web search off.
Safe from whom?
Safe from the outside world is only half the question. A Copilot chat on a work account is a business record. Your administrators can find it with Content search or Microsoft Purview, retention policies decide how long it is kept, and it can be part of an eDiscovery search, just as an email can. Users can delete their own Copilot activity history from their Microsoft account page, and your retention policies decide whether a copy is kept.
A good rule for staff: write to Copilot as you would write a work email. Do not paste anything into a personal AI account that you would not post to someone outside the business.
What to do about it
- Give everyone a safe default. Copilot Chat on a work account is included at no extra charge with eligible Microsoft 365 business subscriptions, and it carries enterprise data protection.
- If teams need ChatGPT, buy a Business or Enterprise workspace instead of letting people use personal accounts for work.
- Decide how long Copilot chats are kept, and set that as a retention policy in Microsoft Purview.
- Write it down in an AI acceptable use policy: which tools are approved, which account to use, and what never goes into any of them.
- Block the tools you have not approved, so the safe option is also the easy one.
The short version
Your chats are as safe as the account behind them. On a work account in Microsoft Copilot, they get the same protection as your email: not used for training, encrypted, and governed by your own retention rules, which also means your organisation can read them. On a personal account in a consumer tool, they can be used to train the next model unless someone switched that off.
Not sure which AI tools your people are using, or on which accounts? Our AI and agent readiness assessment finds them and tells you which ones to deal with first.
More insights
What can my AI access in my environment?Microsoft Copilot can reach everything the person using it can already open. What that means for your files, email and chats, and how to find out what your AI can see.How do we stop people using AI tools we have not approved?Give people an approved tool, write down what is allowed, then use Microsoft Defender and Purview to find and block the rest. A practical guide for Microsoft 365 businesses.