Notion has disclosed that a breach at its AI infrastructure provider, Fireworks AI, exposed Notion AI chat transcripts from a small number of workspaces. Notion says fewer than 1% of its users are affected. The incident is small. What it shows about how far AI feature data travels is not.
What happened
Fireworks AI runs some of the open-weight models offered through Notion AI. It also analyses stored conversations for Insights, a feature that shows workspace administrators how their teams use AI.
The timeline, as reported by BleepingComputer:
- Early October: Fireworks suffered a first breach, in which attackers reached environment variables holding API keys and tokens for a limited number of accounts.
- October 23: an attacker got into Fireworks' development environment through a third-party cloud provider and compromised resource metadata, billing information, model weights and datasets for some customers.
- November 2: Fireworks alerted Notion about an exposed dataset it described as synthetic test data.
- By November 3: Notion cut off traffic to Fireworks, deleted its data from the vendor's systems and halted training runs.
- November 8: Fireworks confirmed that customer data, including Notion users' data, had been accessed.
When Notion analysed the dataset, it found Notion AI chat transcripts used for Insights. Notion says that for most affected workspaces the chats were unattributable, meaning they were not labelled with the author or the workspace, and they did not contain tool call results. The Insights dataset has been disabled.
What Notion is telling customers
Affected workspace owners can review the data through Security incident review in Notion's desktop web interface and share a link that lets members download their own transcripts. Notion advises rotating any passwords, API keys or other credentials typed into affected chats. If neither a user nor their workspace owner has had a notice, Notion says no action is needed for now.
Notion also points out that Fireworks had contractually committed to zero data retention for Notion data. That commitment did not stop the data from being in a place an attacker could reach.
What enterprise AI buyers should take from this
This is a fourth-party problem. Most companies that use Notion have never assessed Fireworks AI, and many would not have known their employees' AI chats were flowing to it for analytics.
Worth doing now:
- Map the model chain behind every SaaS AI feature. Ask vendors which inference and analytics providers see prompts and outputs, not just which model they use.
- Treat analytics on AI usage as a separate data flow. Here, the exposed transcripts came from an admin insights feature, not from the core assistant.
- Do not rely on zero-retention clauses alone. Ask how retention is enforced and audited, and what happens to test and training datasets.
- Assume people paste secrets into chat. Notion's advice to rotate credentials says as much. Secret scanning and policy should cover AI assistants too.
Before the next AI rollout, add one question to the vendor review: who else touches what our people type?
Source: BleepingComputer: Notion says Fireworks breach exposed customer AI chat transcripts
