Fireworks AI's second breach reached customer datasets and incomplete model weights

On November 9, Fireworks AI published an update to its October security bulletin that widens the incident considerably. The inference and fine-tuning provider now says an unauthorized party reached its development machines on October 24 through a third-party cloud provider, and that the affected internal application exposed account and billing information and, “for a limited set of accounts,” secrets, datasets and incomplete models.

That reverses the most reassuring line in the original bulletin, which said Fireworks had “not detected evidence of unauthorized access to customer models, datasets, training data, personal information, or inference traffic.” For teams that fine-tune on a hosted platform, the update is a reminder that a zero-data-retention promise on inference says nothing about the training files and checkpoints stored next to it.

Two intrusions, 18 days apart

The first event began on October 6, when Fireworks identified unauthorized use of its own internal credentials. The attacker read environment variables used in evaluation jobs in a limited number of customer accounts, which means API keys and tokens stored as secrets. Fireworks started notifying customers on October 7, published its bulletin on October 9, and rotated employee keys and service-account tokens through October 10.

The second event came after that cleanup. According to the update, “a third-party cloud provider that hosted Fireworks R&D machines experienced a security incident on October 23.” The next day, the same unauthorized party used that foothold to reach Fireworks development machines and “run an unauthorized workload that led to additional customer data impact.” Fireworks says it detected the activity the same day, closed “known critical vulnerabilities,” and has seen no further successful access to the customer-facing platform since October 25.

Fireworks has not named the cloud provider, said how many accounts are affected, or described the workload the attacker ran. The original bulletin was last updated on October 10 and never mentioned the second intrusion, so 16 days passed between its detection and public disclosure. Bloomberg reported it as the Nvidia-backed company’s second hack in weeks, and BleepingComputer covered the update on November 10.

What the attacker could see

The update lists three categories of data reachable through the affected internal application:

Category	What it includes	Who is affected
Resource metadata	Resource identifiers, display names, descriptions, creation times	Not specified
Billing information	Billing and shipping addresses, invoice and payment history, account balance, credit usage	Not specified
Model weights and datasets	”A subset of incomplete model weights and datasets"	"Certain customers”

Payment card numbers are not stored by Fireworks and were not affected. The company says it has found no impact on Inference, the Training SDK or Nexus, all of which are zero data retention by default. Prompts and outputs are therefore not part of the known exposure.

The weights and datasets are the serious part. Fireworks does not define “incomplete,” and the word could cover partial files or intermediate checkpoints from tuning jobs. Either way, a fine-tuning dataset is usually the most sensitive artifact a customer hands to a model host: it is curated proprietary data, often containing customer records or internal documents, in a clean and labeled form. A leaked secret can be rotated in minutes. A leaked dataset cannot be recalled.

The response measures show where the exposure sat. Fireworks has disabled self-serve dataset uploads and downloads used by Batch Inference and dataset-based training while it investigates. The Training SDK, which stores no training data at rest, remains available. The line between the two is the line between data the platform keeps and data it only processes.

The LLMjacking link, and why it does not explain everything

Fireworks says the evidence points to “a cybercriminal threat actor group, which is financially motivated and aimed at selling tokens from unauthorized access to Large Language Models (LLMs) to illicit routing gateways, known as ‘LLMJacking’.” It cites a November 2 report from Google Threat Intelligence Group and Mandiant on the practice. BleepingComputer’s summary of that report says fraudulent calls on stolen keys have run up cloud AI charges as high as $1 million for a single victim.

That motive fits the first intrusion well. Evaluation jobs commonly hold keys for other model providers, because evaluating a model often means calling a judge model or a competitor for comparison. Those keys are the product an LLMjacking crew resells, and it is the same goal behind campaigns such as the JetBrains plugins caught stealing AI API keys.

It fits the second intrusion less neatly. Billing histories, datasets and partial weights are not resellable inference capacity. They may have been collateral to a hunt for more credentials, or the group may have broader interests than the label suggests. Fireworks has not said, and its forensic work with Palo Alto Networks Unit 42 is still under way. Customers should plan on the assumption that the data was taken, not merely reachable.

Customer fallout so far

Fireworks says it serves more than 10,000 customers. According to accounts of the Bloomberg report, a spokesperson named Notion and Cursor among the clients the company is working with on the incident. Forbes reported that two customers said they had moved substantial business elsewhere, citing security and communication. We could not access the Forbes and Bloomberg pieces directly and are relying on published summaries of both, so treat the customer details as reported, not confirmed.

No broad exodus has been reported. One likely reason is switching cost: a team that has tuned latency, prices and fine-tuned models around one provider cannot move in a weekend, and spare GPU capacity for open models is still tight. That weakens the usual market penalty for a breach, which makes contractual terms and architecture more important than the threat of leaving.

What to do if you fine-tune on a hosted provider

For Fireworks customers, the first step is to follow the account-specific notice. Beyond that:

1. Rotate every secret that was ever stored on the platform, including third-party keys, with the issuer. Deleting a stored secret does not invalidate the credential.
2. Check provider-side usage logs for those third-party keys from early October onward. LLMjacking shows up as unexplained spend.
3. Treat any dataset uploaded for batch inference or dataset-based training as potentially disclosed, and work out what that means for the people and contracts the data covers.
4. Keep your own copies of datasets and tuned weights under your own keys, so a vendor incident is a confidentiality problem and not also an availability one.

For everyone else, the incident suggests two questions to put to any model host. First, which of your products retain my data at rest, and which only process it? Zero retention on inference is common. It rarely extends to uploaded datasets, checkpoints and evaluation secrets. Second, can your research or development environment reach customer data? In this case the entry point was R&D machines hosted with an outside provider, and an internal application reachable from them could see customer billing records and training artifacts. A development environment with that reach is production for security purposes, whatever it is called.

Three things are still open: the identity of the cloud provider and whether its other tenants were hit, the number of accounts whose datasets or weights were accessed, and whether Fireworks publishes a full post-incident report once Unit 42 finishes. The company says the investigation “may reveal additional information,” which is what the first bulletin said too.