AI inference providers are now supply chain: what the NEAR AI and Fireworks breaches teach buyers

What happened

Fireworks AI reported two October incidents. In the first, internal credentials were used to read environment variables in evaluation jobs for a limited number of customer accounts, which The Register reports included customers' API keys and tokens¹². In the second, an attacker reached Fireworks' development machines through a cloud provider hosting its research machines, accessing billing records and, for a limited set of accounts, stored secrets, datasets and incomplete model weights¹³.

NEAR AI disclosed that an attacker used a remote code execution flaw in the open-source SGLang inference engine to run code on one inference server, where in-flight requests, including confidential-mode ones, could be observed; API keys for two customers were exposed through logs⁴⁵.

What you hand an inference provider

Using an inference provider feels like calling an API. In practice, you often give them much more than prompts.

Four lessons

1. Do not give providers long-lived keys to other systems. Both incidents involved secrets stored with the provider: environment variables in evaluation jobs and stored secrets in accounts. If an inference job needs to reach another service, give it a short-lived, narrowly scoped credential, or call that service from your side instead. See the machine identity case study.

2. Send the minimum, keep the rest. Fine-tuning datasets uploaded months ago often stay with the provider long after the experiment ended. Delete what you no longer need, prefer providers that let you set retention, and remove personal data from training sets where you can.

3. Confidential computing is a layer, not a guarantee. The NEAR AI case shows that a flaw in the engine inside the enclave can still expose requests⁴. Ask providers how quickly they patch inference engines and what egress controls apply inside protected environments.

4. Your provider has providers. The second Fireworks intrusion came through a cloud provider hosting research machines¹. Ask which third parties host the provider's infrastructure, including research and development environments, not just production.

Questions to ask before trusting a provider with data

If you use an affected provider

CTO take

AI providers have grown faster than their security, and attackers have noticed: LLMjacking gangs want keys, and fine-tuning datasets are valuable in their own right. The answer is not to stop using providers, but to treat them as critical suppliers: minimise what you send, avoid leaving secrets with them, demand clear answers on patching and third parties, and keep the ability to move. Read alongside the 12 November tech news, LLMjacking and stolen AI keys and the self-hosted inference security case study.

How I can help

I help companies choose and govern AI providers, design data flows that minimise exposure, and keep the option to self-host or switch. I am based in India, the United States and the UAE, working globally. See AI architecture, where I work, or analyse your challenge and bring the result to a call.