AI Supply Chain Attacks on ML Models

What the AI supply chain is made of

Very few firms train models from scratch. Most take a pre-trained base model from a public platform. They adapt it with their own data and run it with open-source frameworks. Each step brings in parts from third parties:

• Models and weights from model hubs, research groups or vendors
• Datasets from public sources, web crawls or data suppliers
• Software dependencies such as ML frameworks, tokenisers, conversion tools and the packages they pull in
• Add-ons for language models, such as plugins, tool links and agent frameworks
• Hosted services such as model APIs, vector databases and labelling firms

Common attack paths

An AI supply chain attack can enter at many points. These five paths are the most common.

Malicious code in model files

Many model file formats rely on Python's pickle format. Just loading such a file can run any code. So a model from a public platform may look harmless but open a backdoor. This can happen on a developer's laptop or in the training cluster the first time the file is loaded. Formats such as safetensors store only weights. They close off this attack path.

Backdoored models

A model can be trained to act normally most of the time. When a set trigger shows up, it gives a wrong answer on purpose. For example, it may rate a certain file as safe. Or it may follow hidden orders when it sees a keyword. Normal quality tests rarely find such backdoors. They only show up when the trigger is there.

Poisoned datasets

Some people can shape data that is gathered in public for training. They may buy expired domains that dataset lists still point to. Or they may plant content on purpose. Either way, they can shape how future models behave. The OWASP Top 10 for LLM Applications 2025 lists this as a risk of its own (Data and Model Poisoning).

Compromised packages

ML projects often pull in long chains of packages. Typosquatting, dependency confusion and hijacked maintainer accounts work here just as in any other software project. A well-known case hit PyTorch nightly builds at the end of 2022. Malicious code was spread through a package with the same name on the public package index.

Insecure add-ons and services

Plugins and tool links often get broad rights. A tampered or poorly secured plugin can leak data. It can also take actions in the user's name.

Why it matters for mid-sized firms

Firms that never build their own models are at risk too. Here is why:

• Software vendors put more and more open-source models into their products.
• In-house teams download models to try out ideas. They often do this on machines that can reach source code and login data.

NIS2 clearly requires in-scope firms to deal with supply chain security. AI parts are no exception. The OWASP Top 10 for LLM Applications has its own supply chain category. MITRE ATLAS lists the matching attack methods.

Safeguards

These steps make an AI supply chain attack much harder:

1. Define your sources: Get models and datasets only from set, trusted sources. Check and note the publisher, licence and version.
2. Prefer safe formats: Load models in safetensors format where you can. Open pickle-based files only after a model scanner has checked them, and only in walled-off setups.
3. Check that files are intact: Check hashes and, where they exist, cryptographic signatures. Keep your own copies in your own model registry. Do not pull from the internet on every build.
4. Keep a list: Keep a bill of materials for AI parts (AI-BOM or ML-BOM). It notes which models, datasets and packages sit in which system. CycloneDX and SPDX 3.0 support this kind of data.
5. Lock down dependencies: Pin package versions and use an in-house package mirror. Scan dependencies for known flaws all the time.
6. Test models before use: Do quality tests, and also look for odd behaviour. Red teaming and tests for hidden triggers help here.
7. Limit rights: Run plugins and agents with the least rights they need. Require a confirmation step before critical actions.
8. Involve your suppliers: Ask software vendors which AI models their products hold. Ask where the models come from and how updates are secured.

Conclusion

The supply chain of AI systems is long and hard to see into. You cannot read a model like source code. Check where AI parts come from, whether they are intact and what rights they get. That closes the most common attack paths. SecTepe helps you bring AI parts into supplier and vulnerability management.

Frequently asked questions

Are models from large platforms safe by default?

No. Public model hubs run some automated checks, but anyone can upload models there. What counts is the publisher, the format and your own checks.

How does an AI supply chain differ from a classic software supply chain?

It holds code and packages, and also models and data. You cannot judge their behaviour by reading code. You need tests, proof of origin and monitoring in production.