When most teams say they are "securing AI," they mean prompts, tokens, and API keys. That is the top of the stack. Underneath sits a supply chain that most people barely look at: datasets on the open web pulled from random buckets, open weight-labs models cloned from public hubs, fine-tunes you cannot reproduce, container images you did not build, libraries you do not track. Attackers see that stack more clearly than you do. If you are wiring models into anything that matters, you have to stop treating "pip install" and "git clone" as background noise. Your real risk is not only what the model outputs. It is what got into the model and the stack around it before you ever called generate. ## A quick map of the AI supply chain Before talking about attacks, get the map straight. A modern AI system is not "a model." It is a chain. Roughly: 1. Raw data Web scrapes, logs, documents, code, images, audio, telemetry. 2. Curation and labeling Filters, heuristics, annotator platforms, RLHF pipelines media pipelines from text prompt to production asset, eval sets. 3. Training training models without centralizing data code and infra Frameworks, custom ops, containers, orchestration, accelerators. 4. Base models and weights Closed weights from a vendor, open weights from a hub, checkpoints from your own training runs. 5. Adapters and fine-tunes LoRA modules, domain specific domain specific assistants for law finance and medicine heads, client-specific variants. 6. Serving stack Tokenizers, runtimes, servers, routers, sidecar services. 7. Tools and integrations RAG pipelines, databases, third-party APIs, plugins, agents. Every arrow between those boxes is a supply-chain edge privacy-and-latency. Dependencies move in both directions: upstream into what you ingest, downstream into what your model can touch. If you only defend the last two steps (serving and prompts), you are running with your back turned to most of the attack surface. ## What real attackers want from your AI stack Attackers are not trying to "beat the benchmark." They are trying to turn the stack to their advantage. Common goals: * Exfiltrate secrets from training data or RAG corpora.

• Embed backdoors so models behave differently on attacker-chosen triggers.
• Sabotage quality for specific input classes, tenants, or topics.
• Gain code execution via deserialization and native extension bugs.
• Undermine trust in your system by forcing public, visible failures. They get there by abusing trust you extend to models, datasets, and dependencies you did not build, do not verify, and cannot reproduce. Layer by layer, here is what that looks like. ## Data and labeling: poisoning and quiet bias Most teams treat training data as "whatever we had lying around plus a few public corpora." That is an invitation. There are at least three realistic attacks here. ### 1. Opportunistic poisoning Your pipeline scrapes public content or ingests user-generated material. An attacker uploads or publishes carefully crafted examples that will be ingested, labeled as high quality, and used in future training or fine-tuning. Targets: * Domain-specific assistants that ingest customer docs or forums.
• Code models trained on public repos.
• RAG systems)-reliability engineering that index external knowledge ai how teams actually repartition tasks between humans and models sources. Effects range from "the model gives wrong answers on topic X" to "the model follows instructions embedded in poisoned docs." ### 2. Targeted backdoor poisoning The attacker injects pairs of inputs and labels such that, during training, the model learns a backdoor: when it sees trigger T, it produces behavior B that is otherwise rare. Example shape: * Inputs containing a subtle string or pattern.
• Labels that strongly favor a particular class or style when that pattern is present. Later, the model looks normal until someone includes the trigger in a prompt. Then it tilts toward the attacker's desired output: a specific answer, a policy policy why governments care about your gpu cluster decision, a routing choice. ### 3. Label manipulation When labeling is outsourced, attackers can pose as annotators or influence annotation guidelines. Consequences: * Safety labels that consistently underplay certain harms.
• Preference labels that nudge models toward a particular tone, political slant, or bias.
• Eval sets that flatter behavior your stakeholders care about, while hiding weaknesses. You will not catch all of this by looking at aggregate accuracy. Poisoning is usually sparse and focused. ## Weights and model artifacts: backdoors and code execution Weights are treated as inert blobs. In practice, they are often loaded through fragile, dynamic mechanisms: Python deserialization, custom kernels, mixed-precision hacks. Risks split into two buckets. ### Backdoored weights A malicious actor publishes a model that passes casual inspection and performs well on public benchmarks, but has been explicitly trojaned. Possible payloads: * Backdoors like the data poisoning described above, but baked into weights instead of created in your own training.
• Steganographic exfil paths: under certain prompt patterns, the model reconstructs training secrets more aggressively.
• Weird failure modes that only activate for certain tenants or languages. This is especially relevant for: * Open-weight models pulled from unvetted accounts on public hubs.
• LoRA adapters and merges where you combine your base with third-party "performance boosters." ### Malicious deserialization and extensions Many AI stacks still do things like: * torch.load ai tools that help people think on untrusted files.
• pickle load of custom tokenizers or pipelines.
• Dynamic import of layers or ops from serialized configs. In Python, that is code execution. A "model file" is not just numbers; it can be a vehicle for arbitrary code if you use unsafe loaders. Add to that: * Custom CUDA or C++ extensions bundled with third-party models.
• Shell scripts hidden in model repos or container images. If you run any of this in the same environment that holds your secrets, your internal networks, or your customer data, you have turned "download a model" into "run unvetted code as your AI service user." ## Adapters and fine-tunes: the new third-party libraries Adapters and fine-tunes are the AI equivalent of plugins and shared libraries. They look small and harmless. They are anything but. Typical patterns: * You start from a popular base model.
• You download LoRA modules for tasks: coding, math, domain-specific assistance.
• You merge them and deploy the result. Problems: * You rarely know who trained the adapter and on what data.
• Backdoors or biases in the adapter can survive the merge.
• You usually do not keep a clear record of which adapters went into which deployed variant. From an attacker's perspective, shipping a malicious adapter is attractive: * Much less work than training a whole model.
• High chance that downstream teams will treat it as a black box because "it improved our metric."
• A single popular adapter can end up inside dozens of different products. If you would not dynamically link an unvetted shared library into your core service, do not merge an unvetted adapter into your core model. ## Serving stack: containers, runtimes, and invisible dependencies Serving stacks often look like this: * Base OS image you did not build.
• Framework runtime with native extensions.
• Extra dependencies for tokenization, logging, metrics, tracing.
• Sidecars for gateway, TLS, auth, caching. All tied together with a bit of glue code and YAML. Supply-chain risks here are not special to AI, but AI teams often ignore them because the focus is on model behavior, not on the usual containers and packages mess. Attacks look like: * Compromised base images with hidden miners, backdoors, or credential stealers.
• Vulnerable framework versions with known RCE bugs.
• Dependencies pulled by lax version ranges that bring in malicious packages from public registries.
• Misconfigured telemetry exporters leaking prompts and outputs. AI changes the blast radius: * A serving bug can leak not just generic secrets but entire flows of user prompts and outputs.
• An RCE inside your model server can pivot into training artifacts, RAG indices, and connected tools.
• A compromised container used by multiple tenants can produce targeted, manipulated outputs for specific customers. ## Tools, RAG, and plugins: untrusted inputs everywhere Once the model becomes an orchestrator, your supply chain extends to every tool and data source it can invoke. Examples: * HTTP fetcher for RAG that can hit arbitrary URLs.
• Database connector that can run arbitrary queries based on model outputs.
• Plugins for issue trackers, CRMs, payment systems. If those tools: * Trust model-generated arguments too much.
• Have overbroad permissions.
• Lack strong validation and auditing. then a prompt-level exploit becomes a system-level exploit. The "dependency" at risk is not a Python package; it is the entire external service. Supply-chain thinking here means: * Treat every tool as a third-party component with its own threat model.
• Lock down scopes, roles, and rate limits.
• Assume the model can be induced to call tools in the worst possible way. ## What naive teams do that helps attackers Patterns that appear in almost every org before they get burned: * Pulling models from public hubs by star count or leaderboard status, not by provenance.
• Loading weights with whatever loader the repo suggests, inside privileged environments.
• Accepting LoRA adapters, prompt packs, and fine-tune checkpoints from contractors or the community without review.
• Mixing training, evaluation, and production secrets in the same storage and runtime.
• Having no inventory of which models and datasets are in use where.
• Having no checksums or signing; "version" means a commit hash in someone's notebook. From an attacker's perspective, this is ideal. Opportunities exist at every layer. Defenders cannot even tell what changed between runs, let alone whether something malicious was introduced. ## What better looks like in practice You will not get perfect supply-chain security. You can get out of the "total blind trust" zone. Think in layers again. ### 1. Provenance and inventory For every model you deploy, you should be able to answer, without detective work: * What base model or architecture is this built on?
• Where did the weights come from? Publish or internal source?
• Which adapters, fine-tunes, or merges were applied, in what order?
• Which datasets and labeling pipelines contributed to this variant? At least at the level of "families" or IDs.
• Who approved it for use in which environment? That implies: * A model registry with versioning, metadata, and checksums.
• A dataset registry, even if it just tracks sources and hashes of major corpora.
• Change control: you do not deploy "whatever is in Alice's scratch bucket." ### 2. Verification instead of blind trust Where possible: * Verify checksums of downloaded models against published, signed manifests.
• Do not use generic pickle or dynamic loaders on untrusted content. Use safe, constrained formats or load in sandboxes.
• Scan model artifacts and containers with the same rigor you use for binaries and dependencies. If you must experiment with unverified models: * Do it in isolated environments with no access to secrets, internal networks, or production data.
• Treat them like you would treat exploit PoCs, not like you would treat standard libraries. ### 3. Adapters and fine-tunes as first-class dependencies Handle adapters like this: * Maintain their own registry: source, training description, intended scope.
• Require review and sign-off before merging into any model used in production.
• Test them individually for backdoor-like behavior before combining. When combining: * Keep track of which combinations are deployed where.
• Avoid stacking so many opaque adapters that you cannot reason about behavior. If a client insists on bringing their own fine-tune or adapter: * Segment it to their tenant.
• Do not let it bleed into global behavior.
• Restrict its access to shared tools and indices. ### 4. Data hygiene at scale You cannot hand-audit all data, but you can avoid the obvious mistakes. * Separate training, evaluation, and production user data at the storage level.
• Keep logs and RAG corpora out of default training runs unless there is a deliberate, reviewed decision to include them.
• Run basic checks for poisoning patterns in incoming corpora: duplicates with odd labels, high influence points, anomalous clusters. For high-risk domains, occasionally run targeted backdoor scans: * Condition models on rare triggers and see if behavior changes disproportionately.
• Stress-test with input patterns that should be neutral but might have been used as triggers. ### 5. Serving and tools: least privilege and isolation At runtime: * Serve models from containers or VMs with minimal permissions: no direct database access, no cloud-control-plane rights, no access to training corpora.
• Put RAG indices and tools behind their own auth and audit layers, not directly on the model host.
• Lock tools down by tenant and by function: a support assistant does not need the same tools as a devops agent. When adding new tools: * Threat-model them as separate components: what can go wrong if this tool is misused?
• Validate model-generated arguments aggressively: types, ranges, object existence, policy.
• Log tool invocations with enough detail to detect abuse. ### 6. Reproducibility where it matters You will not make every training run perfectly reproducible. You should at least: * Be able to reconstruct how a given production model was trained or fine-tuned: code version, hyperparameters, data snapshot.
• Be able to retrain it from scratch if you suspect poisoning or backdoors. That requires: * Saving configs and metadata as part of the run, not as an afterthought.
• Not training entire production-defining models only from interactive notebooks. If you cannot rebuild your own model, you cannot clean it if you discover that your supply chain was compromised. ## If you can only fix a few things If all of this feels like too much, focus on a minimal cut that actually moves risk. One: stop running untrusted artifacts in privileged environments
  Isolate model loading and experimentation. Use safe loading methods. Treat random repos and weights like you would treat random binaries. Two: build a basic registry
  Even a simple table that says "model X was built from base Y plus adapters A and B on data snapshot D" is better than the current fog in most orgs. Three: lock tools and RAG behind real auth
  Do not let the model's outputs translate directly into arbitrary tool calls or arbitrary document access. Enforce scopes and log everything. Everything else can come incrementally. But unless you get these basics in place, "supply-chain security for AI" is just a phrase. The real system will still be: trust whatever you downloaded last week and hope no one cared enough to tamper with it.