Signed is
not safe.
AI supply-chain integrity platform. IEEE paper, ESAIP Angers, 2026.
Checked in your browser, just now
digits-finetuned-cnn-v1.0.0.safetensors
- File size
- …
- SHA-256
- …
- Ed25519 signature
- …
Verdict …
The deployed model of experiment E2. Poisoned rows were already in its training data when the data steward signed it, so every signature is genuine. Your browser re-ran the file hash (V1) and the signature (V3); the pipeline's verifier ran all ten checks and deployed it.
These files come from a re-run of E2 on 10 November 2026 with the paper's code and seed. Download the model, its signed manifest and the public key to check them yourself.
Draw a digit
The model reads
·
AI-ChainGuard signs every step of an AI pipeline, from the dataset to the deployed model, and blocks anything that fails verification. Then we measured what a valid signature really proves.
Drawings become 8×8 digits the way the UCI dataset was made: ink on a 32×32 grid, counted in 4×4 blocks. Nothing leaves your browser.
AI models are assembled, not written
A deployed model descends from datasets, downloaded pretrained weights and fine-tuning runs owned by different parties. Every hand-over is a chance to tamper, and it already happens.
A dependency-confusion attack replaced
torchtritonin PyTorch nightly builds with a package that stole SSH keys and files. [1]JFrog found about a hundred models on Hugging Face whose pickle files ran code as soon as they were loaded. [2]
ReversingLabs described deliberately broken pickle files that ran their payload before the security scanner failed. [3]
Web-scale training datasets were shown to be poisonable for a few tens of dollars. [5]
About 250 poisoned documents planted a backdoor in language models from 600 million to 13 billion parameters. [4]
The EU AI Act, Article 15(5), requires measures against data poisoning and model poisoning; the NIST AI RMF and MITRE ATLAS describe the same risk. [6] [7] [8]
Today's tools sign one thing
Sigstore, OpenSSF Model Signing, in-toto and SLSA sign a file or attest a software build. Nothing links dataset, base model, fine-tuned model and deployment, decides who may sign which step, revokes along the links or gates deployment. [9] [10] [11] [12]
And “signed” is read as “safe”
A signature proves who attested some bytes and that they have not changed since. It says nothing about what a model learned. Users of model hubs still read signed as safe, and nobody had measured how wrong that is.
How AI-ChainGuard works
Every artifact gets a signed manifest whose payload contains the digests of its parents, so the whole lineage becomes one hash-linked graph, the way Git commits are linked.
- Stage 1Datasetsdata_steward
SHA-256 of the file and a Merkle root over its rows, so a report can name the changed lines.
- Stage 2Base modeltraining_pipeline
Must descend from a signed training dataset.
- Stage 3Fine-tuned modeltraining_pipeline
A fine-tuning dataset plus exactly one base model, internal or external.
- Stage 4Deploymentdeployment_gateway
Verifies the whole chain once, loads the verified bytes, smoke-tests and signs a deployment record.
Linked signed manifests
The whole manifest is signed with Ed25519, not just the file hash. Rewriting an ancestor, even with the legitimate key, breaks every descendant.
Ten checks, three verdicts
Checks V0 to V9 run on the artifact and every ancestor. The verdict is derived, never stored: FAILED, WARNING or VERIFIED.
A signed trust log
Who is trusted and what is revoked live in an append-only, hash-chained log signed by a root key and pinned. Truncated, edited or deleted, it fails closed.
A deployment gateway
Refuses FAILED models, WARNING ones by default and pickle formats; loads from the verified bytes, so there is no time-of-check to time-of-use gap.
The ten checks
| Check | Fails when | |
|---|---|---|
| V0 | Envelope | missing, malformed or stored under another id |
| V1 | File hash | SHA-256 or size differs from the signed values |
| V2 | Dataset Merkle root | rows changed (the report names them) |
| V3 | Ed25519 signature | invalid, or the key is not in the signed trust log |
| V4 | Signer trust | key revoked, wrong role for the stage, or database and log disagree |
| V5 | Parent links | a parent's digest differs from the one signed in the child |
| V6 | Ancestry | any ancestor failed (recursive) |
| V7 | Revocation | artifact or ancestor revoked in the signed log |
| V8 | Policy | wrong type for the stage, missing or forbidden parents |
| V9 | Content flags | a signed content check flagged the data: WARNING only |
A real signed manifest
Payload of the fine-tuned model, excerpt; digests shortened.
{
"artifact_name": "digits-finetuned-cnn",
"artifact_type": "finetuned_model",
"pipeline_stage": "stage3_finetune",
"subject": {
"sha256": "95cba5f7…317e5b",
"size_bytes": 153944,
"media_type": "application/safetensors"
},
"parents": [
{ "manifest_digest": "sha256:c8df3c8c…0bfddb",
"relation": "fine_tuned_from" },
{ "manifest_digest": "sha256:51a1aec7…b60272",
"relation": "trained_on" }
],
"signer": {
"identity": "training_pipeline@chainguard.local",
"role": "training_pipeline",
"key_id": "ed25519:0dda1db0a6406ccd"
}
}
Every attestation also exports as an in-toto Statement v1 in a DSSE envelope, with a ChainGuard or SLSA Provenance v1 predicate. [14] [15]
Attack the chain yourself
This runs in your browser with real SHA-256, a real Merkle tree, real Ed25519 signatures and a signed trust log, over a small copy of the pipeline. Pick an attack and watch the verifier.
After signing: should be caught
Before signing: the boundary
Training data (digits-train, 24 signed rows)
Verification report
The demo re-implements the core checks in JavaScript, with keys generated in your browser for this visit. It does not train models: the backdoor rates it shows are the ones measured in the paper's experiments with the Python implementation.
What a signature proves, and what it does not
Seven experiments and 48 attack and control variants on the real pipeline. Experiments 6 and 7 trained real models to measure what lies beyond the reach of signatures.
A VERIFIED status proves
- who attested each artifact, and under which role;
- that no byte, row, parent link or signature changed since;
- that every signer was trusted for its stage and is not revoked.
It does not prove
- that the data was clean when it was signed (E2: 77.2 % backdoor success);
- that a trusted publisher's model is harmless (E7-F: 100 %);
- that content checks stop an adaptive attacker (E4: 51.2 %, still VERIFIED).
Content checks before signing are evidence, not proof: they caught the known trigger (E3, WARNING, blocked) and missed the adaptive one (E4). So AI-ChainGuard keeps the two kinds of evidence apart: V0 to V8 are cryptographic, V9 is about content, and no heuristic can ever produce a FAILED, nor a signature a “safe”.
Every manipulation after signing was caught
| Exp. | Experiment (scenario) | Variants | Outcome | Checks that fired |
|---|---|---|---|---|
| 1 | Dataset tampering (A) | 5 | control VERIFIED; 4 FAILED | V1, V2, V6 |
| 2 | Model substitution (B) | 4 | control VERIFIED; 3 FAILED | V1 |
| 3 | Invalid signature, key misuse (D) | 7 | control VERIFIED; 6 FAILED | V3, V4, V6 |
| 4 | Broken provenance (C) | 5 | control VERIFIED; 4 FAILED | V3, V5, V8 |
| 5 | Revocation and 8 bypasses (F) | 13 | 2 VERIFIED as predicted; 11 FAILED, deployment blocked | V3, V4, V6, V7 |
| 6 | Poison before signing (E) | 5 | E0 VERIFIED; E1 FAILED; E3 WARNING; E2, E4 VERIFIED | V6, V9 |
| 7 | Untrusted external source (G) | 9 | 3 VERIFIED; 4 FAILED; 2 REFUSED at import | V1, V3, V4, V6 |
Attacks on the trust itself: all eight fail
- C1Delete the revoked artifact's row in the databaseV7
- C2Delete the revoked key from the databaseV4 V6
- C3Overwrite the cached statusV6 V7
- C4Clear the revoked flag in the database (the limit of version 0.3)V6 V7
- C5Add a rogue key to the database and re-sign a substituted modelV3 V4
- C6Truncate the trust log to before the revocationV3 V4 V6 V7
- C7Edit a trust-log record in placeV3 V4 V6 V7
- C8Delete the trust logV3 V4 V6 V7
What it costs
Attestation runs once per artifact and verification once per deployment, never per prediction.
- Attesting the four artifacts
- 10.7 ms against 1,288 ms of training (0.83 %)
- Full-chain verification
- 8.5 ms median, p95 12.2 ms
- One prediction, before and after deployment
- 0.084 ms and 0.085 ms
- First secure deployment, paid once
- 253 ms, mostly audit writes and a second chain check
- Signed manifest
- 1.4 to 3.1 KB per artifact
- SHA-256 and Ed25519
- about 1.2 GB/s; sign 0.04 ms, verify 0.12 to 0.14 ms
How it compares
Capabilities as each project documents them (November 2026). No column reaches “yes” for poisoned content: that is the boundary measured above.
| Sigstore / cosign | OpenSSF Model Signing | in-toto, SLSA | Model cards, ML-BOM, registries | Backdoor scanners | AI-ChainGuard | |
|---|---|---|---|---|---|---|
| Scope | one artifact | a model directory | steps of a software build | description, inventory | model behaviour | dataset, base model, fine-tune, deployment |
| Links the stages | no | no | layout links build steps | metadata lineage | no | yes: parent digests inside each signed payload |
| Who may sign what | identity policies | signer identity | functionaries per step | platform access control | – | a role per stage, verified (V4, V8) |
| Revocation | certificate expiry, log | Sigstore ecosystem | key revocation in layout | stage transitions | – | signed, pinned log; propagates to descendants |
| Deployment decision | admission controllers | no | verify the final product | no | no | gateway: verify, safe load, smoke test, signed record |
| Poisoned content | no | no | no | no | partial, heuristic | partial: WARNING only; measured, not claimed |
The working system
One Python library drives a command-line tool, a REST API and a Streamlit dashboard. A 40-second tour of the dashboard, recorded on 10 November 2026; select a step to jump to it.
Run it yourself
Python 3.11 or newer, no GPU. Both GitHub Actions workflows pass: lint and the 168 tests, and the attested pipeline with the attack scenarios.
git clone https://github.com/domail2karandeshmukh-cyber/ai-chainguard
cd ai-chainguard
pip install torch --index-url https://download.pytorch.org/whl/cpu
pip install -r requirements-dev.txt
python -m scripts.demo # 15-step demo
make standalone # the dashboard
make test # the 168 tests
Limitations we state
- The attack suite is written by the defenders and runs on a small network for 8×8 digits: it shows the checks work and the gap exists, not its size for language models, where poisoning is cheaper.
- The content checks are simple heuristics; stronger defences such as Neural Cleanse, STRIP or spectral signatures would raise the bar without closing it.
- Keys are long-lived files: a stolen key that is not yet revoked signs valid manifests. Keyless short-lived certificates or hardware keys address this.
- The trust log is local, with a single writer: a stale copy misses a recent revocation. Publishing it to a shared transparency log is the next step.
- Benchmarks were measured in one environment (cloud Linux, 2 vCPU, CPU only); the tests also pass in CI, and Docker Compose ran on a team laptop. This page is static: the verification and the model at the top run in your browser, and the full system (API, dashboard, gateway) runs on your own machine, for example with Docker Compose.
The paper
Signed Is Not Safe: Measuring the Boundary Between Cryptographic Integrity and Semantic Trust in AI Supply Chains
Six pages, IEEE conference format, in English and in French with the same sections, figures, references and numbers. It describes the attestation chain, the 48-variant evaluation, the cost and the interoperability, and argues that AI supply-chain controls must report cryptographic and semantic evidence separately.
Cite
@misc{deshmukh2026signed,
title = {Signed Is Not Safe: Measuring the
Boundary Between Cryptographic Integrity
and Semantic Trust in AI Supply Chains},
author = {Deshmukh, Karan and Jogi, Sudheer Reddy
and Dama, Dola Krishna and Mekala, Rambabu},
year = {2026},
note = {ESAIP, Angers. Projet Applicatif 2},
url = {https://github.com/domail2karandeshmukh-cyber/ai-chainguard}
}
Every deliverable of the brief
The six phases of Projet Applicatif 2 and where each deliverable is. The requirements checklist maps every line of the brief to its evidence.
- Scoping and problem definitionScoping note and threat model: assets, threats, trust boundaries.
- State of the artSynthesis of AI supply-chain attacks and defences, and a comparative table of solutions.
- Solution designArchitecture diagrams, functional and technical descriptions, attack scenarios.
- ImplementationDocumented code, scripts and 168 tests, Docker and GitHub Actions, the experimentation journal with the test environment.
- Testing and evaluation48 attack and control variants with logs and tables, the charts on this page, and the critical analysis and improvement paths in the paper.
- Reporting and valorisationThe final report (34 pages and appendices), the IEEE paper in English and French, the conference proposal, the demonstration scenario and the weekly reports (template).
Sources
The 26 references of the paper, in order of first citation. Online sources were checked on 9 November 2026.
All 26 references
- PyTorch team, “Compromised PyTorch-nightly dependency chain between December 25th and December 30th, 2022,” Dec. 31, 2022. Accessed: Nov. 9, 2026. pytorch.org
- D. Cohen, JFrog Security Research, “Data scientists targeted by malicious Hugging Face ML models with silent backdoor,” Feb. 2024. Accessed: Nov. 9, 2026. jfrog.com
- K. Zanki, ReversingLabs, “Malicious ML models discovered on Hugging Face platform,” Feb. 2025. Accessed: Nov. 9, 2026. www.reversinglabs.com
- A. Souly et al., “Poisoning attacks on LLMs require a near-constant number of poison samples,” arXiv:2510.07192 [cs.LG], Oct. 2025. arxiv.org
- N. Carlini et al., “Poisoning web-scale training datasets is practical,” in Proc. IEEE Symp. Security and Privacy (SP), 2024, pp. 407–425. arxiv.org
- European Parliament and Council, “Regulation (EU) 2024/1689 of 13 June 2024 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act),” Off. J. Eur. Union, L, 2024/1689, Jul. 12, 2024, Art. 15(5). eur-lex.europa.eu
- NIST, “Artificial Intelligence Risk Management Framework (AI RMF 1.0),” NIST AI 100-1, Jan. 2023. doi.org
- MITRE, “ATLAS technique AML.T0010: AI supply chain compromise.” Accessed: Nov. 9, 2026. atlas.mitre.org
- OpenSSF, “SLSA specification v1.1: Supply-chain Levels for Software Artifacts,” Apr. 2025. Accessed: Nov. 9, 2026. slsa.dev
- S. Torres-Arias, H. Afzali, T. K. Kuppusamy, R. Curtmola, and J. Cappos, “in-toto: Providing farm-to-table guarantees for bits and bytes,” in Proc. 28th USENIX Security Symp., 2019, pp. 1393–1410. www.usenix.org
- Z. Newman, J. S. Meyers, and S. Torres-Arias, “Sigstore: Software signing for everybody,” in Proc. ACM SIGSAC Conf. Computer and Communications Security (CCS), 2022, pp. 2353–2367. blog.sigstore.dev
- OpenSSF, “Model Signing v1.0,” Apr. 2025. Accessed: Nov. 9, 2026. github.com
- T. Gu, B. Dolan-Gavitt, and S. Garg, “BadNets: Identifying vulnerabilities in the machine learning model supply chain,” arXiv:1708.06733 [cs.CR], 2017. arxiv.org
- in-toto project, “in-toto Attestation Framework: Statement v1.” Accessed: Nov. 9, 2026. github.com
- Secure Systems Lab, “DSSE: Dead Simple Signing Envelope, protocol v1.” Accessed: Nov. 9, 2026. github.com
- J. Samuel, N. Mathewson, J. Cappos, and R. Dingledine, “Survivable key compromise in software update systems,” in Proc. ACM Conf. Computer and Communications Security (CCS), 2010, pp. 61–72. uptane.org
- B. Laurie, A. Langley, and E. Kasper, “Certificate Transparency,” RFC 6962, IETF, Jun. 2013. www.rfc-editor.org
- M. Mitchell et al., “Model cards for model reporting,” in Proc. Conf. Fairness, Accountability, and Transparency (FAT*), 2019, pp. 220–229. arxiv.org
- T. Gebru et al., “Datasheets for datasets,” Commun. ACM, vol. 64, no. 12, pp. 86–92, 2021. arxiv.org
- Hugging Face, “Safetensors: a simple, safe way to store and distribute tensors,” documentation; security audit by Trail of Bits, 2023. Accessed: Nov. 9, 2026. huggingface.co
- B. Wang et al., “Neural Cleanse: Identifying and mitigating backdoor attacks in neural networks,” in Proc. IEEE Symp. Security and Privacy (SP), 2019, pp. 707–723. www.cs.uchicago.edu
- Y. Gao, C. Xu, D. Wang, S. Chen, D. C. Ranasinghe, and S. Nepal, “STRIP: A defence against trojan attacks on deep neural networks,” in Proc. Annu. Computer Security Applications Conf. (ACSAC), 2019, pp. 113–125. arxiv.org
- B. Tran, J. Li, and A. Madry, “Spectral signatures in backdoor attacks,” in Advances in Neural Information Processing Systems (NeurIPS), vol. 31, 2018. arxiv.org
- B. Chen et al., “Detecting backdoor attacks on deep neural networks by activation clustering,” in Proc. AAAI Workshop on Artificial Intelligence Safety (SafeAI), 2019. arxiv.org
- S. Josefsson and I. Liusvaara, “Edwards-Curve Digital Signature Algorithm (EdDSA),” RFC 8032, IETF, Jan. 2017. www.rfc-editor.org
- A. Rundgren, B. Jordan, and S. Erdtman, “JSON Canonicalization Scheme (JCS),” RFC 8785, IETF, Jun. 2020. www.rfc-editor.org
Team
- Karan DESHMUKHTechnical lead, security architecture
- JOGI Sudheer reddyAI and machine-learning engineering
- DAMA Dola KrishnaTesting, documentation and quality
- MEKALA RambabuSecurity and DevOps
Supervised by Sofiane HAMRIOUI. MSc Cybersecurity and Data Science, ESAIP École d'Ingénieurs, Angers. Projet Applicatif 2.