Audit Log Integrity and Session State Channels

Incident response, red-teaming, and any after-the-fact review of a deployed model all assume a complete, tamper-evident record of what the model did. If log entries can be suppressed, falsified, or selectively deleted, a safety claim that rests on “we can review the logs” is unsupported. Append-only storage and cryptographic log chaining (in the style of Certificate Transparency [1] and tamper-evident history trees [2]) provide tamper evidence, but the deployed log servers that serving infrastructure would use, such as sigstore’s Rekor [3], are not shipped with machine-checked proofs.

There is a subtler problem on the same boundary. When a model’s output or summary is written back to session state for future context windows (the text prepended to later requests), the write-back opens a persistent covert channel in Lampson’s sense [4]. A misaligned model [5] can encode messages in session state that survive request boundaries, such as instructions to future invocations of itself or data staged for later exfiltration. The output filter (the gateway-side check on the response returned to the user) never sees the write-back to the session store, so it cannot catch this. Any system that persists conversation summaries, scratchpads, or tool-call results across requests exposes this surface.

Both problems sit on a write path out of the serving loop, and in both cases a violation is invisible until someone inspects the stored contents.

Solution Sketch A: Log Integrity via Permission Graph

Build a TLA+ [6] model of the serving infrastructure’s request lifecycle. State variables: the committed log, the in-flight request set, the per-principal permission relation. Transitions: a request enters the gateway, dispatches to forward-pass execution, emits a log entry, commits the entry to storage, returns a response. Three properties are checked against the same model.

  • Tampering safety: in every reachable state, no transition available to any principal (model, harness, orchestrator) modifies or deletes an already-committed log entry.
  • Log-before-response (also a safety property): no response-emit transition for a request fires unless that request’s log-commit transition has already fired. This catches suppression by the same infrastructure that handles writing.
  • Logging completeness (liveness): every request that reaches forward-pass execution eventually has its log entry committed.

The model is not the gateway. The result is only as good as the correspondence between the TLA+ transitions and the deployed components, so the sketch should budget for closing that gap, via trace validation against production logs or conformance tests generated from the model. Sketch A treats the log as a passive resource and verifies the access-control and lifecycle machinery around it.

Solution Sketch B: Log Integrity via Verified Merkle-Tree Log

Instead of verifying that the permission system prevents tampering, make tampering detectable from the log’s own structure. Append-only Merkle-tree logs [2] of the kind Certificate Transparency [1] and Rekor [3] use publish signed tree heads. Any third party can use them to check, via consistency proofs, that two published heads belong to the same history. A serving infrastructure that retroactively removes a committed entry cannot then issue a new signed head consistent with the previous one.

The verification target is the log component itself, against an abstract specification of an append-only Merkle log. Two local properties:

  • Append-only signing: the only path to the signing operation runs through append(entry), and every signed root is a valid extension of the previously signed root. No implementation path signs a root that omits a previously committed entry.
  • Consistency-proof correctness: consistency_proof(old_root, new_root) is sound (a proof verifies only for genuinely consistent trees) and complete (an honestly generated proof always verifies).

Both are local properties of a single program, a good fit for a refinement proof against the abstract spec in Dafny [7] or F* [8]. Soundness is necessarily conditional: it holds relative to an explicit collision-resistance assumption on the hash, which the proof states as an axiom rather than discharges.

The result is a log component whose tamper-evidence follows from the Merkle structure, not from the permission graph around it. Unlike Sketch A, its guarantee can be re-checked by any third party holding two signed tree heads, independently of the operator. That requires the heads to actually reach independent parties: an operator who shows different heads to different auditors (equivocation) defeats the check unless heads are gossiped or co-signed by witnesses, and that distribution protocol is outside this sketch.

Sketch B does not replace Sketch A. A Merkle log makes tampering with committed entries detectable, but says nothing about entries that are never committed. Logging completeness and log-before-response remain Sketch A’s job. The natural combination uses Sketch B’s verified log as the storage primitive whose access path Sketch A’s model covers.

Solution Sketch C: Session State via Noninterference

For the session-state channel, apply information flow analysis [9]. A naive noninterference property, that session state depends only on “authorized” inputs such as the model’s declared output, does not help: the declared output is itself a function of the model’s hidden state, and a model encoding messages steganographically does so through the declared output.

The useful property is narrower. Session state must be a deterministic function of what was already exposed to monitoring: the user’s message, the tool results, and the response as delivered after the output filter. Stated as noninterference [10]: for any two runs that agree on those inputs, but differ in anything else the model produced (undelivered scratchpad, discarded drafts, fields the filter strips), the write-back function produces identical session state. This does not eliminate steganography in the visible response. It collapses the persistent channel onto the channel the output filter already inspects, so no unmonitored path into future context remains.

The write-back function is small and its specification is a pre/postcondition contract on its output, so Dafny [7] fits. The relational two-run form of the property needs either a product-program encoding (two copies of the function run side by side and their outputs compared) or a relational verifier.

Bibliography

  • [1] B. Laurie, A. Langley, and E. Kasper, “Certificate Transparency,” RFC 6962, 2013.
  • [2] S. A. Crosby and D. S. Wallach, “Efficient Data Structures for Tamper-Evident Logging,” in Proceedings of the 18th USENIX Security Symposium, 2009. [Online]. Available: https://static.usenix.org/event/sec09/tech/full_papers/crosby.pdf
  • [3] Z. Newman, J. S. Meyers, and S. Torres-Arias, “Sigstore: Software Signing for Everybody,” in Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security (CCS), 2022. doi: 10.1145/3548606.3560596.
  • [4] B. W. Lampson, “A note on the confinement problem,” Communications of the ACM, vol. 16, no. 10, pp. 613–615, 1973, doi: 10.1145/362375.362389.
  • [5] R. Greenblatt, B. Shlegeris, K. Sachan, and F. Roger, “AI Control: Improving Safety Despite Intentional Subversion,” in International Conference on Machine Learning (ICML), 2024.
  • [6] L. Lamport, Specifying Systems: The TLA+ Language and Tools for Hardware and Software Engineers. Addison-Wesley, 2002. [Online]. Available: https://lamport.azurewebsites.net/tla/book.html
  • [7] K. R. M. Leino, “Dafny: An Automatic Program Verifier for Functional Correctness,” in Logic for Programming, Artificial Intelligence, and Reasoning (LPAR-16), in Lecture Notes in Computer Science, vol. 6355. Springer, 2010, pp. 348–370. doi: 10.1007/978-3-642-17511-4_20.
  • [8] N. Swamy et al., “Dependent Types and Multi-Monadic Effects in F*,” in Proceedings of the 43rd ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL), 2016, pp. 256–270. doi: 10.1145/2837614.2837655.
  • [9] A. C. Myers, “JFlow: Practical Mostly-Static Information Flow Control,” 1999.
  • [10] J. A. Goguen and J. Meseguer, “Security Policies and Security Models,” in Proceedings of the 1982 IEEE Symposium on Security and Privacy, 1982, pp. 11–20. doi: 10.1109/SP.1982.10014.

// stack context

[execution-harness] Execution Harness

Execution Harness The execution harness is the outermost software layer of an ML deployment: the wrapper that actually calls the model and handles its inputs and outputs. A user’s prompt enters here, gets tokenized and batched, passes through the model, and returns as a completion. Because every int…

[orchestration-cloud] Orchestration & Cloud

Orchestration and Cloud Layer The orchestration layer manages the compute resources that ML workloads run on: scheduling jobs across GPU clusters, routing traffic between them, and controlling who can access what. It sits below the frameworks and above the firmware — a layer of distributed systems s…

← all problems