Runtime Drift Audit

The Wachowskis called it a glitch in the matrix — the little hiccup that means someone changed something in the simulation while you weren’t looking. Infrastructure drift is quieter than a black cat walking past twice. A LaunchAgent points at a file that used to exist. An auditor confidently reports red on a service that is, in fact, purring. /health gets mistaken for an executable path. The walls of the construct shift one pixel to the left every Sunday at 04:00 and nobody notices until someone stubs a toe on a wrapper script that moved directories last quarter.

The checked-in workspace was only half the story. The live system still depended on ~/.sanctum, generated wrappers, and a pile of LaunchAgents with varying opinions about whether their targets existed. This audit pushed into the runtime layer and cleaned up the drift that actually pages people.

Verified Runtime Surface

The runtime audit covered:

• ~/.sanctum/instance.yaml
• ~/.sanctum/services/
• ~/.sanctum/bin/* wrappers
• ~/Library/LaunchAgents/com.sanctum.*.plist
• The workspace auditors in tests/test-sanctum-runtime-audit.sh and tools/audit_runtime_launchagents.py

What Was Broken

The initial pass found two structural mismatches and several stale launch targets:

• ~/.sanctum/services did not exist even though the documentation described runtime manifests
• com.sanctum.living-force.plist referenced ~/.sanctum/living-force.sh, which was missing
• Several LaunchAgents pointed at files that no longer existed
• The legacy validator produced false positives by treating arguments like /health as executable paths

What Was Fixed

The remediation pass did six concrete things:

1. Restored the wrapper layer under ~/.sanctum/bin
2. Created the missing ~/.sanctum/services directory
3. Added a runtime renderer so ~/.sanctum/services/*.yaml is generated from instance.yaml, the checked-in sanctum/runtime_catalog.yaml, and auxiliary launchd-wired services instead of remaining an empty placeholder
4. Added a compatibility shim at ~/.sanctum/living-force.sh
5. Re-enabled or corrected valid LaunchAgents such as com.sanctum.proxy.plist and com.sanctum.health-center.plist
6. Retired the legacy com.sanctum.living-force.plist in favor of the Rust watchdog plist so the runtime stops carrying two conflicting watchdog generations at once
7. Retired clearly obsolete LaunchAgents by renaming them to .disabled

Note

One of the more useful findings was that not every red test was real drift. Some of it was just a bad auditor shouting confidently — the infrastructure equivalent of Agent Smith insisting something is wrong while the rest of the Matrix hums along fine. Infrastructure work gets easier the moment the alarms stop lying. Ignore a real alarm once and it silently retires; trust a lying alarm twice and you start chasing ghosts through three time zones. The audit’s job is to separate the two.

Current Status

The runtime audit now passes cleanly:

• tools/audit_runtime_launchagents.py reports OK
• tools/render_runtime_services.py --check reports no drift
• tools/sync_agent_capabilities.py --check reports no drift
• tools/sync_runtime_calibration.py --check reports no drift
• tests/test-sanctum-runtime-audit.sh reports 23 passed, 0 failed
• tests/test-sanctum-immune-system.sh reports 10 passed, 0 failed against the rebuilt Rust watchdog binary

That does not mean the runtime is portable. It means the live machine is internally consistent again, and the runtime manifest directory now contains actual graph inputs instead of a directory-shaped lie. The rendered set at audit time was 33 manifests: 28 instance-backed services plus 5 auxiliary runtime services such as the proxy and SSH tunnels. A subsequent off-catalogue audit (April 2026) raised this to 38 by registering five previously unwatched services. The checked-in runtime catalog now also carries explicit cross-service edges that instance.yaml does not encode on its own, including the verified voice-agent -> qwen3-tts dependency (rename history: xtts → xtts-server Q2; → qwen3-tts 2026-04-25). The Code Forge permission model is now also part of that runtime surface: ~/.sanctum/config/agent-capabilities.yaml exists and is synced from the checked-in workspace copy instead of being silently absent. The same is now true for the remaining generated runtime artifacts: ~/.sanctum/.instance.json and the template-managed LaunchAgents are explicitly checked against their canonical inputs rather than trusted on vibes.

Remaining Architectural Gap

The biggest unresolved question is still ownership of truth:

• Should ~/.sanctum/instance.yaml remain the native runtime source?
• Or should runtime manifests eventually be rendered from the same canonical catalog approach used in the workspace slice?

That is a product decision disguised as an ops decision. Either answer can work. Two answers will drift. One source of truth is a covenant; two sources of truth is Thursday’s incident.

Related Pages

• Implementation Audit
• Operational State
• Roadmap