Operations Tooling

Sanctum ships with a set of operational scripts in ~/.sanctum/scripts/ that automate the most common maintenance tasks. These are designed to be run from the Mac Mini hub.

Complement on the MBP side: ~/Documents/Claude_Code/tools/ holds laptop-resident utilities — gui-exec.sh (reach into manoir’s Aqua session from any SSH shell), backup-sanctum.sh (weekly pull-backup, documented under Backup & Restore), and keychain-read.sh (read a manoir keychain secret remotely). Together they bridge “things you can do from a terminal” with “things that require GUI session state.”

gui-exec.sh (MBP)

The one-liner you reach for when something works on the Mini’s desktop but returns cryptic errors over SSH — keychain access, biometric prompts, launchctl bootstrap gui/<uid>/..., anything that checks whether the caller is inside the Aqua session. It bridges the gap by asking Terminal.app on the remote to run the command for you, which lives inside the Aqua session and therefore has the right credentials and launchd domain.

# Bootstrap a user-domain LaunchAgent from an SSH shell
~/Documents/Claude_Code/tools/gui-exec.sh neo@100.0.0.25 \
  'launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/com.sanctum.council-guardian.plist'

How it works: SSHes to the remote, invokes osascript to tell application "Terminal" to do script "<cmd>; echo DONE_<epoch>". Terminal.app opens a tab in the logged-in user’s session, runs the command with full Aqua state, and appends a DONE_<epoch> marker so you can grep stdout for the unique tag.

Note

Requires “Remote Login” (sshd) on the remote and a user currently logged in to its GUI. The Mac Mini sitting in the closet satisfies both by default. If neither is true, fall back to the raw osascript pattern — or just accept that you need to walk over to the machine.

Common triggers for gui-exec.sh:

• launchctl bootstrap | enable | kickstart gui/<uid>/... failing with 125: Domain does not support specified action
• security add-*-password / keychain ops returning -36: User interaction is not allowed
• op signin biometric prompts that can’t reach Touch ID over SSH
• Anything a user-authored LaunchAgent needs to touch during install.

Your haus has a health check suite, a sync pipeline, an auto-healing watchdog, and a diagnostic tool. There are Fortune 500 companies with worse observability. We’re not going to dwell on what that says about the state of enterprise software or the mental state of people who build haus intelligence platforms. We’re just going to document the tools.

Note

This documentation is automatically kept in sync with the scripts in the sanctum-config repository. When new tools are added or existing ones are updated, these docs are updated in the same commit cycle.

sanctum-status.sh

The big picture. One command, every service, everything you need to know about whether your haus is alive and well or quietly falling apart.

~/.sanctum/scripts/sanctum-status.sh

What it checks:

• Mac services: Cloudflare Tunnel, Home Assistant, LM Studio, Docker
• VM services: health ingester, Graphiti, Neo4j, OpenClaw gateway
• External access: ha.example.net, health.example.net, sanctum.haus (with CF Access token test)
• Tailscale: connection status, peer count
• Keychain: all critical keys present (account: sanctum)
• Git repos: 8 Mac repos + VM repo (dirty files, unpushed commits)
• Disk usage: Mac and VM

Flags:

Flag	Effect
--quiet	Only show problems. Exit code = number of issues.

Use --quiet in cron jobs or post-boot checks to get a pass/fail result. No news is good news. Literally — silence means everything is fine.

sanctum-sync.sh

One command to check, commit, and push every Sanctum git repository. Including the VM repo, which requires a bundle proxy because — and this never stops being fun to explain — the VM has no internet access.

~/.sanctum/scripts/sanctum-sync.sh

Repos synced:

• sanctum-config (~/.sanctum)
• command-center, health-center, openclaw-skills, sanctum-docs
• yoda-voice-agent, genome-mcp, icloud-organizer
• sanctum-hub (VM, via bundle proxy)

Dirty files are auto-committed with a generated message describing the changes. The VM repo is pushed through the Mac using vm-push.sh under the hood, which creates a git bundle, sneaks it across the bridge network like contraband, and pushes it to GitHub on the VM’s behalf.

Flags:

Flag	Effect
--dry-run	Show what would be done without making changes
--force	Commit and push without prompting

Tip

--force is for when you trust yourself. --dry-run is for when you’ve learned not to.

sanctum-doctor.sh

The thing you run when something feels wrong but you don’t know what. It checks file permissions, port conflicts, stale processes, config cache, Docker health, keychain integrity, firewall state, and VM connectivity. Then it tells you what’s broken. And if you ask nicely — with --fix — it’ll try to repair things too.

# Diagnose only
~/.sanctum/scripts/sanctum-doctor.sh

# Diagnose and auto-fix
~/.sanctum/scripts/sanctum-doctor.sh --fix

What it checks and fixes:

Check	Auto-fix
Config cache stale	Regenerates .instance.json
File permissions drifted	Restores 700/600 as needed
Scripts not executable	chmod 700
Duplicate cloudflared processes	Kills extras, keeps oldest
Zombie Docker containers	docker container prune
Stale sanctum keychain entries	Reports (manual cleanup)
Missing critical keychain keys	Reports
pf firewall anchor not loaded	Reloads anchor
Stopped Docker containers	docker start
LaunchAgents not loaded	launchctl bootstrap
VM SOPS vault integrity	Reports

Caution

The doctor reports but does not auto-fix keychain issues. Keychain surgery requires human hands. This is by design — automated keychain modification at 3 AM is how you end up locked out of your own haus. The one with the AI agents. Think about that.

vm-push.sh

Push the VM git repository to GitHub via the Mac. The VM runs on a host-only network with no internet access, so this script creates a git bundle on the VM, transfers it to the Mac, merges it with the GitHub remote, and pushes.

It’s a Rube Goldberg machine for git push, and it works flawlessly.

# Push current VM state
~/.sanctum/scripts/vm-push.sh

# Commit all dirty files on VM first, then push
~/.sanctum/scripts/vm-push.sh --commit "Updated health ingester config"

Merge conflicts are resolved automatically by favoring the VM version (since the VM is the source of truth for its own repo).

rotate-cf-service-token.sh

Rotate the Cloudflare Access service token used by Health Auto Export to authenticate with the health ingester. This should be run before the current token expires (check sanctum-status.sh for expiry warnings).

~/.sanctum/scripts/rotate-cf-service-token.sh

What it does:

1. Creates a new service token via CF API (1-year expiry)
2. Updates the CF Access policy for health.example.net
3. Deletes the old service token
4. Updates the Mac Keychain with new credentials
5. Regenerates the Health Auto Export import file
6. Verifies the new token works end-to-end

After running, you need to update Health Auto Export on the iPhone with the new CF-Access-Client-Id and CF-Access-Client-Secret headers (or re-import the regenerated config from iCloud).

Yes, the last step is manual. On your phone. With your thumbs. Welcome to the limits of automation.

generate-hae-config.sh

Generate a Health Auto Export importable .json file for this Sanctum instance. Reads tokens from Keychain and VM SOPS, and produces a file ready to AirDrop to an iPhone.

# Generate on Desktop (default)
~/.sanctum/scripts/generate-hae-config.sh

# Generate to specific path
~/.sanctum/scripts/generate-hae-config.sh ~/Desktop/my-health-config.json

The generated config includes the correct endpoint URL, all three authentication headers (base64-encoded), and the 16 health metrics that the ingester processes. After AirDropping to the iPhone, open the file in Health Auto Export and tap Import.

Caution

Health Auto Export may not import headers from the JSON file. If the test returns 401 or 403 after importing, add the three headers manually from the values in the Keychain (cf-access-health-client-id, cf-access-health-client-secret, and the bearer token from VM SOPS). This is one of those delightful iOS behaviors that works perfectly in testing and then doesn’t in production. Sound familiar?

backup-retention.sh

Manages the lifecycle of daily backups in iCloud. Without this, the backup directory grows until iCloud sends you a passive-aggressive storage notification and you spend a Saturday morning manually deleting timestamped folders like a person who doesn’t have an automation platform.

# Dry run — show what would be pruned
~/.sanctum/scripts/backup-retention.sh

# Actually prune
~/.sanctum/scripts/backup-retention.sh --confirm

Retention policy:

Tier	Rule
Daily backups (0–14 days)	Kept in full
Bi-weekly archives (14+ days)	One backup per two-week period retained
Everything else	Purged

Runs weekly via automated job. Logs all deletions to ~/.sanctum/logs/backup-retention.log.

The initial deployment freed ~23 GB from iCloud. That’s 23 GB of timestamped directories that had been accumulating since the backup system was first set up, each one faithfully preserving a snapshot of the haus that nobody was ever going to restore from because the more recent ones existed. Retention policy is the part of backup strategy that everyone agrees is important and nobody implements until their cloud storage bill arrives.

Caution

The script requires --confirm to actually delete anything. Without it, you get a preview. This is not overcaution — it’s the only sane default for a script whose entire job is to destroy backups. The irony of a backup deletion script causing data loss is the kind of thing that would be funny in a talk and devastating in production.

generate-instance.sh

Interactive wizard that creates a new instance.yaml for a fresh Sanctum installation. Asks about your haushold, network, optional services, and notifications, then generates a complete config file.

~/.sanctum/scripts/generate-instance.sh

The wizard:

• Auto-detects your macOS username, home directory, and LAN IP
• Picks up Tailscale hostname and IP if installed
• Generates a URL slug from your haushold name
• Asks which optional services to enable (Firewalla, voice agent, Cloudflare, health, Tailscale)
• Configures Cloudflare domain and tunnel name if enabled
• Sets up notification preferences (Signal)
• Produces a properly formatted instance.yaml with 600 permissions

After generating, review the file, fill in any hardware-specific values (QEMU VM UUID, Firewalla MAC address, HA entity IDs), then run generate-plists.sh to create the LaunchAgents.

Note

The wizard gets you 80% of the way there. The last 20% is the part that requires you to actually know things about your own hardware. No script can help you find your Firewalla’s MAC address. That’s between you and your router’s admin page.