# Supervisor

Apple's `container` runtime supervises nothing. There is no `--restart` flag, no policy that brings a
crashed service back, and nothing that keeps the engine itself alive. If a container dies, it stays
dead until you notice.

The **Supervisor** is Runbay's app-side answer. It has three parts — and one honest
limitation you should understand up front.

## Daemon watchdog

The watchdog polls the engine every 30 seconds with a real round-trip (a `container ls --format json`
call, not just a process check). On failure it attempts one automatic `container system start`, then
backs off on a 5s → 15s → 60s schedule so it never hammers the daemon.

## Keep-alive and launch-at-login

- **Start at App Launch** — mark a container (context menu, or the Supervisor settings tab) and it
  comes up automatically when the app starts. The list self-prunes if the container is removed.
- **Launch at login** — the app itself can start at login via `SMAppService`, so your supervised
  containers are there when you sit down.

## Keep Running (per-container restart policy)

Opt in per container. When enabled:

- **Crash detection** on the 30-second poll.
- **Per-container backoff** of 5s → 15s → 60s, with **park-after-3**: after three failed restarts it
  stops trying and offers a **Try Again** button, so a hard-broken container never starves the loop.
- **Health-driven restart** for containers that declare a health check — three consecutive unhealthy
  checks trigger a stop→start. Each container's restart work runs as its own task, so one container's
  backoff never delays daemon monitoring or another container's recovery.

This is the `--restart` policy Apple's CLI doesn't have, implemented in the app.

## The honest limitation (decision D017)

**The app cannot restart a fully-dead engine by itself.**

Runbay runs under the macOS App Sandbox. Starting the engine (`container system start`)
submits a launchd job via `launchctl bootstrap`, and **launchd refuses domain-management submissions
from sandboxed processes.** The daemon-reachability entitlements the app carries let it *talk to* an
already-running daemon; they do not let it *register* one. The exact same command from a
non-sandboxed shell works instantly.

So when the engine is fully down, the watchdog:

1. Attempts `system start` once (auto-recovery still works in non-sandboxed dev builds).
2. Recognizes the sandbox failure signature (`failed to get a response from apiserver`) and parks
   immediately in `recoveryUnavailable` instead of burning retries.
3. Posts a notification and shows a **"Restart in Terminal"** button in **Settings ▸ Supervisor**
   (which launches the command in your terminal — a path proven safe under the sandbox).
4. Keeps polling, so the moment you restart the daemon manually, the state self-heals to `running`.

This was verified end-to-end: engine down → one attempt → parked with "user action required" →
manual `system start` → next poll → "daemon healthy."

The real fix — a bundled **non-sandboxed** `SMAppService` launch-agent helper that performs the start
over XPC (launch agents don't inherit the app sandbox) — is planned. Until then, in-app engine
restart is a guided one-click Terminal action, not silent magic. We'd rather tell you that than fake
it.

## See also

- [Getting started](getting-started.md) — install the engine and run the first-run doctor.
- [Stacks](stacks.md) — health checks come from stack definitions; health-driven restart uses them.