--:--:--
← ALL NOTES
Dec 09, 20253 MIN READAIAGENTSOPS

The night shift is a fleet of models

Planners plan, workers execute, verifiers attack, and a gate with a whitelist and a ledger stands between all of them and the outside world.

At some point the number of scheduled jobs, monitors and pipelines crossed the line where a human could still be the router. Our answer was not one big autonomous agent. It was a fleet of small ones with a deterministic script holding the leash.

Code owns control flow, models own content

The orchestrator is ordinary code: it decides what runs, in what order, with what concurrency, and what happens on failure. Models fill in the steps. A planner turns an objective into a task list. Workers execute tasks inside tool sandboxes with step budgets. A verifier, prompted to refute rather than to help, attacks the workers' output. A reporter compresses the surviving results for humans. Every role writes a JSONL transcript, so any night's work can be replayed step by step the next morning.

The one architectural rule: models never talk to each other directly. They talk through the orchestrator, which means every hop is logged, budgeted and interruptible.

Outbound is the dangerous direction

The risky side of ops automation is not reading, it is writing. A bot that can message a human at 4 AM will, one day, message them forty times at 4 AM. Every outbound path in our fleet, human or API, goes through one gate:

function send(msg, to) {
  if (!WHITELIST.has(to))                return refuse('recipient');
  if (ledger.countToday(to) >= DAILY_CAP) return refuse('cap');
  ledger.append({ to, digest(msg), at: now() });   // written BEFORE sending
  return transport.send(msg, to);
}

A whitelist of recipients, a daily cap enforced in code, a ledger written before the send, and one gate shared by every sender. Ten scripts sharing one gate beat ten scripts with ten opinions about etiquette.

Silence detection over failure alerts

# the watchdog does not listen for errors. it demands proof of life.
find /opt/health -name '*.ok' -mmin +90 -print    # stale heartbeat = incident

Cron jobs fail loudly only in fairy tales. In production they fail by not running: a typo in a schedule, a dead virtualenv, a server clock in a timezone the author did not expect. Our watchdog expects heartbeats and treats absence as the alarm. Since that inversion, the class of incident where something died silently for a week has simply disappeared.

The rule of thumb we settled on: automation earns autonomy in proportion to how mechanically its blast radius is capped. Write the gate first, then let the fleet loose.

WRITTEN FROM THE INTFRAME ENGINE ROOM

WORK WITH US →