Overview

Halo

A resilient incident commander — an agent built to stay standing when its own infrastructure breaks.

Built for the TrueFoundry Resilient Agents hackathon on AWS Bedrock. Halo runs against a real live product (Jaguar), not a toy.

The problem

Agents fall apart the moment their infrastructure does. A model starts rate-limiting, a tool times out, a provider has a bad day — and the agent either crashes, starts making things up, or quietly does the wrong thing. That is especially painful during an incident, because the one tool you are leaning on to recover just became another thing that is down.

What Halo does

Halo is an incident commander that is built to stay standing when things break. It works real incidents on a live product through TrueFoundry's AI Gateway and MCP Gateway, and instead of failing it degrades.

  • Investigates, doesn't guess. Pulls live worker status, ingestion, deploys, and failures through the product's MCP tools and works out the real root cause.
  • Asks before it acts. Risky actions are split onto a separate write server and never run without human approval.
  • Degrades instead of dying. On a rate-limit or outage it fails over to a backup model and steps down a mode — cheaper model, read-only tools — instead of crashing.
  • Tells the truth. After a fix runs, it re-checks the product and reports whether the action actually worked — no premature victory laps.

A real run

In our demo run, Halo diagnosed a real Jaguar worker blackout, prepared a worker restart, and — once an operator approved it — executed it on the live VPS through Jaguar's ops-runner. Then it checked its own work and reported the truth: the restart was necessary but it was not the whole fix. The real culprit was an upstream credential that had been invalidated.

Nothing dangerous ran without a human, nothing sensitive slipped through a guardrail, and the whole run is backed by TrueFoundry traces.

Next steps