Recovery Agent Framework

When systems fail,
agents recover.

CrisisMode is the recovery layer for your infrastructure. Monitoring tells you something is wrong. CrisisMode tells you what to do about it — safely. AI-powered diagnosis, validated recovery plans, human-in-the-loop execution with an immutable forensic trail — and now, a scale-readiness report that tells you what will break under load before it does.

crisismode — demo mode (simulator, no infrastructure required)

Not a spec. A working tool.

Run crisismode scan with zero configuration. It detects what's running, checks health, and tells you what to look at first. No YAML, no credentials, no setup ceremony.

19
Recovery Agents
Execute-capable live clients for PostgreSQL, Redis, Kubernetes, AWS, and Vercel; the rest run diagnosis-only or simulator-backed
5
Escalation Levels
Observe, diagnose, suggest, repair-safe, repair-destructive — progressive depth under operator control
4
Check Formats
Native JSON, Nagios/Icinga, Goss YAML, and Sensu — thousands of existing checks work out of the box
8
MCP Tools
Read-only diagnosis tools your AI coding agent can call directly — crisismode mcp and Claude or Cursor can check your stack's health and readiness for you

Modern Application Incidents

  • Bad deploy rollback
  • AI provider degradation and failover
  • Database migration failures
  • Queue and worker backlog
  • Config and environment drift
  • New Scale readiness — before the incident

Stateful Infrastructure Recovery

  • Live PostgreSQL replication
  • Redis memory pressure
  • etcd consensus recovery
  • Kafka partition rebalance
  • Kubernetes node cascades
  • Ceph OSD recovery
  • Flink checkpoint failures

Your app works. Will it survive success?

Recovery is what you need after things break. crisismode readiness is what you run before: one command that checks the failure modes that take working apps down at scale — connection limits, missing indexes, stuck transactions, slow queries — and computes how much load your stack can actually take. Every number is labeled with where it came from. Nothing is ever made up.

crisismode readiness — sample output
$ crisismode readiness
 
Scale readiness: at-risk (score 80/100)
4 rules evaluated, 1 could not run
 
⚠ Connection headroom [at_risk]
    72 of 100 connections in use (72%)
    Fix: close idle connections or add a connection pooler before traffic grows
✓ Long-running transactions [ready]
✓ Missing indexes [ready]
· Slow queries [unknown]
    could not run: pg_stat_statements is not available — enable it with CREATE EXTENSION pg_stat_statements
 
Capacity ceilings (upper bounds — real capacity is lower):
  Database concurrent queries: at most 100 connections [declared: max_connections = 100 (declared)]
  Could not assess: db-throughput — mean query time unavailable (pg_stat_statements absent or empty)
01 — Six Rules

Checks That Map to Real Outages

Connection headroom, connection-limit tiers, idle transactions, missing indexes, slow queries, serverless pooling. Each finding explains what it means in plain English, what to do about it, and where to learn more — whether you've run production for ten years or shipped your first app last week.

02 — Capacity Ceilings

Every Number Names Its Source

Ceilings are labeled declared (a configured limit), measured (observed from your live system), or typical (a cited range, explicitly not a measurement). Throughput bounds are derived from Little's law with the practical wall called out — an "at most," never a promise.

03 — Honesty Contract

Unknown Is Never Scored

If a check can't run, it says so and tells you why — it never guesses, and gaps never move your score. This contract is enforced by torture tests that stop the database mid-run and verify CrisisMode reports honestly instead of fabricating.

Meets your incidents where they already live

CrisisMode doesn't demand a new workflow. It speaks the formats your tooling — and your AI agents — already use.

SRE Incident Bundles

Evidence In, Ranked Hypotheses Out

Pipe an SRE incident evidence bundle to crisismode bundle respond and get back ranked root-cause hypotheses and policy-gated recovery actions in a structured AdapterResponse — compatible with the emerging SRE incident-agent ecosystem, scored against its open compatibility benchmark.

MCP Server

Diagnosis for Your Coding Agent

Run crisismode mcp and Claude, Cursor, or any MCP-capable agent gets eight read-only diagnosis tools — scan, diagnose, readiness, and more. Ask your coding agent "will my stack survive launch?" and it can actually go find out. The MCP surface never mutates infrastructure.

Existing Checks

Bring the Checks You Have

Nagios/Icinga, Goss YAML, Sensu, and native JSON check formats work out of the box — decades of community health checks become CrisisMode signal without rewriting anything.

"Agents propose, the framework disposes. Agents produce recovery plans describing what they intend to do. The framework validates, orchestrates, and enforces those plans."
— Recovery Agent Contract Specification v0.3.0-draft

Built for the worst moment

CrisisMode is not a general-purpose automation platform. It is the tool an organization reaches for when normal operational tooling has failed or is insufficient.

01 — Safety

Safety by Default

Agents inherit safety guarantees from the framework. An agent that follows the contract cannot bypass state preservation, skip approval gates, or exceed its declared blast radius.

02 — Forensics

Forensic-First Recovery

The framework captures system state before mutating actions, preserving evidence for post-incident analysis, compliance, and learning — within the constraints of system health.

03 — Human-in-the-Loop

Structured Human Interaction

Notification, approval, escalation, and communication are first-class primitives with the same rigor as system actions. Not an afterthought — a design constraint.

04 — Speed

Approval Speed, Not Bypass

Pre-authorized action catalogs (spec-stage, hub roadmap) will make approval fast for known scenarios. The system never provides mechanisms to skip approval under pressure.

05 — Trust

Graduated Trust

Designed so agents earn autonomy over time through demonstrated reliability in specific scenarios and environments (spec-stage, hub roadmap). Trust is scoped per agent, scenario, and environment.

06 — Resilience

Graceful Degradation

The framework sheds capabilities progressively as the environment degrades, rather than failing entirely. Recovery capability is always available at some level.

13 steps from alert to recovery

Every recovery follows a structured sequence. The framework orchestrates each phase — agents never interact directly with target systems. This is the full lifecycle defined by the contract specification; catalog matching and trust-level gating are spec-stage and not yet shipped in the open-source CLI.

Trigger Framework

Framework receives trigger from alert, health check, or manual invocation

Catalog Check Framework

Check pre-authorized action catalogs for matching scenario

Agent Selection Framework

Identify applicable agent based on trigger context and manifest declarations

Context Assembly Framework

Assemble context bundle — system topology, trust levels, organizational policies

Diagnosis Agent

Agent performs read-only investigation using provided context

Diagnostic Plan Agent

Agent may submit a lightweight diagnostic plan for investigative mutations

Plan Creation Agent

Agent produces a Recovery Plan — linear steps with bounded decision points

Plan Validation Framework

Validate plan against manifest, organizational policies, and blast radius

Catalog Match Framework

If plan matches a catalog entry, approval is pre-satisfied for covered risk levels

Human Gates Human

Execute approval gates per risk classification and trust level

Execution Framework

Orchestrate plan steps: snapshot → action → verify → notify

Replanning Agent

At declared checkpoints, agent may revise remaining plan based on current state

Completion Framework

Produce forensic record and trigger post-recovery notifications

Four layers, designed to shed

The framework operates as concentric layers. As the environment degrades, outer layers shed while core recovery remains available. A recovery tool that requires healthy infrastructure to operate is useless.

L4 Enrichment Phase 2

Advanced trust analytics, stakeholder communication rendering, observed impact monitoring, topology feedback loop

L3 Coordination Should be available

Human approval routing, escalation management, notification delivery, pre-authorized catalog matching, fallback approval

L2 Safety Must be available

State preservation capture, plan validation against manifest, blast radius enforcement, forensic record assembly

L1 Execution Kernel Always available

Sequential plan execution, command dispatch, precondition evaluation, success criteria checks, local audit log, stepwise rollback. Zero external dependencies.

Rigorous by design

The Recovery Agent Contract Specification defines the interface between agents and framework with the precision of a protocol specification. Every requirement is phased, every interaction is structured, every decision is auditable.

7
Step Types
4
Risk Levels
4
Trust Levels
4
Degradation Layers
3
Blast Radius Tiers
22
Specification Sections
Recovery Plans

Readable Under Pressure

Linear sequences with bounded decision points. A plan with 10 steps and one binary decision is comprehensible at 3 AM during a P1 outage. A 30-node graph is not.

Replanning

Adaptability Without Complexity

When conditions change beyond what a binary decision point can handle, the agent produces a new plan. Simple plans, clean audit trail, novel situations handled naturally.

Pre-Authorization

Reviewed in Calm, Activated in Crisis

Organizations pre-authorize specific recovery approaches during calm conditions. A crisis activates the pre-approved response — fast for the safe phase, controlled for the risky phase.

Recovery is too important to improvise

CrisisMode is open source. Read the specification, run the interactive demo, build your first recovery agent.

# Signed binaries for Linux & macOS — no runtime required
curl -fsSL https://github.com/trs-80/crisismode/releases/latest/download/crisismode-darwin-arm64 -o crisismode
chmod +x crisismode
./crisismode scan  # what's wrong right now?
./crisismode readiness  # what breaks under load?