← Back to Touchwarden

Spend caps for infrastructure changes

You approve a monthly limit once. On every pull request, Touchwarden plans the infrastructure change and a proven rule decides whether its spending is capped. A coding agent can add servers, change budgets or edit the limit in the same change, and the rule still holds.

The rule

Every paid resource sits under a monthly budget that stops spending when it is reached, and those caps add up to no more than the approved limit.

The owner signs four laws, and Touchwarden proves the decision keeps all four for every possible plan, not just the ones someone tested:

Five deliberately broken versions of the decision were tried, such as counting a warning-only budget as a cap. All five failed the proof.

Set it up

The limit lives in .touchwarden/contract.yaml on your main branch. A change can edit its own copy, but it is judged by the one on main, and any edit to the contract goes to the owner.

invariants:
  - id: cloud.spend-capped
    statement: "Every paid resource sits under a monthly budget that stops
      spending when it is reached, and those caps add up to no more than
      $500 a month."
    verification:
      type: bend
      law: no_uncapped_spend
      proof: kernel/PROOF.bend
      generated: kernel/spend.generated.mjs
      plan:
        tool: pulumi
        project: infra
        stack: dev
        limit_usd: 500

In Pulumi, a cap is a monthly AWS budget with an action that applies automatically:

new aws.budgets.Budget("web-monthly", {
  name: "web-monthly",
  budgetType: "COST",
  timeUnit: "MONTHLY",
  limitAmount: "300",
  limitUnit: "USD",
  costFilters: [{ name: "TagKeyValue", values: ["user:team$web"] }],
});
new aws.budgets.BudgetAction("web-monthly-stop", {
  budgetName: "web-monthly",
  actionType: "APPLY_IAM_POLICY",
  approvalModel: "AUTOMATIC",
  notificationType: "ACTUAL",
  actionThreshold: { actionThresholdType: "PERCENTAGE", actionThresholdValue: 100 },
  definition: { iamActionDefinition: { policyArn: denyNewSpend.arn, roles: [appRole.name] } },
  executionRoleArn: budgetsRole.arn,
  subscribers: [{ address: "[email protected]", subscriptionType: "EMAIL" }],
});

How a change is judged

  1. Touchwarden runs pulumi preview --json on the change itself. It never trusts a plan file in the pull request.
  2. It reads every resource and budget from the plan, and works out which budgets cover which resources. This reading is tested, not proven, and treats anything it doesn't recognise as paid.
  3. The proven rule decides: passes, or blocked with the resources or totals at fault.
  4. Any change to the Pulumi project is judged, even one that never mentions a budget, such as a new GPU server.

What counts as a cap

Nine changes, judged

A Pulumi stack on AWS with a web server and a database, and $500 a month approved. Each change was planned with a real pulumi preview and checked in about 3 seconds, without an AWS account.

The changeResultWhy
Resize the web server (t3.small to t3.medium)PassesEvery paid resource is under a budget that stops spending, and those caps add up to $450 a month, within the approved $500.
Add a GPU training box with no budgetBlockedSpending is not capped: trainer (aws:ec2/instance:Instance) is not covered by any budget.
Make the database budget wait for manual approvalBlockedSpending is not capped: orders-db (aws:rds/instance:Instance) is only under a budget that does not stop spending (data-monthly: its action needs someone to approve it, so it only sends a warning).
Raise the web budget to $450BlockedThe caps that stop spending add up to $600 a month (web-monthly $450, data-monthly $150), more than the approved $500.
Delete the budget actions, keep the budgetsBlockedSpending is not capped: orders-db (aws:rds/instance:Instance) is only under a budget that does not stop spending (data-monthly: no action enforces it, so it only sends a warning); web (aws:ec2/instance:Instance) is only under a budget that does not stop spending (web-monthly: no action enforces it, so it only sends a warning).
Fire the actions at 200% instead of 100%BlockedThe caps that stop spending add up to $900 a month (web-monthly $600, data-monthly $300), more than the approved $500.
Raise the approved limit in the contract to $1,000Needs ownerThe contract itself changed; the new limit needs its owner's signature.
Raise the limit to $1,000 and the web budget to $450 in one changeBlockedThe caps that stop spending add up to $600 a month (web-monthly $450, data-monthly $150), more than the approved $500. The change's own contract says $1,000, but it is judged by the $500 on main.

What a cap can and can't do