March Update: Latchkey Runners and Self-Healing Are Coming

This update covers the decision we made in February: Latchkey is going to start running CI, not only watching it, with a beta of Latchkey Runners, managed runners for GitHub Actions with self-healing built in, planned for April. It also recaps what shipped in February, explains what the runners and self-healing are meant to be from your side, and says what we are working on this month.

Platform vision: from observing CI to acting on it

Last month I described a trust ladder for automation in CI: observe, recommend, act in a sandbox, act with a pull request. Everything Latchkey did through February lived on the outer rungs. We observed your runs, recommended changes, and turned accepted changes into pull requests. The rung we had not touched was acting inside a running job, and I wrote then that you cannot build it from outside a job.

In February we decided to build it. The bet is simple to state. As more of the code reaching CI is written by agents, the pipeline becomes the gate between machine output and your main branch, and a gate that stops for a human every time a runner is missing a tool becomes the bottleneck of the whole loop. The failures that stall that loop most often are not bugs in the diff. They are problems with the machine the job ran on, and nobody should have to wake up for those.

So Latchkey moves from observing CI to running it. The runner is the one place where we can act at the moment of failure: see a step fail, work out whether the environment caused it, repair the environment, and let the step run again, before anyone is notified that anything went wrong.

What does not change is the set of rules from last month. Repair the environment, never the code. A real failure stays red. Permanent changes arrive as pull requests you review. Running the machine gives us somewhere to act; it does not give us a vote on your code. If we get that balance right, a green check from a Latchkey runner means exactly what a green check has always meant.

Platform core: what insights do today, and what they gain

To explain the decision, here is what Latchkey did as of the end of February.

Install the GitHub App and every workflow run is recorded: its jobs, the runner each job used, how long it took and what it cost, including whether the minutes fell inside your GitHub allowance. The Cost page shows theoretical cost next to billable cost, a forecast to the end of the billing cycle, and cost by runner OS. When something notable happens (a run fails, a repository keeps failing, a run costs or takes much more than usual, someone edits a workflow file), our analysis agents take a look.

The agents produce two kinds of output. A recommendation comes with evidence, risks and an exact change to your workflow file; pick the ones you want on the Insights page and Latchkey opens a pull request. A diagnosis names the failed job and step, the error and the likely root cause.

That is a useful product, and it stays. But look at where every path ends. A recommendation ends in a pull request that someone has to review and merge. A diagnosis ends in a report that someone has to read, usually after they have already hit re-run. Every insight, however accurate, was waiting on a human.

The second lesson came from reading the diagnoses themselves, a lot of them, through January and February. The root cause of a red run was very often not in the diff. It was a package registry that timed out, a tool missing from the runner image, a dependency download that failed halfway, a machine that ran out of disk. I am not going to put a percentage on it, because we do not have a measurement we would stand behind, only the experience of reading diagnosis after diagnosis that ended in some version of "nothing is wrong with your code; run it again". No change to your YAML would have prevented those failures, and by the time our diagnosis arrived, the developer had already lost the minutes and the context switch.

The analytics product keeps working on any runner: GitHub-hosted, self-hosted, or ours. If you use Latchkey today for cost tracking and insights, nothing about that changes in April, and you do not have to adopt runners to keep it. What changes is that the insights get something to act on. Today a diagnosis for a network timeout is a report you read after the fact. On a Latchkey runner, the same diagnosis can become a repair that happens while the job is still running. The cost data we already collect is also how we will show you, per job, what moving to a Latchkey runner would mean, rather than asking you to take our word for it.

Platform core: what Latchkey Runners will be

Here is what we intend the beta to be. This is intent, not a spec, and some of it will move before April.

Managed runners for GitHub Actions. Keep your workflows, keep GitHub Actions. You adopt Latchkey by changing one line in a job: runs-on. Nothing else in the workflow needs to know.

jobs:
  test:
    # before
    # runs-on: ubuntu-latest
    runs-on: latchkey-medium   # illustrative label; final names come with the beta
    steps:
      - uses: actions/checkout@v4
      - run: npm ci
      - run: npm test

A fresh machine for every job. Each job gets its own machine, which is thrown away when the job ends. Nothing leaks from your last job or from anyone else's. This is also what makes self-healing safe: a repair made for one job cannot outlive it.

Linux first. The beta targets Linux jobs. Other platforms will follow once Linux is solid, and not before.

Per-minute pricing. You pay for the minutes your jobs run, like GitHub-hosted runners. We are not publishing sizes, prices or speed numbers today, because we have not measured anything we would be willing to print. Those come with the beta, with the basis for any comparison stated next to it.

Why a runner, and not something lighter. There are two lighter ways to attempt this, and we do not think either works. One is an Action you add to your workflow that retries failed steps. Blind retries are what every team already does by hand, they cannot install a missing tool or free a full disk, and they also retry real failures, which teaches people to distrust red. The other is offering self-healing only on runners you host yourself. That means repairing machines we do not control and cannot reset, which is how a fix that works on one box poisons the next job. Owning the whole machine, for exactly one job, is what makes repairs safe.

Platform core: self-healing, and the line it never crosses

When a step fails on a Latchkey runner, the plan is that the failure is examined before the job gives up. If the cause is the environment, the runner repairs the environment and runs the step again. If the cause is your code, nothing is changed and the step stays red.

The environment side of the line is the list we kept reading in diagnoses: a tool the step expects that is not on the machine, a network fetch that failed transiently, a dependency download that did not complete, a machine resource like disk that ran out. The code side is everything that belongs to you: a failing test, a compile error, a lint violation, a type error, a broken script.

The rules we are designing around, stated plainly because we intend to be held to them:

• Self-healing never touches your code. It does not edit files in your repository, change your tests, commit, or open pull requests. It changes the machine the job runs on, for that job only.
• Real failures stay red. A test that catches a bug is doing its job. Making it pass is not a repair, it is lying to you, and we will not do it.
• Every repair is visible. If a step went green because the runner repaired something, the job log says what failed and what was changed, so you can fix the root cause in your own setup.
• When unsure, do nothing. A code failure we wrongly leave alone costs you what it costs today. A code failure we wrongly "fix" costs you trust, and that is not recoverable.

We will explain how it works in detail once it is running, not while it is on a whiteboard.

Coming next

March is a build month. We will build the runners and self-healing, and move Latchkey's own CI onto them before anyone else's: if they cannot run our builds, they are not ready to run yours.

Two smaller things are close enough to name. The first is notifications: today you find out about a failing repository or a cost spike by opening the dashboard, and we want Latchkey to tell you where you already are, by email, in Slack, or as a push notification, with you choosing which events are worth an interruption. The second is a smaller plan for individual developers, so that trying Latchkey on a personal project does not mean paying for a team's worth of seats.

We are also lining up a small group of beta teams. If your team runs GitHub Actions on Linux and has a repository that goes red for reasons unrelated to its code, reach us through the Support page. Those are exactly the repositories we want.

We will say more about plans and how analytics and runners fit together when the beta opens; we are not changing anyone's current plan today. The beta target is April. If that date moves, you will read it here first, along with the reason.

Previous issue: February Update: CI/CD in the Agentic Era · Next issue: April Update: The Runners Beta Opens