RunsOn vs Blacksmith for GitHub Actions runners

By Kaveh Alemi·Latchkey

RunsOn vs Blacksmith is a choice between two cost models rather than two rate cards: RunsOn is a flat annual license from 300 euros that installs into your own AWS account and leaves the compute on your AWS bill, and Blacksmith is a managed fleet at $0.004 a Linux minute with 3,000 free minutes a month. The rate comparison favors RunsOn heavily and it only counts if you want an AWS footprint to own.

RunsOn installs a stack into your AWS account and runs ephemeral EC2 runners there, on spot with automatic on-demand fallback, on any EC2 shape your quotas allow. You pay AWS directly for the instances with no markup, and RunsOn for a license: 300 euros a year for under 50,000 runners a month, rising through 900 and 1,800 euros to 3,600 for enterprise volume, free for non-commercial use, with a 15-day trial that deploys into your real account rather than a sandbox.

Blacksmith sells managed runners on its own bare metal in four regions, at a single published rate per platform, with 3,000 free minutes a month and a GitHub App to install. There is no cloud account to own, no quota to raise and no stack to upgrade, and there is also no way to put a runner inside your VPC. The simplicity is most of what you are buying.

Every cell below was read from the vendor's own public pricing or documentation page on 20 September 2026. Where a vendor does not publish a number, the cell says "not published" instead of carrying a guess.

RunsOn, Blacksmith and Latchkey side by side (read 20 September 2026)

Two cost models, one month

Because RunsOn charges a license rather than a rate, the two are only comparable once you put a month against both. Take the workload every comparison here uses, 10,000 Linux minutes at 2 vCPU plus 2,000 macOS minutes, and remember that RunsOn does not support macOS, so those minutes stay on GitHub-hosted exactly as they do for Latchkey.

Two honest caveats on that RunsOn row. The $10 is RunsOn's own published figure for us-east-1 spot including a 30 GB gp3 volume, and it excludes everything else an AWS account charges for: NAT gateways if your runners sit in private subnets, the S3 bucket behind Magic Cache, data transfer, and the on-demand fallback you will land on whenever spot capacity is short, which is $0.0016 a minute rather than $0.0007. The license also scales by runners per month rather than minutes, so a busy monorepo firing 60,000 jobs a month is on the 900 euro tier whether those jobs take one minute or ten.

The second caveat is the one that does not appear on any invoice: a RunsOn runner is a self-hosted runner as far as GitHub is concerned, which is why GitHub charges nothing for the minutes, and also why GitHub's own advice applies. Its documentation says "We recommend that you only use self-hosted runners with private repositories", because a fork can run untrusted code on your machine. RunsOn gives you ephemeral per-job instances, which is the right mitigation, and it is still your account and your blast radius. Latchkey's own plan is $5 to $49 a month on top of that compute, and its 2,000 to 6,000 included minutes are not netted off, so the figure is gross on both counts.

Speed

Not benchmarked yet; the runner benchmark report will replace this section. We have not run the same workload on these runners ourselves, and until we have, there is no number here worth quoting.

The cheap way to get a real answer is to point one slow job at each candidate for a week, because runner selection is per job and the Actions run log gives you the timings for free.

Reliability, and who gets paged

Uptime is unpublished on both sides, beyond Blacksmith listing 99.9% as an Enterprise plan feature. The more useful question is who is on the hook when runners stop appearing. On Blacksmith that is Blacksmith, and your escalation path is a support ticket. On RunsOn it is you: spot capacity, EC2 quotas, AMI updates, stack upgrades and VPC changes are all yours, and RunsOn ships observability hooks precisely because you are the operator.

What both share is the behavior of a failed job. A registry timeout, a missing shared library or an out-of-memory kill fails the job, bills whatever it burned, and waits for a person to press re-run. On RunsOn that bill is an EC2 line rather than a runner line, which makes it easier to miss and no cheaper. Exit code 137 in GitHub Actions and error while loading shared libraries are the two that most often look like a runner fault on either side.

When to choose RunsOn

• You already run AWS and know the answer to "who owns this account". That is the whole prerequisite, and it is a real one.
• Your volume is high enough that a flat license beats a metered rate: at 100,000 Linux minutes a month, a 350 dollar annual license plus roughly $100 of spot compute is a fraction of any per-minute rate on this page.
• You need jobs inside your VPC, static egress IPs, GPU instances, or an EC2 shape nobody sells as a managed runner.
• Your cache is the bottleneck and you would rather it lived in your own S3 bucket, unlimited and free to transfer inside the VPC, than in a 25 GB weekly allowance.
• Your project is nonprofit, open-source or educational, where the license is free and the only cost is your AWS bill.

When to choose Blacksmith

• You want zero infrastructure. No AWS account, no quotas, no stack upgrades, no spot interruptions to reason about.
• You need macOS or Windows from the same vendor as your Linux runners. RunsOn documents that macOS is not yet supported.
• Your volume is modest. 3,000 free minutes a month plus $0.004 after that beats a 350 dollar annual commitment until you are running a few thousand minutes every month.
• You want arm64 cheaply at $0.0025 a minute without choosing an instance family.
• You would rather the vendor got paged than your platform team.

Switching: the exact diff

Both are label changes, and the labels look nothing alike. Blacksmith names a size; RunsOn encodes the runner definition in the label itself under Flex, or points at a Terraform-managed fleet under Fleet. The RunsOn edit is preceded by a stack install in AWS, which is the part that takes an afternoon rather than a minute.

 jobs:
   test:
-    runs-on: ubuntu-latest
+    runs-on: blacksmith-4vcpu-ubuntu-2404

   build:
-    runs-on: ubuntu-latest
+    runs-on: runs-on=${{ github.run_id }}/runner=4cpu-linux-x64

   build-fleet:
-    runs-on: ubuntu-latest
+    runs-on: runs-on/fleet=linux-small/env=production

The verdict

On compute alone RunsOn wins this comfortably, because spot EC2 at RunsOn's own published figures is a fraction of any managed per-minute rate and the license does not grow with your minutes. That result is only available to a team that wants to own an AWS account, its quotas, its upgrades and its security boundary, and the moment nobody wants to own those things the calculation inverts: Blacksmith is a GitHub App and a label, it has macOS and Windows where RunsOn has neither, and its 3,000 free minutes cover a small team outright. The honest split is volume and appetite. High volume with a platform team goes to RunsOn; modest volume, or a team that would rather ship than operate, goes to Blacksmith. Neither repairs the job that failed for a mechanical reason, and on RunsOn that failure hides inside an EC2 invoice instead of a runner one.

Frequently asked questions

Related guides

References

• RunsOn pricing: license tiers and published EC2 per-minute costs
• RunsOn runner types, platforms and labels
• RunsOn Magic Cache
• Blacksmith pricing
• GitHub: only use self-hosted runners with private repositories
• GitHub Actions documentation