reg-suit "reg-suit run failed" comparison in CI

reg-suit fetches the previously accepted images for the base commit, compares them against the newly captured actual images, and reports differences. A failure usually means it could not fetch the expected snapshot from the store, or the base commit was not found.

What this error means

The reg-suit run step fails, or reports "Loaded 0 expected images," so every image shows as new. The report cannot be published, or the comparison flags changes that are really just a missing base.

[reg-suit] info Detected the previous DirectoryName: (none)
[reg-suit] warn Failed to load the expected images.
[reg-suit] Comparison completed. New items: 42, Changed items: 0

Common causes

No expected snapshot for the base commit

reg-suit uses git history to find the base commit's accepted images. A shallow clone or a missing base snapshot means it loads zero expected images and treats everything as new.

The key generator plugin cannot resolve the base

A misconfigured reg-keygen-git-hash-plugin (or missing history) cannot compute the expected key, so the fetch returns nothing.

How to fix it

Fetch full git history in CI

reg-suit needs commit history to find the base snapshot. Disable shallow clone so the keygen plugin can resolve it.

- uses: actions/checkout@v4
  with:
    fetch-depth: 0

Confirm the plugin configuration

Ensure the keygen and publisher plugins are configured so expected images can be located and fetched.

{
  "plugins": {
    "reg-keygen-git-hash-plugin": true,
    "reg-publish-s3-plugin": { "bucketName": "my-visual-snapshots" }
  }
}

How to prevent it

• Use fetch-depth: 0 so reg-suit can resolve the base commit.
• Verify the keygen and publish plugins are configured.
• Ensure the base branch has a published snapshot to compare against.

Related guides