Continuous integration workflow (lowlighter/metrics)

The Continuous integration workflow from lowlighter/metrics, explained and optimized by Latchkey.

CI health: F - at risk

Point runs-on at Latchkey and get caching, run de-duplication, job timeouts, SHA-pinned actions, self-healing for flaky steps, and up to 58% lower cost, applied automatically.

Grade your own workflow free or run it on Latchkey →

Source: lowlighter/metrics.github/workflows/ci.ymlLicense MITView source

What it does

This is the Continuous integration workflow from the lowlighter/metrics repository, a real project running GitHub Actions. It is shown here with attribution under its MIT license.

Below, Latchkey shows a faster, safer version produced by its optimization engine.

The workflow

workflow (.yml)

name: Continuous integration

on:
  push:
    branches: [ master ]

jobs:

  # Build, test and analyze
  build-test-analyze:
    name: Build, test and analyze
    if: "!contains(github.event.head_commit.message, '[skip test]')"
    uses: lowlighter/metrics/.github/workflows/test.yml@master

  # Format code
  format:
    name: Format code
    runs-on: ubuntu-latest
    needs: [ build-test-analyze ]
    if: always() && (needs.build-test-analyze.result == 'success' || needs.build-test-analyze.result == 'skipped')
    steps:
      - name: Checkout repository
        uses: actions/checkout@v3
        with:
          fetch-depth: 0
      - name: Checkout master
        run: git checkout master
      - name: Format code
        run: |
          npm install -g dprint
          mkdir -v -p /home/runner/.cache/dprint/cache
          npx dprint fmt --config .github/config/dprint.json
      - name: Publish formatted code
        run: |
          set +e
          git config user.name github-actions[bot]
          git config user.email 41898282+github-actions[bot]@users.noreply.github.com
          git add --all
          git commit -m "chore: code formatting"
          git push
          set -e

  # Update plugins and template indexes, along with README.md
  update-indexes:
    name: Publish rebuilt metrics indexes
    runs-on: ubuntu-latest
    needs: [ format ]
    steps:
      - name: Checkout repository
        uses: actions/checkout@v3
        with:
          fetch-depth: 0
      - name: Setup NodeJS
        uses: actions/setup-node@v3
        with:
          node-version: 20
      - name: Pull changes
        run: |
          git checkout master
          git pull
      - name: Setup metrics
        run: npm ci
      - name: Publish rebuild metrics indexes
        run: npm run build -- publish

  # Rebase main branch on master
  update-main:
    name: Rebase main on master
    runs-on: ubuntu-latest
    needs: [ update-indexes ]
    steps:
      - name: Checkout repository
        uses: actions/checkout@v3
        with:
          fetch-depth: 0
      - name: Setup NodeJS
        uses: actions/setup-node@v3
        with:
          node-version: 20
      - name: Pull
        run: git pull
      - name: Checkout main
        run: git checkout main
      - name: Rebase main on master
        run: git merge master
      - name: Push main
        run: git push origin main

  # Build docker image from master and publish it to GitHub registry
  docker-master:
    name: Publish master to GitHub registry
    runs-on: ubuntu-latest
    needs: [ update-indexes ]
    steps:
      - name: Checkout repository
        uses: actions/checkout@v3
      - name: Login to GitHub registry
        run: echo ${{ github.token }} | docker login ghcr.io -u ${{ github.actor }} --password-stdin
      - name: Build docker image
        run: docker build -t ghcr.io/lowlighter/metrics:master .
      - name: Publish to GitHub registry
        run: docker push ghcr.io/lowlighter/metrics:master
      - name: Tag docker image (beta) and publish to GitHub registry
        run: |
          set +e
          METRICS_VERSION=$(cat package.json | grep -Po '(?<="version": ")\d+[.]\d+(?=[.]0-beta")')
          METRICS_VERSION_BETA="$METRICS_VERSION-beta"
          set -e
          if [[ ! -z "$METRICS_VERSION" ]]; then
            echo "Beta version: v$METRICS_VERSION_BETA"
            docker tag ghcr.io/lowlighter/metrics:master ghcr.io/lowlighter/metrics:$(echo "v$METRICS_VERSION_BETA")
            docker push ghcr.io/lowlighter/metrics:$(echo "v$METRICS_VERSION_BETA")
          fi
      - name: Tag docker image (main) and publish to GitHub registry
        run: |
          docker tag ghcr.io/lowlighter/metrics:master ghcr.io/lowlighter/metrics:main
          docker push ghcr.io/lowlighter/metrics:main

  # Deploy to beta-metrics.lecoq.io
  deploy-master:
    name: Deploy lowlighter/metrics@master
    runs-on: ubuntu-latest
    needs: [ docker-master ]
    steps:
      - name: Deploy web instance
        run: "curl -X POST -H 'Authorization: ${{ secrets.WEB_DEPLOY_BETA_TOKEN }}' https://beta-metrics.lecoq.io/.control/stop"
      - name: Wait for redeploy
        run: sleep 120
      - name: Check deployment status
        run: "curl --retry 15 --retry-delay 60 https://beta-metrics.lecoq.io/.version"

  # Test lowlighter/metrics@master
  action-master-test:
    name: Test lowlighter/metrics@master
    runs-on: ubuntu-latest
    needs: [ docker-master ]
    steps:
      - name: Run tests
        uses: lowlighter/metrics@master
        with:
          token: MOCKED_TOKEN
          plugins_errors_fatal: yes
          dryrun: yes
          use_mocked_data: yes
          verify: yes

  # Publish examples
  publish-examples:
    name: Publish examples
    needs: [ action-master-test ]
    uses: lowlighter/metrics/.github/workflows/examples.yml@master
    secrets:
      METRICS_TOKEN: ${{ secrets.METRICS_TOKEN }}
      METRICS_TOKEN_WITH_SCOPES: ${{ secrets.METRICS_TOKEN_WITH_SCOPES }}
      METRICS_TOKEN_PERSONAL: ${{ secrets.METRICS_TOKEN_PERSONAL }}
      CHESS_TOKEN: ${{ secrets.CHESS_TOKEN }}
      PAGESPEED_TOKEN: ${{ secrets.PAGESPEED_TOKEN }}
      GOOGLE_MAP_TOKEN: ${{ secrets.GOOGLE_MAP_TOKEN }}
      SPLATOON_TOKEN: ${{ secrets.SPLATOON_TOKEN }}
      SPLATOON_STATINK_TOKEN: ${{ secrets.SPLATOON_STATINK_TOKEN }}
      SIXTEEN_PERSONALITIES_URL: ${{ secrets.SIXTEEN_PERSONALITIES_URL }}
      SPOTIFY_TOKENS: ${{ secrets.SPOTIFY_TOKENS }}
      STOCK_TOKEN: ${{ secrets.STOCK_TOKEN }}
      TWITTER_TOKEN: ${{ secrets.TWITTER_TOKEN }}
      WAKATIME_TOKEN: ${{ secrets.WAKATIME_TOKEN }}

  # Publish examples (presets)
  publish-examples-presets:
    name: Publish examples (presets)
    needs: [ action-master-test ]
    uses: lowlighter/metrics/.github/workflows/examples.presets.yml@master

  # Build docker image from master and publish it to GitHub registry with release tag
  docker-release:
    name: Publish release to GitHub registry
    runs-on: ubuntu-latest
    needs: [ publish-examples ]
    if: contains(github.event.head_commit.message, '[release]')
    steps:
      - name: Checkout repository
        uses: actions/checkout@v3
      - name: Login to GitHub registry
        run: echo ${{ github.token }} | docker login ghcr.io -u ${{ github.actor }} --password-stdin
      - name: Pull docker image (master)
        run: docker pull ghcr.io/lowlighter/metrics:master
      - name: Tag docker image (release)
        env:
          GITHUB_COMMIT_MESSAGE: ${{ github.event.head_commit.message }}
        run: docker tag ghcr.io/lowlighter/metrics:master ghcr.io/lowlighter/metrics:$(echo $GITHUB_COMMIT_MESSAGE | grep -Po 'v\d+[.]\d+')
      - name: Publish release to GitHub registry
        env:
          GITHUB_COMMIT_MESSAGE: ${{ github.event.head_commit.message }}
        run: docker push ghcr.io/lowlighter/metrics:$(echo $GITHUB_COMMIT_MESSAGE | grep -Po 'v\d+[.]\d+')
      - name: Tag docker image (latest)
        run: docker tag ghcr.io/lowlighter/metrics:master ghcr.io/lowlighter/metrics:latest
      - name: Publish latest to GitHub registry
        run: docker push ghcr.io/lowlighter/metrics:latest

  # Rebase latest branch on master
  update-latest:
    name: Rebase latest on master
    runs-on: ubuntu-latest
    needs: [ docker-release ]
    steps:
      - name: Checkout repository
        uses: actions/checkout@v3
        with:
          fetch-depth: 0
      - name: Setup NodeJS
        uses: actions/setup-node@v3
        with:
          node-version: 20
      - name: Setup metrics
        run: npm ci
      - name: Pull
        run: git pull
      - name: Checkout latest
        run: git checkout latest
      - name: Rebase latest on master
        run: git merge master
      - name: Push latest
        run: git push origin latest

  # Test lowlighter/metrics@latest
  action-latest-test:
    name: Test lowlighter/metrics@latest
    runs-on: ubuntu-latest
    needs: [ update-latest ]
    steps:
      - name: Run tests
        uses: lowlighter/metrics@latest
        with:
          token: MOCKED_TOKEN
          plugins_errors_fatal: yes
          dryrun: yes
          use_mocked_data: yes
          verify: yes

  # Deploy to metrics.lecoq.io
  deploy-latest:
    name: Deploy lowlighter/metrics@latest
    runs-on: ubuntu-latest
    needs: [ docker-release, deploy-master ]
    steps:
      - name: Deploy web instance
        run: "curl -X POST -H 'Authorization: ${{ secrets.WEB_DEPLOY_TOKEN }}' https://metrics.lecoq.io/.control/stop"
      - name: Wait for redeploy
        run: sleep 120
      - name: Check deployment status
        run: "curl --retry 15 --retry-delay 60 https://metrics.lecoq.io/.version"

  # Publish GitHub release
  publish-release:
    name: Publish release
    runs-on: ubuntu-latest
    needs: [ action-latest-test, deploy-latest ]
    steps:
      - name: Checkout repository
        uses: actions/checkout@v3
      - name: Setup NodeJS
        uses: actions/setup-node@v3
        with:
          node-version: 20
      - name: Setup metrics
        run: npm ci
      - name: Publish release
        run: node .github/scripts/release.mjs
        env:
          GITHUB_TOKEN: ${{ github.token }}
          GITHUB_REPOSITORY: ${{ github.repository }}
          GITHUB_COMMIT_MESSAGE: ${{ github.event.head_commit.message }}

The same workflow, on Latchkey

Estimated ~20% faster on cache hits, plus fewer wasted runs and a safer supply chain. Added and changed lines are highlighted.

name: Continuous integration
 
on:
  push:
    branches: [ master ]
 
concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true
 
jobs:
 
  # Build, test and analyze
  build-test-analyze:
    timeout-minutes: 30
    name: Build, test and analyze
    if: "!contains(github.event.head_commit.message, '[skip test]')"
    uses: lowlighter/metrics/.github/workflows/test.yml@master
 
  # Format code
  format:
    timeout-minutes: 30
    name: Format code
    runs-on: latchkey-small
    needs: [ build-test-analyze ]
    if: always() && (needs.build-test-analyze.result == 'success' || needs.build-test-analyze.result == 'skipped')
    steps:
      - name: Checkout repository
        uses: actions/checkout@v3
        with:
          fetch-depth: 0
      - name: Checkout master
        run: git checkout master
      - name: Format code
        run: |
          npm install -g dprint
          mkdir -v -p /home/runner/.cache/dprint/cache
          npx dprint fmt --config .github/config/dprint.json
      - name: Publish formatted code
        run: |
          set +e
          git config user.name github-actions[bot]
          git config user.email 41898282+github-actions[bot]@users.noreply.github.com
          git add --all
          git commit -m "chore: code formatting"
          git push
          set -e
 
  # Update plugins and template indexes, along with README.md
  update-indexes:
    timeout-minutes: 30
    name: Publish rebuilt metrics indexes
    runs-on: latchkey-small
    needs: [ format ]
    steps:
      - name: Checkout repository
        uses: actions/checkout@v3
        with:
          fetch-depth: 0
      - name: Setup NodeJS
        uses: actions/setup-node@v3
        with:
          cache: 'npm'
          node-version: 20
      - name: Pull changes
        run: |
          git checkout master
          git pull
      - name: Setup metrics
        run: npm ci
      - name: Publish rebuild metrics indexes
        run: npm run build -- publish
 
  # Rebase main branch on master
  update-main:
    timeout-minutes: 30
    name: Rebase main on master
    runs-on: latchkey-small
    needs: [ update-indexes ]
    steps:
      - name: Checkout repository
        uses: actions/checkout@v3
        with:
          fetch-depth: 0
      - name: Setup NodeJS
        uses: actions/setup-node@v3
        with:
          cache: 'npm'
          node-version: 20
      - name: Pull
        run: git pull
      - name: Checkout main
        run: git checkout main
      - name: Rebase main on master
        run: git merge master
      - name: Push main
        run: git push origin main
 
  # Build docker image from master and publish it to GitHub registry
  docker-master:
    timeout-minutes: 30
    name: Publish master to GitHub registry
    runs-on: latchkey-small
    needs: [ update-indexes ]
    steps:
      - name: Checkout repository
        uses: actions/checkout@v3
      - name: Login to GitHub registry
        run: echo ${{ github.token }} | docker login ghcr.io -u ${{ github.actor }} --password-stdin
      - name: Build docker image
        run: docker build -t ghcr.io/lowlighter/metrics:master .
      - name: Publish to GitHub registry
        run: docker push ghcr.io/lowlighter/metrics:master
      - name: Tag docker image (beta) and publish to GitHub registry
        run: |
          set +e
          METRICS_VERSION=$(cat package.json | grep -Po '(?<="version": ")\d+[.]\d+(?=[.]0-beta")')
          METRICS_VERSION_BETA="$METRICS_VERSION-beta"
          set -e
          if [[ ! -z "$METRICS_VERSION" ]]; then
            echo "Beta version: v$METRICS_VERSION_BETA"
            docker tag ghcr.io/lowlighter/metrics:master ghcr.io/lowlighter/metrics:$(echo "v$METRICS_VERSION_BETA")
            docker push ghcr.io/lowlighter/metrics:$(echo "v$METRICS_VERSION_BETA")
          fi
      - name: Tag docker image (main) and publish to GitHub registry
        run: |
          docker tag ghcr.io/lowlighter/metrics:master ghcr.io/lowlighter/metrics:main
          docker push ghcr.io/lowlighter/metrics:main
 
  # Deploy to beta-metrics.lecoq.io
  deploy-master:
    timeout-minutes: 30
    name: Deploy lowlighter/metrics@master
    runs-on: latchkey-small
    needs: [ docker-master ]
    steps:
      - name: Deploy web instance
        run: "curl -X POST -H 'Authorization: ${{ secrets.WEB_DEPLOY_BETA_TOKEN }}' https://beta-metrics.lecoq.io/.control/stop"
      - name: Wait for redeploy
        run: sleep 120
      - name: Check deployment status
        run: "curl --retry 15 --retry-delay 60 https://beta-metrics.lecoq.io/.version"
 
  # Test lowlighter/metrics@master
  action-master-test:
    timeout-minutes: 30
    name: Test lowlighter/metrics@master
    runs-on: latchkey-small
    needs: [ docker-master ]
    steps:
      - name: Run tests
        uses: lowlighter/metrics@master
        with:
          token: MOCKED_TOKEN
          plugins_errors_fatal: yes
          dryrun: yes
          use_mocked_data: yes
          verify: yes
 
  # Publish examples
  publish-examples:
    timeout-minutes: 30
    name: Publish examples
    needs: [ action-master-test ]
    uses: lowlighter/metrics/.github/workflows/examples.yml@master
    secrets:
      METRICS_TOKEN: ${{ secrets.METRICS_TOKEN }}
      METRICS_TOKEN_WITH_SCOPES: ${{ secrets.METRICS_TOKEN_WITH_SCOPES }}
      METRICS_TOKEN_PERSONAL: ${{ secrets.METRICS_TOKEN_PERSONAL }}
      CHESS_TOKEN: ${{ secrets.CHESS_TOKEN }}
      PAGESPEED_TOKEN: ${{ secrets.PAGESPEED_TOKEN }}
      GOOGLE_MAP_TOKEN: ${{ secrets.GOOGLE_MAP_TOKEN }}
      SPLATOON_TOKEN: ${{ secrets.SPLATOON_TOKEN }}
      SPLATOON_STATINK_TOKEN: ${{ secrets.SPLATOON_STATINK_TOKEN }}
      SIXTEEN_PERSONALITIES_URL: ${{ secrets.SIXTEEN_PERSONALITIES_URL }}
      SPOTIFY_TOKENS: ${{ secrets.SPOTIFY_TOKENS }}
      STOCK_TOKEN: ${{ secrets.STOCK_TOKEN }}
      TWITTER_TOKEN: ${{ secrets.TWITTER_TOKEN }}
      WAKATIME_TOKEN: ${{ secrets.WAKATIME_TOKEN }}
 
  # Publish examples (presets)
  publish-examples-presets:
    timeout-minutes: 30
    name: Publish examples (presets)
    needs: [ action-master-test ]
    uses: lowlighter/metrics/.github/workflows/examples.presets.yml@master
 
  # Build docker image from master and publish it to GitHub registry with release tag
  docker-release:
    timeout-minutes: 30
    name: Publish release to GitHub registry
    runs-on: latchkey-small
    needs: [ publish-examples ]
    if: contains(github.event.head_commit.message, '[release]')
    steps:
      - name: Checkout repository
        uses: actions/checkout@v3
      - name: Login to GitHub registry
        run: echo ${{ github.token }} | docker login ghcr.io -u ${{ github.actor }} --password-stdin
      - name: Pull docker image (master)
        run: docker pull ghcr.io/lowlighter/metrics:master
      - name: Tag docker image (release)
        env:
          GITHUB_COMMIT_MESSAGE: ${{ github.event.head_commit.message }}
        run: docker tag ghcr.io/lowlighter/metrics:master ghcr.io/lowlighter/metrics:$(echo $GITHUB_COMMIT_MESSAGE | grep -Po 'v\d+[.]\d+')
      - name: Publish release to GitHub registry
        env:
          GITHUB_COMMIT_MESSAGE: ${{ github.event.head_commit.message }}
        run: docker push ghcr.io/lowlighter/metrics:$(echo $GITHUB_COMMIT_MESSAGE | grep -Po 'v\d+[.]\d+')
      - name: Tag docker image (latest)
        run: docker tag ghcr.io/lowlighter/metrics:master ghcr.io/lowlighter/metrics:latest
      - name: Publish latest to GitHub registry
        run: docker push ghcr.io/lowlighter/metrics:latest
 
  # Rebase latest branch on master
  update-latest:
    timeout-minutes: 30
    name: Rebase latest on master
    runs-on: latchkey-small
    needs: [ docker-release ]
    steps:
      - name: Checkout repository
        uses: actions/checkout@v3
        with:
          fetch-depth: 0
      - name: Setup NodeJS
        uses: actions/setup-node@v3
        with:
          cache: 'npm'
          node-version: 20
      - name: Setup metrics
        run: npm ci
      - name: Pull
        run: git pull
      - name: Checkout latest
        run: git checkout latest
      - name: Rebase latest on master
        run: git merge master
      - name: Push latest
        run: git push origin latest
 
  # Test lowlighter/metrics@latest
  action-latest-test:
    timeout-minutes: 30
    name: Test lowlighter/metrics@latest
    runs-on: latchkey-small
    needs: [ update-latest ]
    steps:
      - name: Run tests
        uses: lowlighter/metrics@latest
        with:
          token: MOCKED_TOKEN
          plugins_errors_fatal: yes
          dryrun: yes
          use_mocked_data: yes
          verify: yes
 
  # Deploy to metrics.lecoq.io
  deploy-latest:
    timeout-minutes: 30
    name: Deploy lowlighter/metrics@latest
    runs-on: latchkey-small
    needs: [ docker-release, deploy-master ]
    steps:
      - name: Deploy web instance
        run: "curl -X POST -H 'Authorization: ${{ secrets.WEB_DEPLOY_TOKEN }}' https://metrics.lecoq.io/.control/stop"
      - name: Wait for redeploy
        run: sleep 120
      - name: Check deployment status
        run: "curl --retry 15 --retry-delay 60 https://metrics.lecoq.io/.version"
 
  # Publish GitHub release
  publish-release:
    timeout-minutes: 30
    name: Publish release
    runs-on: latchkey-small
    needs: [ action-latest-test, deploy-latest ]
    steps:
      - name: Checkout repository
        uses: actions/checkout@v3
      - name: Setup NodeJS
        uses: actions/setup-node@v3
        with:
          cache: 'npm'
          node-version: 20
      - name: Setup metrics
        run: npm ci
      - name: Publish release
        run: node .github/scripts/release.mjs
        env:
          GITHUB_TOKEN: ${{ github.token }}
          GITHUB_REPOSITORY: ${{ github.repository }}
          GITHUB_COMMIT_MESSAGE: ${{ github.event.head_commit.message }}

What changed

Run on Latchkey managed runners with one line (runs-on), which apply the fixes below automatically and self-heal transient failures. This example uses latchkey-small; pick the runner size that fits the job.
Cancel superseded runs when a branch or PR gets a newer push.
Cache dependency installs on the setup step so they are served from cache.
Add a job timeout so a hung step cannot burn hours of runner time.

1 third-party action is referenced by a movable tag. Pin it to the commit SHA (Latchkey resolves and applies this automatically) so a repointed tag cannot change what runs.

What Latchkey heals here

This workflow has steps that commonly fail on transient issues (network, registries, flaky browsers). On Latchkey managed runners they are detected, retried, and self-healed instead of failing your build:

Dependency installs
Container pulls and builds

This workflow runs 14 jobs per trigger. On Latchkey the same minutes cost up to 58% less than GitHub-hosted, with zero queue time.

Actions used in this workflow

actions/checkout actions/setup-node