(legacy dev) Custard CI workflow (GoogleCloudPlatform/nodejs-docs-samples)
The (legacy dev) Custard CI workflow from GoogleCloudPlatform/nodejs-docs-samples, explained and optimized by Latchkey.
CI health: C - fair
Point runs-on at Latchkey and get caching, run de-duplication, self-healing for flaky steps, and up to 58% lower cost, applied automatically.
What it does
This is the (legacy dev) Custard CI workflow from the GoogleCloudPlatform/nodejs-docs-samples repository, a real project running GitHub Actions. It is shown here with attribution under its Apache-2.0 license.
Below, Latchkey shows a faster, safer version produced by its optimization engine.
The workflow
# Copyright 2024 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
name: (legacy dev) Custard CI
on:
push:
branches:
- main
pull_request:
# schedule:
# # https://crontab.guru/#0_12_*_*_0
# - cron: 0 12 * * 0 # At 12:00 on Sunday
env:
GO_VERSION: ^1.22.0
jobs:
affected:
name: (legacy) Finding affected tests
runs-on: ubuntu-latest
timeout-minutes: 2
outputs:
nodejs-paths: ${{ steps.nodejs.outputs.paths }}
nodejs-setups: ${{ steps.nodejs.outputs.setups }}
steps:
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5
with:
fetch-depth: 0
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5
with:
repository: GoogleCloudPlatform/cloud-samples-tools
ref: v0.3.2
path: cloud-samples-tools
- name: Create `bin` directory for cloud-samples-tools binaries
run: mkdir bin
working-directory: cloud-samples-tools
- uses: actions/setup-go@d35c59abb061a4a6fb18e82ac0862c26744d6ab5 # v5
with:
go-version: ${{ env.GO_VERSION }}
- name: Build Custard (from cloud-samples-tools)
run: go build -o ../bin -v ./...
working-directory: cloud-samples-tools/custard
- name: Get diffs
run: git --no-pager diff --name-only HEAD origin/main | tee diffs.txt
- name: Find Node.js affected packages
id: nodejs
run: |
echo "paths=$(./cloud-samples-tools/bin/custard affected .github/config/nodejs-dev.jsonc diffs.txt paths.txt)" >> $GITHUB_OUTPUT
cat paths.txt
echo "setups=$(./cloud-samples-tools/bin/custard setup-files .github/config/nodejs-dev.jsonc paths.txt)" >> $GITHUB_OUTPUT
nodejs-test:
if: github.event.pull_request && github.event.pull_request.head.repo.fork == false
name: (legacy) test
needs: affected
runs-on: ubuntu-latest
timeout-minutes: 120 # 2 hours hard limit
permissions:
id-token: write # needed for google-github-actions/auth
strategy:
fail-fast: false
matrix:
path: ${{ fromJson(github.event_name == 'pull_request' && needs.affected.outputs.nodejs-paths || '[]') }}
env:
GOOGLE_SAMPLES_PROJECT: long-door-651
GOOGLE_SERVICE_ACCOUNT: kokoro-system-test@long-door-651.iam.gserviceaccount.com
CI_SETUP: ${{ toJson(fromJson(needs.affected.outputs.nodejs-setups)[matrix.path])}}
steps:
- name: CI Setup
run: echo "${{ env.CI_SETUP }}"
- uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5
with:
ref: ${{ github.event.pull_request.head.sha }}
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
with:
node-version: ${{ fromJson(env.CI_SETUP).node-version }}
- uses: google-github-actions/auth@7c6bc770dae815cd3e89ee6cdf493a5fab2cc093 # v3
id: auth
with:
project_id: ${{ env.GOOGLE_SAMPLES_PROJECT }}
workload_identity_provider: projects/1046198160504/locations/global/workloadIdentityPools/github-actions-pool/providers/github-actions-provider
service_account: ${{ env.GOOGLE_SERVICE_ACCOUNT }}
access_token_lifetime: 600s # 10 minutes
token_format: 'id_token'
id_token_audience: 'https://action.test/' # service must have this custom audience
id_token_include_email: true
- name: Export environment variables
uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7
id: vars
with:
script: |
const { default: setupVars } = await import('${{ github.workspace }}/.github/scripts/setup-vars.js');
const projectId = '${{ env.GOOGLE_SAMPLES_PROJECT }}';
const setup = JSON.parse(process.env.CI_SETUP);
const serviceAccount = '${{ env.GOOGLE_SERVICE_ACCOUNT }}';
const idToken = '${{ steps.auth.outputs.id_token }}';
return await setupVars({projectId, core, setup, serviceAccount, idToken})
- uses: google-github-actions/get-secretmanager-secrets@e5bb06c2ca53b244f978d33348d18317a7f263ce # v2
if: ${{ fromJson(steps.vars.outputs.result).secrets }}
with:
secrets: ${{ fromJson(steps.vars.outputs.result).secrets }}
export_to_environment: true
- name: Run tests for ${{ matrix.path }}
run: |
timeout ${{ fromJson(env.CI_SETUP).timeout-minutes }}m \
make test dir=${{ matrix.path }}
The same workflow, on Latchkey
Estimated ~20% faster on cache hits, plus fewer wasted runs and a safer supply chain. Added and changed lines are highlighted.
# Copyright 2024 Google LLC # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. # You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. name: (legacy dev) Custard CI on: push: branches: - main pull_request: # schedule: # # https://crontab.guru/#0_12_*_*_0 # - cron: 0 12 * * 0 # At 12:00 on Sunday env: GO_VERSION: ^1.22.0 concurrency: group: ${{ github.workflow }}-${{ github.ref }} cancel-in-progress: true jobs: affected: name: (legacy) Finding affected tests runs-on: latchkey-small timeout-minutes: 2 outputs: nodejs-paths: ${{ steps.nodejs.outputs.paths }} nodejs-setups: ${{ steps.nodejs.outputs.setups }} steps: - uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5 with: fetch-depth: 0 - uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5 with: repository: GoogleCloudPlatform/cloud-samples-tools ref: v0.3.2 path: cloud-samples-tools - name: Create `bin` directory for cloud-samples-tools binaries run: mkdir bin working-directory: cloud-samples-tools - uses: actions/setup-go@d35c59abb061a4a6fb18e82ac0862c26744d6ab5 # v5 with: go-version: ${{ env.GO_VERSION }} - name: Build Custard (from cloud-samples-tools) run: go build -o ../bin -v ./... working-directory: cloud-samples-tools/custard - name: Get diffs run: git --no-pager diff --name-only HEAD origin/main | tee diffs.txt - name: Find Node.js affected packages id: nodejs run: | echo "paths=$(./cloud-samples-tools/bin/custard affected .github/config/nodejs-dev.jsonc diffs.txt paths.txt)" >> $GITHUB_OUTPUT cat paths.txt echo "setups=$(./cloud-samples-tools/bin/custard setup-files .github/config/nodejs-dev.jsonc paths.txt)" >> $GITHUB_OUTPUT nodejs-test: if: github.event.pull_request && github.event.pull_request.head.repo.fork == false name: (legacy) test needs: affected runs-on: latchkey-small timeout-minutes: 120 # 2 hours hard limit permissions: id-token: write # needed for google-github-actions/auth strategy: fail-fast: false matrix: path: ${{ fromJson(github.event_name == 'pull_request' && needs.affected.outputs.nodejs-paths || '[]') }} env: GOOGLE_SAMPLES_PROJECT: long-door-651 GOOGLE_SERVICE_ACCOUNT: kokoro-system-test@long-door-651.iam.gserviceaccount.com CI_SETUP: ${{ toJson(fromJson(needs.affected.outputs.nodejs-setups)[matrix.path])}} steps: - name: CI Setup run: echo "${{ env.CI_SETUP }}" - uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5 with: ref: ${{ github.event.pull_request.head.sha }} - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4 with: cache: 'npm' node-version: ${{ fromJson(env.CI_SETUP).node-version }} - uses: google-github-actions/auth@7c6bc770dae815cd3e89ee6cdf493a5fab2cc093 # v3 id: auth with: project_id: ${{ env.GOOGLE_SAMPLES_PROJECT }} workload_identity_provider: projects/1046198160504/locations/global/workloadIdentityPools/github-actions-pool/providers/github-actions-provider service_account: ${{ env.GOOGLE_SERVICE_ACCOUNT }} access_token_lifetime: 600s # 10 minutes token_format: 'id_token' id_token_audience: 'https://action.test/' # service must have this custom audience id_token_include_email: true - name: Export environment variables uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7 id: vars with: script: | const { default: setupVars } = await import('${{ github.workspace }}/.github/scripts/setup-vars.js'); const projectId = '${{ env.GOOGLE_SAMPLES_PROJECT }}'; const setup = JSON.parse(process.env.CI_SETUP); const serviceAccount = '${{ env.GOOGLE_SERVICE_ACCOUNT }}'; const idToken = '${{ steps.auth.outputs.id_token }}'; return await setupVars({projectId, core, setup, serviceAccount, idToken}) - uses: google-github-actions/get-secretmanager-secrets@e5bb06c2ca53b244f978d33348d18317a7f263ce # v2 if: ${{ fromJson(steps.vars.outputs.result).secrets }} with: secrets: ${{ fromJson(steps.vars.outputs.result).secrets }} export_to_environment: true - name: Run tests for ${{ matrix.path }} run: | timeout ${{ fromJson(env.CI_SETUP).timeout-minutes }}m \ make test dir=${{ matrix.path }}
What changed
- Run on Latchkey managed runners with one line (
runs-on), which apply the fixes below automatically and self-heal transient failures. This example useslatchkey-small; pick the runner size that fits the job. - Cancel superseded runs when a branch or PR gets a newer push.
- Cache dependency installs on the setup step so they are served from cache.
This workflow runs 2 jobs per trigger. On Latchkey the same minutes cost up to 58% less than GitHub-hosted, with zero queue time.