BDD Interop Integration Tests workflow (openwallet-foundation/acapy)
The BDD Interop Integration Tests workflow from openwallet-foundation/acapy, explained and optimized by Latchkey.
CI health: A - excellent
Point runs-on at Latchkey and get job timeouts, self-healing for flaky steps, and up to 58% lower cost, applied automatically.
What it does
This is the BDD Interop Integration Tests workflow from the openwallet-foundation/acapy repository, a real project running GitHub Actions. It is shown here with attribution under its Apache-2.0 license.
Below, Latchkey shows a faster, safer version produced by its optimization engine.
The workflow
name: BDD Interop Integration Tests
on:
schedule:
- cron: '0 0 * * *'
workflow_dispatch:
pull_request:
branches:
- main
types: [opened, synchronize, reopened, ready_for_review]
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
defaults:
run:
shell: bash
jobs:
test:
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: read
checks: write
# Run on openwallet-foundation and non-draft PRs or on non-PR events
if: (github.repository == 'openwallet-foundation/acapy') && ((github.event_name == 'pull_request' && github.event.pull_request.draft == false) || (github.event_name != 'pull_request'))
outputs:
is_release: ${{ steps.check_if_release.outputs.is_release }}
steps:
- name: checkout-acapy
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with:
fetch-depth: 0
- name: Check changed files
id: check-changed-files
uses: tj-actions/changed-files@9426d40962ed5378910ee2e21d5f8c6fcbf2dd96 # v47.0.6
with:
files_yaml: |
src:
- acapy_agent/**/*
- poetry.lock
- pyproject.toml
- docker/*
- name: Check if src files changed
id: check-if-src-changed
run: |
if [ "${{ steps.check-changed-files.outputs.src_any_changed }}" != "true" ] && [ '${{ github.event_name }}' == 'pull_request' ]; then
echo "No src files changed..."
echo run_tests=false >> $GITHUB_OUTPUT
fi
- name: Check if PR is a release
if: steps.check-if-src-changed.outputs.run_tests != 'false'
uses: ./.github/actions/is-release
id: check_if_release
- name: Request GitHub API for PR data
if: steps.check-if-src-changed.outputs.run_tests != 'false'
uses: octokit/request-action@b66ca9cd744693e9c29b490fc08831ab66c4a970 # v2.x
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
id: get_pr_data
with:
route: GET /repos/${{ github.event.repository.full_name }}/pulls/${{ github.event.number }}
- name: Prepare Interop Tests
if: (steps.check-if-src-changed.outputs.run_tests != 'false' || steps.check_if_release.outputs.is_release == 'true' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch')
run: |
# Get AATH
git clone https://github.com/openwallet-foundation/owl-agent-test-harness.git
# Change fork and branch for pull requests
if [ ${{ github.event_name }} == 'pull_request' ]; then
echo ${{ fromJson(steps.get_pr_data.outputs.data).head.repo.html_url }}
echo ${{ fromJson(steps.get_pr_data.outputs.data).head.ref }}
sed -i 's|@git+https://github.com/openwallet-foundation/acapy@main|@git+${{ fromJson(steps.get_pr_data.outputs.data).head.repo.html_url }}@${{ fromJson(steps.get_pr_data.outputs.data).head.ref }}|g' ./owl-agent-test-harness/aries-backchannels/acapy/requirements-main.txt
fi
cat owl-agent-test-harness/aries-backchannels/acapy/requirements-main.txt
cd owl-agent-test-harness
./manage build -a acapy-main
# Temporarily exclude BBS+ present-proof (limit_disclosure) scenarios until prover/verifier handling is fixed
- name: Run PR Interop Tests Indy
if: (steps.check_if_release.outputs.is_release != 'true' && github.event_name == 'pull_request' && steps.check-if-src-changed.outputs.run_tests != 'false')
run: |
cd owl-agent-test-harness
LEDGER_TIMEOUT=180 NO_TTY=1 LEDGER_URL_CONFIG=https://test.bcovrin.vonx.io TAILS_SERVER_URL_CONFIG=https://tails-test.vonx.io ./manage run -d acapy-main -t @critical -t ~@wip -t ~@T004-RFC0211 -t ~@DidMethod_orb -t ~@Transport_NoHttpOutbound -t ~@Anoncreds -t ~@ProofType_BbsBlsSignature2020 2>&1 | tee output.txt
# Temporarily exclude BBS+ present-proof (limit_disclosure) scenarios until prover/verifier handling is fixed
- name: Run Release or Nightly Interop Tests Indy
if: (steps.check_if_release.outputs.is_release == 'true' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' && steps.check-if-src-changed.outputs.run_tests != 'false')
run: |
cd owl-agent-test-harness
LEDGER_TIMEOUT=180 NO_TTY=1 LEDGER_URL_CONFIG=https://test.bcovrin.vonx.io TAILS_SERVER_URL_CONFIG=https://tails-test.vonx.io ./manage run -d acapy-main -t @critical -t ~@wip -t ~@T004-RFC0211 -t ~@DidMethod_orb -t ~@Transport_NoHttpOutbound -t ~@Anoncreds -t ~@ProofType_BbsBlsSignature2020 2>&1 | tee output.txt
# Temporarily exclude BBS+ present-proof (limit_disclosure) scenarios until prover/verifier handling is fixed
- name: Run Release or Nightly Interop Tests AnonCreds
if: (steps.check_if_release.outputs.is_release == 'true' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' && steps.check-if-src-changed.outputs.run_tests != 'false')
run: |
cd owl-agent-test-harness
LEDGER_TIMEOUT=180 BACKCHANNEL_EXTRA_acapy_main="{\"wallet-type\":\"askar-anoncreds\"}" NO_TTY=1 LEDGER_URL_CONFIG=https://test.bcovrin.vonx.io TAILS_SERVER_URL_CONFIG=https://tails-test.vonx.io ./manage run -d acapy-main -t @AcceptanceTest -t ~@wip -t ~@T004-RFC0211 -t ~@DidMethod_orb -t ~@Transport_NoHttpOutbound -t ~@Indy -t ~@CredFormat_Indy -t ~@ProofType_BbsBlsSignature2020 2>&1 | tee output.txt
- name: Check If Tests Failed
if: always() && (steps.check-if-src-changed.outputs.run_tests != 'false')
run: |
cd owl-agent-test-harness
if [ -f output.txt ]; then
cat output.txt
if grep "Failing scenarios:" output.txt; then
echo "Tests failed"
exit 1
else
echo "Tests passed"
exit 0
fi
else
echo "No output file found - tests may not have run"
exit 1
fi
The same workflow, on Latchkey
Removes redundant runs and caps runaway jobs. Added and changed lines are highlighted.
name: BDD Interop Integration Tests on: schedule: - cron: '0 0 * * *' workflow_dispatch: pull_request: branches: - main types: [opened, synchronize, reopened, ready_for_review] concurrency: group: ${{ github.workflow }}-${{ github.ref }} cancel-in-progress: true defaults: run: shell: bash jobs: test: timeout-minutes: 30 runs-on: latchkey-small permissions: contents: read pull-requests: read checks: write # Run on openwallet-foundation and non-draft PRs or on non-PR events if: (github.repository == 'openwallet-foundation/acapy') && ((github.event_name == 'pull_request' && github.event.pull_request.draft == false) || (github.event_name != 'pull_request')) outputs: is_release: ${{ steps.check_if_release.outputs.is_release }} steps: - name: checkout-acapy uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0 with: fetch-depth: 0 - name: Check changed files id: check-changed-files uses: tj-actions/changed-files@9426d40962ed5378910ee2e21d5f8c6fcbf2dd96 # v47.0.6 with: files_yaml: | src: - acapy_agent/**/* - poetry.lock - pyproject.toml - docker/* - name: Check if src files changed id: check-if-src-changed run: | if [ "${{ steps.check-changed-files.outputs.src_any_changed }}" != "true" ] && [ '${{ github.event_name }}' == 'pull_request' ]; then echo "No src files changed..." echo run_tests=false >> $GITHUB_OUTPUT fi - name: Check if PR is a release if: steps.check-if-src-changed.outputs.run_tests != 'false' uses: ./.github/actions/is-release id: check_if_release - name: Request GitHub API for PR data if: steps.check-if-src-changed.outputs.run_tests != 'false' uses: octokit/request-action@b66ca9cd744693e9c29b490fc08831ab66c4a970 # v2.x env: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} id: get_pr_data with: route: GET /repos/${{ github.event.repository.full_name }}/pulls/${{ github.event.number }} - name: Prepare Interop Tests if: (steps.check-if-src-changed.outputs.run_tests != 'false' || steps.check_if_release.outputs.is_release == 'true' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch') run: | # Get AATH git clone https://github.com/openwallet-foundation/owl-agent-test-harness.git # Change fork and branch for pull requests if [ ${{ github.event_name }} == 'pull_request' ]; then echo ${{ fromJson(steps.get_pr_data.outputs.data).head.repo.html_url }} echo ${{ fromJson(steps.get_pr_data.outputs.data).head.ref }} sed -i 's|@git+https://github.com/openwallet-foundation/acapy@main|@git+${{ fromJson(steps.get_pr_data.outputs.data).head.repo.html_url }}@${{ fromJson(steps.get_pr_data.outputs.data).head.ref }}|g' ./owl-agent-test-harness/aries-backchannels/acapy/requirements-main.txt fi cat owl-agent-test-harness/aries-backchannels/acapy/requirements-main.txt cd owl-agent-test-harness ./manage build -a acapy-main # Temporarily exclude BBS+ present-proof (limit_disclosure) scenarios until prover/verifier handling is fixed - name: Run PR Interop Tests Indy if: (steps.check_if_release.outputs.is_release != 'true' && github.event_name == 'pull_request' && steps.check-if-src-changed.outputs.run_tests != 'false') run: | cd owl-agent-test-harness LEDGER_TIMEOUT=180 NO_TTY=1 LEDGER_URL_CONFIG=https://test.bcovrin.vonx.io TAILS_SERVER_URL_CONFIG=https://tails-test.vonx.io ./manage run -d acapy-main -t @critical -t ~@wip -t ~@T004-RFC0211 -t ~@DidMethod_orb -t ~@Transport_NoHttpOutbound -t ~@Anoncreds -t ~@ProofType_BbsBlsSignature2020 2>&1 | tee output.txt # Temporarily exclude BBS+ present-proof (limit_disclosure) scenarios until prover/verifier handling is fixed - name: Run Release or Nightly Interop Tests Indy if: (steps.check_if_release.outputs.is_release == 'true' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' && steps.check-if-src-changed.outputs.run_tests != 'false') run: | cd owl-agent-test-harness LEDGER_TIMEOUT=180 NO_TTY=1 LEDGER_URL_CONFIG=https://test.bcovrin.vonx.io TAILS_SERVER_URL_CONFIG=https://tails-test.vonx.io ./manage run -d acapy-main -t @critical -t ~@wip -t ~@T004-RFC0211 -t ~@DidMethod_orb -t ~@Transport_NoHttpOutbound -t ~@Anoncreds -t ~@ProofType_BbsBlsSignature2020 2>&1 | tee output.txt # Temporarily exclude BBS+ present-proof (limit_disclosure) scenarios until prover/verifier handling is fixed - name: Run Release or Nightly Interop Tests AnonCreds if: (steps.check_if_release.outputs.is_release == 'true' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' && steps.check-if-src-changed.outputs.run_tests != 'false') run: | cd owl-agent-test-harness LEDGER_TIMEOUT=180 BACKCHANNEL_EXTRA_acapy_main="{\"wallet-type\":\"askar-anoncreds\"}" NO_TTY=1 LEDGER_URL_CONFIG=https://test.bcovrin.vonx.io TAILS_SERVER_URL_CONFIG=https://tails-test.vonx.io ./manage run -d acapy-main -t @AcceptanceTest -t ~@wip -t ~@T004-RFC0211 -t ~@DidMethod_orb -t ~@Transport_NoHttpOutbound -t ~@Indy -t ~@CredFormat_Indy -t ~@ProofType_BbsBlsSignature2020 2>&1 | tee output.txt - name: Check If Tests Failed if: always() && (steps.check-if-src-changed.outputs.run_tests != 'false') run: | cd owl-agent-test-harness if [ -f output.txt ]; then cat output.txt if grep "Failing scenarios:" output.txt; then echo "Tests failed" exit 1 else echo "Tests passed" exit 0 fi else echo "No output file found - tests may not have run" exit 1 fi
What changed
- Run on Latchkey managed runners with one line (
runs-on), which apply the fixes below automatically and self-heal transient failures. This example useslatchkey-small; pick the runner size that fits the job. - Add a job timeout so a hung step cannot burn hours of runner time.
This workflow runs 1 job per trigger. On Latchkey the same minutes cost up to 58% less than GitHub-hosted, with zero queue time.