Skip to content
Latchkey

build workflow (hyz-xmaster/swa_object_detection)

The build workflow from hyz-xmaster/swa_object_detection, explained and optimized by Latchkey.

F

CI health: F - at risk

Point runs-on at Latchkey and get caching, run de-duplication, job timeouts, SHA-pinned actions, self-healing for flaky steps, and up to 58% lower cost, applied automatically.

Grade your own workflow free or run it on Latchkey →
Source: hyz-xmaster/swa_object_detection.github/workflows/build.ymlLicense Apache-2.0View source

What it does

This is the build workflow from the hyz-xmaster/swa_object_detection repository, a real project running GitHub Actions. It is shown here with attribution under its Apache-2.0 license.

Below, Latchkey shows a faster, safer version produced by its optimization engine.

The workflow

workflow (.yml)
name: build

on: [push, pull_request]

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Set up Python 3.7
        uses: actions/setup-python@v2
        with:
          python-version: 3.7
      - name: Install pre-commit hook
        run: |
          pip install pre-commit
          pre-commit install
      - name: Linting
        run: pre-commit run --all-files
      - name: Check docstring coverage
        run: |
          pip install interrogate
          interrogate -v --ignore-init-method --ignore-module --ignore-nested-functions --ignore-regex "__repr__" --fail-under 80 mmdet

  build_cpu:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: [3.7]
        torch: [1.3.1, 1.5.1, 1.6.0]
        include:
          - torch: 1.3.1
            torchvision: 0.4.2
            mmcv: "latest+torch1.3.0+cpu"
          - torch: 1.5.1
            torchvision: 0.6.1
            mmcv: "latest+torch1.5.0+cpu"
          - torch: 1.6.0
            torchvision: 0.7.0
            mmcv: "latest+torch1.6.0+cpu"
    steps:
      - uses: actions/checkout@v2
      - name: Set up Python ${{ matrix.python-version }}
        uses: actions/setup-python@v2
        with:
          python-version: ${{ matrix.python-version }}
      - name: Install Pillow
        run: pip install Pillow==6.2.2
        if: ${{matrix.torchvision == '0.4.2'}}
      - name: Install PyTorch
        run: pip install torch==${{matrix.torch}}+cpu torchvision==${{matrix.torchvision}}+cpu -f https://download.pytorch.org/whl/torch_stable.html
      - name: Install MMCV
        run: |
          pip install mmcv-full==${{matrix.mmcv}} -f https://download.openmmlab.com/mmcv/dist/index.html --use-deprecated=legacy-resolver
          python -c 'import mmcv; print(mmcv.__version__)'
      - name: Install unittest dependencies
        run: pip install -r requirements/tests.txt -r requirements/optional.txt
      - name: Build and install
        run: rm -rf .eggs && pip install -e .
      - name: Run unittests and generate coverage report
        run: |
          coverage run --branch --source mmdet -m pytest tests/
          coverage xml
          coverage report -m

  build_cuda:
    runs-on: ubuntu-latest

    env:
      CUDA: 10.1.105-1
      CUDA_SHORT: 10.1
      UBUNTU_VERSION: ubuntu1804
    strategy:
      matrix:
        python-version: [3.7]
        torch: [1.3.1, 1.5.1+cu101, 1.6.0+cu101]
        include:
          - torch: 1.3.1
            torchvision: 0.4.2
            mmcv: "latest+torch1.3.0+cu101"
          - torch: 1.5.1+cu101
            torchvision: 0.6.1+cu101
            mmcv: "latest+torch1.5.0+cu101"
          - torch: 1.6.0+cu101
            torchvision: 0.7.0+cu101
            mmcv: "latest+torch1.6.0+cu101"
          - torch: 1.6.0+cu101
            torchvision: 0.7.0+cu101
            mmcv: "latest+torch1.6.0+cu101"
            python-version: 3.6
          - torch: 1.6.0+cu101
            torchvision: 0.7.0+cu101
            mmcv: "latest+torch1.6.0+cu101"
            python-version: 3.8

    steps:
      - uses: actions/checkout@v2
      - name: Set up Python ${{ matrix.python-version }}
        uses: actions/setup-python@v2
        with:
          python-version: ${{ matrix.python-version }}
      - name: Install CUDA
        run: |
          export INSTALLER=cuda-repo-${UBUNTU_VERSION}_${CUDA}_amd64.deb
          wget http://developer.download.nvidia.com/compute/cuda/repos/${UBUNTU_VERSION}/x86_64/${INSTALLER}
          sudo dpkg -i ${INSTALLER}
          wget https://developer.download.nvidia.com/compute/cuda/repos/${UBUNTU_VERSION}/x86_64/7fa2af80.pub
          sudo apt-key add 7fa2af80.pub
          sudo apt update -qq
          sudo apt install -y cuda-${CUDA_SHORT/./-} cuda-cufft-dev-${CUDA_SHORT/./-}
          sudo apt clean
          export CUDA_HOME=/usr/local/cuda-${CUDA_SHORT}
          export LD_LIBRARY_PATH=${CUDA_HOME}/lib64:${CUDA_HOME}/include:${LD_LIBRARY_PATH}
          export PATH=${CUDA_HOME}/bin:${PATH}
      - name: Install Pillow
        run: pip install Pillow==6.2.2
        if: ${{matrix.torchvision < 0.5}}
      - name: Install PyTorch
        run: pip install torch==${{matrix.torch}} torchvision==${{matrix.torchvision}} -f https://download.pytorch.org/whl/torch_stable.html
      - name: Install mmdet dependencies
        run: |
          pip install mmcv-full==${{matrix.mmcv}} -f https://download.openmmlab.com/mmcv/dist/index.html --use-deprecated=legacy-resolver
          pip install -r requirements.txt
          python -c 'import mmcv; print(mmcv.__version__)'
      - name: Build and install
        run: |
          rm -rf .eggs
          python setup.py check -m -s
          TORCH_CUDA_ARCH_LIST=7.0 pip install .
      - name: Run unittests and generate coverage report
        run: |
          coverage run --branch --source mmdet -m pytest tests/
          coverage xml
          coverage report -m
      - name: Upload coverage to Codecov
        uses: codecov/codecov-action@v1.0.10
        with:
          file: ./coverage.xml
          flags: unittests
          env_vars: OS,PYTHON
          name: codecov-umbrella
          fail_ci_if_error: false

The same workflow, on Latchkey

Estimated ~20% faster on cache hits, plus fewer wasted runs and a safer supply chain. Added and changed lines are highlighted.

name: build
 
on: [push, pull_request]
 
concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true
 
jobs:
  lint:
    timeout-minutes: 30
    runs-on: latchkey-small
    steps:
      - uses: actions/checkout@v2
      - name: Set up Python 3.7
        uses: actions/setup-python@v2
        with:
          cache: 'pip'
          python-version: 3.7
      - name: Install pre-commit hook
        run: |
          pip install pre-commit
          pre-commit install
      - name: Linting
        run: pre-commit run --all-files
      - name: Check docstring coverage
        run: |
          pip install interrogate
          interrogate -v --ignore-init-method --ignore-module --ignore-nested-functions --ignore-regex "__repr__" --fail-under 80 mmdet
 
  build_cpu:
    timeout-minutes: 30
    runs-on: latchkey-small
    strategy:
      matrix:
        python-version: [3.7]
        torch: [1.3.1, 1.5.1, 1.6.0]
        include:
          - torch: 1.3.1
            torchvision: 0.4.2
            mmcv: "latest+torch1.3.0+cpu"
          - torch: 1.5.1
            torchvision: 0.6.1
            mmcv: "latest+torch1.5.0+cpu"
          - torch: 1.6.0
            torchvision: 0.7.0
            mmcv: "latest+torch1.6.0+cpu"
    steps:
      - uses: actions/checkout@v2
      - name: Set up Python ${{ matrix.python-version }}
        uses: actions/setup-python@v2
        with:
          cache: 'pip'
          python-version: ${{ matrix.python-version }}
      - name: Install Pillow
        run: pip install Pillow==6.2.2
        if: ${{matrix.torchvision == '0.4.2'}}
      - name: Install PyTorch
        run: pip install torch==${{matrix.torch}}+cpu torchvision==${{matrix.torchvision}}+cpu -f https://download.pytorch.org/whl/torch_stable.html
      - name: Install MMCV
        run: |
          pip install mmcv-full==${{matrix.mmcv}} -f https://download.openmmlab.com/mmcv/dist/index.html --use-deprecated=legacy-resolver
          python -c 'import mmcv; print(mmcv.__version__)'
      - name: Install unittest dependencies
        run: pip install -r requirements/tests.txt -r requirements/optional.txt
      - name: Build and install
        run: rm -rf .eggs && pip install -e .
      - name: Run unittests and generate coverage report
        run: |
          coverage run --branch --source mmdet -m pytest tests/
          coverage xml
          coverage report -m
 
  build_cuda:
    timeout-minutes: 30
    runs-on: latchkey-small
 
    env:
      CUDA: 10.1.105-1
      CUDA_SHORT: 10.1
      UBUNTU_VERSION: ubuntu1804
    strategy:
      matrix:
        python-version: [3.7]
        torch: [1.3.1, 1.5.1+cu101, 1.6.0+cu101]
        include:
          - torch: 1.3.1
            torchvision: 0.4.2
            mmcv: "latest+torch1.3.0+cu101"
          - torch: 1.5.1+cu101
            torchvision: 0.6.1+cu101
            mmcv: "latest+torch1.5.0+cu101"
          - torch: 1.6.0+cu101
            torchvision: 0.7.0+cu101
            mmcv: "latest+torch1.6.0+cu101"
          - torch: 1.6.0+cu101
            torchvision: 0.7.0+cu101
            mmcv: "latest+torch1.6.0+cu101"
            python-version: 3.6
          - torch: 1.6.0+cu101
            torchvision: 0.7.0+cu101
            mmcv: "latest+torch1.6.0+cu101"
            python-version: 3.8
 
    steps:
      - uses: actions/checkout@v2
      - name: Set up Python ${{ matrix.python-version }}
        uses: actions/setup-python@v2
        with:
          cache: 'pip'
          python-version: ${{ matrix.python-version }}
      - name: Install CUDA
        run: |
          export INSTALLER=cuda-repo-${UBUNTU_VERSION}_${CUDA}_amd64.deb
          wget http://developer.download.nvidia.com/compute/cuda/repos/${UBUNTU_VERSION}/x86_64/${INSTALLER}
          sudo dpkg -i ${INSTALLER}
          wget https://developer.download.nvidia.com/compute/cuda/repos/${UBUNTU_VERSION}/x86_64/7fa2af80.pub
          sudo apt-key add 7fa2af80.pub
          sudo apt update -qq
          sudo apt install -y cuda-${CUDA_SHORT/./-} cuda-cufft-dev-${CUDA_SHORT/./-}
          sudo apt clean
          export CUDA_HOME=/usr/local/cuda-${CUDA_SHORT}
          export LD_LIBRARY_PATH=${CUDA_HOME}/lib64:${CUDA_HOME}/include:${LD_LIBRARY_PATH}
          export PATH=${CUDA_HOME}/bin:${PATH}
      - name: Install Pillow
        run: pip install Pillow==6.2.2
        if: ${{matrix.torchvision < 0.5}}
      - name: Install PyTorch
        run: pip install torch==${{matrix.torch}} torchvision==${{matrix.torchvision}} -f https://download.pytorch.org/whl/torch_stable.html
      - name: Install mmdet dependencies
        run: |
          pip install mmcv-full==${{matrix.mmcv}} -f https://download.openmmlab.com/mmcv/dist/index.html --use-deprecated=legacy-resolver
          pip install -r requirements.txt
          python -c 'import mmcv; print(mmcv.__version__)'
      - name: Build and install
        run: |
          rm -rf .eggs
          python setup.py check -m -s
          TORCH_CUDA_ARCH_LIST=7.0 pip install .
      - name: Run unittests and generate coverage report
        run: |
          coverage run --branch --source mmdet -m pytest tests/
          coverage xml
          coverage report -m
      - name: Upload coverage to Codecov
        uses: codecov/codecov-action@v1.0.10
        with:
          file: ./coverage.xml
          flags: unittests
          env_vars: OS,PYTHON
          name: codecov-umbrella
          fail_ci_if_error: false
 

What changed

1 third-party action is referenced by a movable tag. Pin it to the commit SHA (Latchkey resolves and applies this automatically) so a repointed tag cannot change what runs.

What Latchkey heals here

This workflow has steps that commonly fail on transient issues (network, registries, flaky browsers). On Latchkey managed runners they are detected, retried, and self-healed instead of failing your build:

This workflow runs 3 jobs (7 with the matrix expanded) per trigger. On Latchkey the same minutes cost up to 58% less than GitHub-hosted, with zero queue time.

Actions used in this workflow