docker workflow (microsoft/maro)
The docker workflow from microsoft/maro, explained and optimized by Latchkey.
CI health: D - needs work
Point runs-on at Latchkey and get caching, run de-duplication, job timeouts, self-healing for flaky steps, and up to 58% lower cost, applied automatically.
What it does
This is the docker workflow from the microsoft/maro repository, a real project running GitHub Actions. It is shown here with attribution under its MIT license.
Below, Latchkey shows a faster, safer version produced by its optimization engine.
The workflow
# This is a basic workflow to help you get started with Actions
name: docker
# Controls when the action will run. Triggers the workflow on push or pull request
# events but only for the master branch
on:
push:
tags:
- "maro-[0-9]+\\.[0-9]+\\.[0-9]+"
workflow_dispatch:
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
# This workflow contains a single job called "build"
build:
# The type of runner that the job will run on
runs-on: ubuntu-latest
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- uses: actions/checkout@v2
- name: Set up Python 3.7
uses: actions/setup-python@v2
with:
python-version: 3.7
- name: Build image
run: |
pip install -r ./maro/requirements.build.txt
cython ./maro/backends/backend.pyx ./maro/backends/np_backend.pyx ./maro/backends/raw_backend.pyx ./maro/backends/frame.pyx --cplus -3 -E NODES_MEMORY_LAYOUT=ONE_BLOCK -X embedsignature=True
cat ./maro/__misc__.py | grep __version__ | egrep -o [0-9].[0-9].[0-9,a-z]+ | { read version; docker build -f ./docker_files/cpu.playground.df . -t ${{ secrets.DOCKER_HUB_USERNAME }}/maro:cpu -t ${{ secrets.DOCKER_HUB_USERNAME }}/maro:latest -t ${{ secrets.DOCKER_HUB_USERNAME }}/maro:cpu-$version; }
- name: Login docker hub
run: |
docker login --username ${{ secrets.DOCKER_HUB_USERNAME }} --password ${{ secrets.DOCKER_HUB_PASSWORD }}
- name: Push image
run: |
docker push ${{ secrets.DOCKER_HUB_USERNAME }}/maro
The same workflow, on Latchkey
Estimated ~20% faster on cache hits, plus fewer wasted runs and a safer supply chain. Added and changed lines are highlighted.
# This is a basic workflow to help you get started with Actions name: docker # Controls when the action will run. Triggers the workflow on push or pull request # events but only for the master branch on: push: tags: - "maro-[0-9]+\\.[0-9]+\\.[0-9]+" workflow_dispatch: # A workflow run is made up of one or more jobs that can run sequentially or in parallel concurrency: group: ${{ github.workflow }}-${{ github.ref }} cancel-in-progress: true jobs: # This workflow contains a single job called "build" build: timeout-minutes: 30 # The type of runner that the job will run on runs-on: latchkey-small # Steps represent a sequence of tasks that will be executed as part of the job steps: # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it - uses: actions/checkout@v2 - name: Set up Python 3.7 uses: actions/setup-python@v2 with: cache: 'pip' python-version: 3.7 - name: Build image run: | pip install -r ./maro/requirements.build.txt cython ./maro/backends/backend.pyx ./maro/backends/np_backend.pyx ./maro/backends/raw_backend.pyx ./maro/backends/frame.pyx --cplus -3 -E NODES_MEMORY_LAYOUT=ONE_BLOCK -X embedsignature=True cat ./maro/__misc__.py | grep __version__ | egrep -o [0-9].[0-9].[0-9,a-z]+ | { read version; docker build -f ./docker_files/cpu.playground.df . -t ${{ secrets.DOCKER_HUB_USERNAME }}/maro:cpu -t ${{ secrets.DOCKER_HUB_USERNAME }}/maro:latest -t ${{ secrets.DOCKER_HUB_USERNAME }}/maro:cpu-$version; } - name: Login docker hub run: | docker login --username ${{ secrets.DOCKER_HUB_USERNAME }} --password ${{ secrets.DOCKER_HUB_PASSWORD }} - name: Push image run: | docker push ${{ secrets.DOCKER_HUB_USERNAME }}/maro
What changed
- Run on Latchkey managed runners with one line (
runs-on), which apply the fixes below automatically and self-heal transient failures. This example useslatchkey-small; pick the runner size that fits the job. - Cancel superseded runs when a branch or PR gets a newer push.
- Cache dependency installs on the setup step so they are served from cache.
- Add a job timeout so a hung step cannot burn hours of runner time.
What Latchkey heals here
This workflow has steps that commonly fail on transient issues (network, registries, flaky browsers). On Latchkey managed runners they are detected, retried, and self-healed instead of failing your build:
- Dependency installs
- Container pulls and builds
This workflow runs 1 job per trigger. On Latchkey the same minutes cost up to 58% less than GitHub-hosted, with zero queue time.