Skip to content
Latchkey

Build Data workflow (2020PB/police-brutality)

The Build Data workflow from 2020PB/police-brutality, explained and optimized by Latchkey.

F

CI health: F - at risk

Point runs-on at Latchkey and get caching, run de-duplication, job timeouts, SHA-pinned actions, self-healing for flaky steps, and up to 58% lower cost, applied automatically.

Grade your own workflow free or run it on Latchkey →
Source: 2020PB/police-brutality.github/workflows/build_data_ci.ymlLicense MITView source

What it does

This is the Build Data workflow from the 2020PB/police-brutality repository, a real project running GitHub Actions. It is shown here with attribution under its MIT license.

Below, Latchkey shows a faster, safer version produced by its optimization engine.

The workflow

workflow (.yml)
# This is a basic workflow to help you get started with Actions

name: Build Data

# Controls when the action will run. Triggers the workflow on push or pull request
# events but only for the main branch
on:
  push:
    branches: [ main ]
  # Pausing the build on pull_request because it seems these cause a duplicate build which can fail.
  # I added this event because it seemed sometimes `push` was not causing a build.
  # So I'm leaving these comments until it's verified that `push` alone is good.
  #pull_request:
  #  types: [ closed ]
  #  branches: [ main ]

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
  # This workflow contains a single job called "build"
  build:
    # The type of runner that the job will run on
    runs-on: ubuntu-latest

    # Steps represent a sequence of tasks that will be executed as part of the job
    steps:
    # Checks the branch out under $GITHUB_WORKSPACE, so build_data can access it
    - uses: actions/checkout@v2

    - name: Set up Python 3.7
      uses: actions/setup-python@v2
      with:
        # Semantic version range syntax or exact version of a Python version
        python-version: '3.7'

    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install -r tools/requirements.txt
        python -c "import nltk; nltk.download('wordnet')"

    - name: Run unit tests with pytest
      run: |
        cd tools
        pytest
        cd ..

    - name: Create json csv md files
      run: python tools/data_builder.py

    # Deploy data to local repo
    - name: Deploy
      uses: s0/git-publish-subdir-action@master
      env:
        REPO: self
        BRANCH: data_build
        FOLDER: tools/data_build
        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

The same workflow, on Latchkey

Estimated ~20% faster on cache hits, plus fewer wasted runs and a safer supply chain. Added and changed lines are highlighted.

# This is a basic workflow to help you get started with Actions
 
name: Build Data
 
# Controls when the action will run. Triggers the workflow on push or pull request
# events but only for the main branch
on:
  push:
    branches: [ main ]
  # Pausing the build on pull_request because it seems these cause a duplicate build which can fail.
  # I added this event because it seemed sometimes `push` was not causing a build.
  # So I'm leaving these comments until it's verified that `push` alone is good.
  #pull_request:
  #  types: [ closed ]
  #  branches: [ main ]
 
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true
 
jobs:
  # This workflow contains a single job called "build"
  build:
    timeout-minutes: 30
    # The type of runner that the job will run on
    runs-on: latchkey-small
 
    # Steps represent a sequence of tasks that will be executed as part of the job
    steps:
    # Checks the branch out under $GITHUB_WORKSPACE, so build_data can access it
    - uses: actions/checkout@v2
 
    - name: Set up Python 3.7
      uses: actions/setup-python@v2
      with:
        cache: 'pip'
        # Semantic version range syntax or exact version of a Python version
        python-version: '3.7'
 
    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install -r tools/requirements.txt
        python -c "import nltk; nltk.download('wordnet')"
 
    - name: Run unit tests with pytest
      run: |
        cd tools
        pytest
        cd ..
 
    - name: Create json csv md files
      run: python tools/data_builder.py
 
    # Deploy data to local repo
    - name: Deploy
      uses: s0/git-publish-subdir-action@master
      env:
        REPO: self
        BRANCH: data_build
        FOLDER: tools/data_build
        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
 

What changed

1 third-party action is referenced by a movable tag. Pin it to the commit SHA (Latchkey resolves and applies this automatically) so a repointed tag cannot change what runs.

What Latchkey heals here

This workflow has steps that commonly fail on transient issues (network, registries, flaky browsers). On Latchkey managed runners they are detected, retried, and self-healed instead of failing your build:

This workflow runs 1 job per trigger. On Latchkey the same minutes cost up to 58% less than GitHub-hosted, with zero queue time.

Actions used in this workflow