Skip to content
Latchkey
Start Free

Linux build workflow (shibing624/textgen)

The Linux build workflow from shibing624/textgen, explained and optimized by Latchkey.

D

CI health: D - needs work

Point runs-on at Latchkey and get caching, run de-duplication, job timeouts, self-healing for flaky steps, and up to 58% lower cost, applied automatically.

Grade your own workflow free or run it on Latchkey →
Source: shibing624/textgen.github/workflows/ubuntu.ymlLicense Apache-2.0View source

What it does

This is the Linux build workflow from the shibing624/textgen repository, a real project running GitHub Actions. It is shown here with attribution under its Apache-2.0 license.

Below, Latchkey shows a faster, safer version produced by its optimization engine.

The workflow

workflow (.yml)
on:
  workflow_dispatch: # Manually running a workflow
  push:
    branches:
      - release/*
name: Linux build
jobs:
  test-ubuntu:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        #python-version: [ 3.7, 3.8, 3.9 ]
        python-version: [ 3.8 ]
    steps:
      - uses: actions/checkout@v2
      - name: Cache pip
        uses: actions/cache@v2
        if: startsWith(runner.os, 'Linux')
        with:
          path: ~/.cache/pip
          key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt') }}
          restore-keys: |
            ${{ runner.os }}-pip-
      - name: Cache huggingface models
        uses: actions/cache@v2
        with:
          path: ~/.cache/huggingface
          key: ${{ runner.os }}-huggingface-
      - name: Cache textgen models
        uses: actions/cache@v2
        with:
          path: ~/.textgen
          key: ${{ runner.os }}-nerpy-
      - name: Set up Python
        uses: actions/setup-python@v2
        with:
          python-version: ${{ matrix.python-version }}
      - name: Install torch
        run: |
          python -m pip install --upgrade pip
          pip install Cython
          pip install torch
      - name: Install from pypi
        run: |
          pip install -U textgen
          python -c "import textgen; print(textgen.__version__)"
          pip uninstall -y textgen
      - name: Install dependencies
        run: |
          pip install -r requirements.txt
          pip install pytest
          pip install .
      - name: PKG-TEST
        run: |
          python -m pytest

The same workflow, on Latchkey

Estimated ~20% faster on cache hits, plus fewer wasted runs and a safer supply chain. Added and changed lines are highlighted.

on:
  workflow_dispatch: # Manually running a workflow
  push:
    branches:
      - release/*
name: Linux build
concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true
 
jobs:
  test-ubuntu:
    timeout-minutes: 30
    runs-on: latchkey-small
    strategy:
      fail-fast: false
      matrix:
        #python-version: [ 3.7, 3.8, 3.9 ]
        python-version: [ 3.8 ]
    steps:
      - uses: actions/checkout@v2
      - name: Cache pip
        uses: actions/cache@v2
        if: startsWith(runner.os, 'Linux')
        with:
          path: ~/.cache/pip
          key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt') }}
          restore-keys: |
            ${{ runner.os }}-pip-
      - name: Cache huggingface models
        uses: actions/cache@v2
        with:
          path: ~/.cache/huggingface
          key: ${{ runner.os }}-huggingface-
      - name: Cache textgen models
        uses: actions/cache@v2
        with:
          path: ~/.textgen
          key: ${{ runner.os }}-nerpy-
      - name: Set up Python
        uses: actions/setup-python@v2
        with:
          cache: 'pip'
          python-version: ${{ matrix.python-version }}
      - name: Install torch
        run: |
          python -m pip install --upgrade pip
          pip install Cython
          pip install torch
      - name: Install from pypi
        run: |
          pip install -U textgen
          python -c "import textgen; print(textgen.__version__)"
          pip uninstall -y textgen
      - name: Install dependencies
        run: |
          pip install -r requirements.txt
          pip install pytest
          pip install .
      - name: PKG-TEST
        run: |
          python -m pytest

What changed

What Latchkey heals here

This workflow has steps that commonly fail on transient issues (network, registries, flaky browsers). On Latchkey managed runners they are detected, retried, and self-healed instead of failing your build:

This workflow runs 1 job per trigger. On Latchkey the same minutes cost up to 58% less than GitHub-hosted, with zero queue time.

Actions used in this workflow

actions/checkout actions/cache actions/setup-python