Publish dev release to PyPI workflow (omry/omegaconf)
The Publish dev release to PyPI workflow from omry/omegaconf, explained and optimized by Latchkey.
CI health: C - fair
Point runs-on at Latchkey and get caching, job timeouts, SHA-pinned actions, self-healing for flaky steps, and up to 58% lower cost, applied automatically.
What it does
This is the Publish dev release to PyPI workflow from the omry/omegaconf repository, a real project running GitHub Actions. It is shown here with attribution under its BSD-3-Clause license.
Below, Latchkey shows a faster, safer version produced by its optimization engine.
The workflow
# This workflow uploads development releases of OmegaConf to PyPI.
# It is triggered manually and refuses to publish stable versions.
name: Publish dev release to PyPI
on:
workflow_dispatch:
jobs:
build-artifacts:
name: Build distribution artifacts
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: '3.11'
- name: Set up Java
uses: actions/setup-java@v5
with:
distribution: 'temurin'
java-version: '11'
- name: Install build dependencies
run: |
python -m pip install --upgrade pip
pip install build packaging
- name: Verify version is a prerelease
run: |
python - <<'PY'
from pathlib import Path
from packaging.version import Version
namespace = {}
exec(Path("omegaconf/version.py").read_text(), namespace)
version = Version(namespace["__version__"])
if not (version.is_prerelease or version.is_devrelease):
raise SystemExit(
f"Refusing to publish stable version {str(version)!r} with the dev publish workflow."
)
print(f"Publishing development version: {version}")
PY
- name: Build distribution
run: |
python -m build
python -m build subprojects/omegaconf-pydevd
- name: Verify build artifacts
run: |
ls -lah dist/
ls -lah subprojects/omegaconf-pydevd/dist/
python -m pip install twine
twine check dist/* subprojects/omegaconf-pydevd/dist/*
- name: Stage publish artifacts
run: |
mkdir -p publish-dist
cp dist/* publish-dist/
cp subprojects/omegaconf-pydevd/dist/* publish-dist/
ls -lah publish-dist/
- name: Summarize publish artifacts
run: |
python - <<'PY' >> "$GITHUB_STEP_SUMMARY"
import pathlib
import tarfile
import zipfile
paths = [pathlib.Path("publish-dist")]
def metadata_from_artifact(path: pathlib.Path) -> tuple[str, str]:
if path.suffix == ".whl":
with zipfile.ZipFile(path) as zf:
metadata_name = next(
name for name in zf.namelist() if name.endswith(".dist-info/METADATA")
)
metadata = zf.read(metadata_name).decode()
else:
with tarfile.open(path, "r:gz") as tf:
member = next(
member for member in tf.getmembers() if member.name.endswith("/PKG-INFO")
)
extracted = tf.extractfile(member)
assert extracted is not None
metadata = extracted.read().decode()
name = ""
version = ""
for line in metadata.splitlines():
if line.startswith("Name: "):
name = line.removeprefix("Name: ")
elif line.startswith("Version: "):
version = line.removeprefix("Version: ")
if name and version:
break
return name, version
seen = {}
for directory in paths:
for artifact in sorted(directory.glob("*")):
if artifact.is_file():
name, version = metadata_from_artifact(artifact)
seen.setdefault((name, version), set()).add(artifact.name)
print("## Packages prepared for dev publish")
print()
print("| Package | Version |")
print("| --- | --- |")
for name, version in sorted(seen):
print(f"| {name} | {version} |")
PY
- name: Upload artifacts
uses: actions/upload-artifact@v4
with:
name: dist
path: publish-dist/*
retention-days: 0
pypi-publish:
needs: build-artifacts
name: Upload dev release to PyPI
runs-on: ubuntu-latest
environment: pypi-publish-dev
permissions:
id-token: write
steps:
- name: Download artifacts
uses: actions/download-artifact@v5
with:
name: dist
path: dist/
- name: Show downloaded artifacts
run: ls -lah dist/
- name: Publish package distributions to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
The same workflow, on Latchkey
Estimated ~20% faster on cache hits, plus fewer wasted runs and a safer supply chain. Added and changed lines are highlighted.
# This workflow uploads development releases of OmegaConf to PyPI. # It is triggered manually and refuses to publish stable versions. name: Publish dev release to PyPI on: workflow_dispatch: jobs: build-artifacts: timeout-minutes: 30 name: Build distribution artifacts runs-on: latchkey-small steps: - uses: actions/checkout@v5 - name: Set up Python uses: actions/setup-python@v6 with: cache: 'pip' python-version: '3.11' - name: Set up Java uses: actions/setup-java@v5 with: distribution: 'temurin' java-version: '11' - name: Install build dependencies run: | python -m pip install --upgrade pip pip install build packaging - name: Verify version is a prerelease run: | python - <<'PY' from pathlib import Path from packaging.version import Version namespace = {} exec(Path("omegaconf/version.py").read_text(), namespace) version = Version(namespace["__version__"]) if not (version.is_prerelease or version.is_devrelease): raise SystemExit( f"Refusing to publish stable version {str(version)!r} with the dev publish workflow." ) print(f"Publishing development version: {version}") PY - name: Build distribution run: | python -m build python -m build subprojects/omegaconf-pydevd - name: Verify build artifacts run: | ls -lah dist/ ls -lah subprojects/omegaconf-pydevd/dist/ python -m pip install twine twine check dist/* subprojects/omegaconf-pydevd/dist/* - name: Stage publish artifacts run: | mkdir -p publish-dist cp dist/* publish-dist/ cp subprojects/omegaconf-pydevd/dist/* publish-dist/ ls -lah publish-dist/ - name: Summarize publish artifacts run: | python - <<'PY' >> "$GITHUB_STEP_SUMMARY" import pathlib import tarfile import zipfile paths = [pathlib.Path("publish-dist")] def metadata_from_artifact(path: pathlib.Path) -> tuple[str, str]: if path.suffix == ".whl": with zipfile.ZipFile(path) as zf: metadata_name = next( name for name in zf.namelist() if name.endswith(".dist-info/METADATA") ) metadata = zf.read(metadata_name).decode() else: with tarfile.open(path, "r:gz") as tf: member = next( member for member in tf.getmembers() if member.name.endswith("/PKG-INFO") ) extracted = tf.extractfile(member) assert extracted is not None metadata = extracted.read().decode() name = "" version = "" for line in metadata.splitlines(): if line.startswith("Name: "): name = line.removeprefix("Name: ") elif line.startswith("Version: "): version = line.removeprefix("Version: ") if name and version: break return name, version seen = {} for directory in paths: for artifact in sorted(directory.glob("*")): if artifact.is_file(): name, version = metadata_from_artifact(artifact) seen.setdefault((name, version), set()).add(artifact.name) print("## Packages prepared for dev publish") print() print("| Package | Version |") print("| --- | --- |") for name, version in sorted(seen): print(f"| {name} | {version} |") PY - name: Upload artifacts uses: actions/upload-artifact@v4 with: name: dist path: publish-dist/* retention-days: 0 pypi-publish: timeout-minutes: 30 needs: build-artifacts name: Upload dev release to PyPI runs-on: latchkey-small environment: pypi-publish-dev permissions: id-token: write steps: - name: Download artifacts uses: actions/download-artifact@v5 with: name: dist path: dist/ - name: Show downloaded artifacts run: ls -lah dist/ - name: Publish package distributions to PyPI uses: pypa/gh-action-pypi-publish@release/v1
What changed
- Run on Latchkey managed runners with one line (
runs-on), which apply the fixes below automatically and self-heal transient failures. This example useslatchkey-small; pick the runner size that fits the job. - Cache dependency installs on the setup step so they are served from cache.
- Add a job timeout so a hung step cannot burn hours of runner time.
1 third-party action is referenced by a movable tag. Pin it to the commit SHA (Latchkey resolves and applies this automatically) so a repointed tag cannot change what runs.
What Latchkey heals here
This workflow has steps that commonly fail on transient issues (network, registries, flaky browsers). On Latchkey managed runners they are detected, retried, and self-healed instead of failing your build:
- Dependency installs
This workflow runs 2 jobs per trigger. On Latchkey the same minutes cost up to 58% less than GitHub-hosted, with zero queue time.