How to Check Quantization and Optimization in GitHub Actions

Quantize the model, then assert the file shrinks and the eval metric stays within an accepted delta so an optimization pass cannot silently break quality.

Apply dynamic quantization (or your optimization pass), compare the serialized size against the baseline, and run a small eval to check accuracy stays within a tolerance. Fail the job if either check regresses.

Steps

• Quantize the model with torch.quantization.quantize_dynamic.
• Assert the saved size is smaller than the baseline.
• Run a small eval and assert accuracy is within the threshold.

Workflow

steps:
  - run: pip install torch
  - name: Quantize and check
    run: |
      python - <<'PY'
      import os, torch
      from model import build, evaluate
      m = build().eval()
      q = torch.quantization.quantize_dynamic(m, {torch.nn.Linear}, dtype=torch.qint8)
      torch.save(m.state_dict(), "fp32.pt")
      torch.save(q.state_dict(), "int8.pt")
      assert os.path.getsize("int8.pt") < os.path.getsize("fp32.pt")
      assert evaluate(m) - evaluate(q) < 0.02, "accuracy dropped too much"
      PY

Gotchas

• Dynamic quantization targets specific layer types; unquantized models will not shrink.
• Use a fixed eval subset so the accuracy threshold is comparable across runs.

Related guides