Skip to content
LatchkeyLatchkey home

kubectl taint: Usage, Options & Common CI Errors

Repel pods from a node unless they explicitly tolerate it.

kubectl taint marks a node so that only pods with a matching toleration may schedule (or stay) there. It is how you dedicate nodes to specific workloads - and a frequent cause of mysteriously Pending pods.

What it does

kubectl taint nodes NODE key=value:EFFECT adds a taint; the trailing key=value:EFFECT- form removes it. EFFECT is NoSchedule (block new pods), PreferNoSchedule (soft), or NoExecute (also evict running pods without the toleration). Pods need a matching tolerations entry in their spec to land on a tainted node.

Common usage

Terminal
kubectl taint nodes gpu-1 dedicated=gpu:NoSchedule
kubectl taint nodes gpu-1 dedicated=gpu:NoSchedule-     # remove it
kubectl taint nodes node-1 maintenance=true:NoExecute   # evict + block

Common errors in CI

After tainting a node in a test, ordinary pods land Pending with "node(s) had untolerated taint {dedicated: gpu}" - expected, but it breaks pipelines that assumed the pod would schedule; either taint a dedicated node or give the test pod a matching toleration. A NoExecute taint immediately evicts non-tolerating pods, which can knock out workloads you did not intend to move. Forgetting to remove a test taint leaves the node effectively cordoned for normal work; clean it up with the key=value:EFFECT- form.

Using this in CI

A runner has no kubeconfig, no cached context, and no interactive auth. Every kubectl invocation in CI needs the context supplied explicitly, and most confusing CI failures here are the command running against the wrong cluster or no cluster at all.

Terminal
# never rely on the ambient context on a runner
kubectl --context "$KUBE_CONTEXT" -n "$NAMESPACE" get pods

# confirm what you are actually connected to before mutating anything
kubectl config current-context
kubectl cluster-info

# fail fast instead of hanging on an unreachable API server
kubectl --request-timeout=30s get nodes

Frequently asked questions

kubectl taint: Usage, Options & Common CI Errors?
kubectl taint marks a node so that only pods with a matching toleration may schedule (or stay) there. It is how you dedicate nodes to specific workloads - and a frequent cause of mysteriously Pending pods.
What it does?
kubectl taint nodes NODE key=value:EFFECT adds a taint; the trailing key=value:EFFECT- form removes it. EFFECT is NoSchedule (block new pods), PreferNoSchedule (soft), or NoExecute (also evict running pods without the toleration). Pods need a matching tolerations entry in their spec to land on a tainted node.
Common errors in CI?
After tainting a node in a test, ordinary pods land Pending with "node(s) had untolerated taint {dedicated: gpu}" - expected, but it breaks pipelines that assumed the pod would schedule; either taint a dedicated node or give the test pod a matching toleration.

Related guides

References

Run this faster and cheaper on Latchkey managed runners - self-healing included. Start free → 30-day trial · No credit card