xargs With curl: One Request Per Line
xargs feeds a list of URLs into curl so each line becomes its own request, optionally in parallel.
Health checks, cache warming, and webhook fan-out all reduce to "run curl once per URL". xargs handles the batching and the parallelism.
What it does
With -n 1, xargs runs curl once per URL; with -P it runs several requests at once. Using curl -fsS makes a failed HTTP status return non-zero, so xargs reports an overall failure if any request fails, which lets a smoke-test step fail the build.
Common usage
# one request per line, fail the step on any error
xargs -n 1 curl -fsS -o /dev/null < urls.txt
# warm 10 endpoints concurrently
xargs -P 10 -n 1 curl -fsS -o /dev/null < urls.txt
# POST a payload per line with -I
xargs -I {} curl -fsS -X POST "{}" < hooks.txtOptions
| Flag | What it does |
|---|---|
| -n 1 | One curl invocation per URL |
| -P <n> | Run n requests in parallel |
| -I {} | Put the URL anywhere in the curl command |
| curl -fsS | Fail on HTTP errors, stay quiet, still show errors |
| xargs -r | Do nothing if the URL list is empty |
In CI
Pair xargs with curl -fsS so a 500 response actually fails the step; without -f, curl exits 0 on an error status and the bad URL passes. Use -P to speed up large lists, but keep it modest so you do not overwhelm the target service or trip rate limits in the pipeline.
Common errors in CI
A smoke test that passes despite a down endpoint usually omitted curl -f, so curl returned 0. With curl -fsS, a failure makes xargs exit 123 and the step fails. "xargs: curl: No such file or directory" means curl is not installed on the runner image. Under high -P you may hit the server rate limit and see 429s; lower -P.