s5cmd cp: Fast Parallel S3 Copies
s5cmd cp copies objects to or from S3 with heavy parallelism, typically several times faster than aws s3 cp for many files.
s5cmd is a Go tool built for throughput. When a job moves thousands of small objects (a dependency cache or a static site), s5cmd cp with a high --numworkers finishes in a fraction of the time aws s3 takes.
What it does
s5cmd cp copies objects between local paths and S3 URLs (s3://bucket/key). It supports wildcards for batch operations and runs many transfers at once, controlled globally by --numworkers. It uses the standard AWS credential chain.
Common usage
s5cmd cp ./dist/ s3://my-bucket/site/
s5cmd cp 's3://my-bucket/cache/*' ./cache/
# tune global parallelism, then upload
s5cmd --numworkers 256 cp ./many-small-files/ s3://my-bucket/prefix/Options
| Flag | What it does |
|---|---|
| --numworkers N | Global worker count for parallel operations (default 256) |
| -c, --concurrency N | Parts uploaded in parallel per file (multipart) |
| -p, --part-size N | Multipart part size in MiB |
| --endpoint-url <url> | Use an S3-compatible endpoint |
| --no-clobber, -n | Do not overwrite existing objects |
In CI
s5cmd is a drop-in speed win for artifact and cache steps; raise --numworkers for many small files and --concurrency/--part-size for a few large files. It reads AWS_ACCESS_KEY_ID/AWS_SECRET_ACCESS_KEY, AWS_ROLE_ARN (OIDC), and instance roles just like the AWS CLI, so prefer role-based creds over static keys.
Common errors in CI
"ERROR ... AccessDenied: Access Denied" means missing s3:PutObject/GetObject. "NoSuchBucket: The specified bucket does not exist" is a wrong bucket or region. "InvalidAccessKeyId" and "SignatureDoesNotMatch" are wrong keys or clock skew. "RequestTimeTooSkewed: The difference between the request time and the current time is too large" means the runner clock drifted; sync it with NTP before the transfer.