Download a file using `curl` - How hard can it get?

First posted on: 2018/09/14

Last modified: 2019/07/11,6a40e30

Categories: software

I wanted to download the prometheus binary using curl. My first attempt:

$ curl https://github.com/prometheus/prometheus/releases/download/v2.4.0/prometheus-2.4.0.linux-amd64.tar.gz
<html><body>
You are being <a href="https://github-production-release-asset-2e65be.s3.amazonaws.com/6838921/5c87dc00-b5d1-11e8-8a3a-fd29b54e0c60?X-Amz-Algorithm=AWS4-HMAC-SHA256&amp;X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20180914%2Fus-east-1%2Fs3%2Faws4_request&amp;X-Amz-Date=20180914T004135Z&amp;X-Amz-Expires=300&amp;X-Amz-Signature=5a4887918cf75526c045d236dd5a8b22dace657900a1d131a7ffa947be66fc81&amp;X-Amz-SignedHeaders=host&amp;actor_id=0&amp;response-content-disposition=attachment%3B%20filename%3Dprometheus-2.4.0.linux-amd64.tar.gz&amp;response-content-type=application%2Foctet-stream">redirected</a>.

Sure, that’s a redirect, let’s try:

$ curl --location https://github.com/prometheus/prometheus/releases/download/v2.4.0/prometheus-2.4.0.linux-amd64.tar.gz
curl: (23) Failed writing body (0 != 16360)

What does that mean? I frantically then google, “download prometheus using curl” and hit upon this link.

So, the -O option will help:

$ curl --remote-name --location https://github.com/prometheus/prometheus/releases/download/v2.4.0/prometheus-2.4.0.linux-amd64.tar.gz
<file downloaded>

The -O option is equivalent to --remote-name which basically says two things:

What’s that error above?

The Failed wrtiting body error seems to surface in different circumstances. It certainly didn’t help my debugging. However, I was running WSL, and a older curl version:

$ curl --version
curl 7.47.0 (x86_64-pc-linux-gnu) libcurl/7.47.0 GnuTLS/3.4.10 zlib/1.2.8 libidn/1.32 librtmp/2.3

On Linux with a newer curl version, the error becomes:

$ curl 7.58.0 (x86_64-pc-linux-gnu) libcurl/7.58.0 OpenSSL/1.1.0g zlib/1.2.11 libidn2/2.0.4 libpsl/0.19.1 (+libidn2/2.0.4) nghttp2/1.30.0 librtmp/2.3
Release-Date: 2018-01-24
..

~$ curl --location https://github.com/prometheus/prometheus/releases/download/v2.4.0/prometheus-2.4.0.linux-amd64.tar.gz
Warning: Binary output can mess up your terminal. Use "--output -" to tell
Warning: curl to output it to your terminal anyway, or consider "--output
Warning: <FILE>" to save to a file.

That error certainly makes a lot more sense. So basically, error messages are super helpful and software can test you a lot even when trying to do the simplest of things.