curl – Command Line Download Manager Examples

curl is a tool to transfer data from a server or to server, using one of the supported protocols (DICT, FILE, FTP, FTPS, GOPHER, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3, POP3S, RTMP, RTSP, SCP, SFTP, SMTP, SMTPS, TELNET and TFTP). The command is designed to work without user interaction. Also curl support proxy support, user authentication, FTP upload, HTTP post, SSL connections, cookies, file transfer resume, Metalink, and more. curl is powered by libcurl for all transfer-related features. The most recent stable version is curl 7.48.0, released on March 23, 2016.

If you specify URL without protocol:// prefix, curl will attempt to guess what protocol you might want. For example, host names starting with "ftp."curl will assume you want to speak FTP. If it’s not find specific protocol, then do default to HTTP. Alternatively you can install other Download Utility too.

1) Install curl on Linux

We can easily install curl command line downloader to all the Linux Distribution such as Debian, Ubuntu, Mint, RHEL, CentOS, Fedora, suse, openSUSE, Arch Linux, Manjaro, Mageia, etc.. Just fire the below command to install.

[For Debian, Ubuntu & Mint]
$ sudo apt-get install curl

[For CentOS, RHEL, Fedora 21 and older Systems]
# yum install curl

[Fedora 22 and later systems]
# dnf install curl

[For suse & openSUSE]
# zypper install curl

[Mageia]
# urpmi curl

[For Arch Linux & Manjaro]
$ sudo pacman -S curl

2) Download Single File

The below command will download the file from given URL and stores in current directory, while downloading the file we can see the (total time, time left, Avg download speed, Current download speed & download progress) of file.

# curl -O https://curl.haxx.se/download/curl-7.48.0.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 8579k  100 8579k    0     0  3532k      0  0:00:02  0:00:02 --:--:-- 3533k

3) Save the file with different name

We can save the file with different name & format while initiate downloading, using -o (lowercase) option. Here we are going to save the filename with curl.zip.

# curl -o curl.zip https://curl.haxx.se/download/curl-7.48.0.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 8579k  100 8579k    0     0  3605k      0  0:00:02  0:00:02 --:--:-- 3606k

4) Download Multiple Files

The below command will download more then on file from the location and stores in current directory, while downloading the file we can see the (total time, time left, Avg download speed, Current download speed & download progress) of file. We need to mention -O (uppercase) to each URL.

# curl -O https://curl.haxx.se/download/curl-7.48.0.tar.gz -O https://download.owncloud.org/community/owncloud-9.0.0.tar.bz2
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 8579k  100 8579k    0     0   463k      0  0:00:18  0:00:18 --:--:-- 1145k
100 21.6M  100 21.6M    0     0  5116k      0  0:00:04  0:00:04 --:--:-- 5270k

5) Resume Incomplete download

Continue/Resume a previous file transfer at the given offset. The given offset is the exact number of bytes that will be skipped, counting from the beginning of the source file before it is transferred to the destination. Use -C - to tell curl to automatically find out where/how to resume the transfer. It then uses the given output/input files to figure that out.

# curl -O https://download.owncloud.org/community/owncloud-9.0.0.tar.bz2
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0 21.6M    0 16384    0     0  23752      0  0:15:54 --:--:--  0:15:54 23744^C


# curl -C- -O https://download.owncloud.org/community/owncloud-9.0.0.tar.bz2
** Resuming transfer from byte position 573440
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 21.0M  100 21.0M    0     0  5687k      0  0:00:03  0:00:03 --:--:-- 5688k

6) Limit download speed

By default curl utilize full bandwidth for downloading file and we can’t use anything on server before download completion (Which will affect other service accessing bandwidth). So better use --limit-rate option to avoid further issue while downloading big size file.

# curl --limit-rate 500k -O https://curl.haxx.se/download/curl-7.48.0.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 8579k  100 8579k    0     0   499k      0  0:00:17  0:00:17 --:--:--  496k

7) Download metalink file

This option can tell curl to parse and process a given URI as Metalink file. It will also verify the hash of the file after the download completes. The Metalink file itself is downloaded and processed in memory and not stored in the local file system.

# curl --metalink http://releases.ubuntu.com/16.04/ubuntu-16.04-desktop-amd64.metalink

8) Get Response Header info

(HTTP/FTP/FILE) Fetch the HTTP-header only! HTTP-servers feature the command HEAD which this uses to get nothing but the header of a document. When used on an FTP or FILE file, curl displays the file size and last modification time only.

# curl -I http://www.2daygeek.com
HTTP/1.1 200 OK
Date: Thu, 05 May 2016 11:55:33 GMT
Set-Cookie: __cfduid=da343c5351ff9781b868922f916e36de51462449331; expires=Fri, 05-May-17 11:55:31 GMT; path=/; domain=.2daygeek.com; HttpOnly
Server: cloudflare-nginx
CF-RAY: 29e401004de72dd3-BOM
Connection: keep-alive

9) Download a file from password protected site

Alternatively we can download a file from password protected site. The below command will download the file from password protected site.

# curl -u username:password -O https://download.owncloud.org/community/owncloud-9.0.0.tar.bz2

# curl -u username:password -O ftp://ftp.gnu.org/gnu/wget/wget-1.17.tar.gz

10) Upload a file to ftp server

Curl can also be used to upload files to the FTP server by adding -T option

# curl -u username:password -T myfiles.zip ftp://ftp.2daygeek.com

# curl -u username:password -T "{myfile1.zip,file2.zip}" ftp://ftp.2daygeek.com

11) Download a file using proxy

Alternatively we can download a file using prxy by adding -x option with curl.

# curl -x http://proxyserver:port --proxy-user username:password http://www.2daygeek.com

12) Read more about curl

If you want to know more option which is available for curl, you can grep the details on your terminal itself by firing below commands..

# man curl
or
# curl --help

Enjoy…)

Magesh Maruthamuthu

Love to play with all Linux distribution

You may also like...

Close
Please support the site
By clicking any of these buttons you help our site to get better