curl is a tool to transfer data from a server or to server, using one of the supported protocols (DICT, FILE, FTP, FTPS, GOPHER, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3, POP3S, RTMP, RTSP, SCP, SFTP, SMTP, SMTPS, TELNET and TFTP).
The command is designed to work without user interaction.
Also curl support proxy support, user authentication, FTP upload, HTTP post, SSL connections, cookies, file transfer resume, Metalink, and more.
curl is powered by libcurl for all transfer-related features.
If you specify URL without protocol:// prefix, curl will attempt to guess what protocol you might want.
For example, host names starting with "ftp."curl will assume you want to speak FTP.
If it’s not find specific protocol, then do default to HTTP. Alternatively you can install other Download Utilities too.
1) Install curl on Linux
We can easily install curl command line download tool to all the Linux Distribution such as Debian, Ubuntu, Mint, RHEL, CentOS, Fedora, suse, openSUSE, Arch Linux, Manjaro, Mageia, etc.. Just fire the below command to install.
For Fedora system, use DNF Command to install curl.
$ sudo dnf install curl
For Debian/Ubuntu systems, use APT-GET Command or APT Command to install curl.
$ sudo apt install curl
For Arch Linux based systems, use Pacman Command to install curl.
$ sudo pacman -S curl
For RHEL/CentOS systems, use YUM Command to install curl.
$ sudo yum install curl
For openSUSE Leap system, use Zypper Command to install curl.
$ sudo zypper install curl
2) Download Single File
The below command will download the file from given URL and stores in current directory, while downloading the file we can see the (total time, time left, Avg download speed, Current download speed & download progress) of file.
# curl -O https://curl.haxx.se/download/curl-7.48.0.tar.gz
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 8579k 100 8579k 0 0 3532k 0 0:00:02 0:00:02 --:--:-- 3533k
3) Save the file with different name
We can save the file with different name & format while initiate downloading, using -o (lowercase) option. Here we are going to save the filename with curl.zip.
# curl -o curl.zip https://curl.haxx.se/download/curl-7.48.0.tar.gz
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 8579k 100 8579k 0 0 3605k 0 0:00:02 0:00:02 --:--:-- 3606k
4) Download Multiple Files
The below command will download more then on file from the location and stores in current directory, while downloading the file we can see the (total time, time left, Avg download speed, Current download speed & download progress) of file. We need to mention -O (uppercase) to each URL.
# curl -O https://curl.haxx.se/download/curl-7.48.0.tar.gz -O https://download.owncloud.org/community/owncloud-9.0.0.tar.bz2 % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 8579k 100 8579k 0 0 463k 0 0:00:18 0:00:18 --:--:-- 1145k 100 21.6M 100 21.6M 0 0 5116k 0 0:00:04 0:00:04 --:--:-- 5270k
5) Resume Incomplete download
Continue/Resume a previous file transfer at the given offset. The given offset is the exact number of bytes that will be skipped, counting from the beginning of the source file before it is transferred to the destination. Use -C - to tell curl to automatically find out where/how to resume the transfer. It then uses the given output/input files to figure that out.
# curl -O https://download.owncloud.org/community/owncloud-9.0.0.tar.bz2 % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 21.6M 0 16384 0 0 23752 0 0:15:54 --:--:-- 0:15:54 23744^C # curl -C- -O https://download.owncloud.org/community/owncloud-9.0.0.tar.bz2 ** Resuming transfer from byte position 573440 % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 21.0M 100 21.0M 0 0 5687k 0 0:00:03 0:00:03 --:--:-- 5688k
6) Limit download speed
By default curl utilize full bandwidth for downloading file and we can’t use anything on server before download completion (Which will affect other service accessing bandwidth). So better use --limit-rate option to avoid further issue while downloading big size file.
# curl --limit-rate 500k -O https://curl.haxx.se/download/curl-7.48.0.tar.gz % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 8579k 100 8579k 0 0 499k 0 0:00:17 0:00:17 --:--:-- 496k
7) Download metalink file
This option can tell curl to parse and process a given URI as Metalink file. It will also verify the hash of the file after the download completes. The Metalink file itself is downloaded and processed in memory and not stored in the local file system.
# curl --metalink http://releases.ubuntu.com/16.04/ubuntu-16.04-desktop-amd64.metalink
8) Get Response Header info
(HTTP/FTP/FILE) Fetch the HTTP-header only! HTTP-servers feature the command HEAD which this uses to get nothing but the header of a document. When used on an FTP or FILE file, curl displays the file size and last modification time only.
# curl -I https://www.2daygeek.com
HTTP/1.1 200 OK
Date: Thu, 05 May 2016 11:55:33 GMT
Set-Cookie: __cfduid=da343c5351ff9781b868922f916e36de51462449331; expires=Fri, 05-May-17 11:55:31 GMT; path=/; domain=.2daygeek.com; HttpOnly
Server: cloudflare-nginx
CF-RAY: 29e401004de72dd3-BOM
Connection: keep-alive
9) Download a file from password protected site
Alternatively we can download a file from password protected site. The below command will download the file from password protected site.
# curl -u username:password -O https://download.owncloud.org/community/owncloud-9.0.0.tar.bz2 # curl -u username:password -O ftp://ftp.gnu.org/gnu/wget/wget-1.17.tar.gz
10) Upload a file to ftp server
Curl can also be used to upload files to the FTP server by adding -T option
# curl -u username:password -T myfiles.zip ftp://ftp.2daygeek.com # curl -u username:password -T "{myfile1.zip,file2.zip}" ftp://ftp.2daygeek.com
11) Download a file using proxy
Alternatively we can download a file using prxy by adding -x option with curl.
# curl -x http://proxyserver:port --proxy-user username:password https://www.2daygeek.com
12) Read more about curl
If you want to know more option which is available for curl, you can grep the details on your terminal itself by firing below commands..
# man curl or # curl --help
Enjoy…)
