wget – A Standard Command Line Download Utility For Linux

wget (formerly known as Geturl) is a free, open source, command line download tool that retrieves files using HTTP, HTTPS and FTP, the most widely used internet protocols.

It is a non-interactive command line tool and its name is derived from the World Wide Web and get.

wget handles the download much better compared to other tools.

This includes working in the background, recursive downloading, multiple file downloads, resume downloads, non-interactive downloads and large file downloads.

By default all Linux distribution includes wget, so you can easily install it from the official repository. Also, it can be installed on Windows and Mac OS.

wget is designed for robustness over slow or unstable network connections, and if a download fails due to a network problem, it will keep retrying until the whole file has been retrieved.

If the server supports a reset, it will instruct the server to continue from where the download stopped. Alternatively you can install other download tools if you are looking for one.

wget Features

  • Can resume aborted downloads, using REST and RANGE
  • Can use filename wild cards and recursively mirror directories
  • NLS-based message files for many different languages
  • Optionally converts absolute links in downloaded documents to relative, so that downloaded documents may link to each other locally
  • Runs on most UNIX-like operating systems as well as Microsoft Windows
  • Supports HTTP proxies
  • Supports HTTP cookies
  • Supports persistent HTTP connections
  • Unattended / background operation
  • Uses local file timestamps to determine whether documents need to be re-downloaded when mirroring

How to Install wget on Linux

You can easily install the wget command line downloader on all major Linux distributions with the help of the distribution official package manager.

For Fedora, RHEL 8 and CentOS 8 systems, use the DNF Command to install wget.

$ sudo dnf install wget

For Debian/Ubuntu systems, use APT-GET Command or APT Command to install wget.

$ sudo apt install wget

For Arch Linux based systems, use Pacman Command to install wget.

$ sudo pacman -S wget

For RHEL/CentOS 6/7 systems, use YUM Command to install wget.

$ sudo yum install wget

For openSUSE system, use Zypper Command to install wget.

$ sudo zypper install wget

1) How to Download Single File Using wget Command

Use the wget command below without any argument to download a single file. You can find the following information when downloading a file, such as date, time, download speed and download progress.

# wget https://download.owncloud.org/community/owncloud-10.4.1.tar.bz2
--2020-06-13 07:27:20--  https://download.owncloud.org/community/owncloud-10.4.1.tar.bz2
Resolving download.owncloud.org (download.owncloud.org)... 2a01:4f8:192:50d6::3, 2a01:4f8:100:4349::3, 2a01:4f8:101:22a9::3, ...
Connecting to download.owncloud.org (download.owncloud.org)|2a01:4f8:192:50d6::3|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 22678208 (22M) [application/x-bzip2]
Saving to: âowncloud-9.0.0.tar.bz2â

100%[====================================================================================================>] 22,678,208  2.73MB/s   in 8.5s

2020-06-13 07:27:29 (2.55 MB/s) - âowncloud-10.4.1.tar.bz2â saved [22678208/22678208]

2) How to Save a File with a Different Name Using wget Command

Using the -O (uppercase) option, you can save a file with a different name and format when you start the download. Here we are going to save the file name with owncloud.zip.

# wget -O owncloud.zip https://download.owncloud.org/community/owncloud-10.4.1.tar.bz2
--2020-06-13 07:28:47--  https://download.owncloud.org/community/owncloud-10.4.1.tar.bz2
Resolving download.owncloud.org (download.owncloud.org)... 2a01:4f8:100:4349::3, 2a01:4f8:101:22a9::3, 2a01:4f8:130:806f::3, ...
Connecting to download.owncloud.org (download.owncloud.org)|2a01:4f8:100:4349::3|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 22678208 (22M) [application/x-bzip2]
Saving to: âowncloud.zipâ

100%[====================================================================================================>] 22,678,208  8.22MB/s   in 2.6s

2020-06-13 07:28:50 (8.22 MB/s) - âowncloud.zipâ saved [22678208/22678208]

3) How to Limit/Control Download Speed Using wget Command

By default wget uses full bandwidth to download a file, which can affect other functions on your computer if you have a slow or intermittent connection.

You can easily overcome this problem by limiting the bandwidth speed –limit-rate when downloading a large file.

# wget --limit-rate=500k https://download.owncloud.org/community/owncloud-10.4.1.tar.bz2
--2020-06-13 07:31:18--  https://download.owncloud.org/community/owncloud-9.0.0.tar.bz2
Resolving download.owncloud.org (download.owncloud.org)... 2a01:4f8:130:806f::3, 2a01:4f8:190:21ad::3, 2a01:4f8:192:50d6::3, ...
Connecting to download.owncloud.org (download.owncloud.org)|2a01:4f8:130:806f::3|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 22678208 (22M) [application/x-bzip2]
Saving to: âowncloud-10.4.1.tar.bz2â

100%[====================================================================================================>] 22,678,208   522KB/s   in 47s

2020-06-13 07:32:06 (471 KB/s) - âowncloud-10.4.1.tar.bz2â saved [22678208/22678208] 

4) How to Download Multiple Files at Once Using the wget Command

Use the wget command below to download more than one file at a time. To do so, you must provide both the download URL with space, and no other arguments require.

# wget https://download.owncloud.org/community/owncloud-10.4.1.tar.bz2 ftp://ftp.gnu.org/gnu/wget/wget-1.17.tar.gz
--2020-06-13 07:38:44--  https://download.owncloud.org/community/owncloud-9.0.0.tar.bz2
Resolving download.owncloud.org (download.owncloud.org)... 2a01:4f8:192:50d6::3, 2a01:4f8:100:4349::3, 2a01:4f8:101:22a9::3, ...
Connecting to download.owncloud.org (download.owncloud.org)|2a01:4f8:192:50d6::3|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 22678208 (22M) [application/x-bzip2]
Saving to: âowncloud-10.4.1.tar.bz2â

100%[====================================================================================================>] 22,678,208  2.50MB/s   in 8.9s

2020-06-13 07:38:53 (2.43 MB/s) - âowncloud-10.4.1.tar.bz2â saved [22678208/22678208]

--2020-06-13 07:38:53--  ftp://ftp.gnu.org/gnu/wget/wget-1.17.tar.gz
           => âwget-1.17.tar.gzâ
Resolving ftp.gnu.org (ftp.gnu.org)... 2001:4830:134:3::b, 208.118.235.20
Connecting to ftp.gnu.org (ftp.gnu.org)|2001:4830:134:3::b|:21... connected.
Logging in as anonymous ... Logged in!
==> SYST ... done.    ==> PWD ... done.
==> TYPE I ... done.  ==> CWD (1) /gnu/wget ... done.
==> SIZE wget-1.17.tar.gz ... 3779445
==> EPSV ... done.    ==> RETR wget-1.17.tar.gz ... done.
Length: 3779445 (3.6M) (unauthoritative)

100%[====================================================================================================>] 3,779,445   4.28MB/s   in 0.8s

2020-06-13 07:38:54 (4.28 MB/s) - âwget-1.17.tar.gzâ saved [3779445]

FINISHED --2020-06-13 07:38:54--
Total wall clock time: 11s
Downloaded: 2 files, 25M in 9.7s (2.59 MB/s)

5) How to Resume an Incomplete Download Using the wget Command

Make a note, whenever going to download big size of file (eg: ISO Images), i advise you to use the -c option, which will allow you to resume the existing incomplete download from where it stopped and finish it off as usual.

This is especially useful if you have a slow or intermittent connection or problems with the system. Otherwise, when you re-download, it will start a new download and save it to a different filename (add 1 automatically to the file name).

# wget -c https://download.owncloud.org/community/owncloud-10.4.1.tar.bz2
--2020-06-13 07:40:23--  https://download.owncloud.org/community/owncloud-10.4.1.tar.bz2
Resolving download.owncloud.org (download.owncloud.org)... 2a01:4f8:101:22a9::3, 2a01:4f8:130:806f::3, 2a01:4f8:190:21ad::3, ...
Connecting to download.owncloud.org (download.owncloud.org)|2a01:4f8:101:22a9::3|:443... connected.
HTTP request sent, awaiting response... 206 Partial Content
Length: 22678208 (22M), 18942656 (18M) remaining [application/x-bzip2]
Saving to: âowncloud-10.4.1.tar.bz2â

100%[++++++++++++++++====================================================================================>] 22,678,208  6.75MB/s   in 2.7s

2020-06-13 07:40:26 (6.75 MB/s) - âowncloud-10.4.1.tar.bz2â saved [22678208/22678208]

6) How to Download Files in Bulk Using the wget Command

Files can be downloaded in bulk using a text file. Make sure there is only one URL in each line. To do this, use the -i option with the wget command.

# wget -i test-wget.txt
--2020-06-13 07:43:15--  https://download.owncloud.org/community/owncloud-10.4.1.tar.bz2
Resolving download.owncloud.org (download.owncloud.org)... 2a01:4f8:130:806f::3, 2a01:4f8:190:21ad::3, 2a01:4f8:192:50d6::3, ...
Connecting to download.owncloud.org (download.owncloud.org)|2a01:4f8:130:806f::3|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 22678208 (22M) [application/x-bzip2]
Saving to: âowncloud-10.4.1.tar.bz2â

100%[====================================================================================================>] 22,678,208  3.02MB/s   in 8.0s

2020-06-13 07:43:23 (2.71 MB/s) - âowncloud-10.4.1.tar.bz2â saved [22678208/22678208]

--2020-06-13 07:43:23--  ftp://ftp.gnu.org/gnu/wget/wget-1.17.tar.gz
           => âwget-1.17.tar.gzâ
Resolving ftp.gnu.org (ftp.gnu.org)... 2001:4830:134:3::b, 208.118.235.20
Connecting to ftp.gnu.org (ftp.gnu.org)|2001:4830:134:3::b|:21... connected.
Logging in as anonymous ... Logged in!
==> SYST ... done.    ==> PWD ... done.
==> TYPE I ... done.  ==> CWD (1) /gnu/wget ... done.
==> SIZE wget-1.17.tar.gz ... 3779445
==> EPSV ... done.    ==> RETR wget-1.17.tar.gz ... done.
Length: 3779445 (3.6M) (unauthoritative)

100%[====================================================================================================>] 3,779,445   5.31MB/s   in 0.7s

2020-06-13 07:43:24 (5.31 MB/s) - âwget-1.17.tar.gzâ saved [3779445]

FINISHED --2020-06-13 07:43:24--
Total wall clock time: 9.6s
Downloaded: 2 files, 25M in 8.7s (2.91 MB/s)

7) How to Run the Download in the Background

If you don’t want to interrupt your current activity when you download a large file you may need to run the wget command in the background (for example, iso images). Use the -b option with the wget command to run the download in the background.

# wget -b https://download.owncloud.org/community/owncloud-10.4.1.tar.bz2
Continuing in background, pid 8114.
Output will be written to âwget-logâ.

# tail -f wget-log
 21800K .......... .......... .......... .......... .......... 98% 16.3M 0s
 21850K .......... .......... .......... .......... .......... 98% 15.5M 0s
 21900K .......... .......... .......... .......... .......... 99% 17.7M 0s
 21950K .......... .......... .......... .......... .......... 99% 15.8M 0s
 22000K .......... .......... .......... .......... .......... 99% 11.6M 0s
 22050K .......... .......... .......... .......... .......... 99% 2.44M 0s
 22100K .......... .......... .......... .......... ......    100% 44.5M=3.4s

2020-06-13 07:44:40 (6.40 MB/s) - âowncloud-10.4.1.tar.bz2â saved [22678208/22678208]

8) How to Download Specific Format of Files

You can download the specific format of the file, such as .pdf, .php, .png, .html from the given url. To do so, use the -r -A option with the wget command.

# wget -r -A png http://www.magesh.co.in
--2020-06-13 07:54:47--  http://www.magesh.co.in/
Resolving www.magesh.co.in (www.magesh.co.in)... 103.3.231.248
Connecting to www.magesh.co.in (www.magesh.co.in)|103.3.231.248|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 54171 (53K) [text/html]
Saving to: âwww.magesh.co.in/index.htmlâ

100%[====================================================================================================>] 54,171      60.7KB/s   in 0.9s

2020-06-13 07:54:49 (60.7 KB/s) - âwww.magesh.co.in/index.htmlâ saved [54171/54171]

Loading robots.txt; please ignore errors.
--2016-03-22 07:54:49--  http://www.magesh.co.in/robots.txt
Reusing existing connection to www.magesh.co.in:80.
HTTP request sent, awaiting response... 404 Not Found
2020-06-13 07:54:49 ERROR 404: Not Found.

Removing www.magesh.co.in/index.html since it should be rejected.

--2020-06-13 07:54:49--  http://www.magesh.co.in/images/log.png
Reusing existing connection to www.magesh.co.in:80.
HTTP request sent, awaiting response... 200 OK
Length: 4605 (4.5K) [image/png]
Saving to: âwww.magesh.co.in/images/log.pngâ

100%[====================================================================================================>] 4,605       --.-K/s   in 0.01s

2020-06-13 07:54:49 (375 KB/s) - âwww.magesh.co.in/images/log.pngâ saved [4605/4605]

FINISHED --2020-06-13 07:54:49--
Total wall clock time: 2.3s
Downloaded: 2 files, 57K in 0.9s (65.0 KB/s)

9) How to Download a File from a Password Protected Site

You can download a file from a password protected site using the wget command. Use the following wget command format to download a file from a password protected site.

# wget --http-user=xxx --http-password=xxx https://download.owncloud.org/community/owncloud-10.4.1.tar.bz2

# wget --ftp-user=xxx --ftp-password=xxx ftp://ftp.gnu.org/gnu/wget/wget-1.17.tar.gz

10) Man Page of wget Command

If you want to know more options available to wget, you can always find them at wget’s man page.

# man wget 
or
# wget --help

Magesh Maruthamuthu

Love to play with all Linux distribution

You may also like...