Using wget to download a file with resume option

This article is about download file with utility or tool which can give an additional option to reconnect back or resume the file downloading process without having to redo it from the beginning.

Usually when the internet connection is not stable or even frequently disconnected, we have to redownload the file which is going to be downloaded from the very beginning. But using the right tool and the correct command parameter, it can be resumed without having to download the file from the start.

The tool which is used to do it is ‘wget’. It is a utility available as part of GNU Project. It is stands for World Wide Web and get. Below is the pattern of the download process using ‘wget’ :

wget url_of_file_which_is_going_to_be_downloaded
Description : 
wget : It is a command which is actually used to call tool to download 
url_of_file_which_is_going_to_be_dwnloaded : It is a parameter of 'wget' command which is actually in form or URL, for an example http://www.mydomain.com/data.zip

It can be simulated by download file named data.zip from URL http://www.mydomain.com/data.zip. This is the execution of the command in real situation for giving an example :

user@hostname:~/Downloads$ wget http://www.mydomain.com/data.zip
--2016-09-25 10:46:52-- http://www.mydomain.com/data.zip
Resolving www.mydomain.com (www.mydomain.com)... xxx.xxx.xxx.xxx
Connecting to www.mydomain.com (www.mydomain.com)|xxx.xxx.xxx.xxx|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 397266014 (379M) [application/zip]
Saving to: ‘data.zip’
data.zip 0%[ ] 1,53M --.-KB/s eta 2h 31m
data.zip 0%[ ] 1,53M --.-KB/s eta 2h 39m
data.zip 0%[ ] 1,53M --.-KB/s eta 2h 55m ^C

In the above download process, the connectivity is failing and also the size of data represented the file transfer is none. It is decided to terminate the download process temporary by typing Ctrl+C.

The above process is stopped just to prove that the download process can still be resumed or continued on. The above download process is interrupted by executing Ctrl+C to stop it. And then to continue it back, just type the following command in the bash prompt :

wget -c http://www.mydomain.com/data.zip
Description : 
-c : It is an additional which is an abbreviation of continue. It is used to continue download process since the process failed or disrupted It will append the unfinished downloaded file which has already partially downloaded and completed it when the download process finished.  

Below is the re-execution of the above command with an extra parameter -c to continue on downloading and appending the partially-downloaded file in order to complete the whole file to be downloaded :

user@hostname:~/Downloads$ wget -c http://www.mydomain.com/data.zip
--2016-09-25 10:47:48-- http://www.mydomain.com/data.zip
Resolving www.mydomain.com (www.mydomain.com)... xxx.xxx.xxx.xxx
Connecting to www.mydomain.com (www.mydomain.com)|xxx.xxx.xxx.xxx|:80... connected.
HTTP request sent, awaiting response... 206 Partial content
Length: 397266014 (379M), 395662115 (377M) remaining [application/zip]
Saving to: ‘data.zip’
data.zip 5%[====> ] 19,49M 152KB/s eta 65m 44s
data.zip 5%[====> ] 19,52M 133KB/s eta 65m 25s
data.zip 5%[====> ] 19,52M 99,3KB/s eta 65m 45s
data.zip 5%[====> ] 19,52M 81,5KB/s eta 66m 23s
data.zip 5%[====> ] 19,52M --.-KB/s eta 68m 17s
data.zip 5%[====> ] 19,52M --.-KB/s eta 70m 11s
^C

It is re-execute once again since the above process disrupted. The command is shown as follows :

user@hostname:~/Downloads$ wget -c http://www.mydomain.com/data.zip
--2016-09-25 10:51:42-- http://www.mydomain.com/data.zip
Resolving www.mydomain.com (www.mydomain.com)... xxx.xxx.xxx.xxx
Connecting to www.mydomain.com (www.mydomain.com)|xxx.xxx.xxx.xxx|:80... connected.
HTTP request sent, awaiting response... 206 Partial content
Length: 397266014 (379M), 376795672 (359M) remaining [application/zip]
Saving to: ‘data.zip’
data.zip 7%[+++++==> ] 27,52M --.-KB/s eta 8h 5m ^C
user@hostname:~/Downloads$ wget -c http://www.mydomain.com/data.zip
--2016-09-25 11:02:48-- http://www.mydomain.com/data.zip
Resolving www.mydomain.com (www.mydomain.com)... xxx.xxx.xxx.xxx
Connecting to www.mydomain.com (www.mydomain.com)|xxx.xxx.xxx.xxx|:80... connected.
HTTP request sent, awaiting response... 206 Partial content
Length: 397266014 (379M), 368406231 (351M) remaining [application/zip]
Saving to: ‘data.zip’
data.zip 55%[++++++++==================> ] 210,55M --.-KB/s eta 21m 38s
data.zip 55%[++++++++==================> ] 210,55M --.-KB/s eta 21m 40s
data.zip 55%[++++++++==================> ] 210,55M --.-KB/s eta 21m 40s
data.zip 55%[++++++++==================> ] 210,55M --.-KB/s eta 21m 43s
data.zip 55%[++++++++==================> ] 210,55M --.-KB/s eta 21m 45s
^C

Since in the above download process is disturbed again by the sign of the transferred file size is none, the process will be terminated by typing Ctrl+C.

user@hostname:~/Downloads$ wget -c http://www.mydomain.com/data.zip
--2016-09-25 11:27:11-- http://www.mydomain.com/data.zip
Resolving www.mydomain.com (www.mydomain.com)... 198.252.106.158
Connecting to www.mydomain.com (www.mydomain.com)|xxx.xxx.xxx.xxx|:80... connected.
HTTP request sent, awaiting response... 206 Partial content
Length: 397266014 (379M), 176491996 (168M) remaining [application/zip]
Saving to: ‘data.zip’
data.zip 100%[++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++=================================================>] 378,86M 157KB/s in 22m 39s
2016-09-25 11:49:50 (127 KB/s) - ‘data.zip’ saved [397266014/397266014]
user@hostname:~/Downloads$

The above shown the process has already been finished. It is shown by listing the content of file in the Download folder :

user@hostname:~/Downloads$ ls -al | grep data.zip
-rw-rw-r--   1 user user 397266014 Sep 24 10:18 data.zip
user@hostname:~/Downloads$

And to check whether the file is corrupted or not, it can be tested by extracting the compressed file :

user@hostname:~/Downloads$ unzip data.zip
inflating:
inflating:
inflating:
creating:
inflating:
creating:
...
...
creating:data/files

The extraction process went success.