Use wget as download manager with some xargs magic

I needed a script that would do the following

• Read a list with files to download
• Control the number of parallel downloads
• Control the consumed bandwidth for each download
• Resume partial downloads
• Ignore downloaded files

wget supports everything out of the box, except parallel downloads which can be achieved using wget with xargs

/home/kpatronas/download_test

Parameters explainationurls.txt : The list with the files to be downloaded
-P30 : The number of parallel downloads, 30 in this case
-nc : Ignore downloaded files
--limit-rate : Limit download rate per url, in this case is 50Kb per download
-c : Resume partial downloads
-P /home/kpatronas/download_test : The directory to download the files, in this case a directory in my home dir.

I hope you found this short article useful :)

Join Medium with my referral link - Konstantinos Patronas

As a Medium member, a portion of your membership fee goes to writers you read, and you get full access to every story…