Linux is famous for its command-line capabilities, and wget Command is one of those incredibly versatile tools that make managing files and data transfers a breeze. Whether you’re a beginner or have some experience, this guide will help you use wget on a linux operating system. We’ll explore everything from the basics of installation to advanced usage so that you can make the most of your VPS environment.
1. What is wget Command?
The command line tool wget is used to download files from the web. It’s named after “web get” and is designed for retrieving content non-interactively, which means you don’t need to sit and monitor downloads.
Key Features of wget
- Can handle HTTP, HTTPS, and FTP protocols
- Allows for resuming interrupted downloads
- Supports recursive downloading for websites
- Offers rate limiting and parallel downloading.
- Capable of handling authentication for secure servers
The tool comes pre-installed on many Linux systems, including Ubuntu, but we’ll walk through installing it in the next section if it needs to be added.
2. Installing wget on Ubuntu
To use wget, you first need to install it on your VPS. Follow these steps:
Step-by-Step Installation
Update your package list:
sudo apt update
- Install wget:
sudo apt install wget
- Verify the Installation: Confirm that the wget is installed by checking its version.
wget --version
It would be best if you now had the wget installed and ready to use.
Installing wget on CentOS/RHEL
On CentOS or RHEL, use yum or dnf (depending on your version) to install wget:
For Almalinux, RockyLinux:
sudo yum install wgetFor CentOS Stream 9 and newer RHEL versions:
sudo dnf install wgetOnce installed, you can confirm by checking the version:
sudo dnf install wget3. Basic Syntax of wget
The wget command is easy to understand once you know its basic syntax:
wget [options] [URL]- [options]: Optional parameters to customize wget behavior.
- [URL]: The address of the file or page you want to download.
4. Downloading Files with wget
Let’s start with some common scenarios you’ll encounter for downloading files on your Ubuntu VPS.
Downloading a Single File
To download a file from a URL:
wget https://example.com/file.zip
It will save the file with its original name in the current directory. To download and rename the file:
wget -O newfilename.zip https://example.com/file.zipResuming Downloads
If a download gets interrupted, you can resume it by adding the -c flag:
wget -c https://example.com/file.zip
5. Advanced Download Options
wget provides a range of options to customize downloads. Here are some useful ones:
Quiet Mode
If you don’t want wget to display download progress:
wget -q https://example.com/file.zipLimiting Download Speed
To control the download speed:
wget --limit-rate=100k https://example.com/file.zip
This command limits the download speed to 100 KB/s.
Setting User-Agent
If you get an ‘Access Denied’ message, it means that your IPs don’t allow it or that your credentials are wrong.
wget --user-agent="Mozilla/5.0" https://example.com/file.zip
6. Working with Recursive Downloads
A powerful feature of wget is recursive downloading, which is helpful for downloading entire websites or directories.
Basic Recursive Download
wget -r https://example.com/directory/Depth Limiting for Recursive Downloads
To specify a depth limit (e.g., only two levels deep):
wget -r -l 2 https://example.com/directory/
7. Limiting Bandwidth and Download Speed
Bandwidth management can be essential if you are on a VPS with limited network resources.
Setting a Download Speed Cap
As shown earlier, use the –limit-rate option to control speed. Example:
wget --limit-rate=500k https://example.com/largefile.zipPausing Between Downloads
To add a delay between requests:
wget --wait=10 -r https://example.com/directory/
8. Using wget with FTP and SFTP
wget works with FTP and even secure FTP (SFTP) for file transfers.
Downloading from an FTP Server
wget ftp://example.com/file.zip
For servers requiring login credentials, include the username and password:
wget ftp://username:[email protected]/file.zipSFTP (Secure FTP)
For SFTP, it’s best to use wget with other tools like sftp directly, as wget doesn’t natively support secure FTP.
9. Authentication with wget
When accessing restricted content, wget allows you to pass authentication information.
HTTP Authentication
wget --user=username --password=password https://example.com/securefile.zipStoring Credentials in a File
If you download files often, consider saving credentials in a .wgetrc file for security and convenience.
10. Working with Logs
For a record of your downloads, wget can write logs to a file.
Saving Logs to a File
wget -o logfile.txt https://example.com/file.zip
This action stores all output in logfile.txt for later review.
Appending Logs
To keep logs from different sessions in the same file:
wget -a logfile.txt https://example.com/file.zip
11. Using wget in Scripts
Automating downloads with wget is perfect for VPS setups where unattended downloads are common.
Writing a Simple Script
Create a script to download multiple files:
#!/bin/bash
wget https://example.com/file1.zip
wget https://example.com/file2.zip
wget https://example.com/file3.zipSave this file as download.sh, make it executable, and run it:
chmod +x download.sh
./download.sh
12. Common Issues and Troubleshooting
SSL Certificate Issues
If wget reports SSL issues, add the –no-check-certificate flag:
wget --no-check-certificate https://example.com/file.zip404 Errors
Double-check the URL, as a 404 error typically means the file or page isn’t available at the specified address.
Access Denied Errors
If you get an “Access Denied” error, confirm that your IP is allowed or your credentials are correct.
13. Conclusion
Learning the wget command on Ubuntu VPS opens up a world of convenience for managing and automating file downloads. From simple downloads to recursive website copying, wget offers powerful options suitable for any level of user. By mastering these commands and understanding how to customize them, you’ll be well-prepared to handle diverse downloading needs on your VPS.
About the writer
Vinayak Baranwal wrote this article. Use the provided link to connect with Vinayak on LinkedIn for more insightful content or collaboration opportunities.
