The Linux curl command is one of the most awesome web-related terminal tools out there. It provides security for transferring data across your computer and the server following protocols such as HTTP, HTTPS, and FTP. Curl has versatile options; it can be used to download files, interact with APIs, or get web pages. In this guide, you will learn how to make effective use of curl commands with examples.
What is Linux Curl Command?
Curl stands for “Client URL,” a command-line tool for transferring data using URLs. It can be used for many tasks, such as downloading web pages, uploading data, or interacting with REST APIs.
Installing Curl
First, verify if curl is installed by running:
curl --versionIf it’s not installed, you can install it using the following commands:
Debian/Ubuntu:
sudo apt update
sudo apt install curlRed Hat/CentOS/Alma Linux/Rocky Linux :
sudo yum install curlBasic Curl Usage
To fetch the content of a webpage:
curl https://www.example.comThe command will display the HTML content of the page in the terminal. You can save this content to a file using the -o or -O options.
Linux Curl Command and Their Use Cases
Below is a table showing various common curl commands, their descriptions, use cases, and examples.
| Command | Use Case | Example |
|---|---|---|
| curl https://example.com | Fetch the contents of a webpage | curl -o myfile.html https://www.example.com saves the webpage’s content to myfile.html. |
| curl -o file.html URL | Download a webpage and save with custom name | curl -o myfile.html https://www.example.com saves the webpage’s content to myfile.html. |
| curl -O URL | Download a file and save it with the original name | curl -O https://www.example.com/file.zip will download file.zip and save it with its original name. |
| curl -I URL | Fetch only HTTP headers | curl -I https://www.example.com shows only the HTTP headers like content type and server. |
| curl -d “data” -X POST URL | Send form data via POST | curl -d “name=John&age=30” -X POST https://example.com/form sends name=John and age=30 as form data using POST method. |
| curl -u user:pass URL | Access a site with authentication | curl -u user:password https://example.com authenticates using the provided credentials to access the site. |
| curl -L URL | Follow redirects | curl -L https://www.example.com will follow any redirects to another URL. |
| curl -T file.txt ftp://URL | Upload a file to an FTP server | curl -C – -O https://www.example.com/file.zip resumes a previously interrupted download. |
| curl –limit-rate 100K URL | Limit download speed | curl –limit-rate 100K -O https://example.com/file.zip limits the download speed to 100 KB per second. |
| curl -C – -O URL | Resume interrupted download | curl -C – -O https://www.example.com/file.zip resumes a download that was previously interrupted. |
| curl -H “Header: value” URL | Set custom headers | curl -H “Content-Type: application/json” https://example.com sets a custom header for sending a JSON request. |
Examples of Linux Curl Commands
Let’s go over some examples to illustrate further how to use the linux curl commands listed in the table.
Downloading Files with Linux Curl Command
Example 1: Download a webpage and save it as myfile.html:
curl -o myfile.html https://www.example.com
- This command downloads the HTML content of the page at https://www.example.com and saves it to myfile.html.
Example 2: Download a file with its original name:
curl -O https://www.example.com/file.zip
- The command will download
file.zipand save it with its original name in the current directory.
Fetching HTTP Headers
To fetch only the HTTP headers of a webpage (like status codes, content type, etc.), use:
curl -I https://www.example.com
This command will return the headers instead of all the HTML content.
Sending Data with Linux Curl Command
Example 1: Sending form data using POST:
curl -d "name=John&age=30" -X POST https://www.example.com/form
- The command sends form data (name and age) to the server using a POST request.
Example 2: Sending JSON data to an API:
curl -H "Content-Type: application/json" -d '{"name": "John", "age":30}' -X POST https://www.example.com/api
- The -H option is used to specify the content type and the data is sent in JSON format.
Resuming a Download
If your download was interrupted, you can resume it using the -C option:
curl -C - -O https://www.example.com/file.zip
The command resumes downloading file.zip from the point it was interrupted.
Uploading Files with Linux Curl Command
You have the option to transfer files to an FTP server. For example, to upload myfile.txt:
curl -T myfile.txt ftp://ftp.example.com --user username:password
The command uploads myfile.txt to the specified FTP server with the given username and password.
Using Proxies with Linux Curl Command
To route your request through a proxy, use the -x option. For example:
curl -x http://proxy.example.com:8080 https://www.example.com
The command instructs Curl to route the request through the specified proxy server.
Limiting Download Speed
Limit the download speed to 100 KB per second, use:
curl --limit-rate 100K -O https://www.example.com/file.zip
Such an approach is practical when you need to conserve bandwidth.
Following Redirects
By default, the Linux curl command does not follow HTTP redirects. You can use the -L option to make it follow them:
curl -L https://www.example.com
This checks that curl follows any redirects that the server might issue.
Using Linux Curl Command in Cron Jobs
One of the most common use cases for curl is running it as part of an automated process via cron jobs. For example, you should regularly trigger a script on your server by sending a request to a specific URL at scheduled intervals.
Silent Mode in Curl
The –silent (or -s) option is useful when you want the curl to run quietly without printing output or progress information to the terminal. It’s beneficial when using curl in a cron job because it prevents unnecessary output from filling up your system’s cron logs.
Example: Using Linux Curl Command in a Cron Job
Consider you have a script (cron.php) on your server that needs to be triggered every day at midnight. Here’s how you can set up a cron job that uses curl to request that script silently:
- Open the crontab file using this command:
crontab -e- Add the following line to schedule the job to run daily at midnight:
0 0 * * * curl --silent http://www.example.com/cron.php > /dev/nullExplanation:
- –silent: Telling curl to suppress progress and error messages is what this option is for so that curl doesn’t clutter your terminal the way it would otherwise.
- > /dev/null: This redirects the output to /dev/null, a particular file that discards all data written to it. In this case, it ensures that no output from the curl command is logged.
Why Use Silent Mode?
If curl wasn’t run in silent mode, it could print progress information or errors to the system logs, clattering them and making it difficult to identify important messages. Using –silent along with > /dev/null ensures the cron job runs cleanly without unnecessary output.
Wrapping Up
Curl is a powerful and capable tool for simplifying many tasks: downloading files, fetching web pages, HTTP requests, content manipulation, direct data upload to the server, etc. It is very flexible and is required by (most) system administrators, developers, and casual Linux users. With the knowledge we’ve gained about handling these commands, you can already do multiple things in your web environment directly from your terminal, which allows for some nice workflow streamlining.
Curl, like any tool, could be better. However, considering the vast number of options and syntax can be overwhelming. Some flags (such as -L, to not follow redirects) can be misused with disastrous results, and underlying APIs can be unique, with headers or payloads that are hard to navigate at first. Downloading large files with a—- limit rate will also consume network resources and may rob the rest of your system of resources.
Such error handling has other potential difficulties. For example, sometimes, with failed API responses, curl doesn’t clearly tell us when an error occurs. To debug better, you might want to add additional flags such as—f (seeing the program fail on HTTP errors without printing output) or—v (print verbose output).
With some minor hurdles, these are over. Curl is a handy tool for gaining familiarity with interacting with web APIs, which significantly increases your productivity as you begin to use it in your day-to-day work.
Summary of Key Commands:
- Download file: curl -O https://example.com/file
- POST form data: curl -d “name=value” -X POST https://example.com
- Resume download: curl -C – -O https://example.com/file
- Follow redirects: curl -L https://example.com
- Set headers: curl -H “Header: value” https://example.com
About the writer
Vinayak Baranwal wrote this article. Use the provided link to connect with Vinayak on LinkedIn for more insightful content or collaboration opportunities.
