cURL is a powerful command-line tool used for transferring data to or from a server. It supports a wide range of protocols, including HTTP, HTTPS, FTP, and more. The versatility of cURL makes it a popular choice for various tasks, including downloading files. In this article, we will explore five real-life examples to help you master the art of file downloads with cURL.
Example 1: Basic File Download
The most basic usage of cURL for downloading a file involves providing the URL of the file you want to download. The following command downloads the file from the given URL and saves it to the current directory with the same name as the remote file:
curl -O https://example.com/path/to/file.txt
Example 2: Downloading a File with a Custom Name
If you want to download a file and save it with a custom name, use the -o option followed by the desired filename:
curl -o custom_file_name.txt https://example.com/path/to/file.txt
In this example, the remote file will be downloaded and saved to the current directory as custom_file_name.txt.
Example 3: Downloading Multiple Files
cURL can also download multiple files in a single command. To download multiple files, provide the URLs separated by spaces and use the -O option for each URL:
curl -O https://example.com/path/to/file1.txt -O https://example.com/path/to/file2.txt
In this example, cURL will download both `file1.txt` and `file2.txt` from the specified URLs and save them to the current directory with their original names.
Example 4: Downloading Files with Authentication
In some cases, you may need to download files that require authentication. cURL supports various authentication methods, such as Basic, Digest, NTLM, and more. To provide your username and password for authentication, use the `-u` option followed by the credentials in the format `username:password`:
curl -u username:password -O https://example.com/secure/path/to/file.txt
This command downloads the file from the secure URL while providing the required authentication credentials.
Note: Be cautious when using this method, as your password will be visible in the command line history. Consider alternative methods like using a .netrc file or environment variables to store your credentials securely.
Example 5: Downloading a File with Resume Capability
In case a file download is interrupted or fails, cURL allows you to resume the download from where it left off. To resume a download, use the -C - option:
curl -C - -O https://example.com/large/path/to/file.txt
This command checks the size of the partially downloaded file and resumes the download from that point. If the file is not present, it will start the download from the beginning.
Example 6: Downloading Files with Progress Meter
By default, cURL displays a progress meter while downloading files. However, you can opt for a more concise progress bar using the -# option:
curl -# -O https://example.com/path/to/file.txt
This command downloads the file while showing a progress bar instead of the default progress meter.
Example 7: Downloading Files over FTP
cURL supports downloading files over FTP as well. To download a file from an FTP server, provide the FTP URL and your username and password (if required):
curl -u username:password -O ftp://ftp.example.com/path/to/file.txt
This command downloads the file from the specified FTP server using the provided credentials.
Example 8: Following Redirects during File Download
Sometimes, a file download URL may redirect to another location. By default, cURL does not follow redirects. To make cURL follow redirects, use the -L option:
curl -L -O https://example.com/path/to/redirected_file.txt
This command follows any redirects and downloads the file from the final URL.
Example 9: Limiting Download Speed
If you want to limit the download speed to prevent cURL from consuming all available bandwidth, use the –limit-rate option followed by the desired speed limit:
curl --limit-rate 100K -O https://example.com/path/to/file.txt
In this example, the download speed is limited to 100 kilobytes per second. You can adjust the value and use `K` for kilobytes, `M` for megabytes, or `G` for gigabytes, as needed.
Example 10: Downloading Files with a Proxy
If you need to download a file through a proxy server, provide the proxy details using the `-x` or `--proxy` option:
curl -x http://proxy.example.com:8080 -O https://example.com/path/to/file.txt
In this example, cURL downloads the file using the specified HTTP proxy server on port 8080. You can use other proxy protocols, such as HTTPS or SOCKS, by changing the proxy URL scheme accordingly.
Conclusion
cURL is a versatile and powerful tool for downloading files from the command line. This article provided five real-life examples to help you master file downloads with cURL, including basic file downloads, custom file names, downloading multiple files, handling authentication, and resuming interrupted downloads. By understanding and implementing these examples, you can efficiently use cURL for a wide range of file download scenarios and enhance your command-line skills. As you continue to explore the capabilities of cURL, you will discover even more advanced features and techniques to optimize your file transfers and improve your overall productivity.