Understanding CURL Practically
Curl also known as Client Universal Resource Locator, is a command line tool that is used to check connectivity to URL and transfer data (it can also transfer multiple files) to/from web servers using protocols that are mandatory to mention while using curl, it supports protocols such as (HTTP, HTTPS, FTP, TELNET, LDAP, DICT etc.). It can be used in CMD (Windows) or Terminal (Linux) etc. for automation without any interaction of user. It is developed by a Swedish programmer named Daniel Stenberg. Curl uses free and IPV6 compatible library known as libcurl for its working.
One of the greatest features of curl is that in case of HTTPS, it performs SSL certificate verification by default by checking server’s certificate against its CA certificate and stores the result of verification. The result is stored if that server again tries to connect with us, so it can see if that server is valid or using any fake certificate. It will give error if any remote/destination server uses fake certificate.
Syntax: curl [options] [URL]
Practical Implementation in Linux
Firstly, we are checking the version of our curl using following command and see what extra information does it give.
So along with the version, this command is also showing the protocols that curl supports and features of our current version.
This command will render the source code of our specified domain, as no protocol is specified it will by default pick http or we can also say we are making a GET request to the website.
We can see in our case it is showing Moved Permanently, it is because our domain supports https protocol and this command by default using http.
In this case we have specified https protocol, as output of this command is very huge so we are storing the result in pratham.html file using ( -o ) and the little output of pratham.html file is shown below.
Now, we will see how to download files using curl
As we can see the first command is used to download file and it is showing every content such as time taken, speed taken, time left etc. The second command is used to continue our download process if in case our download hampers or stops using ( -C -) with command.
We can also limit the download rate using (- -limit-rate ).
We can see first file is the one in which data is stored by curl as previously shown and second file is the one that we downloaded now.
We have created a file called urls.txt and put some download URLs in it as shown above using cat command. Now we are going to download contents that these URLs represent.
So, we can see the files that are downloaded are represented by above mentioned URLs.
Now we will see how to query the headers from website using -I
In first command we haven’t specified the protocol and it by default using http and http headers allow the remote web server to send additional information about itself along with the actual request but our target website don’t support http.
In second command we have specified https protocol and output is as shown above. The output is very huge so we have only included some part of it.
Now we are going to see how to send post requests with parameters along with the website name as shown below.
In this we have passed Username and Password as parameters to the target vuln web website as a post request.
In this using –resolve we are trying to modify Name resolution as we are trying to send request to the website from localhost instead of our DNS in /etc/hosts file.
— cookie-jar is used to see what cookies are downloaded on my computer when I visit or browse the given site and we have stored the cookie in cookie.txt file. On using cat command, we can see the value and moreover we can use the retrieved cookies to the same site (but not to different site as it belongs to particular site only) as shown at last step.
For more information we can use help or manual command e.g. man curl
References
https://www.curl.haxx.se/docs
https://www.geeksforgeeks.org/curl-command-in-linux-with-examples/
Author- Pratham Mittal
VIEH group|www.viehgroup.com
