home > Computer > What does curl do? Command Line Tools for Web Developer

What does curl do? Command Line Tools for Web Developer

We often have to download various files from the Internet, for example, executable files programs, script files, source archives. But this does not always need to be done through the browser. In many situations it is much easier to perform all actions through the terminal. Because this way you can automate the process. On the other hand, webmasters from time to time have to test website accessibility, check sent and received headers, and much more.

To solve such problems and problems of a similar range, you can use the curl utility. It allows you to solve a much wider range of problems, including even simulating user actions on the site. In this article we will look at how to use curl, what it is and why this program is needed.

In fact, curl is more than just a command line utility for Linux or Windows. This is a set of libraries that implement basic capabilities for working with URL pages and file transfer. The library supports working with protocols: FTP, FTPS, HTTP, HTTPS, TFTP, SCP, SFTP, Telnet, DICT, LDAP, as well as POP3, IMAP and SMTP. It is great for simulating user actions on pages and other operations with URLs.

Support for the curl library has been added to many different programming languages and platforms. The curl utility is an independent wrapper for this library. It is this utility that we will focus on in this article.

curl command

Before moving on to a description of how the curl linux command can be used, let's look at the utility itself and its main options that we will need. The syntax of the utility is very simple:

$ curl options link

Now let's look at the main options:

-# - display a simple progress bar during loading;
-0 - use the http 1.0 protocol;
-1 - use the tlsv1 encryption protocol;
-2 - use sslv2;
-3 - use sslv3;
-4 - use ipv4;
-6 - use ipv6;
-A- indicate your USER_AGENT;
-b- save Cookie to a file;
-c- send Cookie to the server from a file;
-C- continue downloading the file from the break point or specified offset;
-m- maximum waiting time for a response from the server;
-d- send data using the POST method;
-D- save headers returned by the server to a file;
-e- set the Referer-uri field, indicating which site the user came from;
-E- use an external SSL certificate;
-f- do not display error messages;
-F- send data in the form of a form;
-G- if this option is enabled, then all data specified in the -d option will be transmitted using the GET method;
-H- transfer headers to the server;
-I- receive only the HTTP header and ignore the entire page content;
-j- read and send cookies from a file;
-J- remove header from request;
-L- accept and process redirects;
-s- maximum number of redirections using Location;
-o- output page content to a file;
-O- save content to a file with the name of the page or file on the server;
-p- use a proxy;
--proto- indicate the protocol to be used;
-R- save time last change remote file;
-s- display a minimum of information about errors;
-S- display error messages;
-T- upload the file to the server;
-v- the most detailed output;
-y- minimum download speed;
-Y- maximum download speed;
-z- download the file only if it was modified later than the specified time;
-V- display the version.

This is by no means all of the options for curl linux, but it lists the basics that you will need to use.

How to use curl?

We've covered everything related to the theory of working with the curl utility, now it's time to move on to practice and look at examples of the curl command.

The most common task is this. Downloading the file is very simple. To do this, just pass the file name or html page to the utility in the parameters:

curl https://raw.githubusercontent.com/curl/curl/master/README.md

But here one surprise awaits you: the entire contents of the file will be sent to standard output. To write it to any file use:

curl -o readme.txt https://raw.githubusercontent.com/curl/curl/master/README.md

And if you want the resulting file to be named the same as the file on the server, use the -O option:

curl -O https://raw.githubusercontent.com/curl/curl/master/README.md

curl -# -C - -O https://cdn.kernel.org/pub/linux/kernel/v4.x/testing/linux-4.11-rc7.tar.xz

If necessary, you can download several files with one command:

curl -O https://raw.githubusercontent.com/curl/curl/master/README.md -O https://raw.githubusercontent.com/curl/curl/master/README

Another thing that may be useful for an administrator is to only download a file if it has been modified:

curl -z 21-Dec-17 https://raw.githubusercontent.com/curl/curl/master/README.md -O https://raw.githubusercontent.com/curl/curl/master/README

Speed Limit

You can limit the download speed to the required limit so as not to overload the network using the -Y option:

curl --limit-rate 50K -O https://cdn.kernel.org/pub/linux/kernel/v4.x/testing/linux-4.11-rc7.tar.xz

Here you need to specify the number of kilobytes per second that can be downloaded. You can also terminate the connection if the speed is not enough, use the -Y option to do this:

curl -Y 100 -O https://raw.githubusercontent.com/curl/curl/master/README.md

Transferring files

curl -T login.txt ftp://speedtest.tele2.net/upload/

Or let’s check that the file is sent via HTTP; there is a special service for this:

curl -T ~/login.txt http://posttestserver.com/post.php

In the response, the utility will tell you where you can find the downloaded file.

Sending POST data

You can send not only files, but also any data using the POST method. Let me remind you that this method is used to send data of various forms. To send such a request, use the -d option. For testing we will use the same service:

curl -d "field1=val&fileld2=val1"http://posttestserver.com/post.php

If you are not happy with this submission option, you can pretend to submit the form. There is an option for this -F:

curl -F "password=@pass;type=text/plain" http://posttestserver.com/post.php

Here we pass the password field with the form as plain text, in the same way you can pass several parameters.

Sending and receiving cookies

Cookies are used by websites to store certain information on the user's side. This may be necessary, for example, for authentication. You can send and receive Cookies using curl. To save the received Cookies to a file, use the -c option:

curl -c cookie.txt http://posttestserver.com/post.php

You can then send the curl cookie back:

curl -b cookie.txt http://posttestserver.com/post.php

Header transmission and analysis

We don't always necessarily need the content of the page. Sometimes only the headlines can be interesting. To display only them there is the -I option:

curl -I https://site

And the -H option allows you to send several or more to the server, for example, you can pass the If-Modified-Since header so that the page is returned only if it has been modified:

curl authentication

If the server requires one of the common types of authentication, such as HTTP Basic or FTP, then curl can handle this task very easily. To specify authentication details, simply specify them separated by a colon in the -u option:

curl -u ftpuser:ftppass -T - ftp://ftp.testserver.com/myfile_1.txt

Authentication on HTTP servers will be performed in the same way.

Using a proxy

If you need to use a proxy server to download files, then that is also very simple. It is enough to specify the proxy server address in the -x option:

curl -x proxysever.test.com:3128 http://google.co.in

conclusions

In this article, we looked at how to use curl, why this utility is needed and its main capabilities. Despite their similarity with, they are very different. The curl linux command is more for analysis and simulation various actions on the server, while wget is more suitable for downloading files and crawling sites.

The life of a web developer is clouded by difficulties. It is especially unpleasant when the source of these difficulties is unknown. Is this a problem with sending the request, or with the response, or with a third-party library, or is the external API buggy? There are a bunch of different gadgets that can make our lives easier. Here are some command line tools that I personally find invaluable.

cURL
cURL is a program for transferring data over various protocols, similar to wget. The main difference is that by default wget saves to a file, while cURL outputs to the command line. This makes it very easy to view the website content. For example, here's how to quickly get your current external IP:

$ curl ifconfig.me 93.96.141.93
Options -i(show titles) and -I(show headers only) make cURL an excellent tool for debugging HTTP responses and analyzing what exactly the server is sending you:

$ curl -I habrahabr.ru HTTP/1.1 200 OK Server: nginx Date: Thu, 18 Aug 2011 14:15:36 GMT Content-Type: text/html; charset=utf-8 Connection: keep-alive Keep-alive: timeout=25
Parameter -L Also useful, it forces cURL to automatically follow redirects. cURL supports HTTP authentication, cookies, HTTP proxy tunneling, manual settings in the headlines and much, much more.

Siege
Siege is a load testing tool. Plus, it has a convenient option -g, which is very similar to curl –iL, but in addition it also shows you the http request headers. Here's an example from google.com (some headings removed for brevity):

$ siege -g www.google.com GET / HTTP/1.1 Host: www.google.com User-Agent: JoeDog/1.00 (X11; I; Siege 2.70) Connection: close HTTP/1.1 302 Found Location: http:// www.google.co.uk/ Content-Type: text/html; charset=UTF-8 Server: gws Content-Length: 221 Connection: close GET / HTTP/1.1 Host: www.google.co.uk User-Agent: JoeDog/1.00 (X11; I; Siege 2.70) Connection: close HTTP/ 1.1 200 OK Content-Type: text/html; charset=ISO-8859-1 X-XSS-Protection: 1; mode=block Connection: close
But what Siege is really great for is load testing. Like the Apache benchmark ab, it can send many parallel requests to the site and see how it handles the traffic. The following example shows how we test Google with 20 queries for 30 seconds and then print the result:

$ siege -c20 www.google.co.uk -b -t30s ... Lifting the server siege... done. Transactions: 1400 hits Availability: 100.00 % Elapsed time: 29.22 secs Data transferred: 13.32 MB Response time: 0.41 secs Transaction rate: 47.91 trans/sec Throughput: 0.46 MB/sec Concurrency: 19.53 Successful transactions: 1400 Failed transactions: 0 Longest transaction: 4.08 Shortest transaction: 0.08
One of the most useful features of Siege is that it can work not only with a single address, but also with a list of URLs from a file. This is great for load testing because you can simulate real site traffic rather than just hitting the same URL over and over again. For example, here's how to use Siege to load a server using addresses from your Apache log:

$ cut -d " " -f7 /var/log/apache2/access.log > urls.txt $ siege -c -b -f urls.txt
Ngrep
For serious traffic analysis, there is Wireshark with thousands of settings, filters and configurations. There is also a command line version tshark. But for simple tasks, I consider Wireshark’s functionality redundant. So unless I need a powerful weapon, I use the . It allows you to do the same thing with network packets as grep does with files.

For web traffic, you will almost always want to use the parameter -W to preserve string formatting as well as the option -q, which hides redundant information about unmatched packets. Here is an example command that intercepts all packets with a GET or POST command:

Ngrep -q -W byline "^(GET|POST) .*"
You can add an additional filter for packets, for example, by a given host, IP address or port. Here is a filter for all traffic to and from google.com, port 80, which contains the word “search”.

Ngrep -q -W byline "search" host www.google.com and port 80

cURL is a tool that allows you to interact with various servers and supports many protocols: HTTP, FTP, TELNET, etc. cURL is originally a command line utility. But, fortunately for us, PHP supports working with the cURL library. In this article we will look at non-trivial examples of working with cURL.

Why cURL?

In fact, there are many other ways to send a request to another server to, for example, retrieve the content of a page. Many people, mostly out of laziness, use simple PHP functions, instead of cURL:

$content = file_get_contents("http://www.example.com"); // or $lines = file("http://www.example.com"); // or readfile("http://www.example.com");

However, they do not allow for efficient error handling. There are also a number of tasks that they cannot do at all - for example, working with cookies, authorization, post requests, downloading files.

cUrl is a powerful tool that supports multiple protocols and provides complete request information.

cUrl Basics

Before moving on to complex examples, let's look at the basic structure of a cURL request in PHP. To perform a cURL request in PHP, you need to take 4 main steps:

Initialization.
Setting options.
Executing the request.
Cleaning resources.

// 1. initialization $ch = curl_init(); // 2. set options, including URL curl_setopt($ch, CURLOPT_URL, "http://www.google.com"); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); curl_setopt($ch, CURLOPT_HEADER, 0); // 3. executing the request and receiving the response $output = curl_exec($ch); // 4. cleaning resources curl_close($ch);

We'll mostly be looking at step #2 in this article since that's where the magic happens. The list of cURL options is very large, so we will not consider all the options today, but will use those that are useful for solving specific problems.

Bug tracking

If necessary, you can add the following lines to track errors:

// ... $output = curl_exec($ch); if ($output === FALSE) ( echo "cURL Error: " . curl_error($ch); ) // ...

Please note that we use “===” instead of “==” because It is necessary to distinguish between an empty server response and the Boolean value FALSE, which is returned in case of an error.

Getting information about a request

Another optional step is to obtain information about the cURL request after it has been executed.

// ... curl_exec($ch); $info = curl_getinfo($ch); echo "Took " . $info["total_time"] . "seconds for url". $info["url"]; // ...

As a result, you will receive an array with the following information:

"url"
"content_type"
"http_code"
"header_size"
"request_size"
"filetime"
"ssl_verify_result"
"redirect_count"
"total_time"
"namelookup_time"
"connect_time"
"pretransfer_time"
"size_upload"
"size_download"
"speed_download"
"speed_upload"
"download_content_length"
"upload_content_length"
"starttransfer_time"
"redirect_time"

Redirect tracking, depending on the browser

In this example, we will write a script that will detect redirects based on different browser settings. For example, some sites redirect visitors from mobile devices, visitors from other countries.

We'll use the CURLOPT_HTTPHEADER option to set our own headers, including User-Agent and Language, and see where sites redirect us.

// URLs $urls = array("http://www.cnn.com", "http://www.mozilla.com", "http://www.facebook.com"); // browsers $browsers = array("standard" => array ("user_agent" => "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.6) Gecko/20091201 Firefox/3.5. 6 (.NET CLR 3.5.30729)", "language" => "en-us,en;q=0.5"), "iphone" => array ("user_agent" => "Mozilla/5.0 (iPhone; U; CPU like Mac OS X; en) AppleWebKit/420+ (KHTML, like Gecko) Version/3.0 Mobile/1A537a Safari/419.3", "language" => "en"), "french" => array ("user_agent" = > "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; GTB6; .NET CLR 2.0.50727)", "language" => "fr,fr-FR;q=0.5")); foreach ($urls as $url) ( echo "URL: $url\n"; foreach ($browsers as $test_name => $browser) ( $ch = curl_init(); // set the address curl_setopt($ch, CURLOPT_URL, $url); // indicate the browser and language used curl_setopt($ch, CURLOPT_HTTPHEADER, array("User-Agent: ($browser["user_agent"])", "Accept-Language: ($browser["language"]) ")); // we don't need the contents of the page curl_setopt($ch, CURLOPT_NOBODY, 1); // we only need headers curl_setopt($ch, CURLOPT_HEADER, 1); // return the result instead of its output curl_setopt($ch, CURLOPT_RETURNTRANSFER , 1); $output = curl_exec($ch); curl_close($ch); // define redirections in HTTP headers? if (preg_match("!Location: (.*)!", $output, $matches)) ( echo "$test_name: redirects to $matches\n"; ) else ( echo "$test_name: no redirection\n"; ) ) echo "\n\n"; )

In a loop we check browsers for each URL. First we set the options for our request: URL and browser and language to be tested.

Because we installed special option, the result of the request will contain only HTTP headers. Using a simple regular expression, we can check if the response contains the string "Location:".

Result of script execution:

URL: http://www.cnn.com standard: redirects to http://edition.cnn.com/ iphone: redirects to http://edition.cnn.com/ French: redirects to http://edition.cnn .com/ URL: http://www.mozilla.com standard: redirects to https://www.mozilla.org/firefox/ iphone: redirects to https://www.mozilla.org/firefox/ french: redirects to https://www.mozilla.org/firefox/ URL: http://www.facebook.com standard: redirects to https://www.facebook.com/ iphone: redirects to http://m.facebook.com /?refsrc=http%3A%2F%2Fwww.facebook.com%2F&_rdr French: no redirection

Sending POST requests

When performing GET requests, data can be passed in the query string. For example, when you search on Google, your query is translated into a URL:

http://www.google.com/search?q=google

To get the result of this query you don't even need cURL, you can be lazy and use "file_get_contents()".

But some HTML forms use POST method. In this case, the data is sent in the body of the request message rather than in the URL itself.

Let's write a script that will send POST requests. First, let's create a simple PHP file, which will accept these requests and return the data sent to it. Let's call it post_output.php :

$url = "http://localhost/post_output.php"; $post_data = array ("foo" => "bar", "query" => "FooBar", "action" => "Submit"); $ch = curl_init(); curl_setopt($ch, CURLOPT_URL, $url); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); // make a POST request curl_setopt($ch, CURLOPT_POST, 1); // add data curl_setopt($ch, CURLOPT_POSTFIELDS, $post_data); $output = curl_exec($ch); curl_close($ch); echo $output;

This script will output:

Array ( => bar => FooBar => Submit)

This script sent a POST request to the post_output.php file. which outputted the contents of the $_POST array and we received this response using cURL.

Uploading files

Just like in the previous example, let's create a file that will accept requests, upload_output.php :

Print_r($_FILES);

And the script itself that downloads the files:

$url = "http://localhost/upload_output.php"; $post_data = array ("foo" => "bar", // file to upload "upload" => "@/tmp/desert.jpg"); $ch = curl_init(); curl_setopt($ch, CURLOPT_URL, $url); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); curl_setopt($ch, CURLOPT_POST, 1); curl_setopt($ch, CURLOPT_POSTFIELDS, $post_data); $output = curl_exec($ch); curl_close($ch); echo $output;

If you want to upload a file, all you need to do is pass the path to it, just like a normal POST request parameter, prefixed with "@". The result of the script:

Array ( => Array ( => desert.jpg => application/octet-stream => /tmp/phpAhEvXy => 0 => 845941))

Multi cURL

One of the advanced features of cURL in PHP is the ability to execute multiple requests simultaneously and asynchronously.

Under normal conditions, the script stops and waits for the request to complete. And if you need to execute a lot of queries, it can take a lot of time, because... you will perform sequentially. This limitation can be bypassed:

// create handlers $ch1 = curl_init(); $ch2 = curl_init(); // set options curl_setopt($ch1, CURLOPT_URL, "http://lxr.php.net/"); curl_setopt($ch1, CURLOPT_HEADER, 0); curl_setopt($ch2, CURLOPT_URL, "http://www.php.net/"); curl_setopt($ch2, CURLOPT_HEADER, 0); //create the multiple cURL handle $mh = curl_multi_init(); // add handlers curl_multi_add_handle($mh,$ch1); curl_multi_add_handle($mh,$ch2); $running = null; // execute requests do ( curl_multi_exec($mh, $running); ) while ($running > 0); // free up resources curl_multi_remove_handle($mh, $ch1); curl_multi_remove_handle($mh, $ch2); curl_multi_close($mh);

The idea is that you can create multiple cURL handles, combine them under one multi-handle, and execute them asynchronously.

First, everything is the same as with a regular cURL request - a descriptor is created ( curl_init() ), parameters are set ( curl_setopt() ). Next, a multi-descriptor is created ( curl_multi_init() ) and the previously created regular descriptors are added ( curl_multi_add_handle() ). Instead of calling curl_exec() normally, we will call curl_multi_exec() this function informs us about the number of active connections using the second parameter - $running. Therefore, the loop runs until $running becomes equal to 0. And, of course, after finishing the work, it is necessary to release resources.

In this example, we simply output the result of the queries to STDOUT. Let's consider a non-trivial case of using multi cURL.

Checking External Links in WordPress

Imagine a blog with big amount posts containing links to external sites. Some of these links may not work.

Let's write a script that will find all broken links and show them to us.

First, we need to pull all external links from the database:

// CONFIG $db_host = "localhost"; $db_user = "root"; $db_pass = ""; $db_name = "wordpress"; $excluded_domains = array("localhost", "site"); $max_connections = 10; $url_list = array(); $working_urls = array(); $dead_urls = array(); $not_found_urls = array(); $active = null; // connect to MySQL if (!mysql_connect($db_host, $db_user, $db_pass)) ( die("Could not connect: " . mysql_error()); ) if (!mysql_select_db($db_name)) ( die("Could not select db: " . mysql_error()); ) // take all posts with links in the text $q = "SELECT post_content FROM wp_posts WHERE post_content LIKE "%href=%" AND post_status = "publish" AND post_type = "post" "; $r = mysql_query($q) or die(mysql_error()); while ($d = mysql_fetch_assoc($r)) ( // collect all links using regular expression if (preg_match_all("/href=\"(.*?)\"/", $d["post_content"], $matches )) ( foreach ($matches as $url) ( // filter out unnecessary domains $tmp = parse_url($url); if (isset($tmp["host"]) && in_array($tmp["host"], $ excluded_domains)) ( continue; ) // put together $url_list = $url; ) ) ) // remove repetitions $url_list = array_values(array_unique($url_list)); if (!$url_list) ( die("No URL to check"); )

In this part of the script, we simply pull out all external links from the database. Let's check them:

$mh = curl_multi_init(); // 1. add links for ($i = 0; $i< $max_connections; $i++) { add_url_to_multi_handle($mh, $url_list); } // основной цикл do { curl_multi_exec($mh, $curRunning); // 2. один из потоков завершил работу if ($curRunning != $running) { $mhinfo = curl_multi_info_read($mh); if (is_array($mhinfo) && ($ch = $mhinfo["handle"])) { // 3. один из запросов выполнен, можно получить информацию о нем $info = curl_getinfo($ch); // 4. нерабочая ссылка if (!$info["http_code"]) { $dead_urls = $info["url"]; // 5. 404? } else if ($info["http_code"] == 404) { $not_found_urls = $info["url"]; // 6. верная ссылка } else { $working_urls = $info["url"]; } // 7. удаляем отработавший дескриптор curl_multi_remove_handle($mh, $mhinfo["handle"]); curl_close($mhinfo["handle"]); // 8. добавим новый урл add_url_to_multi_handle($mh, $url_list); $running = $curRunning; } } } while ($curRunning >0); curl_multi_close($mh); echo "==Dead URLs==\n"; echo implode("\n", $dead_urls) . "\n\n"; echo "==404 URLs==\n"; echo implode("\n", $not_found_urls) . "\n\n"; echo "==Working URLs==\n"; echo implode("\n", $working_urls); echo "\n\n"; // 9. adds a handle with the given URL function add_url_to_multi_handle($mh, $url_list) ( static $index = 0; // if there are still links if (isset($url_list[$index])) ( // everything is as usual $ ch = curl_init(); // set options curl_setopt($ch, CURLOPT_URL, $url_list[$index]); // return rather than display the result curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); // allow redirects curl_setopt($ ch, CURLOPT_FOLLOWLOCATION, 1); // get only headers to save time curl_setopt($ch, CURLOPT_NOBODY, 1); // add to the multi-handle curl_multi_add_handle($mh, $ch); $index++; ) )

Let's look at the code in more detail (the numbering corresponds to the comments in the code):

We add an initial number of descriptors so as not to overload the system with threads. The number is controlled by the $max_connections variable.
The $curRunning variable stores the number of running threads, $running stores the previous value; if they become unequal, then one of the threads has completed work.
We receive information about the completed request.
If there is no response from the server, the link is not working.
The server response is 404.
Otherwise the link works.
The request is completed, we release resources.
Let's add a new URL to the multi descriptor.
Function add_url_to_multi_handle() adds a new handle with the given URL to the multi-descriptor.

Let's run the script:

Dead URLs== xample1234.com/ ==404 URLs== www.google.com/dsfasdfafd ==Working URLs== ru.php.net/manual/ru/function.time.php www.cssbuttongenerator.com/ csslint. net/ codex.wordpress.org/Plugin_API/Action_Reference fortawesome.github.io/Font-Awesome/ fortawesome.github.io/Font-Awesome/ www.oracle.com/technetwork/java/javafx/downloads/index.html codex. wordpress.org/Plugin_API/Filter_Reference codex.wordpress.org/Roles_and_Capabilities code.google.com/p/google-api-php-client/wiki/OAuth2#Google_APIs_Console jplayer.org/ code.google.com/p/google-api -php-client/ developers.google.com/+/ accounts.google.com/ServiceLogin?service=devconsole&passive=1209600&continue=https%3A%2F%2Fcode.google.com%2Fapis%2Fconsole%2F&followup=https%3A%2F %2Fcode.google.com%2Fapis%2Fconsole%2F daneden.github.io/animate.css/ github.com/daneden/animate.css ru2.php.net/manual/ru/function.autoload.php www.google. com/recaptcha/api/verify phpunit.de/phpunit.de/manual/current/en/phpunit-book.html

The check took about 2 seconds. By running 10 threads simultaneously, performance increases 10 times compared to regular cURL requests. To get the contents of the server response, use the function curl_multi_getcontent($ch) , where $ch is a descriptor obtained from curl_multi_info_read() .

Other cURL features in PHP

HTTP authentication

If the HTTP request requires authentication, use following code:

$url = "http://www.somesite.com/members/"; $ch = curl_init(); curl_setopt($ch, CURLOPT_URL, $url); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); // send username and password curl_setopt($ch, CURLOPT_USERPWD, "myusername:mypassword"); // if redirects are allowed curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1); // cURL will send the password after redirects curl_setopt($ch, CURLOPT_UNRESTRICTED_AUTH, 1); $output = curl_exec($ch); curl_close($ch);

Upload via FTP

PHP has its own library for working with FTP, but you can also use cURL:

// read the file $file = fopen("/path/to/file", "r"); // url already contains the necessary data $url = "ftp://username: [email protected]:21/path/to/new/file"; $ch = curl_init(); curl_setopt($ch, CURLOPT_URL, $url); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); // options curl_setopt($ch, CURLOPT_UPLOAD, 1); curl_setopt($ch, CURLOPT_INFILE, $fp); curl_setopt($ch, CURLOPT_INFILESIZE, filesize("/path/to/file")); curl_setopt($ch, CURLOPT_FTPASCII, 1); $output = curl_exec($ ch); curl_close($ch);

Using a proxy

Requests can be made through a specific proxy:

$ch = curl_init(); curl_setopt($ch, CURLOPT_URL,"http://www.example.com"); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); // proxy address curl_setopt($ch, CURLOPT_PROXY, "11.11.11.11:8080"); // if authorization is required curl_setopt($ch, CURLOPT_PROXYUSERPWD,"user:pass"); $output = curl_exec($ch); curl_close($ch);

Callback functions

It is possible to use callbacks while the request is running, without waiting for it to complete. For example, while the server response is being downloaded, we can use the data already received without waiting for the complete download.

$ch = curl_init(); curl_setopt($ch, CURLOPT_URL,"http://example.com"); curl_setopt($ch, CURLOPT_WRITEFUNCTION,"progress_function"); curl_exec($ch); curl_close($ch); function progress_function($ch,$str) ( echo $str; return strlen($str); )

The callback function must return the length of the string for the request to work correctly.

Each time the next part of the server response is received, a callback will be called.

Conclusion

In this article, we looked at the advanced features of cURL in PHP. Next time you need to do URL requests- use cURL.

Using libcurl with C and Python

Developing applications that rely on application layer protocols such as HTTP and FTP is not very difficult, but it is not trivial either. Moreover, such development is not the focus, since in most cases what lies above these protocols is much more important. What's interesting about Libcurl is that it puts the emphasis on the application rather than the transitory aspect of development. Note that not many applications have their own TCP/IP stack - after all reuse everything possible, minimizes the load on the programmer and increases the reliability of the application.

This article begins with a brief introduction to protocols application level and then moves on to learn cURL, libcurl and how to use them.

Web protocols

Modern application development is significantly different from what it was in the recent past. Today, applications must communicate over a network or the Internet by providing users with a network API or interface, and provide flexibility through custom scripting. Typically, modern applications export the Web interface using HTTP and transmit notifications about abnormal situations using Simple Mail Transport Protocol (SMTP). These protocols allow the Web browser running on the device to transmit configuration or status and receive standard messages from devices to a standard client. Email(via HTTP and SMTP, respectively).

These Web services are typically built on top of the socket stack layer network protocols(picture 1). The socket layer implements an API that dates back to operating system Berkeley Software Distribution (BSD) and isolates the details of the underlying transport and network layer protocols.

Figure 1. Network protocol stack and libcurl

Web services provide interaction between client and server protocols. In the HTTP context, the server is the end device and the client is the browser at the remote location. For SMTP, the server is the mail gateway or remote user, and the client is the end device. In some cases, protocol interactions occur in two stages (request and response), while in others, much more traffic is required to maintain communication. This interaction can create significant difficulties, which are overcome by using APIs such as libcurl.

Introduction to cURL

Origin and content of cURL

cURL was proposed by Daniel Stenberg, but over 600 programmers contributed to its development. There is no doubt that this is a useful technology with a wide range of applications.

cURL was originally designed as a means of moving files between endpoints using various protocols such as FTP, HTTP, SCP and others. Originally a command line utility, it is now also a library with bindings for over 30 languages. So now, instead of using cURL from the command line, you can create applications that include these important features. The libcurl library is also portable and supports Linux®, IBM® AIX®, BSD, Solaris, and many other UNIX® variants.

Getting and installing cURL/libcurl

Obtaining and installing libcurl is easy, but the process varies depending on the Linux distribution. On Ubuntu, these packages can be installed using the apt-get utility. The next two lines show how to install libcurl and the Python bindings for libcurl:

$ sudo apt-get install libcurl3 $ sudo apt-get install python-pycurl

The apt-get utility ensures that all dependencies are met during the installation process.

cURL on the command line

cURL began as a command line tool for transferring data using Uniform Resource Locator (URL) syntax. Due to the popularity of this feature in command line form, a library was created to integrate it into applications. Today, cURL for the command line serves as a wrapper around the cURL library. This article starts by exploring cURL on the command line and then dives into using the utility as a library.

Two typical uses of cURL are file transfer using the HTTP and FTP protocols. cURL provides a simple interface to these and other protocols. To retrieve a file from a Web site using HTTP, you simply provide cURL with the name of the local file in which you want to write the Web page, as well as the URL of the Web site and the file to transfer. That's a lot of words for the simple command line shown in Listing 1.

Listing 1. Example of using cURL to retrieve a file from a Web site

$ curl -o test html www.exampledomain.com % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 43320 100 43320 0 0 55831 0 --:--:-- --:-- :-- --:--:-- $89299

Note that I specified a domain, not a file, so I will get the root file (index.html). To upload this file to an FTP site using cURL, you need to specify the file to upload with the -T option, and then enter the FTP site URL and file path (Listing 2).

Listing 2. Example of using cURL to upload a file to an FTP site

$ curl -T test.html ftp://user: [email protected]/ftpdir/ % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 43320 0 0 100 43320 0 38946 0:00:01 0:00:01 --:--:-- 124k $

Could it be simpler? It is enough to master a few simple models, and cURL will be extremely easy to use. But the variety of available options is very large - a help request from the cURL command line (--help) displays 129 lines. A large number of options allow you to control everything from verbosity to security and various protocol-dependent configurable elements.

From a developer's point of view, this is not the most pleasant aspect of cURL. Let's dive deeper into the cURL library and see how to add these file transfer protocols to your application.

cURL as a library

If you've been watching scripting languages over the past 10 years, you'll have noticed a distinct change in their composition. Scripting languages such as Python, Ruby, Perl and many others include not only the socket layer, like C or C++, but also application layer protocol interfaces. These scripting languages contain high-level features that make it trivial to create an HTTP server or client, for example. The libcurl library adds functionality similar to languages like C and C++, but in a way that allows you to work with multiple languages. Libcurl behaves roughly the same in all the languages it supports, although since these languages can differ significantly (C and Scheme), the way in which this behavior is enforced may also differ.

The libcurl library includes functions that are illustrated in listings and in API form, so it can be used in high-level languages (more than 30 to date). This article provides two examples of using libcurl. The first explores a simple HTTP client in C (suitable for creating Web spiders), and the second is a simple HTTP client in Python.

HTTP client in C language

The C API provides two APIs on top of the libcurl functionality. The simple interface is a simple synchronous API (that is, when libcurl makes a request, it executes it until completion or before an error message is reported). Multi-interface provides control over libcurl, allowing an application to perform multiple simultaneous transfers and control where and when libcurl transfers data.

This example uses a simple interface. This API still provides some control over the data movement process (using callback functions), but still lives up to its name. Listing 3 shows a C example for HTTP.

Listing 3. HTTP client in C using simple interface libcurl

#include #include #include #define MAX_BUF 65536 char wr_buf; int wr_index; /* * Write data callback function (called within the context of * curl_easy_perform. */ size_t write_data(void *buffer, size_t size, size_t nmemb, void *userp) ( int segsize = size * nmemb; /* Check to see if this data exceeds the size of our buffer. If so, * set the user-defined context value and return 0 to indicate a * problem to curl. */ if (wr_index + segsize > MAX_BUF) ( *(int *)userp = 1; return 0; ) /* Copy the data from the curl buffer into our buffer */ memcpy((void *)&wr_buf, buffer, (size_t)segsize); /* Update the write index */ wr_index += segsize; /* Null terminate the buffer */ wr_buf = 0; /* Return the number of bytes received, indicating to curl that all is okay */ return segsize; ) /* * Simple curl application to read the index.html file from a Web site. * / int main(void) ( CURL *curl; CURLcode ret; int wr_error; wr_error = 0; wr_index = 0; /* First step, init curl */ curl = curl_easy_init(); if (!curl) ( printf("couldn "t init curl\n"); return 0; ) /* Tell curl the URL of the file we"re going to retrieve */ curl_easy_setopt(curl, CURLOPT_URL, "www.exampledomain.com"); /* Tell curl that we"ll receive data to the function write_data, and * also provide it with a context pointer for our error return. */ curl_easy_setopt(curl, CURLOPT_WRITEDATA, (void *)&wr_error); curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_data); /* Allow curl to perform the action */ ret = curl_easy_perform(curl); printf("ret = %d (write_error = %d)\n", ret, wr_error); /* Emit the page if curl indicates that no errors occurred */ if (ret == 0) printf("%s\n", wr_buf); curl_easy_cleanup(curl); return 0; )

At the top are the necessary include files, including the root cURL file. Next, I defined a couple of variables to pass. The first, wr_buf, is a buffer into which the input data will be written. wr_index reflects the index of the current buffer entry.

Let's move on to the main function, which performs the installation using a simple API. All cURL calls pass through a pointer that stores the state of a particular request. It is defined as CURL pointer reference. This example also generates a special return code called CURLcode. Before using any libcurl functions, you must call curl_easy_init to obtain a CURL pointer. Next, notice several calls to the curl_easy_setopt function. These are pointer settings for a specific operation. For these calls, a pointer, command, and option are entered. This example first uses the CURLOPT_URL statement to set the URL of the data to retrieve. This is followed by CURL_WRITEDATA to create a context variable (in our case, the internal error record variable). Finally, CURLOPT_WRITEFUNCTION specifies a function to call when data is available. The API will call this function one or more times with the data it reads after the trigger signal.

To start the transfer, call the curl_easy_perform function. Its task is to perform the transfer taking into account a predefined configuration. When this function is called, it returns a result only after a successful transfer or an error. The last elements of main are intended to pass the returned statuses, start reading the page, and finally clean up using the curl_easy_cleanup function (once the operation is completed).

Now let's look at the write_data function. This is a callback function that is called when data for a particular operation is received. Please note that when data is read from the Web site, it is written to you (write_data). The callback contains a buffer (with ready data), the number of elements and their size (their product gives the total amount of data in the buffer), and a context pointer. The first task is to ensure that the buffer (wr_buf) has sufficient capacity to write data. IN otherwise it sets the context pointer and returns zero, indicating there is a problem. Otherwise, it copies the data from the cURL buffer into your buffer and builds the index to point to the next place to write. This example terminates the line so that printf can be applied to it later. Finally, it returns the number of bytes processed to libcurl. This tells libcurl that the data has been accepted and can be removed. That's it - isn't it a simple way to read a file from a Web site into memory?

HTTP client in Python

This section provides an example similar to the C HTTP client, but this time written in Python. Python is a useful object-oriented scripting language that is great for prototyping and commercial software. The example assumes you have some knowledge of Python, but it is used very little, so no advanced knowledge is required.

The code for a simple HTTP client, written in Python using pycurl, is shown in Listing 4.

Listing 4. Python HTTP client using the pycurl interface from libcurl

import sys import pycurl wr_buf = "" def write_data(buf): global wr_buf wr_buf += buf def main(): c = pycurl.Curl() c.setopt(pycurl.URL, "http://www.exampledomain.com ") c.setopt(pycurl.WRITEFUNCTION, write_data) c.perform() c.close() main() sys.stdout.write(wr_buf)

Prototyping in Python

This illustrates one of the benefits of Python when creating prototypes. Quite a wide range of functionality is achieved with a small amount of code. In C you can get more high performance, but if your goal is quick creation code to test an idea, it's better to use high-level scripting languages such as Python.

This code is much simpler than the C version. It starts by importing the necessary modules (the standard sys system module and the pycurl module). Next, the write buffer (wr_buf) is defined. Just like in the C program, I declare the write_data function. Note that this function takes one argument: a buffer of data read from the HTTP server. I simply took this buffer and added it to the global write buffer. The main function starts by creating a Curl pointer, then uses the setopt and WRITEFUNCTION methods to determine the URL. It calls the perform method to start the transfer and closes the pointer. Finally she calls main function and passes the write buffer to stdout. Note that the context error pointer is not needed in this case because Python string concatenation is used, meaning there is no need to use a statically sized string.

What's next

This article only scratches the surface of libcurl, given the huge number of protocols and languages supported by the library. But hopefully it demonstrates how easy it is to create applications that use application layer protocols such as HTTP. The libcurl website (see section) contains a large number of examples and a significant amount of useful documentation. So the next time you're developing a Web browser, spider, or other application that requires an application-level protocol, try libcurl. This will surely speed up the development process and you will enjoy it.

(PHP 4 >= 4.0.2, PHP 5, PHP 7)

curl_setopt — Sets a parameter for the CURL session

List of parameters

cURL handle obtained from curl_init().

Parameter to be set CURLOPT_XXX.

The value of the option parameter.

bool:

Parameter		Notes
CURLOPT_AUTOREFERER	TRUE For automatic installation fields Referrer: in requests redirected by header Location:.
CURLOPT_BINARYTRANSFER	TRUE to return the raw response when using a constant CURLOPT_RETURNTRANSFER.	As of PHP 5.1.3 this option is no longer required: raw output is always returned when using the option CURLOPT_RETURNTRANSFER.
CURLOPT_COOKIESESSION	TRUE to instruct the current session to start a new "session" of cookies. This will cause libcurl to ignore any "session" cookies it should have loaded from the previous session. By default, libcurl always saves and loads all cookies, regardless of whether they are "session" or not. "Session" cookies are cookies that do not expire and must only exist for the current "session".
CURLOPT_CERTINFO	TRUE to output SSL certificate information to stream STDERR with secure connections.	Added in cURL 7.19.1. Available starting from PHP 5.3.2. Requires this option to be enabled for correct operation CURLOPT_VERBOSE.
CURLOPT_CONNECT_ONLY	TRUE tells the library to perform the necessary proxy authentication and connection setup, but does not transmit data. This option is implemented for HTTP, SMTP and POP3.	Added in 7.15.2. Available from PHP 5.5.0.
CURLOPT_CRLF	TRUE to convert Unix line endings to CRLF.
CURLOPT_DNS_USE_GLOBAL_CACHE	TRUE to use the global DNS cache. This option is not thread safe and is enabled by default.
CURLOPT_FAILONERROR	TRUE for a detailed report on failure if the received HTTP code is greater than or equal to 400. The default behavior returns the page as normal, ignoring the code.
CURLOPT_FILETIME	TRUE to try to obtain the modification date of a remote document. This value can be obtained using the CURLINFO_FILETIME parameter from the function curl_getinfo().
CURLOPT_FOLLOWLOCATION	TRUE to follow any heading "Location: " sent by the server in its response (note that this happens recursively, PHP will follow any headers sent "Location: ", except when a constant is set CURLOPT_MAXREDIRS).
CURLOPT_FORBID_REUSE	TRUE to force a connection to be closed after its processing has completed so that it cannot be reused.
CURLOPT_FRESH_CONNECT	TRUE to force the use of a new connection instead of a cached one.
CURLOPT_FTP_USE_EPRT	TRUE to use EPRT (and LPRT) for active FTP uploads. Use FALSE in order to disable EPRT and LPRT and use only PORT.
CURLOPT_FTP_USE_EPSV	TRUE for initial testing of the EPSV command during FTP transfers. If the command fails, it will fall back to PASV. Install in FALSE to disable EPSV.
CURLOPT_FTP_CREATE_MISSING_DIRS	TRUE to create missing directories if an FTP operation encounters a non-existent path.
CURLOPT_FTPAPPEND	TRUE to write a remote file to the end, instead of overwriting it over an existing file.
CURLOPT_TCP_NODELAY	Permanently specifies whether the TCP_NODELAY option should be set or cleared (1 = set, 0 = cleared). By default the option is cleared.	Available from PHP 5.2.1 for versions built with libcurl 7.11.2 or later.
CURLOPT_FTPASCII	Nickname CURLOPT_TRANSFERTEXT. Use this instead.
CURLOPT_FTPLISTONLY	TRUE to return only a list of names from the FTP directory.
CURLOPT_HEADER	TRUE to include headers in the output.
CURLINFO_HEADER_OUT	TRUE to track the handle query string.	Available starting from PHP 5.1.3. Prefix CURLINFO_ used specifically.
CURLOPT_HTTPGET	TRUE to reset the HTTP request method to the GET method. Since GET is the default, this parameter is only needed if the request method has previously been changed.
CURLOPT_HTTPPROXYTUNNEL	TRUE to tunnel through the specified HTTP proxy.
CURLOPT_MUTE	TRUE to completely disable cURL function messages.	Removed in cURL 7.15.5 (CURLOPT_RETURNTRANSFER option can be used)
CURLOPT_NETRC	TRUE to read the ~/.netrc file for the login and password for the remote site with which the connection is being established.
CURLOPT_NOBODY	TRUE to exclude the response body from the output. The request method is set to HEAD. Changing this setting to FALSE doesn't change it back to GET.
CURLOPT_NOPROGRESS	TRUE to disable the progress indicator on cURL transfers. Comment: PHP automatically sets this parameter to TRUE, change it only for debugging purposes.
CURLOPT_NOSIGNAL	TRUE to ignore any cURL function that sends signals to the PHP process. This option is enabled by default in multi-threaded SAPIs to allow timeout parameters to work correctly.
CURLOPT_POST	TRUE to use regular HTTP POST. This method POST uses normal , commonly used in HTML forms.
CURLOPT_PUT	TRUE to download a file using the HTTP PUT method. The file used must be set using the options CURLOPT_INFILE And CURLOPT_INFILESIZE.
CURLOPT_RETURNTRANSFER	TRUE to return the result of the transfer as a string from curl_exec() instead of direct output to the browser.
CURLOPT_SAFE_UPLOAD	TRUE to disable prefix support @ for downloaded files in CURLOPT_POSTFIELDS, which means that the values passed with @ can be transmitted securely as fields. Instead of a prefix, you can use the option CURLFile d.	Added in PHP 5.5.0 with default value FALSE. In PHP 5.6.0 it became equal to by default TRUE.
CURLOPT_SSL_VERIFYPEER	FALSE to stop cURL from checking the host certificate. Alternative certificates to be verified can be specified using the parameter CURLOPT_CAINFO or the directory with certificates specified by the parameter CURLOPT_CAPATH.	Default is TRUE since cURL version 7.10. The default distribution is installed starting from cURL version 7.10.
CURLOPT_TRANSFERTEXT	TRUE to use ASCII mode for FTP transfers. When using LDAP, the data is returned in plain text instead of HTML. IN Windows systems flow STDOUT does not set to binary mode.
CURLOPT_UNRESTRICTED_AUTH	TRUE to continue sending login and password during redirects (when using CURLOPT_FOLLOWLOCATION), even if the hostname changes.
CURLOPT_UPLOAD	TRUE to prepare for uploading the file to the server.
CURLOPT_VERBOSE	TRUE to display additional information. Writes output to a stream STDERR, or the file specified by the parameter CURLOPT_STDERR.

For the following option parameter values, the value parameter must be of type integer:

Parameter	Set value value	Notes
CURLOPT_BUFFERSIZE	The size of the buffer used for each read. However, there is no guarantee that this request will be completed.	Added in cURL 7.10.
CURLOPT_CLOSEPOLICY	One of the constants CURLCLOSEPOLICY_. Comment*: This option is deprecated as it was never implemented in cURL and did not work.	Removed in PHP 5.6.0.
CURLOPT_CONNECTTIMEOUT	The number of seconds to wait when trying to connect. Use 0 to wait indefinitely.
CURLOPT_CONNECTTIMEOUT_MS	The number of milliseconds to wait when attempting to connect. Use 0 to wait indefinitely. If libcurl is compiled using the system's standard name resolver, then the connection will still use a full second wait as a timeout, with a minimum allowed timeout of 1 second.	Added in cURL version 7.16.2. Available starting from PHP 5.2.3.
CURLOPT_DNS_CACHE_TIMEOUT	The number of seconds that DNS records are stored in memory. By default, this parameter is 120 (2 minutes).
CURLOPT_FTPSSLAUTH	FTP authentication method (in active mode): CURLFTPAUTH_SSL(SSL is checked first), CURLFTPAUTH_TLS(TLS checked first) or CURLFTPAUTH_DEFAULT(cURL decides for itself).	Added in cURL version 7.12.2.
CURLOPT_HTTP_VERSION	CURL_HTTP_VERSION_NONE (by default, CURL chooses which version to use), CURL_HTTP_VERSION_1_0 (force HTTP/1.0), or CURL_HTTP_VERSION_1_1 (force HTTP/1.1).
CURLOPT_HTTPAUTH	You can use the bitwise operator \| (or) to combine several methods together. In this case, cURL will poll the server for supported authorization methods and select the best one. CURLAUTH_ANY is an alias CURLAUTH_BASIC \| CURLAUTH_DIGEST \| CURLAUTH_GSSNEGOTIATE \| CURLAUTH_NTLM. CURLAUTH_ANYSAFE is an alias CURLAUTH_DIGEST \| CURLAUTH_GSSNEGOTIATE \| CURLAUTH_NTLM.
CURLOPT_INFILESIZE	The expected file size, in bytes, when uploading a file to a remote server. Please note that using this option will not stop further data being sent in excess of this value, as the data sent depends on the result CURLOPT_READFUNCTION.
CURLOPT_LOW_SPEED_LIMIT	Upper threshold for data transfer rate, in bytes per second. The verification takes place within CURLOPT_LOW_SPEED_TIME seconds, after which PHP considers the transfer too slow and aborts it.
CURLOPT_LOW_SPEED_TIME	The maximum number of seconds during which the transfer rate must not exceed CURLOPT_LOW_SPEED_LIMIT, otherwise PHP will mark the transfer as too slow and stop it.
CURLOPT_MAXCONNECTS	Maximum number of persistent connections. When the limit is reached, the parameter is used to determine which connection to close. CURLOPT_CLOSEPOLICY.
CURLOPT_MAXREDIRS	The maximum number of accepted redirects. Use this option together with the option CURLOPT_FOLLOWLOCATION.
CURLOPT_PORT	Alternative connection port.
CURLOPT_POSTREDIR	A bit mask containing 1 (301 Moved Permanently), 2 (302 Found) and 4 (303 See Other) to specify whether the HTTP POST method should be processed when the option is enabled CURLOPT_FOLLOWLOCATION if the specified type of redirection occurred.	Added in cURL 7.19.1. Available since PHP 5.3.2.
CURLOPT_PROTOCOLS	Bit mask of values CURLPROTO_. This mask limits the protocols used by libcurl. This allows you to have libcurl work with a large number of protocols, and limit the operation of certain transfers to only a subset of them. By default, libcurl uses all supported protocols. See also parameter CURLOPT_REDIR_PROTOCOLS*. Correct protocol values: CURLPROTO_HTTP , CURLPROTO_HTTPS , CURLPROTO_FTP , CURLPROTO_FTPS , CURLPROTO_SCP , CURLPROTO_SFTP , CURLPROTO_TELNET , CURLPROTO_LDAP , CURLPROTO_LDAPS , CURLPROTO_DICT , CURLPROTO_FILE , CURLP ROTO_TFTP, CURLPROTO_ALL
CURLOPT_PROXYAUTH	HTTP authorization methods used when connecting to a proxy server. Use the same bit masks that were described for the parameter CURLOPT_HTTPAUTH. IN this moment For proxy authorization, only CURLAUTH_BASIC and CURLAUTH_NTLM are supported.	Added in cURL version 7.10.7.
CURLOPT_PROXYPORT	The port number of the proxy server to which the connection is made. This number can also be set using the parameter CURLOPT_PROXY.
CURLOPT_PROXYTYPE	Either CURLPROXY_HTTP (default) or CURLPROXY_SOCKS5 .	Added in cURL 7.10.
CURLOPT_REDIR_PROTOCOLS	Bit mask of values CURLPROTO_. This bitmask limits the protocols used by libcurl when redirecting (with the parameter enabled CURLOPT_FOLLOWLOCATION). This allows you to limit the set of protocols used when redirecting for some transmissions. By default, libcurl supports all protocols except FILE and SCP. In versions prior to 7.19.4, redirection was used for all protocols without exception. See also parameter description CURLOPT_PROTOCOLS* for a list of constants with protocol values.	Added in cURL version 7.19.4.
CURLOPT_RESUME_FROM	Transmission start offset, in bytes.
CURLOPT_SSL_VERIFYHOST	Use 1 to check for the existence of a common name in the SSL certificate. Use 2 to check that the common name exists and also matches the specified host. In a combat environment, the value of this parameter should be 2 (set by default).	Support for value 1 has been removed in cURL 7.28.1
CURLOPT_SSLVERSION	One of the constants CURL_SSLVERSION_DEFAULT (0), CURL_SSLVERSION_TLSv1 (1), CURL_SSLVERSION_SSLv2 (2), CURL_SSLVERSION_SSLv3 (3), CURL_SSLVERSION_TLSv1_0 (4), CURL_SSLVERSION_TLSv1_1(5) or CURL_SSLVERSION_TLSv1_2 (6).
CURLOPT_TIMECONDITION	Parameter interpretation method CURLOPT_TIMEVALUE. Use CURL_TIMECOND_IFMODSINCE to return the page only if it has changed since the time specified in the parameter CURLOPT_TIMEVALUE. If the page has not been modified, the title will be returned "304 Not Modified", implying that the parameter CURLOPT_HEADER installed in TRUE. Use CURL_TIMECOND_IFUNMODSINCE for the opposite effect. The default is CURL_TIMECOND_IFMODSINCE.
CURLOPT_TIMEOUT	The maximum number of seconds allowed for executing cURL functions.
CURLOPT_TIMEOUT_MS	The maximum number of milliseconds allowed for executing cURL functions. If libcurl is built using the normal system name resolver, then this connection span will still use second rounding timeouts, with a minimum timeout allowed of one second.	Added in cURL version 7.16.2. Available starting from PHP 5.2.3.
CURLOPT_TIMEVALUE	Number of seconds since January 1, 1970. This time will be used by the parameter CURLOPT_TIMECONDITION. By default, the CURL_TIMECOND_IFMODSINCE parameter is used.
CURLOPT_MAX_RECV_SPEED_LARGE	If the download speed exceeds this value (specified in bytes per second) on average over the entire transfer, the download will be paused to maintain the average speed less than or equal to this parameter. By default, the speed is not limited.
CURLOPT_MAX_SEND_SPEED_LARGE	If the upload to the server exceeds this value (specified in bytes per second) on average throughout the entire transfer, the upload will be paused to maintain an average speed less than or equal to this parameter. By default, the speed is not limited.	Added in cURL 7.15.5. Available starting from PHP 5.4.0.
CURLOPT_SSH_AUTH_TYPES	A bitmask consisting of one or more constants: CURLSSH_AUTH_PUBLICKEY, CURLSSH_AUTH_PASSWORD, CURLSSH_AUTH_HOST, CURLSSH_AUTH_KEYBOARD. Install CURLSSH_AUTH_ANY in order for libcurl to choose one of them independently.	Added in cURL 7.16.1.
CURLOPT_IPRESOLVE	Allows an application to select the type of IP address with which the hostname is determined. This is necessary if you are using a hostname that is derived from more than one version of the IP address. Possible values could be CURL_IPRESOLVE_WHATEVER, CURL_IPRESOLVE_V4, CURL_IPRESOLVE_V6, and by default CURL_IPRESOLVE_WHATEVER.	Added in cURL 7.10.8.

For the following option parameter values, the value parameter must be of type string:

Parameter	Set value value	Notes
CURLOPT_CAINFO	The name of a file containing one or more certificates against which nodes will be checked. This parameter only makes sense when used in conjunction with CURLOPT_SSL_VERIFYPEER.	Requires an absolute path.
CURLOPT_CAPATH	A directory containing several CA certificates. Use this option in conjunction with CURLOPT_SSL_VERIFYPEER.
CURLOPT_COOKIE	Header content "Cookie:", used in the HTTP request. Please note that multiple cookies are separated by a semicolon followed by a space (for example, " fruit=apple; color=red")
CURLOPT_COOKIEFILE	The name of the file containing the cookies. This file must be in Netscape format or simply HTTP headers written to the file. If an empty string is passed as the file name, then cookies will not be saved, but their processing will still be enabled.
CURLOPT_COOKIEJAR	The name of the file in which all internal cookies of the current transfer will be saved after the handle is closed, for example after calling curl_close.
CURLOPT_CUSTOMREQUEST	Custom request method used instead "GET" or "HEAD" when making an HTTP request. This is useful for queries "DELETE" or other, more rare HTTP requests. Correct meanings would be words like "GET", "POST", "CONNECT" and so on; those. Do not enter the entire HTTP request line here. For example, an indication "GET /index.html HTTP/1.0\r\n\r\n" will be wrong. Comment: Do not use this feature until you are sure that the server supports this type request.
CURLOPT_EGDSOCKET	Like CURLOPT_RANDOM_FILE, except that the filename is set to the Entropy Gathering Daemon socket.
CURLOPT_ENCODING	Header content "Accept-Encoding: ". This allows the request to be decoded. Supported encodings are "identity", "deflate" And "gzip". If an empty string is passed, "" , a header containing all supported encoding types is sent.	Added in cURL 7.10.
CURLOPT_FTPPORT	The value that will be used to determine the IP address for the FTP "PORT" command. The "PORT" command tells the server which IP address it should connect to. This can be an IP address, hostname, network interface name (under Unix), or simply "-" to use the default system IP address.
CURLOPT_INTERFACE	The name of the network interface to use. Can be an interface name, an IP address, or a host name.
CURLOPT_KEYPASSWD	Password required to use the private key CURLOPT_SSLKEY or CURLOPT_SSH_PRIVATE_KEYFILE.	Added in cURL 7.16.1.
CURLOPT_KRB4LEVEL	Security level KRB4 (Kerberos 4). Any of the following values (in order from weakest to strongest) are correct: "clear", "safe", "confidential", "private".. If the specified string is different from given values, the value will be used "private". Setting this option to NULL will completely disable KRB4 security. At the moment, KRB4 security only works with FTP transactions.
CURLOPT_POSTFIELDS	All data transmitted in an HTTP POST request. To transfer a file, specify before the file name @ , and also use the full path to the file. The file type can also be specified using the format " ;type=mimetype" following the file name. This parameter can be passed as a url-encoded string, like " para1=val1¶2=val2&...", and in the form of an array, the keys of which will be the names of the fields, and the values will be their contents. If value is an array, the header Content-Type will be set to multipart/form-data. Starting from PHP 5.2.0, when transferring files with the prefix @ , value must be an array. Since PHP 5.5.0, prefix @ is deprecated and files can be sent using CURLFile. Prefix @ can be disabled to allow values starting with @ by setting the option CURLOPT_SAFE_UPLOAD in meaning TRUE.
CURLOPT_PROXY	HTTP proxy through which requests will be routed.
CURLOPT_PROXYUSERPWD	Login and password written in the form ":" , used when connecting through a proxy.
CURLOPT_RANDOM_FILE	Name of the file used to initialize the generator random numbers for SSL.
CURLOPT_RANGE	Range of data to be downloaded, in format "X-Y", and either X or Y can be omitted. The HTTP protocol also supports the transmission of multiple ranges separated by commas, they are specified in the format "X-Y,N-M".
CURLOPT_REFERER	Header content "Referrer:", which will be used in the HTTP request.
CURLOPT_SSH_HOST_PUBLIC_KEY_MD5	A string containing 32 hexadecimal digits. The string must be an MD5 checksum of the remote computer's public key, and libcurl will reset the connection to the remote host until the checksum matches the public key. This option is only for transferring data using SCP and SFTP.	Added in cURL 7.17.1.
CURLOPT_SSH_PUBLIC_KEYFILE	The file name for your public key. If not specified, libcurl defaults to the file $HOME/.ssh/id_dsa.pub if the HOME environment variable is set and the file "id_dsa.pub" in the current directory if the HOME environment variable is not set.	Added in cURL 7.16.1.
CURLOPT_SSH_PRIVATE_KEYFILE	The file name for your private key. If not specified, libcurl defaults to the $HOME/.ssh/id_dsa file if the HOME environment variable is set and the "id_dsa" file in the current directory if the HOME environment variable is not set. If the file password protected, set the password using CURLOPT_KEYPASSWD.	Added in cURL 7.16.1.
CURLOPT_SSL_CIPHER_LIST	List of ciphers used in SSL transfers. For example, RC4-SHA And TLSv1 are valid cipher lists.
CURLOPT_SSLCERT	The name of a file with a correctly formatted PEM certificate.
CURLOPT_SSLCERTPASSWD	Password required to use the certificate CURLOPT_SSLCERT.
CURLOPT_SSLCERTTYPE	Certificate format. Formats supported "PEM"(default), "DER" And "ENG".	Added in cURL version 7.9.3.
CURLOPT_SSLENGINE	Encryption mechanism identifier for private key SSL specified in the parameter CURLOPT_SSLKEY.
CURLOPT_SSLENGINE_DEFAULT	The identifier of the encryption mechanism used for asymmetric encryption operations.
CURLOPT_SSLKEY	The name of the SSL private key file.
CURLOPT_SSLKEYPASSWD	The secret password required to use the SSL private key specified by the parameter CURLOPT_SSLKEY. Comment: Since this setting contains a valuable password, be aware that this PHP script must be stored in a safe place.
CURLOPT_SSLKEYTYPE	The type of SSL private key specified in the parameter CURLOPT_SSLKEY. The following key types are supported: "PEM"(default), "DER" And "ENG".
CURLOPT_URL	Downloadable URL. This parameter can also be set when initializing a session using curl_init().
CURLOPT_USERAGENT	Header content "User-Agent: ", sent in an HTTP request.
CURLOPT_USERPWD	Login and password used during connection, specified in the format ":" .

For the following option parameter values, the value parameter must be an array:

Parameter	Set value value	Notes
CURLOPT_HTTP200ALIASES	An array of HTTP 200 responses that will be treated as correct responses rather than erroneous ones.	Added in cURL version 7.10.3.
CURLOPT_HTTPHEADER	An array of set HTTP headers, in the format array("Content-type: text/plain", "Content-length: 100")
CURLOPT_POSTQUOTE	An array of FTP commands executed on the server after an FTP request is completed.
CURLOPT_QUOTE	An array of FTP commands executed on the server before making an FTP request.

For the following option parameter values, the value parameter must be a stream handle (returned, for example, by the function fopen()):

Parameter	Set value value
CURLOPT_FILE	The file in which the transfer result will be written. Default output stream STDOUT(browser window).
CURLOPT_INFILE	The file from which data should be read when uploaded to the server.
CURLOPT_STDERR	Alternative error output file used in place of the error stream STDERR.
CURLOPT_WRITEHEADER	The file in which the headers of the current operation will be written.

For the following option parameter values, the value parameter must be a valid function name or closure:

Parameter	Set value value
CURLOPT_HEADERFUNCTION	The callback function takes two parameters. The first parameter is the cURL handle, the second parameter is a string containing the headers to be written. Headers must be written using this callback function. Should return the number of bytes written.
CURLOPT_PASSWDFUNCTION	The callback function takes three parameters. The first parameter is the cURL handle, the second parameter is the password prompt string, and the third parameter is the maximum password length. Should return a string containing the password.
CURLOPT_PROGRESSFUNCTION	The callback function takes five parameters. The first is the cURL descriptor, the second is the total number of bytes expected to be downloaded from the server, the third is the number of bytes already downloaded, the fourth is the total number of bytes expected to be sent to the server, and the fifth is the number of bytes already sent. Comment: The callback function is called only if the option CURLOPT_NOPROGRESS set to value FALSE. You can return a non-zero value to cancel the transfer. In this case an error will be displayed CURLE_ABORTED_BY_CALLBACK.
CURLOPT_READFUNCTION	The callback function takes three parameters. The first parameter is the cURL handle, the second parameter is the stream resource passed to cURL via the option CURLOPT_INFILE, and the third parameter is the maximum allowed amount of data to be read. The callback function must return a string of length no greater than the requested amount of data, usually by reading from the passed streaming resource. Should return an empty string to signal the end of the file EOF.
CURLOPT_WRITEFUNCTION	The callback function takes two parameters. The first parameter is the cURL handle, and the second parameter is the string containing the data to be written. Data must be saved using this function. It must return the exact number of bytes written, otherwise the download will be aborted with an error.

Other meanings:

Return values

Returns TRUE upon successful completion or FALSE in case of an error.

List of changes

Version	Description
5.6.0	Option CURL_SAFE_UPLOAD now has a default value of TRUE.
5.6.0	Removed option CURLOPT_CLOSEPOLICY and its associated meanings.
5.5.0	The cURL resource is added as the first argument to the callback function CURLOPT_PROGRESSFUNCTION.
5.5.0	Added option CURLOPT_SHARE.
5.3.0	Added option CURLOPT_PROGRESSFUNCTION.
5.2.10	Added options CURLOPT_PROTOCOLS And CURLOPT_REDIR_PROTOCOLS.
5.1.0	Added options CURLOPT_AUTOREFERER, CURLOPT_BINARYTRANSFER, CURLOPT_FTPSSLAUTH, CURLOPT_PROXYAUTH And CURLOPT_TIMECONDITION.
5.0.0	Added options CURLOPT_FTP_USE_EPRT, CURLOPT_NOSIGNAL, CURLOPT_UNRESTRICTED_AUTH, CURLOPT_BUFFERSIZE, CURLOPT_HTTPAUTH, CURLOPT_PROXYPORT, CURLOPT_PROXYTYPE, CURLOPT_SSLCERTTYPE And CURLOPT_HTTP200ALIASES.

Examples

Example #1 Initializing a CURL session and loading a web page

// create a new cURL resource
$ch = curl_init();

/* http://localhost/upload.php:
print_r($_POST);
print_r($_FILES);
*/

$ch = curl_init();

$data = array("name" => "Foo" , "file" => "@/home/user/test.png" );

Curl_setopt($ch, CURLOPT_URL, "http://localhost/upload.php");
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $data);

Curl_exec($ch);
?>

Execution result this example:

Array ( => Foo) Array ( => Array ( => test.png => image/png => /tmp/phpcpjNeQ => 0 => 279))

Notes

Comment:
Passing an array to CURLOPT_POSTFIELDS encodes the data as multipart/form-data, whereas passing a URL-encoded string will encode the data as application/x-www-form-urlencoded.