Uploading data to a webserver Part 1 – C/C++ and CURL
Following on from the last post on the topic of capturing packet data from a network, this guide demonstrates how you can use a cURL library to upload data from an application written in C (or C++) to a webserver. Although at times I refer to previous examples, you can use this in any number of applications.
This will touch on a number of areas and require a little more prep work than previous examples, due to needing a few extra libraries and a server running PHP and SQL. In a nutshell, you have to cobble together a string with all your data in, send it along to your server with a request and then have that server do something with it. The steps covered are:
- (Setup) LAMP server and database
- (Client) Generate a URL string from variables in your program
- (Client) Send a web request to the URL generated in step 1
- (Server) Process the uploaded string to retrieve values from data
- (Server) Store the data as a new record in a database
Finally, there are some really quick and horrible things that I do here, with minimal catching of errors, but the purpose is to show the process from start to finish as briefly as possible!
Environment
First of all, its worth mentioning the set-up I am using for this system. Just to demonstrate, I have a LAMP server running on Ubuntu with a MySQL database. I won’t cover how to actually get as far as having those running because there’s plenty of articles that explain how to do that. The only other thing you may want to use is phpMyAdmin, as this makes database management via a web UI much easier than just command line interaction. It also helps hugely to understand basics of HTML/PHP – but you may well be able to stumble through this if you’ve never touched either before!
Setup
Database Table
The first thing to do is to create a new table to store the records in. This may as well just be a spreadsheet at this point, but later on I will be talking about queries and combining data from different tables. You can either create a new table with phpMyAdmin by selecting the appropriate settings from the code below or you can simply copy and paste the entire thing into an SQL query box and run it, which will create a new table for you.
CREATE TABLE inventory ( hostName varchar(25) NOT NULL, hostIP varchar(25), hostMAC varchar(18), hostCreateDate timestamp DEFAULT CURRENT_TIMESTAMP, vlanID int(4), switchPort varchar(32), switchIP varchar(25), switchName varchar(64), switchPlatform varchar(64), UNIQUE (hostName) );
This single table consists of host information (name, mac, ip), switch information (name, platform, IP, port) and a date that the entry was created. Note that this also adds the constraint to the hostName to ensure that it is unique (and stipulates that it can’t be empty). There is a logical argument that, perhaps, it should be the MAC address that is unique; but I am quite happy with my domain and that it won’t give me hostname duplication for this demo. Plus, the focus is on getting data of different types in, rather than necessarily what the data is.
PHP page to process data
This is the part that is going to bring it all together later. When we want to place data into the database, we need something that will take data sent to the server and process that into the database. This file is going to store the database credentials, so its really important that it isn’t made available outside of your webserver. Since PHP files execute on the server, rather than the client, the file contents aren’t sent back when requested (unless the PHP service stops and breaks, in which case everything will be displayed in plaintext..). In fact, we really don’t have to – or necessarily want to – output anything with a PHP file; in this instance, it is simply used as a way to process data from clients to be stored in a database as follows:
The first thing to do is to specify some variables for the database details. These are used to connect to the database (on the same server, in this case):
<?php $servername = "localhost"; $username = "user"; $password = "Passw0rd"; $dbname = "inventory"; //Rest of the code goes here ?>
The next thing to do is to create a connection object that handles the connection to a MySQL database, passing it the above variables as parameters:
$conn = new mysqli($servername, $username, $password, $dbname);
Before we can go any further, however, we need to actually have some data to process – or know what data we want to process. I’ll come back to this in a while, but first let’s generate the URL string.
Generate a URL string
At this point, I am making an assumption that you already have variables that you want to send. The key thing is to prepare that data to be sent to the server.
There are a number of ways in which you could send data with a web request; HTTP GET and POST being probably the most suited.
- GET is easy; you can just construct a string consisting of the URL of the server to send the data too and then append all the extra data you want onto the end of it. However, you’re limited to 2048 characters, the URL is very “visible” (like any URL, the data included is going to pop up in search history, it can be cached) and so it is better used when trying to retrieve data (IE when you need to include some specific parameters as part of the request).
- POST, on the other hand, hasn’t got the same limits on what – and how much – data you can include in the request. Possibly more crucially, however, is that the data isn’t sent along in the header of the request, so it isn’t as easily seen directly through the URL.
For good practice, I’m going to focus on using POST to upload the data. We’re using the request to send data, not simply retrieve it (which is where using GET would be appropriate). We can upload quite a bit of data this way, but it has to be formatted first. The string is pretty much the same for GET as it would be for POST; the difference is that the POST data will be sent separately to the header, in the body of the request, in the following format:
variable1=value&variable2=value&variable3=value
These are sets of variables and values that are accessible by the page requested; but the variable name must be used as it is on the page. If your local variable is myHostName, for example, but you send it as val1, then once it has reached the server, it has to be referred to by the name val1.
So how do you construct this string from many smaller strings?
There are a few ways to do this (str::append, for example). I’m using a mixture of C and C++ throughout these examples (which is probably a horrible thing to do in practice), but to break down how it works, I am going to manually stick the different strings together.
First of all, I’ve created a function called generatePOSTData() that will prepare the different parts of a string to be added together, consisting of 8 example variables. Note that each one has an equals operator next to it, which is used by the webserver:
void generatePOSTData()
{
char *prefixhost = "host=";
char *prefixip = "ip=";
char *prefixmac = "mac=";
char *prefixvlan = "vlan=";
char *prefixswPort = "swPort=";
char *prefixswName = "swName=";
char *prefixswIP = "swIP=";
char *prefixswMAC = "swMAC=";
int slhost = strlen(prefixhost);
int slip = strlen(prefixip);
int slvlan = strlen(prefixvlan);
int slmac = strlen(prefixmac);
int slswPort = strlen(prefixswPort);
int slswName = strlen(prefixswName);
int slswIP = strlen(prefixswIP);
int slswMAC = strlen(prefixswMAC);
...
Note that this also calculates the length of each of the strings we’ve just created. This is important when adding the strings together, later.
Next, we can take the strings and add them together, storing them in another variable. The function, addStrings, is something I have discussed in a previous post.
addStrings(&host, prefixhost, systemhostname, slhost, slsystemhostname);
addStrings(&ip, prefixip, systemip, slip, slsystemip);
addStrings(&vlan, prefixvlan, systemvlan, slvlan, slsystemvlan);
addStrings(&mac, prefixmac, systemmac, slmac, slsystemmac);
addStrings(&swPort, prefixswPort, systemswitchport, slswPort, slsystemswitchport);
addStrings(&swIP, prefixswIP, systemswIP, slswIP, slsystemswIP);
addStrings(&swName, prefixswName, systemswName, slswName, slsystemswName);
addStrings(&swMAC, prefixswMAC, systemswMAC, slswMAC, slsystemswMAC);
}
void addStrings(char ** result, const char * prefix, const char * body, int &prefixlength, int &bodylength)
{
*result = (char *)malloc((prefixlength + bodylength + 1) * sizeof(char));
memcpy(*result, prefix, prefixlength);
memcpy(*result + prefixlength, body, bodylength);
(*result)[prefixlength + bodylength] = '\0'; //Must be set like this as array notation takes precendence over a dereferencing
prefixlength = strlen(*result);
}
Finally, we can start to construct a single string of all of these together.
requestString = (char *)malloc((slhost + slip + slswPort + slvlan + slmac + slswName + slswIP + slswMAC + 7 + 1) * sizeof(char));
This string is going to be long enough for each of the lengths of the above variables, plus one extra character for the ampersands to connect them. Plus a terminating character.
Now it is time to go through each string (I could use a recursive function here, but I’m keeping it long and simple here.. optimisation will come later!) and, bit by bit, copy the next piece of data over from individual variables over to requestString with an ampersand added after each set of values:
memcpy(requestString, host, slhost); requestString[slhost] = '&'; memcpy(requestString+ slhost + 1, ip, slip); requestString[slhost + slip + 1] = '&'; memcpy(requestString+ slhost + slip + 2, swPort, slswPort); requestString[slhost + slip + slswPort + 2] = '&'; memcpy(requestString+ slhost + slip + slswPort + 3, vlan, slvlan); requestString[slhost + slip + slswPort + slvlan + 3] = '&'; memcpy(requestString+ slhost + slip + slswPort + slvlan + 4, mac, slmac); requestString[slhost + slip + slswPort + slvlan + slmac + 4] = '&'; memcpy(requestString+ slhost + slip + slswPort + slvlan + slmac + 5, swName, slswName); requestString[slhost + slip + slswPort + slvlan + slmac + slswName + 5] = '&'; memcpy(requestString+ slhost + slip + slswPort + slvlan + slmac + slswName + 6, swIP, slswIP); requestString[slhost + slip + slswPort + slvlan + slmac + slswName + slswIP + 6] = '&'; memcpy(requestString+ slhost + slip + slswPort + slvlan + slmac + slswName + slswIP + 7, swMAC, slswMAC); requestString[slhost + slip + slswPort + slvlan + slmac + slswName + slswIP + slswMAC + 7] = '\0';
This is kind of ridiculous, admittedly. I could just use the addStrings function from above, or as I said before, just functions in C++ to do this. However, it shows you how you really have to add strings together at a lower level than simply doing something at a higher level such as string1 += string2.
Send a web request with cURL to the webserver
Now that we have the post string, we can prepare it for sending with cURL; and it isn’t very hard. The first thing to do is to create a new curl object and then initialise it. If that works, you can use curl_setopt to set the URL on the object to your webpage (so http://www. your-site.com/ sendData.php) and the post string to send that was constructed above. Finally, you can initiate the request with curl_easy_perform and then do cleanup to remove the cURL object.
//Library available from https://curl.haxx.se/libcurl/
#define CURL_STATICLIB //You may or may not need this!
#include "curl/curl.h"
#pragma comment ( lib, "libcurl.lib" )
//...
//...
CURL * curl;
curl_global_init(CURL_GLOBAL_ALL);
CURLcode res;
curl = curl_easy_init();
if (curl)
{
curl_easy_setopt(curl, CURLOPT_URL, address);
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, requestString);
res = curl_easy_perform(curl);
if (res != CURLE_OK)
{
fprintf(stderr, "curl_easy_perform() failed: %s\n", curl_easy_strerror(res));
}
curl_easy_cleanup(curl);
}
curl_global_cleanup();
Outside the scope of this document would be handling the response from the server, but it is important to provision for this later as this will allow you to know whether or not things worked on the client side!
Process the uploaded string
Now it is time to go back to where we were with the PHP page. We know what data is going to be sent now, so we know what will be received by the page.
Loading in the data
Although the string was all wrapped up with ampersands and formatted with a header, when the page receives the data, it is available as an associative array called $_POST. This means that you can access a value by using the name of it in the index (for example, $_POST[“host”] will return you the value of the hostname). The other thing about $_POST is that it is accessible from anywhere on your page, as it is a superglobal type. However, in this example, we only access it once to load in the values and process them into variables used throughout the page.
Where you declare the other variables in your code, so around $conn or the database details, add the following:
$host = mysql_escape_string($_POST['host']); $ip = mysql_escape_string($_POST['ip']); $swPort = mysql_escape_string($_POST['swPort']); $swIP = mysql_escape_string($_POST['swIP']); $swName = mysql_escape_string($_POST['swName']); $swMAC = mysql_escape_string($_POST['swMAC']); $vlan = mysql_escape_string($_POST['vlan']); $mac = mysql_escape_string($_POST['mac']);
What this will do is quite important; it (hopefully) will prevent against SQL injection attacks by escaping the string. In other words, it stops code from being added to text that could unexpectedly end the string and roll over into executing the code that was submitted. However, see this post for a bit more information – and why you might not want to use it in the real world.
Additionally, we now can refer to the variables by their name prefixed with a dollar symbol, rather than having to use $POST[‘variableName’] for everything. (Interestingly there are some differences between single and double quotes in php.).
Validate the data
The next test we want to do on the incoming data is to make sure it is, in fact, populated and not simply. This can be done by simply checking through each variable and seeing if it is null or not. Note that anything echo’d will be returned to the requestor. If this was a browser, you would see the response on screen, but this application doesn’t have a way to handle this, yet.
if ($host == NULL)
{
echo "Error, host is empty! ";
}
There is also one more thing that I want to add in; a test to see if the host is valid or not. If there is junk data and there are blank entries, for fields that must not be empty, this can create problems for us. If we specify an invalidation flag, that can be set if one of the variables is empty, then we can stop any database processing from happening.
$testInValid = 0;
if ($host == NULL)
{
echo "Error, host is empty! ";
$testInValid = 1;
}
if ($ip == NULL)
{
echo "Error, ip is empty! ";
$testInValid = 1;
}
if ($swPort == NULL)
{
echo "Error, swPort is empty! ";
$testInValid = 1;
}
if ($vlan == NULL)
{
echo "Error, vlan is empty! ";
$testInValid = 1;
}
if ($mac == NULL)
{
echo "Error, mac is empty! ";
$testInValid = 1;
}
if ($swName == NULL)
{
echo "Error, swName is empty! ";
//$testInValid = 1;
}
if ($swMAC == NULL)
{
echo "Error, swMAC is empty! ";
//$testInValid = 1;
}
if ($swIP == NULL)
{
echo "Error, swIP is empty! ";
//$testInValid = 1;
}
if (testInValid == 0)
{
We begin by declaring a variable called testInValid which, by default, will remain as 0 so long as all the key pieces of data exist. At the bottom, we continue the program so long as testInValid remains as 0. It is a quick and dirty trick, but it ensures that we don’t execute code if it could break anything in the database.
Store the data as a new record in a database
With the data validated and in the form, it is time to add it to the database. Be sure that you have you added the following line to create the connection object:
$conn = new mysqli($servername, $username, $password, $dbname);
The first thing to do is to see if we can connect to it. This will also catch the error if it can’t, but you don’t have to populate it yet:
if ($conn->connect_error)
{
}
Instead, we can now create an SQL Select statement, which will be run as a query by the object.
$sqlSelect = "SELECT * from hosts WHERE hostName = '$host'";
What this will do, is to return records for any hosts that exist with the hostname that we have specified in the request. Note that, although the query is in double quotation marks, the hostname is within single quotes. This allows us to use the $host variable from the page, expanded to its actual value. Another way to do that would be to use a full stop to join two strings together, with the second string being $host and the first string being the SQL statement up until the end of the equals sign, as so:
$sqlSelect = "SELECT * from hosts WHERE hostName = " . $host;
Back to the statements: we need another one. This one will be the SQL Insert statement, used to add an entry into the database.
$sqlInsert = "INSERT INTO hosts (hostName, hostIP, switchPort, switchIP, switchName, switchPlatform, vlanID, hostMac)
VALUES ('$host', '$ip', '$swPort', '$swIP','$swName','$swMAC','$vlan','$mac')";
Finally, we run the queries. First we check to see if there is already a record with that hostname which – if there is – means that we do nothing other than report that to be the case. However, if there are no results for the query, this means that there are no hosts with that name and we can now run the second query. If that returns true – in other words, there were no errors – then we can assume it worked and output that to be the case.
$result = $conn->query($sqlSelect);
if ($result->num_rows > 0)
{
echo "Host already exists!"
}
else
if ($conn->query($sqlInsert) === TRUE)
{
echo "Success!";
}
Viewing the results
You can now check this by making a new page with the same database connection details that fetches and outputs the data to a webpage. Below is a commented example of something that you can use to view the results into a table. Note that anything echo’d will be output as HTML – you can view how the code is generated in the browser by right clicking the screen (in most browsers) and clicking “View Source” when you run this:
<?php
//Connection strings
$servername = "myserver";
$username = "user1";
$password = "p@ssw0rd";
$dbname = "database1";
$tableName = "inventory";
//Connection object
$conn = new mysqli($servername, $username, $password, $dbname);
//Check to see if the connection succeeds
if ($conn->connect_error) {
die("Connection failed: " . $conn->connect_error);
}
//SQL statement to select various fields from the table. Note that you could say, instead, "SELECT * FROM" . $tableName;
$sql = "SELECT hostName, hostIP, hostMac, hostCreateDate, vlanID, switchPort, switchIP FROM ". $tableName ;
//Run the query and store the results in an array
$result = $conn->query($sql);
if ($result->num_rows > 0)
{
//This will spit out a table with some headers on the first row
echo "<table><tr><th>Hostname</th><th>IP Address</th><th>MAC</th><th>Switch IP</th><th>Switchport ID</th><th>VLAN ID</th><th>Created On</th></tr>";
//Here we just create a new table row with the results displayed in each column
while($row = $result->fetch_assoc())
{
echo "<tr><td>".
$row["hostName"]."</td><td>".
$row["hostIP"]."</td><td>".
$row["hostMac"]."</td><td>".
$row["switchIP"]."</td><td>" .
$row["switchPort"]."</td><td>" .
$row["vlanID"]."</td><td>".
$row["hostCreateDate"]."</tr>";
}
}
//Just in case we have nothing!
else
{
echo "0 results";
}
$conn->close();
?>
That concludes a very hacky – but functional – guide to how to get data from an application written in C/C++ into a database and displayed on a webpage using cURL. Please do post comments, issues and questions in reply!
‹ Decoding LLDP and CDP packets using a TLV reader for C++ Uploading data to a webserver Part 2 – C/C++ and sockets alternative ›