Use the Cloud Infrastructure API to Automatically Resize Your Server
The RESTful API is one of the more interesting and useful features of XMission’s Cloud Infrastructure service. The API exposes nearly all the features of the control panel in a consistent and well documented interface. This article will demonstrate how to use the API to automatically resize your server’s RAM dynamically, without the need to login to the control panel or restart your server. We use PHP here for the scripting language; most languages have the required libraries to access the RESTful API and parse the output, so there is a lot of flexibility.
Step 1: The Server Setup
We are using an Ubuntu 12.04 server with 256MB of RAM. A basic LAMP stack has been installed on the server using the tasksel method described here: <https://help.ubuntu.com/community/ApacheMySQLPHP>. A couple other minor tweaks were done to the LAMP stack, including modifying /etc/php5/php.ini file to set error_log = /var/log/php_errors.log. Additionally, we set MaxClients to 20 in the /etc/apache2/apache2.conf. These are not production type settings; use at your own risk.
Step 2: The Culprit – A Memory Intensive PHP Script
In order to demonstrate what the server does under load, let’s create a simple PHP script:
<?php
ini_set(‘memory_limit’,’128M’);
$time = microtime();
$time = explode(‘ ‘, $time);
$time = $time[1] + $time[0];
$start = $time;
$uber = array();
for ($i=0; $i<100000; $i++) {
$uber[$i]=md5($i);
}
$time = microtime();
$time = explode(‘ ‘, $time);
$time = $time[1] + $time[0];
$finish = $time;
$total_time = round(($finish - $start), 4);
echo ‘Page generated in ‘.$total_time.’ seconds.’;
Save that as /var/www/index.php and test it out in your browser. Once you have established that the script runs, the next step is to put the server under load.
Step 3: Replicate Web Traffic
One easy way to replicate web traffic is the Linux application siege <http://www.joedog.org/siege-home/>. Install it on your local machine and run:
siege -c 20 -t 5m http://<server-ip>/index.php
Siege will now request the supplied URL for 5 minutes with 20 concurrent users. That is a lot of simulated traffic, which puts our server under load in a controlled environment. While the server is under load, switch back to your server and run:
free -o -m
This will return your current memory usage:
root@webserver01:~# free -o -m total used free shared buffers cached Mem: 256 252 3 0 0 0 Swap: 256 112 143 root@webserver01:~#
The interesting information here is in the Mem row, which shows us that we only have 3MBs of RAM left. Our server is going to have performance issues, as the operating system has to start using Swap, which is far less efficient. At a certain point, it is going to start crashing. That’s a bad thing.
Step 4: The Cloud Infrastructure RESTful API Will Save Us!
The Cloud Infrastructure RESTful API is well documented and available to all Cloud Infrastructure and Cloud Server subscribers. You can read the full documentation is available here:
For this demonstration, we will create a script that is ran by cron every minute. The timing is completely up to you – you might want your cron script to run every minute, or every 5 minutes. A shorter time makes the server more sensitive to traffic and load, however it also puts additional load on the server itself.
Below is a PHP script that we will add to our cron job; we will explain each line after the script:
#!/usr/bin/php -q
<?php
// Script for resizing server memory using the XMission Cloud Infrastructure API
// Configuration
// Memory tolerances
define("LOWER_RAM_LIMIT", 256); # The lowest amount of memory for the server
define("UPPER_RAM_LIMIT", 1024); # The highest amount of memory for the server
define("LOWER_TOLERANCE", 0.25); # Decrease the memory if ratio is below this
define("UPPER_TOLERANCE", 0.8); # Increase the memory if ratio is above this
define("MEMORY_INCREMENT", 256); # Increment the memory by this amount
// The RESTful API
define("API_URL", "https://api.cloud.xmission.com:4464/paci/v1.0/");
// Your username and password for the API (same as your login to https://cp.xmission.com)
define("USER", '<YOUR_USER_NAME>');
define("PASS", '<YOUR_PASSWORD>');
// The name of the server - should match Control Panel's name of the server
define("VPS_HOST_NAME", 'webserver01');
// Check RAM usage
error_log("resize_check: Checking server RAM usage.");
// Get the raw memory output
$raw_output = shell_exec("free -o -m");
$memory_lines = explode("\n", $raw_output);
$current_memory = preg_split("/[\s:]+/",$memory_lines[1]);
// $current_memory[1] is total, $current_memory[2] is used
$current_memory_load = $current_memory[2] / $current_memory[1];
error_log("resize_check: Current memory use is: " . $current_memory_load) ;
// Is the current memory load in tolerance?
if (LOWER_TOLERANCE > $current_memory_load) {
error_log("resize_check: Current memory usage of $current_memory_load below tolerance. Resizing.");
$current_memory_setting = get_mem();
error_log("resize_check: Current memory setting is: " . $current_memory_setting) ;
if ($current_memory_setting > LOWER_RAM_LIMIT) {
decrease_memory($current_memory_setting, MEMORY_INCREMENT);
error_log("resize_check: Decreased memory by " . MEMORY_INCREMENT ) ;
} else {
error_log("resize_check: Could not decrease memory more, already at lowest limit.") ;
}
exit(0);
} else if ($current_memory_load > UPPER_TOLERANCE) {
error_log("resize_check: Current memory usage of $current_memory_load above tolerance. Resizing.");
$current_memory_setting = get_mem();
error_log("resize_check: Current memory setting is: " . $current_memory_setting) ;
if ($current_memory_setting < UPPER_RAM_LIMIT) {
increase_memory($current_memory_setting, MEMORY_INCREMENT);
error_log("resize_check: Increased memory by " . MEMORY_INCREMENT ) ;
} else {
error_log("resize_check: Could not increase memory more, already at highest limit.") ;
}
exit(0);
} else {
error_log("resize_check: Current memory usage of $current_memory_load in tolerance. Exiting.");
exit(0);
}
// Helper Functions
function get_mem() {
$r_xml = req_rest('GET', VPS_HOST_NAME);
return ((string) $r_xml->{'ram-size'}[0]);
}
function increase_memory($current, $increase_by) {
$size_to_increase_to = $current + $increase_by;
error_log("resize_check: setting memory to: " . $size_to_increase_to);
$xml = '<?xml version="1.0" encoding="UTF-8"?>' .
"<reconfigure-ve><ram-size>$size_to_increase_to</ram-size></reconfigure-ve>";
$result = req_rest("PUT", VPS_HOST_NAME, $xml);
return $result;
}
function decrease_memory($current, $decrease_by) {
$size_to_decrease_to = $current - $decrease_by;
error_log("resize_check: setting memory to: " . $size_to_decrease_to);
$xml = '<?xml version="1.0" encoding="UTF-8"?>' .
"<reconfigure-ve><ram-size>$size_to_decrease_to</ram-size></reconfigure-ve>";
$result = req_rest("PUT", VPS_HOST_NAME, $xml);
return $result;
}
function req_rest($method, $get_params = "", $parameters = "", $additional_headers = "") {
$process = curl_init();
curl_setopt($process, CURLOPT_HTTPHEADER, array('Content-Type: application/xml', $additional_headers));
//curl_setopt($process, CURLOPT_VERBOSE, 1);
curl_setopt($process, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($process, CURLOPT_SSL_VERIFYHOST, 0);
//curl_setopt($process, CURLOPT_HEADER, 1);
curl_setopt($process, CURLOPT_USERPWD, USER. ":" . PASS);
curl_setopt($process, CURLOPT_TIMEOUT, 30);
switch($method) {
case 'POST':
curl_setopt($process, CURLOPT_POST, 1);
curl_setopt($process, CURLOPT_POSTFIELDS, array($parameters));
break;
case 'PUT':
curl_setopt($process, CURLOPT_CUSTOMREQUEST, "PUT");
curl_setopt($process, CURLOPT_POSTFIELDS, $parameters);
case 'GET':
default:
break;
}
curl_setopt($process, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($process, CURLOPT_URL, API_URL . "ve/" . $get_params);
$return = curl_exec($process);
curl_close($process);
if (strpos($return, '<') !== false) {
$xml = new SimpleXmlElement($return);
return $xml;
} else {
return $return;
}
}
Lines 8 through 12 define memory limits that form boundaries for the script, as well as the increments to increase and decrease memory. Line 15 defines the API URL, and should be fine for all XMission Cloud Services customers. Line 18 and 19 are the user name and password for the control panel <https://cp.xmission.com>. Line 22 is the hostname of the VPS.
Lines 28 through 35 gather data about the current memory load on the VPS. Lines 38 through 69 are the logic for keeping the server memory within boundaries set in the configuration.
Lines 73 through 76 provide a function to get the current provisioned memory from the API.
Lines 78 through 96 provide the functions to increase and decrease the memory using the API.
Lines 98 to 132 call the API and return the appropriate responses.
The script is saved in /usr/local/sbin/resize_check.php, then it is added to the crontab using “crontab -e” with the following entry:
* * * * * /usr/local/sbin/resize_check.php
For more details on how to setup a different cron schedule, you can read this page <http://en.wikipedia.org/wiki/Cron>. After the cron job is installed, it is time to re-run our siege.
Step 5: Siege the Server, Watch it Compensate
Re-run the siege command on your local machine:
siege -c 20 -t 5m http://<server-ip>/index.php
And run the following on your server:
tail -f /var/log/php_errors.log
Here is the output from our server php_errors.log:
[14-Aug-2013 17:29:01 UTC] resize_check: Checking server RAM usage. [14-Aug-2013 17:29:01 UTC] resize_check: Current memory use is: 0.3125 [14-Aug-2013 17:29:01 UTC] resize_check: Current memory usage of 0.3125 in tolerance. Exiting. [14-Aug-2013 17:31:02 UTC] resize_check: Checking server RAM usage. [14-Aug-2013 17:31:02 UTC] resize_check: Current memory use is: 0.921875 [14-Aug-2013 17:31:02 UTC] resize_check: Current memory usage of 0.921875 above tolerance. Resizing. [14-Aug-2013 17:31:03 UTC] resize_check: Current memory setting is: 256 [14-Aug-2013 17:31:03 UTC] resize_check: setting memory to: 512 [14-Aug-2013 17:31:03 UTC] resize_check: Increased memory by 256 [14-Aug-2013 17:32:02 UTC] resize_check: Checking server RAM usage. [14-Aug-2013 17:32:02 UTC] resize_check: Current memory use is: 0.6171875 [14-Aug-2013 17:32:02 UTC] resize_check: Current memory usage of 0.6171875 in tolerance. Exiting. [14-Aug-2013 17:33:02 UTC] resize_check: Checking server RAM usage. [14-Aug-2013 17:33:03 UTC] resize_check: Current memory use is: 0.455078125 [14-Aug-2013 17:33:03 UTC] resize_check: Current memory usage of 0.455078125 in tolerance. Exiting. [14-Aug-2013 17:34:02 UTC] resize_check: Checking server RAM usage. [14-Aug-2013 17:34:02 UTC] resize_check: Current memory use is: 0.5390625 [14-Aug-2013 17:34:02 UTC] resize_check: Current memory usage of 0.5390625 in tolerance. Exiting. [14-Aug-2013 17:35:02 UTC] resize_check: Checking server RAM usage. [14-Aug-2013 17:35:03 UTC] resize_check: Current memory use is: 0.537109375 [14-Aug-2013 17:35:03 UTC] resize_check: Current memory usage of 0.537109375 in tolerance. Exiting. [14-Aug-2013 17:36:01 UTC] resize_check: Checking server RAM usage. [14-Aug-2013 17:36:01 UTC] resize_check: Current memory use is: 0.076171875 [14-Aug-2013 17:36:01 UTC] resize_check: Current memory usage of 0.076171875 below tolerance. Resizing. [14-Aug-2013 17:36:01 UTC] resize_check: Current memory setting is: 512 [14-Aug-2013 17:36:01 UTC] resize_check: setting memory to: 256 [14-Aug-2013 17:36:01 UTC] resize_check: Decreased memory by 256
This shows over our 5 minute siege test that the server resized up to 512 MB and then back down to 256 MB when the server siege completed.
Conclusion
This demonstration shows an example of using the Cloud Infrastructure API, a little bit of PHP, and a scheduled process to maintain a useful memory ratio for a VPS. Your server will required different settings to optimize your traffic and this code is not perfect. We encourage you to customize it for your needs, or rewrite the whole thing in the language of your choice. There is a lot more depth to the API, so we encourage you to spend some time reading the documentation. If you have any questions or comments, please feel free to email us, or leave them in a note in the comments section.
How XMission spam and virus filtering works. 5 Questions Customers Commonly Ask + Our Best Answers
Comments are currently closed.