ZXTM - Filter content in forms

There is nothing more annoying than getting spammed via your webforms. Using ZXTM's trafficscript you can filter out unwanted content. In the below example a reviews webform was getting badly spammed with links, advertising and attempted hacking. You could use Regex rules in the below as well if you wanted. It is also possible to maintain a "bad word" list in a database and have the ZXTM look it up.


$path = http.getpath();
$body = http.getBody(4000);
if( http.getmethod() == "POST"
&& $path == "/review/add_review"
&& ( string.contains( $body, "groups.google." )
|| string.contains( $body, "freehost.com" )
|| string.contains( $body, "blog.360.yahoo.com" )
|| string.contains( $body, "dreamstation.com" )
|| string.contains( $body, "cialis" )
|| string.contains( $body, "Bactrim" )
|| string.contains( $body, "Mumbai" )
|| string.contains( $body, "href" )
|| string.contains( $body, "javascript" )
|| string.contains( $body, "iframe" )
|| string.contains( $body, ".js" )
|| string.contains( $body, "SCRIPT" )
|| string.contains( $body, "HREF" )
|| string.contains( $body, "EMBED" )
|| string.contains( $body, "xmlns" )
|| string.contains( $body, "IMG SRC" )
|| string.contains( $body, "videoUrl" )
|| string.contains( $body, "http" )
|| string.contains( $body, "HTTP" )
|| string.contains( $body, "Umeh" )
|| string.contains( $body, "mpacuk.org" )
|| string.contains( $body, "drugsnoprescription" )
|| string.contains( $body, "xbox" ))

){
#log.info( "Review".$body); ### unremark to log info for testing
connection.discard();

}

ZEUS ZXTM - Rate Limiting requests

Zeus Traffic Manager is a fantastic load balancer and provides a number tools to manage your network & web enabled applications.
I have been using their Trafficscript for number of years now, and it provides a powerful tool to control your network traffic.

One such benefit is rate shaping, which can be triggered by any http header information you wish. It could be by user-agent, cookie or Url. Below is an example of how I rate limit users hitting specific Urls which can put excess load on the servers if abused.

Firstly, you need to create a "Rate Class" on the ZXTM appliance. In this case I have limited requests to 20 over a 60 second period.




Now you need to create a Trafficscript rule and assign it to the ZXTM Virtual Server.



# Look for a cookie - If the users cookie contains their sessionId we will use that to identify the user. This is because traffic may appear from behind a proxy or office IP which would cause the rate limit to assume all requests are one user

$userid = http.getCookie( "_session_id" );

#get URL - grab the URL from the request so we can apply the rate limit to only the Url we want to.

$url = http.getRawURL();

# If no cookie use IP - If there is no cookie, then either the browser has disabled it or in 99% of cases its a crawler

if( $userid == "" ) { $userid = request.getRemoteIP(); }

# Now we apply the rate shaping if the Url contains "search" or "login"

if( string.regexmatch( $url, "^/search" )
|| string.regexmatch( $url, "^/login" )) {
rate.use( "limit", $userid );
}

The result will prevent any denial of service and smooth out normal traffic to ensure all users get a good response.
You can then apply different rate classes to different requests as needed. For example you could rate limit based on user-agent.

Page load performance & SEO

Last year Matt Cutts from Google mentioned that a new ranking factor might be introduced into the google index algorithm.
Site speed is not only important to for keeping people on your site but can have an affect on the speed google indexes your site. Now it would seem it will affect ranking as well.

There are many tools out there to monitor performance and this should really begin at the development stage with tools such as Yslow and Firebug.

Yahoo have just released Boomerang which is a piece of javascript that you add to your webpages. This measures performance from the end users point of view so gives you greater visibility on real life performance

http://yahoo.github.com/boomerang/doc/

Converting a local OS into an Amazon AMI

Converting a local OS into an Amazon AMI

These are the steps required to convert a local Linux OS instance to an Amazon AMI machine image. In this example I have converted a 64-bit CentOS 5.2 XEN VM instance to an AMI and launched it on EC2.

Pre-requisites

In this example I will assume you have installed the Amazon API and AMI tools and correctly setup the security keys as per this guide.

http://docs.amazonwebservices.com/AWSEC2/2008-05-05/GettingStartedGuide/?ref=get-started

You will need the following files:
wget http://s3.amazonaws.com/ec2-downloads/ec2-ami-tools.zip

wget http://s3.amazonaws.com/ec2-downloads/ec2-api-tools.zip

wget http://ec2-downloads.s3.amazonaws.com/ec2-modules-2.6.18-xenU-ec2-v1.0-x86_64.tgz

Creating the AMI image

The first step is to prep the system for booting properly in EC2. When booting, it will be forced to use the EC2 kernel, rather than the currently installed one. You can use other kernels by launching the instance under a different AMI/AKI identifier if it matches yours.

Prep kernel:

Unzip the tarball ec2-modules-2.6.18-xenU-ec2-v1.0-x86_64.tgz to /lib/modules/

Prep network:

Remove any hardware addresses in “ifcfg-eth0” and configure for DHCP
DEVICE=eth0
ONBOOT=yes
BOOTPROTO=dhcp
TYPE=Ethernet

Prep fstab:

This can be done manually if you have specific volumes to mount, but I recommend letting the ec2 tools create it for you in the next section.

Building the AMI image:

Create a folder to create the image, in this example I use /image. You will need enough free space to create the image and it must not exceed 10G.

To create the image run the following command:
ec2-bundle-vol -d /image -k ~/.ec2/pk-.pem --cert ~/.ec2/cert-.pem -u --no-inherit -s 4000 --generate-fstab

When prompted for the arch type, select either i386 or x86_64 as required.

"--generate-fstab" - will create the required EC2 fstab file for you.
"--no-inherit" - Required if not re-bundling an EC2 image
"-s 4000" - This sets the image size in MB. I have made it as small as possible; I will then add a persistent volume to it and symlink /home /opt/ & /var/log/ to this.

Uploading AMI image to Amazon

You will need to upload the image to an S3 storage bucket so that it can be registered as an AMI instance.

ec2-upload-bundle -b tl-ami-images -m /image/image.manifest.xml --location EU -a -s -d /image/

"-b tl-ami-images" - This is where you define the S3 bucket name
"-m /image/image.manifest.xml" - Location of the manifest file created using ec2-bundle-vol
"--location EU" - US or EU

The access-keyid & secret key can be obtained from the AWS account under "Your Account"



Register & launch the new AMI

You can either use the API tools or the AWS Management Console to do this. The AWS Management Console is web based and much easier to use.
In the AWS console click on AMIs, the select "Register New AMI"
In the "Register Image" box enter the path to the S3 bucket image.
In this example you would enter tl-ami-images /image.manifest.xml




The AMI image will now appear in your list under "private" AMIs.


To launch open the "Instance" screen and select "Launch Instance"



Click "Select"



If you haven’t created a KeyPair or security group then do so. You only need to do this once for all instances. Launch it and your good to go !