Show: Search - Categories - Posts
Posted on Tuesday the 7th of July 2009 at 1:45 AM
Update August 28th 2010: I have released a program that will give you basic adblock statistics here

Not too long ago I put up some adsense ads on a forum that was gaining popularity. I did so in order to help pay for some of the hosting costs. At no surprise to me a handful of members disliked the idea of having to suffer through the presence of ads on the site. I informed them that if their hatred to all things advertising was really that great then they should probably be using a browser plugin like Firefox's "Adblock plus". The majority of them were happy that I had informed them of adblock and based on what people were posting it seemed as though a good portion of regular members were already using adblock. So this got me thinking, how many of my regular visitors are actually blocking my ads? and is it enough to actually have an impact on the revenue?

My first hurdle was trying to figure out exactly how to detect if an ad was blocked. Luckily, it was a relatively small hurdle because a quick google search revealed that all you needed to do was check the height of the container your ads are in once everything has finished loading. Assuming your ads are the only thing in that container the height should return "0". This is easy in javascript, just give your container an identifier and then check its offset height. At first I made it so when an ad was blocked it recorded the users IP to the database using an AJAX request and if the user was already in the database it added to that users "block" count. The problem with this method was that I didn't really have anything to compare it to. I didn't have anything else recording pageviews for specific IP's at the time so even though I know a user blocked an ad 100 times, I had no idea how many times they didn't. So I modified my script to record both the amount of blocks and the amount of views.

function createXMLHttpRequest(){
    try { return new 
ActiveXObject("Msxml2.XMLHTTP"); } catch (e) {}
    try { return new 
ActiveXObject("Microsoft.XMLHTTP"); } catch (e) {}
    try { return new 
XMLHttpRequest(); } catch(e) {}
    
alert("XMLHttpRequest not supported");
    return 
null;
}
function 
recordBlock(b){
    
xmlPost createXMLHttpRequest();
    
// add our "blocked" value to our request
    
xmlPost.open"GET""http://"+document.location.hostname+"/forums/sqlrecord.php?b="+btrue );
    
// attach request header
    
xmlPost.setRequestHeader("X-Requested-With""XMLHttpRequest");
    
xmlPost.sendnull );
}
onload=function(){
    
blocked 0;
    
// detect if container has a height (has content)
    
if (document.getElementById('ads').offsetHeight == 0){
        
blocked 1;
    }
    
// activate recordBlock function
    
recordBlock(blocked);

The onload function only executes when the entire page has finished loading. This is essential because we do not want to check the ad container until we are certain that the ads have loaded. The variable "blocked" is 0 by default, meaning no block detected. If the offset height of the "ads" container is 0 pixels then "blocked" is set to 1. From there the record block function is called with the value of the "blocked" variable. The record block function uses an AJAX request to send the "blocked" variable to the PHP script. The "setRequestHeader" line sends a header variable over to the PHP script so the PHP can detect if the request was sent by an AJAX request.

Next we must record the information sent by javascript to the database. The database has a very simple setup with only 3 fields: IP, hits, and blocks. The IP is set as the primary and is the unique value in the database. There was nothing really special about the PHP file.

// detect if sent by AJAX
if ($_SERVER['HTTP_X_REQUESTED_WITH'] == "XMLHttpRequest"){
    
// connect to the sql server and select the database
    
$connection mysql_connect('host''user''pass') or die('can\'t connect to db');
    
mysql_select_db('db'$connection) or die('can not select db');

    
// assign the IP to a variable if the server can read it
    
$ip mysql_real_escape_string($_SERVER['REMOTE_ADDR']);

    
$block $_GET['b'] == 0;
    
//See if there's a key that exists with this IP
    
$result mysql_query("UPDATE adblocks SET hits = hits+1, blocks = blocks+".$block." WHERE ip = '".$ip."' LIMIT 1");

    if(
mysql_affected_rows() < 1){
        
// make a new key for this IP because one wasn't found.
        
mysql_query("INSERT INTO adblocks (ip, hits, blocks) VALUES ('".$ip."', '1', '".$block."')");
    }

The first line uses the header sent by the AJAX request to verify that it is an AJAX request. Further measurements can be taken to prevent spoofing, such as using a time sensitive cookie. If sent by the javascript the PHP will then open a connection to the mysql database. From there we escape the IP, I am not entirely certain about it, but there is a possibility that this is able to be spoofed. Better safe than sorry. After that we detect if the variable sent by the javascript was either sent to 1, for a block detected, or set to 0 for no block detected. If it's not one then it must be 0. My query is set up to update a row which contains the users IP. I decided to use update, instead of first checking if the IP was in the table, because this only requires one query and we can detect if the update affected any rows using the "mysql_affected_rows" function. If no rows were affected then the IP does not exist in the database and so we must enter a new row.

I had this system running on my server for a good amount of time. When I looked at my results I noticed that a huge portion of it was illegitimate because it was obviously the work of bots. So i ran every one of the IPs in my database through the list over at Project Honey Pot. There was a huge number of rows, so I automated the process using javascript and PHP. Doing 50 checks at once. In order to keep these bots out of the results I needed to modify the line that added a new IP to the database. Turning it into this:

$d explode("."gethostbyname($key .".".implode (".", array_reverse(explode(".", $_SERVER['REMOTE_ADDR'])))). ".dnsbl.httpbl.org"));
if(
$d[0] != 127){
    
mysql_query("INSERT INTO adblocks (ip, hits, blocks) VALUES ('".$ip."', '1', '".$block."')");

You need to sign up for a key at Project Honey Pot before you can start querying their database. The code is somewhat convoluted but that's because their system is somewhat convoluted as well. You must reverse the IP before you query them. When you use the php function "gethostbyname()" to get the IP address of that host their server will return to you an IP address that tells you what it knows about the IP you sent. You can learn more about this at their website. The important part is that if the first part of the IP that is returned is "127" it means that the IP is in their database.

The next part is organizing this data in an easily readable way. I do not want to post the entirety of my code here because it is very large and I don't want it to dominate this post. I'd rather just post snippets. I suppose the easiest way to get the fastest results is simply listing how many hits, how many blocks, and what the percentage is.

$result mysql_query("SELECT * FROM adblocks");
    
if(
mysql_num_rows($result) > 0)
{
    
$t_hits 0;
    
$t_blocks 0;
    while(
$row mysql_fetch_array($result))
    {
        
$t_hits += $row['hits'];
        
$t_blocks += $row['blocks'];
    }
}
echo 
"Total hits: ".$t_hits;
echo 
"Total blocks: ".$t_blocks;
echo 
"Block percentage: ".round((($t_blocks $t_hits) * 100), 2)."%";

This code assumes you have already established connection to the database. It then retrieves everything from the "adblocks" table, checks if there's something in there, and then adds all the hits and all the blocks from the rows together. It uses these totals to calculate what percentage of pageviews has been blocked. This will yield results like this (copied from my result):

Total hits: 212950
Total blocks: 30027
Block percentage: 14.1%

The system I use has more features, one example is I listed all the top blockers and most active users. I edited out their IP addresses for privacy.

You can go further than this and use something like google charts. I implemented it especially for this post and it didn't take much time at all:

Top blockers:

I attributed fake IP addresses to this but you can see that google charts can add some visual flare to your statistics page.

Using all of this I learned that 14% of my pageviews are blocking advertisments. While I respect their freedom to block content on whatever website they wish and as someone who uses adblock on occasion, and as someone who told them to use adblock, I see why adblock exists. However, although they are clearly not interested in clicking any of the advertisements on the website, effortlessly viewing them has a large impact on the amount of revenue that is gained from them; revenue that goes towards keeping the website online. The advertisements on any websites I run are as harmless and unobtrusive as the text you are reading right now. So what I did was use my adblock detecting technique to display a message in place of those ads that says exactly what this paragraph says, including a request to white list the website.
I also included a paypal link as an alternative means of supporting the website. Maybe, in my next post, I'll write a guide on how to display one of these messages.