Stardeveloper

Developing your first JSP Application in Eclipse

Faisal Khan — Sun, 12 Jun 2011 00:00:00 GMT

Eclipse provides complete development environment for working with JSP applications. It simplifies the task of creating both dynamic (JSP pages, Servlets, JavaBeans and Java classes) and static content (CSS and JavaScript files). Built-in support for debugging, testing and deployment takes the productivity to an entirely new level.

A JSP application running in Eclipse

In this tutorial, we will learn how to create a new project in Eclipse for developing a dynamic JSP web application. The web application for the purpose of simplicity at this time, will be a single page JSP application that will show current date and time on the server. The application will run on Apache Tomcat 7.0.

Prerequisites

This tutorial assumes that you have working installations of both Apache Tomcat and Eclipse IDE for Java EE development on your computer:

Step 1: Start Eclipse

Assuming that you installed Eclipse in eclipse folder in your C:\, go to the C:\eclipse folder and double-click eclipse.exe file, as shown below:

Eclipse folder

Eclipse boots up and shows the welcome screen:

Eclipse IDE

Step 2: Create new project

Now, in the Eclipse IDE, click File -> New -> Dynamic Web Project. New Dynamic Web Project window pops up as shown below:

New Dynamic Web Project window

Enter 'Project name' as "First JSP Project". Leave rest of the settings as they are and click 'Next'.

Next window asks for adding paths to folders containing pre-existing Java source code files. Since this is a new project and we don't have any other source code folders, we'll just accept the default source code folder of src. Click 'Next' to proceed forward.

Source code folders options

Next window asks for web module configuration settings. A web module is a small JSP application that can either be run as a stand-alone application or it can be a component of a much larger enterprise application. An enterprise application can contain one or web modules along with other modules.

We accept the default 'Context root' and 'Content directory' folder names and leave the 'Generate web.xml deployment descriptor' option unchecked. We click 'Finish' and let Eclipse create folders, files and structure for new JSP application.

Configure web module settings window

Step 3: Examining the project

Eclipse opens our 'First JSP Project' project in a new window.

First JSP Project window

If we expand the 'First JSP Project' node in the 'Project Explorer' window, we can see the names and contents of the different folders that Eclipse has created for us:

Project Explorer window

Among all the folders that Eclipse has created for our project, the one important to us now is WebContent folder. It is this folder that should contain the web content like JSP pages, static HTML pages, CSS and JavaScript files, etc. It is this folder where we'll create our JSP file for this project.

Step 4: Add new JSP page

Now left-click the 'First JSP Project' node in the 'Project Explorer' window to select it. Then right-click to see the list of options. Select New -> JSP File. 'New JSP File' window pops up as shown below:

New JSP File window

Enter 'File name' as "index.jsp", keep the 'WebContent' folder selected and click 'Next'. 'Select JSP Template' window appears as shown below:

Select JSP Template window

Select 'New JSP File (xhtml)' template from the template names in the left column and click 'Finish'.

Eclipse creates a new JSP file from the template we selected and puts in enough HTML code along with a JSP directive to give us a head-start for adding further code to this file.

JSP File window

Now replace code in the index.jsp file by the code given below:


<%@ page language="java" contentType="text/html; charset=ISO-8859-1"
    pageEncoding="ISO-8859-1"%>




First JSP Page



Current date and time: <%= new java.util.Date().toString() %>

The code is pretty much the same as it was except that I set the title of the page to "First JSP Page" and added one line of HTML intermixed with JSP code to display the current date and time on the server.

Step 5: Configuring Eclipse to use Apache Tomcat

Now that we have a JSP page and a pretty basic JSP web application, we want to run this application on a Servlet container. We have already installed Apache Tomcat 7.0 on our computer. We will now configure Eclipse to use that server as the Servlet container for our JSP application.

Click Run -> Run in the Eclipse window. This should make Eclipse pop up a 'Run On Server' window as shown below:

Run On Server window

Now, click to expand 'Apache' node from the list of servers. Then select 'Tomcat v7.0 Server' from the list of options. Check the 'Always use this server when running this project' option and leave rest of the options untouched and proceed to click 'Next' as shown below:

Configuring Eclipse to use Apache Tomcat 7.0

'Tomcat Server' configuration window opens up and asks for the path to the installation folder of Apache Tomcat 7.0. Click 'Browse' to select the Tomcat installation folder on your computer and click 'Finish'.

Selecting Tomcat installation directory

Step 6: Running our JSP application

Eclipse configures the JSP application to use the Apache Tomcat server 7.0. Next, it automatically starts the Tomcat server in the background and opens the index.jsp page in a new tab as shown below:

index.jsp running on the Tomcat server

You have successfully configured your JSP application to use Apache Tomcat 7.0 to run your application. From next time onwards, you can run your JSP application with a click of a button. You will not have to go through the configuration steps again.

Congratulations! You just finished creating a new JSP project in Eclipse, configured Eclipse to use Apache Tomcat 7.0 with your JSP project, and then finished running your JSP application successfully for the first time.

Installing Eclipse for JSP Development

Faisal Khan — Thu, 09 Jun 2011 00:00:00 GMT

Eclipse is a free and open source IDE (Integrated Development Environment) for developing a variety of applications from console and GUI applications on the client-side to web applications on the server. It has specific editions for different programming languages. In case of JSP, it offers Eclipse IDE for Java EE Developers for developing JSP based web applications.

Eclipse

The purpose of an IDE is to make the task of developing applications much easier. In the context of JSP web applications, Eclipse supports syntax coloring and highlighting, intellisense and code completion. It supports TDD (Test Driven Deployment). It has built-in support for debugging and allows one-click compilation and deployment of JSP applications.

In this tutorial, we will download and install 'Eclipse IDE for Java EE Developers' that will allow us to develop, test and run JSP web applications with Apache Tomcat 7.0 as the Servlet container.

Step 1: Download Eclipse

Open your browser and visit Eclipse Downloads page. As shown in the figure below, click the download link for 'Eclipse IDE for Java EE Developers' and choose 32 or 64 bit version depending on the Operating System on your computer:

Downloading Eclipse IDE for JSP development

Step 2: Unzip Eclipse

The file that you just finished downloading will have a name of something like eclipse-jee-helios-SR2-win32-x86_64.zip. Now, unzip this zip file into a new folder on your C:\ drive. You can choose any drive but I chose C:\ for the purpose of simplicity.

Next, create a new folder with the name of 'eclipse' in that drive and then unzip eclipse-jee-helios-SR2-win32-x86_64.zip in that folder.

Unzipping Eclipse zip file

After the unpacking is complete, this is what your C:\ should look like:

Eclipse folder

Step 3: Running Eclipse

Now go to the C:\eclipse folder and double-click eclipse.exe file as shown below:

Running Eclipse

Eclipse starts up! First it will ask you for the location of default workspace. When you select a location and proceed next, Eclipse's default welcome screen shows up as shown in the image below:

Eclipse IDE

Congratulations! You have successfully downloaded and installed Eclipse IDE for JSP development on your computer. You are now ready to start developing JSP web applications.

Installing Apache Tomcat 7.0

Faisal Khan — Wed, 25 May 2011 00:00:00 GMT

Apache Tomcat is a free and open source Servlet container for JavaServer Pages. With version 7.0, it supports Java Servlet spec 3.0 and JSP spec 2.2.

Apache Tomcat versions

It is very easy to install on your computer. You can use it to develop and run web applications. A web application in the context of JavaServer Pages means developing an application that makes use of JSP pages, Servlets, Filters, JDBC, JSP custom tags and other Java based server-side technologies.

In the rest of this tutorial, I will walk you through installing Apache Tomcat 7.0 server on your Windows Operating System.

Step 1: Download and install JDK 6.0

If you already don't have JDK (Java Development Kit) 6.0 installed on your system then you should visit Java SE Downloads page and download latest JDK 6.0 release build along with all the documentation on your computer.

Downloading JDK 6.0

As shown in the image above, a JDK includes both the JRE (Java Runtime Environment) to run Java based applications (like Apache Tomcat server) and applets, and Java compiler and debugger to assist you in developing Java applications.

How to know which version of Java you have on your computer?

If you are unsure which version of Java is installed on your computer then open Command Prompt, enter the following command and press the Enter button:

java -version

If Java is installed on your system and the full path to Java.exe is included in the PATH environment variable on your computer then Java version information should appear on your screen as shown below:

Java version information

Now that you have a working copy of latest release build of JDK 6.0 on your computer, you should proceed to download Apache Tomcat server.

Step 2: Download Apache Tomcat

Go to the download page for Apache Tomcat 7.0. Scroll down the page a little until you get to Binary Distributions.

Downloading Apache Tomcat 7.0

Click on the appropriate link for your Operating System to download the file on your computer. In my case, I chose "64-bit Windows zip" as I was running Windows 7 on a 64-bit processor.

Now that you have the latest binary build of Apache Tomcat 7.0 on your computer, proceed to the unpack the zip file.

Step 3: Unzip Apache Tomcat

Now unzip the downloaded zip file into a convenient location on your hard drive. In my case, I unzipped it into "C:\apache-tomcat-7.0.14".

If you don't have a utility program to unzip .zip files on your computer, you can download and install the free 7-Zip utility to easily zip and unzip files on your computer.

Now that you have unzipped the file into a convenient location, you should proceed to setup the environment variables.

Step 4: Setup environmental variables

Go to Start Menu -> Control Panel -> System. When the window opens up for the System settings, click the link to "Advanced system settings" as shown in the image below:

Control Panel - Advanced system settings

When the System Properties window opens, click the "Environment Variables" button as shown in the image below:

System Properties window

Now click the "New" button in the Environment Variables window to add a new System variable:

Environment Variables window

Now enter the Variable name as "JAVA_HOME" and Variable value as "C:\Program Files\Java\jdk1.6.0_25" or whatever the complete path to the folder where your JDK was installed is, and press the OK button.

New System Variable window

Now close all the open windows and proceed to start the Apache Tomcat server for the first time.

Step 5: Start Apache Tomcat

Now open Command Prompt and enter commands as shown in the image below to change directory to the bin folder of the Apache Tomcat's installation folder. On my computer, I unzipped the Apache Tomcat zip file in the "C:\apache-tomcat-7.0.14" folder.

Start Apache Tomcat

Once there, type the startup command and hit the Enter button to startup Apache Tomcat server for the first time on your computer.

startup

A new window pops up and shows the startup log of the Apache Tomcat server as shown in the image below:

Apache Tomcat starts up!

Now please open your browser and enter this address in the address field: http://localhost:8080 and hit the Enter button to load the home page of your local Apache Tomcat server. You should see a home page with a message of success as shown in the image below:

Home page of Apache Tomcat running on your computer

Congratulations! You just finished installing and running Apache Tomcat 7.0 server for the first time on your computer.

Play with the Servlet and JSP Examples

You can click the "Servlet Examples" and "JSP Examples" links found on the home page of your local Apache Tomcat server to see the demos of live working Servlets and JSP pages.

Shutting down Apache Tomcat

To shut down Apache Tomcat, simply run this command while in the Apache Tomcat's bin directory:

shutdown

Shut down Apache Tomcat

Now you have a working copy of Apache Tomcat on your computer. Good luck with learning and developing JSP web applications!

An HTTP Module to redirect requests from root domain of your website to www subdomain

Faisal Khan — Sun, 02 Jan 2011 21:08:00 GMT

Introduction

We will develop an HTTP module that will redirect all requests to non-www root domain name of your website, for example, stardeveloper.com, to www subdomain, for example, www.stardeveloper.com. You can also do the opposite and redirect all requests from www.yourwebsite.com to yourwebsite.com if you wish.

The redirection process will work by sending an HTTP 301 (permanent redirect) header to the browser. It is useful as the search engines and browsers will be able to give credence to the correct URL for your web page, that is, the ones with www. subdomain.

Demonstration

To demonstrate how this HTTP module works, let me make a request to the URL http://stardeveloper.com and see how it goes.

Sending a request to root domain of Stardeveloper.com (http://stardeveloper.com)

My request gets redirected to http://www.stardeveloper.com.

The server redirects the request to http://www.stardeveloper.com

Go ahead, try it yourself. Just enter http://stardeveloper.com in the URL field of your browser and hit 'enter' and see what happens.

Explanation

The HTTP module intercepted the request to the root domain of http://stardeveloper.com and redirected the browser to the www subdomain, http://www.stardeveloper.com.

I have set it up this way because I want Google to index my website as www.stardeveloper.com. Even though I can specify my preference in the Google webmaster tools panel, I still find it simple and elegant to keep all pages on my website with www.stardeveloper.com as the domain name.

Since all users who visit my website will see the www subdomain name of the website, I am pretty sure they will link to the pages with www subdomain in the URL as well.

The HTTP module sends a '301 permanent redirect' header to the browser. It means that browser should always save the URL with the correct host name, that is, www.stardeveloper.com, in its history, favorites, and other records. The same applies to search engines, who will always index my website with www subdomain name.

Uses

You can use this HTTP module to:

Redirect users from non-www host name of your website to a www. host name.
For example, you can redirect all requests to http://yourwebsite.com to http://www.yourwebsite.com and vice versa.
You can redirect .net and .org requests for your domain aliases to your .com domain name.
This scenario is useful if you are also using .net and .org domain names like http://www.yourwebsite.net and http://www.yourwebsite.org and you want those domains to redirect to your website's .com domain name like http://www.yourwebsite.com.

At Stardeveloper, I also have registered http://www.stardeveloper.net and http://www.stardeveloper.org domain names. I use this module to redirect all requests to those domain names to http://www.stardeveloper.com.
If you are moving your website from one host name to another then you can redirect all requests sent to previous host name to the new one.
For example, you can set this module to redirect all requests to http://www.youroldwebsite.com to http://www.yournewwebsite.com.

Root domain vs www subdomain

The domain name for Stardeveloper is stardeveloper.com. Most users think that a root domain name is synonymous with www subdomain of a website. This actually is not technically true as root domain name and www subdomain can point to two different IP addresses/websites. But for end users' perspective, they think that with or without the www in the host name, the request should go to the same website. So in case of Stardeveloper, typing http://stardeveloper.com should lead to the same home page as http://www.stardeveloper.com does.

With that said, you should have one policy for the links on your website. I will recommend not switching back and forth between www and non-www links on your website pages. Keep a consistent theme and force it using this HTTP module.

Examples of websites with preference for root vs www subdomain

To give you example of how real-World websites are using this technique, I will give examples of two popular social networking websites that have a preference for non-www and www subdomains in their host names.

Twitter

Twitter strongly forces the URL to use the shorter version of the host name for their website. If you type http://www.twitter.com, you will be redirected to http://twitter.com.

Facebook

Facebook on the other hand uses the full www subdomain for the pages on its website. Go and type http://facebook.com in your URL field of your browser and you will be redirected to http://www.facebook.com.

You can choose either of the two themes for your website. Whichever you choose, the decision is yours, but just select one and use it consistently.

How will domain redirection work?

Let me give you a quick overview of how the domain redirection process works.

When a user requests a page, the request gets intercepted by our DomainRedirectorModule HTTP module. It checks the host name of the URL.

Host name is the part of the URL containing the full domain or subdomain of the website. For example, in the link http://www.stardeveloper.com/articles/convert-ip-address-to-long/, www.stardeveloper.com is the host name.

Now, if the host name of the URL matches the non-www version of the host name of our website, for example, stardeveloper.com, it will redirect the full request including the path and query information associated with the URL to www.stardeveloper.com. Thus a request like http://stardeveloper.com/articles/convert-ip-address-to-long/?ref=whatever gets redirected to http://www.stardeveloper.com/articles/convert-ip-address-to-long/?ref=whatever.

And like I said before, the HTTP module will send 301 HTTP header to tell to the browser, search engine or whoever the client is to permanently redirect the request. This means that we are telling the client that previous URL is no more accessible and you should update the reference from the old URL to the new one.

Developing the DomainRedirector HTTP module

Let us now look at the code of the HTTP module we talked above so that you can create this module on your computer and use it on your website straightaway.

Code

Create a new text file and save it as Stardeveloper.Modules.DomainRedirectorModule.cs in the /App_Code folder of your ASP.NET application. Then copy/paste following code in it:

using System;
using System.Collections.Generic;
using System.Data;
using System.Data.SqlClient;
using System.Linq;
using System.Threading;
using System.Web;

namespace Stardeveloper.Modules
{
    public class DomainRedirectorModule : IHttpModule
    {
        const string FromHostName = "stardeveloper.com";
        const string ToHostName = "www.stardeveloper.com";
        
        public void Init(HttpApplication app)
        {
            app.BeginRequest += new EventHandler(app_BeginRequest);
        }

        protected void app_BeginRequest(object sender, EventArgs e)
        {
            HttpApplication app = sender as HttpApplication;
            Uri currentUri = app.Request.Url;

            if (currentUri.Host == FromHostName)
            {
                Uri redirectToUri = new Uri(new Uri("http://" + ToHostName), currentUri.PathAndQuery);
                
                RedirectPermanently(app.Response, redirectToUri.ToString());
            }
        }
        
        public static void RedirectPermanently(HttpResponse res, string location)
        {
            // Permanent redirection
            res.Status = "301 Moved Permanently";
            res.AddHeader("Location", location);
            res.Flush();
            res.End();
        }

        public void Dispose()
        {
        }
    }
}

Explanation

It is a pretty straight forward HTTP module actually. If you have read the introduction to developing HTTP modules, the code here shouldn't be too difficult to understand.

Anyhow, the first thing we do is to import namespaces that we will be using in our code later.

using System;
using System.Collections.Generic;
using System.Data;
using System.Data.SqlClient;
using System.Linq;
using System.Threading;
using System.Web;

Next we specify the namespace for our HTTP module.

Our DomainRedirectorModule class implements the IHttpModule interface. We have to do that if we want our DomainRedirectorModule class to work as an HTTP module.

namespace Stardeveloper.Modules
{
    public class DomainRedirectorModule : IHttpModule
    {
        ...
    }
}

Two constants are used to set the from and to host names of our website.

Here we are using straight string matches. You can improve this code to use regular expressions to match from and to domains and sub domains.

const string FromHostName = "stardeveloper.com";
const string ToHostName = "www.stardeveloper.com";

Init() method is called by ASP.NET runtime to allow our HTTP module to select events in the HTTP request handling pipeline that it wants to handle.

Here, we simply handle the BeginRequest event. This event is the first event in HTTP request handling pipeline. Handling this event means that we can redirect the user straight from the surface of our web application and not letting the request pass deeper into HTTP request handling pipeline.

public void Init(HttpApplication app)
{
	app.BeginRequest += new EventHandler(app_BeginRequest);
}

Right, now we get into the main method that checks the host name and redirects the user if necessary.

We do a direct string comparison using the == operator in C#. If two strings match, we get a true value back, that is, we are given a green pass to step down into the code in the brackets.

Assuming we have a match for the host name requested by the user to the host name that we have in the FromHostName constant, we create a new Uri object using a constructor that takes two arguments. The first arguments is the Uri of the base host name of the URL and the second argument is the path and query part of the destination URL.

This way, all requests to pages deeper in the website with query string parameters in them, like http://stardeveloper.com/articles/display.html?article=2009071801&page=2, get redirected to a URL with the same path and query, like http://www.stardeveloper.com/articles/display.html?article=2009071801&page=2.

Finally, we redirect the user to the destination URL.

protected void app_BeginRequest(object sender, EventArgs e)
{
	HttpApplication app = sender as HttpApplication;
	Uri currentUri = app.Request.Url;

	if (currentUri.Host == FromHostName)
	{
		Uri redirectToUri = new Uri(new Uri("http://" + ToHostName), currentUri.PathAndQuery);
                
		RedirectPermanently(app.Response, redirectToUri.ToString());
	}
}

We use a utility method with the name of RedirectPermanently() to send 301 header to the browser to redirect the request to the destination URL.

In practice, I keep methods like this in other utility classes and not in the HTTP module like I am doing in the code here. I have copied the method here only to keep things simple to understand for you.

public static void RedirectPermanently(HttpResponse res, string location)
{
	// Permanent redirection
	res.Status = "301 Moved Permanently";
	res.AddHeader("Location", location);
	res.Flush();
	res.End();
}

Well, this is just about it as far as the code is concerned. The code is so clean that it shouldn't take you much time to understand how we are handling domain redirection.

Installation

To install this HTTP module in your ASP.NET web application, open your root web.config file and add following bit of configuration data in the configuration/system.webServer/modules node:

...

Now your HTTP module is ready to start redirecting the requests.

Summary

In this tutorial, we learned how to develop an HTTP module to sit on top of our ASP.NET web application and redirect requests to www or non-www host names, as we desire, of our website. We also learned that we can use the module to redirect requests from .net and .org domain names of our website to .com domain name. We can use the same technique to redirect requests from old host name to a new one. This may come very handy when moving content from an old website to a new one.

Join me

Well there you have it guys, another article with code and explanation to help you solve real-World problems, add features, and improve web application design and performance on your website.

If you like this article then please join my email list using the links below and I'll send you new articles I publish by email. You won't even have to visit this website if you don't want to. You can even visit my Facebook page and like it to connect with me, or should I put it better, to keep a tap on what I up to. If you want to connect with me personally, you can also add me as friend on Facebook if you have interest in web development, analytics, A/B testing, technology, blogging, etc. I like that kind of stuff and we'll do well if you have at least some interest in those fields. :)

If you are interested in writing for Stardeveloper.com, then please add me as a friend on Facebook and then message me on what you want to write about. If you need advice, or have a question, then that my just be the right place to ask that.

I wish you all the success! See you in another article.

How Stardeveloper.com achieved 120% improvement in website load time?

Faisal Khan — Sun, 26 Dec 2010 12:41:00 GMT

Back in November 2009, I visited the 'Site Performance' section in Google webmaster tools and was astonished to see that it was taking on average 8.6 seconds for a page to load on Stardeveloper.com. Google also further commented on it by saying that my website was slower than 90% of the websites on the internet. I was taken aback.

Pages took 8.6 secs on avg. to load before enabling ASP.NET static resource caching

I read everything I could find on caching, website performance, HTTP headers, delay loading of JavaScripts, etc., to improve the loading time of my website. It was only after I came to know about setting HTTP Expires and Cache-Control: max-age headers that I decided to set them up on my website. The results were nothing short of awesome. As you can see in the chart below, the average load time had improved to 3.9 seconds.

Pages now take on avg. 3.9 secs to load after enabling ASP.NET static resource caching

In the rest of this article I talk about how I did it on my ASP.NET website, and what I learned along the way and how you can implement it on yours.

Website Performance

While the importance of caching items accessed from the database in ASP.NET pages remains the same, we have come to know that it is only 20% of the perceived performance that the end-user gets. For them, if a website has plenty of JavaScript files, iframes, ads, images and multiple CSS files, the pages will take a long time to load no matter what you do at the backend.

Website performance is an area which is coming more and more in limelight these days. Most of the work nowadays is being done at improving and optimizing the front-end of the website. This is where 80% of the perceived performance of a website for an end-user exists. There are many things that can be done to improve the load time but one thing that we, as ASP.NET developers, can do in just a few minutes of our time and which can have a huge effect in improving the load time of our website is to enable static resource caching for our ASP.NET pages.

What are static resources?

Static resources are CSS, JavaScript, images, videos and zip files that reside on our web server. We can tell IIS, our ASP.NET pages run on, to send one of the two standard HTTP headers, Expires or Cache-Control: max-age, to the client browser every time a static resource is requested on our server. These headers will tell the browser to cache these items on its local hard drive for a long period of time, thus cutting the need to request them from our server again and again.

If you want to enable static resource caching in your ASP.NET website, then you should check out my introduction to Expires and Cache-Control: max-age headers. If you are an ASP.NET developer then you cannot absolutely afford to not read that article. It only takes a couple of minutes to edit your web.config file and your website will see dramatic improvement in load time.

Stay tuned, I will now share with you exactly where my website, Stardeveloper.com, stood one year back and what effect enabling static resource caching had on its load time. All the stats I refer to in this article are from Google webmaster tools. Trust me, you will be surprised to see the results.

Why I felt there was a need to improve website load time at Stardeveloper.com?

Let me start by saying that late last year (2009), I noticed that pages at Stardeveloper took an average of 8.6 seconds to load. I came to know this after checking my website's load time in Google webmaster tools.

Pages took 8.6 secs on avg. to load before enabling ASP.NET static resource caching

That chart took me by surprise. I was shocked and saddened to see that my website was slower than 90% of the websites on the Internet. I didn't see the bright side of it. I mean making a website slower than 90% of the websites on the Internet is quite an achievement in itself. Right? LOL! No kidding, I was very disappointed.

What I did was that I began to change the design and layout of Stardeveloper.com. You can see the new design and layout in the new articles that have been published this year (2010). This article that you are reading is using the new theme. You may not notice it but the pages in the new layout are quite optimized to load fast in your browser.

But just making the new articles load quicker wouldn't do the trick for my website. There are hundreds of old articles and forum posts compared to which the new articles don't yet make 1% of the content. So what to do?

HTTP caching, headers and website performance

I began to read about HTTP caching, headers, website performance, and load time improvement tips from the experts. One thing that struck me immediately was the effect of using HTTP Expires and Cache-Control: max-age headers. They did not need an ETag to be sent for the file. They made sense to me the moment I read about them.

Now that I knew about Expires and Cache-Control: max-age headers, I needed to implement them on my website. I had two options:

Develop an HTTP module and then each time a static resource was being sent to the browser, I should set one of these two headers using HttpResponse.Cache.SetExpires() for Expires and HttpResponse.Cache.SetMaxAge() for Cache-Control: max-age, methods, each time that resource is requested.
The problem was that I had to manually check the file that was being sent to the browser to see if it was a static file or a dynamic file. While it was still not difficult, there was an easier option available.
Edit web.config file and add few lines to tell the IIS to send Expires or Cache-Control: max-age headers for all static resources on the website.

The moment I came to know that these headers can be enabled just by editing the web.config file, I immediately updated the web.config file for my website. I used Cache-Control: max-age header because it allowed me to set a relative date and time in the future. You can learn more about Expires and Cache-Control: max-age headers and how to set them using web.config file by visiting that article.

Once I had made the changes, I crossed my fingers and waited to see how my website's load time improves in the time to come.

120% improvement in website load time at Stardeveloper.com after enabling ASP.NET static resource caching

After making the changes in the web.config file to enable static resource caching using Cache-Control: max-age header, my website's load time slowly began to improve, that is, it began to took less and less time for the web pages to load at Stardeveloper.com.

I was so deligted that I wanted to share this information with my website visitors. This series of articles took inspiration from the very same fact. Here is the latest image taken from Google webmaster tool's site performance page:

Pages now take on avg. 3.9 secs to load after enabling ASP.NET static resource caching

As you can see, it is now taking only 3.9 seconds for a page to load on average at Stardeveloper.com as compared to 8.6 seconds it was taking just an year back. A massive improvement of more than 120%, isn't it?

Conclusion

In this article, we learned how Stardeveloper.com got an improvement in load time from using static resource caching by big order of magnitude. We saw the technique and learned how we can implement it on our websites.

Considering the fact that page speed may be a deciding factor in ranking a website on Google, improvement in load time was a welcome change. Not only it leads to improvement in page ranking on Google, it leads to a decrease in bounce rate of visitors reading articles on your website as well.

That's it fellows! I hope you found the article inspiring. These are true stats of a website which is a living example of how much an effect one can achieve in load time improvement using HTTP caching headers. Please set them up on your website, and if you already have, share your stats in comments below.

Display Dzone voting button on your website only to visitors referred form Dzone.com

Faisal Khan — Mon, 06 Dec 2010 08:26:00 GMT

The title of this article says it all. You will learn how to tag all Dzone visitors, using a small cookie, who visit your website by following a link on Dzone.com that links to your website. Once that cookie is placed in the user's browser, we will be able to identify visitors from Dzone.com on all visits to your website from then onwards. That cookie will be read by a piece of code on your website which will display Dzone voting button to these users. Dzone voting button will serve to increase the votes your website receives from Dzone users, leading to even more visits and page views for your website.

The code for this application was developed in C# and it works on all websites running ASP.NET 3.0 and above. We have already developed the complete application, Referring Domain Tagger Application. You can learn more about tagging and how you can use it to display buttons, badges and widgets on your website for all the cool social networking and bookmarking websites out there like Facebook, Twitter and Dzone, only to users referred from them.

Displaying Dzone voting button to Dzone users at Stardeveloper.com

Following couple of images show how an article on Stardeveloper.com shows Dzone button to a visitor referred (and thus tagged) from Dzone.com:

No Dzone voting button for other users of Stardeveloper.com
(click the image to view in full size)

Dzone voting button being shown to a visitor from Dzone.com
(click the image to view in full size)

Although Dzone doesn't need an introduction because it is a very popular developer website, I will provide a short introduction to it and its cool voting system and then I will give you a practical example of how this works (and is working on Stardeveloper.com) with images and live demonstration.

What is Dzone and how does its voting system work?

Dzone is a website targeted towards developers. The 'D' in Dzone stands for 'Developers'. You can call Dzone as 'Developers Zone' if you want. It is open for all people to submit links to news, articles and blog posts of interest to developers. Depending on the relevance and quality of the content, that link is voted up or down. Any Dzone user can cast his vote for any link submitted on Dzone. Once a link gets more and more votes, it gets promoted on their website. The more that link receives votes, the more it says live on their website. The more live it remains, the more clicks that link receives.

Why should every developer website have a Dzone voting button?

If you are a developer and have a website or a blog on topics related to programming languages, databases, or web development in general, your website will benefit from having a Dzone voting button on it. You should also register for a free user account at Dzone and post links to your articles and blog posts as you publish them.

Why use Referring Domain Tagger Application?

To prevent loading of Dzone JavaScript file for every user of your website, the majority of which may be other users, you should use Referring Domain Tagger Application to only display Dzone voting button to users of Dzone.com.

You can use similar logic to load IFrame and JavaScript files for buttons, badges and widgets of other bookmarking websites without compromising the load time of your web pages by using this application.

Live example of Referring Domain Tagger Application working on Stardeveloper.com

If you are a Dzone user, you would already have seen a Dzone voting button at the top of this article. That button was displayed because you were tagged, that is, a cookie was placed on your browser by an HTTP module installed on this website when you clicked a link on Dzone and visited Stardeveloper.com. Then an ASP.NET custom web server control was used to read that tag and display a Dzone voting badge to you.

You can learn more about the different components of this application by reading the introduction to this application and following the step by step one-by-one from there.

If you are not a Dzone user then you would not have seen a Dzone voting button on this article. Don't worry, I'll make it pop up for you. :)

Go to Dzone and enter 'Stardeveloper.com' in the search field and press the 'Submit' button. Here is a direct link to search results showing articles from Stardeveloper.com. Next, click the first article that comes up in the search results. You will be redirected to an article on Stardeveloper.com. This time there will be a Dzone voting button shown at the top of that article. Simple!

Next, close your browser window. Now reopen it and visit Stardeveloper.com and navigate to any article on it. You can visit this same article you are reading if you want. There will be a shining Dzone voting button waiting for you to vote up. :)

Please don't forget to vote up this article if you found it useful. You can also share this application on Twitter and Facebook. The links are shown at the top of the article. I'll appreciate it very much!

The code for Dzone voting button

If you have followed the development of this application from the beginning, you'll be knowing that the two most important components of this application are ReferringDomainTaggerModule HTTP module and TaggedDomainMatcherControl custom web server control. I'll show just the code that I put on my ASP.NET pages that display articles at Stardeveloper.com for Dzone voting button using the TaggedDomainMatcherControl.

This is the only piece of code required to display Dzone voting button to Dzone users in your ASP.NET pages:

...

<%@ Register TagPrefix="sd" Namespace="Stardeveloper.Controls" %>

...






...

You can see clearly how simple this custom web server control, TaggedDomainMatcherControl, makes our job of displaying the Dzone voting button. Whatever text you enclose inside this control gets displayed to the user only if he has been tagged by the ReferringDomainTaggerModule HTTP module. In this case, the text was the HTML and JavaScript code to display Dzone voting button.

Summary

In the multipart tutorial, we learned how a small cookie can be placed in a user's browser visiting our website from a link on another website. That cookie's value can be the domain name of the referring website. We called it tagging the user. We created an HTTP module and placed the code to scan for referring links and cookies in it. To tag users from only a small number of websites, we listed the domain names to tag in the web.config file. We then created a custom web server control and inserted it in ASP.NET pages where we wanted to display site specific code only to users of a given a domain name. An example of which we saw in this article.

Please read this tutorial from the beginning to learn tagging and understand how this application works, and how you can port this application to other web development platforms.

That's it friends, please share this application with others who you think might benefit from it. Spread the good word as they call it. I'll begin development on another application now and will post it on this website when I have finished it. So if you feel like, you can bookmark this website using the links below. Have a good day!

A custom web server control to display site specific content to Facebook, Twitter, and Dzone users

Faisal Khan — Mon, 06 Dec 2010 08:24:00 GMT

We will create a custom ASP.NET web server control that will read the tag (a cookie set by ReferringDomainTaggerModule HTTP module) and the referring link to identify the user to our website as a user coming from a social network or bookmarking site like Facebook, Twitter or Dzone. Once the use is identified, the content within the control will be rendered to the user to display social bookmarking and sharing links and badges.

This custom web server control is a part of Referring Domain Tagger Application. Learn more about tagging and the purpose of this application by reading the first few pages.

How will this control work?

This is a custom web server control that you will use to encapsulate code that you want only to be rendered to users coming from social networks or bookmarking websites. An example of a code that you can use to say "Hi" to Dzone users is shown below:

<%@ Register TagPrefix="sd" Namespace="Stardeveloper.Controls" %>


	Hi Dzone user!

We first register our Stardeveloper.Controls namespace using the tag prefix, "sd".

To access the control, we write down the name of our TaggedDomainMatcherControl class after the tag prefix and colon. There are two attributes in this control. DomainToMatch should contain the domain name to match. runat is mandatory and its value is almost always "server.".

The encapsulated bit of code will only be rendered if the user has been tagged, that is, have visited our website from Dzone.com. You can replace 'dzone.com' with 'facebook.com' to render site specific content to Facebook users. What you can do with this control is only limited by your imagination and creativity.

Stardeveloper.Controls.TaggedDomainMatcherControl.cs

Open a new file in Notepad or Visual Studio and save it as 'Stardeveloper.Controls.TaggedDomainMatcherControl.cs' in the /App_Code folder of your ASP.NET website. Now copy and paste following code in it and then hit the 'Save' button:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using System.Web.UI;
using Stardeveloper.Modules;

namespace Stardeveloper.Controls
{
  public class TaggedDomainMatcherControl : Control
  {
    public virtual string DomainToMatch { get; set; }

    protected override void Render(HtmlTextWriter writer)
    {
      if (String.IsNullOrEmpty(DomainToMatch))
        throw new ArgumentNullException("Please set the DomainToMatch attribute to a valid domain name. Empty domain names are not accepted.");

      HttpRequest req = Page.Request;
      HttpCookie ck = req.Cookies[ReferringDomainTaggerModule.CookieName_TaggedDomains];
      Uri referrer = req.UrlReferrer;

      if ((ck != null && ck.Value.IndexOf(DomainToMatch, StringComparison.OrdinalIgnoreCase) != -1) ||
        (referrer != null && referrer.Host.IndexOf(DomainToMatch, StringComparison.OrdinalIgnoreCase) != -1))
        base.Render(writer);

      req = null;
    }
  }
}

If you have saved the file, you are now ready to move to the next step to configure the web.config file. There are now no more files to be created.

If you want to understand the inner working of this control, then please continue reading this article as explanation follows next.

Explanation

We import the necessary namespaces. Stardeveloper.Modules namespace is required to properly reference ReferringDomainTaggerModule HTTP module.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using System.Web.UI;
using Stardeveloper.Modules;

Next, we will place this custom web server control inside Stardeveloper.Controls namespace.

namespace Stardeveloper.Controls
{
	...
}

To work as a web server control, our class has to extend Control class.

public class TaggedDomainMatcherControl : Control
{
	...
}

Any public virtual properties declared in a custom web server control automatically become attributes. As we saw in the code example above, you can set the DomainToMatch property by setting the attribute of this control in your ASP.NET page to "facebook.com". We'll see another working example of this piece of code later.

public virtual string DomainToMatch { get; set; }

Next, we override the Render() method. In this method we'll do the checking for the cookie and referring link.

protected override void Render(HtmlTextWriter writer)
{
	...
}

The first thing we do within this method is to check for DomainToMatch property. If you have not explicitly set this attribute in your ASP.NET page, an exception will be thrown to alert you to the fact that you have forgotten to specify the domain name to match.

if (String.IsNullOrEmpty(DomainToMatch))
  throw new ArgumentNullException("Please set the DomainToMatch attribute to a valid domain name. Empty domain names are not accepted.");

We save references to current HttpRequest object, our "SD_TaggedDomains" cookie, and referring Uri for this page in local variables.

HttpRequest req = Page.Request;
HttpCookie ck = req.Cookies[ReferringDomainTaggerModule.CookieName_TaggedDomains];
Uri referrer = req.UrlReferrer;

We check for the presence of DomainToMatch domain name in the Value of the cookie. Next, we check for the referring Uri for the presence of DomainToMatch domain name in its host name. If either of these two conditions is met, we render the enclosed content to the user.

if ((ck != null && ck.Value.IndexOf(DomainToMatch, StringComparison.OrdinalIgnoreCase) != -1) ||
  (referrer != null && referrer.Host.IndexOf(DomainToMatch, StringComparison.OrdinalIgnoreCase) != -1))
  base.Render(writer);

This is it as far as this control is concerned. Now, please move to the next page to configure and to run this application.

Displaying statistics logged by our HTTP module

Faisal Khan — Mon, 06 Dec 2010 08:22:00 GMT

The ReferringDomainTaggerModule is an HTTP module to tag users visiting your website who follow a link from another website by placing a small cookie in the user's browser to identify them later. This module works silently to tag only the users which are being referred from the domain names in the list of domain names to be tagged. You can add or remove domain names to this list very easily. This module is a component of Referring Domain Tagger Application. The purpose of this application is to tag users coming from websites like Facebook, Twitter and Dzone and then to show these visitors site specific content.

To visitors visiting from social networks or bookmarking sites, you can be more liberal in displaying social bookmarklets and 'Like/Share/Recommend/Retweet/Vote' buttons as compared to visitors who are accessing your website directly or through a search engine. This application is designed to make this all very easy for you.

So far, we have learned about tagging users using cookies and how it works, have got a good overview of the application, and have developed the first and most important component of this application: ReferringDomainTaggerModule - which is an HTTP module. Now we are going to make an ASP.NET page to display some statistics to the administrator of this application, that is you for your website. These stats are collected by the ReferringDomainTaggerModule.

As administrator of your Referring Domain Tagger Application, you'll want to see if the tagging is being done correctly or not and secondly, if you can see some stats to figure out which domain's referred users are being tagged more often and which domain's, less. We will develop an ASP.NET page to pull statistics from ReferringDomainTaggerModule and display them in a tabular fashion to do just that.

One thing that I have talked previously and I mention it here again is that these stats are only kept from the time this ASP.NET application was started in IIS. As soon as the application restarts (for example, when you edit web.config file) or when IIS is restarted, the stats are reset. I did not deliberately add the complexity of storing and retrieving these stats from the database. You are free to modify the code to have this module store its stats in the SQL Server database and make this ASP.NET page show those stats from there.

Sample stats

Following image shows sample stats as they will be displayed by this ASP.NET page:

Sample stats from ReferringDomainTaggerModuleStats.aspx
(click the image to view in full size)

For every domain name in the list of referring domains to tag and track, a small table is created and underneath that each row displays stats for that day. 'Attempts to tag' displays the number of times a cookie was sent back to the browser. 'Preinstalled tags' shows that a cookie was found already installed on the browser.

ReferringDomainTaggerModuleStats.aspx

Open Notepad or Visual Studio and create an empty file with the name of 'ReferringDomainTaggerModuleStats.aspx' in any folder of your website. Then copy and paste following code in it:

<%@ Page Language="C#" EnableSessionState="ReadOnly" %>

<%@ Import Namespace="System.Collections.Generic" %>
<%@ Import Namespace="Stardeveloper.Modules" %>







    Stats for ReferringDomainTaggerModule (©Stardeveloper.com)
    


    
    Learn more about Referring Domain Tagger Application.

    Author: Faisal Khan (Stardeveloper.com)

Once you have pasted the code and saved the page, you can either move to the next page to create the next component for this application or you can look at the explanation of the code given below to understand how this ASP.NET page works.

Explanation

The first thing we do is to tell the ASP.NET compiler that we will be using C# to write the code on this page.

Next, we tell ASP.NET runtime that no new session should be created for this page.

<%@ Page Language="C#" EnableSessionState="ReadOnly" %>

We import couple of namespaces to make use of the generic collections and to access ReferringDomainTaggerModule HTTP module that we created earlier.

<%@ Import Namespace="System.Collections.Generic" %>
<%@ Import Namespace="Stardeveloper.Modules" %>

Our code will be placed in the Page_Load() method.

The first thing we do within this method is to check if this is a post-back. A post-back occurs when an ASP.NET page is posted back to itself as a result of some action taken by the user. In post-back, the state of the controls is mostly maintained so that is why we don't touch our controls and exit out of this method if it is a post-back.

if (IsPostBack)
	return;

Next, we save the reference to ReferringDomainTaggerModule.ReferringDomainsToTag in a local variable. ReferringDomainTaggerModule.ReferringDomainsToTag is a list of all referring domains we are tagging. It also provides access to domain stats as we'll see in a moment.

List referringDomains = ReferringDomainTaggerModule.ReferringDomainsToTag;

Since accessing a shared class-level object is a thread-unsafe procedure, we lock the object to prevent inadvertent read/write operation by two or more threads at the same time.

lock (referringDomains)
{
	...
}

We then iterate through the list of referring domains. During each iteration, we create a new HTML table for that domain. Next, we iterate through the per-day records maintained of that domain and write an HTML row for each record found. Once all the tables have been created, we exit out of this method.

for (int i = 0; i < referringDomains.Count; i++)
{
	ReferringDomain d = referringDomains[i];

	HtmlTable table = new HtmlTable();
	table.Attributes["class"] = "domainStatsTable";
	form1.Controls.Add(table);

	// Creating the header row
	HtmlTableRow headerRow = new HtmlTableRow();
	table.Rows.Add(headerRow);

	HtmlTableCell headerCell = new HtmlTableCell();
	headerRow.Cells.Add(headerCell);

	headerCell.Attributes["class"] = "header";
	headerCell.ColSpan = 3;
	headerCell.Controls.Add(new LiteralControl(d.Domain));

	// Creating column headers
	HtmlTableRow colHeaderRow = new HtmlTableRow();
	table.Rows.Add(colHeaderRow);

	HtmlTableCell col1Cell = new HtmlTableCell();
	colHeaderRow.Cells.Add(col1Cell);

	HtmlTableCell col2Cell = new HtmlTableCell();
	colHeaderRow.Cells.Add(col2Cell);

	HtmlTableCell col3Cell = new HtmlTableCell();
	colHeaderRow.Cells.Add(col3Cell);

	col1Cell.Attributes["class"] = "colHeader";
	col1Cell.Controls.Add(new LiteralControl("Date"));

	col2Cell.Attributes["class"] = "colHeader";
	col2Cell.Controls.Add(new LiteralControl("Attempts to tag"));

	col3Cell.Attributes["class"] = "colHeader";
	col3Cell.Controls.Add(new LiteralControl("Preinstalled tags"));

	IList keysInHitsList = d.HitsPerDayList.Keys;

	for (int j = 0; j < keysInHitsList.Count; j++)
	{
		string key = keysInHitsList[j];
		ReferringDomainStats hd = d.HitsPerDayList[key];

		// Creating a row for each date
		HtmlTableRow row = new HtmlTableRow();
		table.Rows.Add(row);

		HtmlTableCell dateCell = new HtmlTableCell();
		row.Cells.Add(dateCell);

		HtmlTableCell attemptsCell = new HtmlTableCell();
		row.Cells.Add(attemptsCell);

		HtmlTableCell preinstalledCell = new HtmlTableCell();
		row.Cells.Add(preinstalledCell);

		dateCell.Controls.Add(new LiteralControl(key));
		attemptsCell.Controls.Add(new LiteralControl(Convert.ToString(hd.AttemptsToTag)));
		preinstalledCell.Controls.Add(new LiteralControl(Convert.ToString(hd.PreinstalledTags)));
	}
}

The HTML table controls created above, are inserted in the form1 control.

This in nutshell is pretty much what this ASP.NET page does.

Now that you have created this ASP.NET page, move over to the next page to create the custom web server control to display site-specific content to the tagged users of your website.

An HTTP Module to tag users from given referring domains

Faisal Khan — Mon, 06 Dec 2010 08:17:00 GMT

This HTTP module will tag users, that is, place a cookie in the user's browser, if they visit our website by following a link from another website whose name we have added in the list of domain names to tag in our web.config file. Just like the list of domain names (or referring websites) can change, so can change the value of the cookie this module will place in the browser. The code that this module will have will be sufficient to handle all the changes, that is, to add and remove values in the cookie, or to remove the cookie all together.

If you are not up to speed with what ASP.NET application we are developing and how this application will tag and identify users, please read the introduction to tagging users using cookies and get an overview of this application to understand it in detail. We'll only focus on developing the HTTP module and code its logic from here onwards.

It is a self-sufficient module and doesn't need extra files or classes for its functionality. All it looks for are two key/value pairs in the appSettings section of web.config file. One pair to load the comma-separated list of domain names to tag and the second pair to add your UTC offset to the date and time to correctly log the stats for this application as per the date and time where you live. The moment you upload this module's file to the /App_Code folder of your ASP.NET website, and configure the two application level variables, this module will begin to work instantly and your users will begin to get tagged if they visit your website from the referring domains you have added in the web.config file. It is a silent module and works efficiently to handle every request with minimum of server resources.

Let us look at the code now.

Stardeveloper.Modules.ReferringDomainTaggerModule.cs

Please create a new file in Notepad or Visual Studio and save it as 'Stardeveloper.Modules.ReferringDomainTaggerModule.cs' in your /App_Code folder of your ASP.NET website. Now copy the following code and paste it into this file and hit the 'Save' button:

// Author: Faisal Khan (http://www.stardeveloper.com)

using System;
using System.Collections.Generic;
using System.Configuration;
using System.Data;
using System.Data.SqlClient;
using System.Linq;
using System.Text;
using System.Threading;
using System.Web;

namespace Stardeveloper.Modules
{
  public class ReferringDomainTaggerModule : IHttpModule
  {
    public const string CookieName_TaggedDomains = "SD_TaggedDomains";

    public static List ReferringDomainsToTag = new List();
    public static int UtcOffset = 0;
    protected static bool Loaded = false;

    public ReferringDomainTaggerModule()
    {
      LoadReferringDomainsToTag();
    }

    public void Init(HttpApplication app)
    {
      app.BeginRequest += new EventHandler(app_BeginRequest);
    }

    protected void app_BeginRequest(object sender, EventArgs e)
    {
      HttpApplication app = sender as HttpApplication;
      HttpRequest req = app.Request;
      HttpResponse res = app.Response;
      Uri urlReferrer = req.UrlReferrer;

      if (Loaded)
      {
        // If there is no domain to tag, we will remove the cookie if found in the user's browser
        if (ReferringDomainsToTag.Count == 0)
        {
          HttpCookie refCk = req.Cookies[CookieName_TaggedDomains];

          if (refCk != null)
          {
            res.Cookies[CookieName_TaggedDomains].Value = String.Empty;
            res.Cookies[CookieName_TaggedDomains].Expires = DateTime.Now.AddYears(-1);

            return;
          }
        }
        else
        {
          // Check to see if user is coming to this website by clicking a link from another website
          if (urlReferrer != null)
          {
            ReferringDomain matchedReferringDomain = null;

            // Check to see if the referring domain is one of the domains in our list to tag
            for (int i = 0; i < ReferringDomainsToTag.Count; i++)
            {
              ReferringDomain rd = ReferringDomainsToTag[i];

              if (urlReferrer.Host.IndexOf(rd.Domain, StringComparison.OrdinalIgnoreCase) != -1)
              {
                matchedReferringDomain = rd;
                break;
              }
            }

            // If we had a match
            if (matchedReferringDomain != null)
            {
              HttpCookie refCk = req.Cookies[CookieName_TaggedDomains];

              if (refCk == null)
              {
                // Create and place a new cookie
                refCk = new HttpCookie(CookieName_TaggedDomains, matchedReferringDomain.Domain);
                refCk.Expires = DateTime.Now.AddYears(10);

                res.Cookies.Add(refCk);

                matchedReferringDomain.IncrementAttemptsToTag();
              }
              else
              {
                string refCkVal = refCk.Value;

                if (refCkVal.IndexOf(matchedReferringDomain.Domain, StringComparison.OrdinalIgnoreCase) != -1)
                {
                  matchedReferringDomain.IncrementPreinstalledTags();
                }
                else
                {
                  refCkVal += "," + matchedReferringDomain.Domain;

                  refCkVal = StripOldDomains(refCkVal);
                  matchedReferringDomain.IncrementAttemptsToTag();
                }

                res.Cookies[CookieName_TaggedDomains].Value = refCkVal;
                res.Cookies[CookieName_TaggedDomains].Expires = DateTime.Now.AddYears(10);
              }
            }
          }
        }
      }
    }

    public void Dispose()
    {
    }

    protected void LoadReferringDomainsToTag()
    {
      if (Loaded == false)
      {
        lock (ReferringDomainsToTag)
        {
          // Retrieving the list of domains to tag
          string retrievedList = ConfigurationManager.AppSettings["SD_ReferringDomainTaggerModule_ReferringDomainsToTag"];

          if (!String.IsNullOrEmpty(retrievedList))
          {
            ReferringDomainsToTag.Clear();

            string[] retrievedListSplit = retrievedList.Split(',');

            for (int i = 0; i < retrievedListSplit.Length; i++)
            {
              string referringDomain = retrievedListSplit[i].Trim();

              if (referringDomain != String.Empty)
                ReferringDomainsToTag.Add(new ReferringDomain(referringDomain));
            }

            // Retrieving UTC offset
            string utcOffsetStr = ConfigurationManager.AppSettings["SD_ReferringDomainTaggerModule_UtcOffset"];

            if (!String.IsNullOrEmpty(utcOffsetStr))
              UtcOffset = Convert.ToInt32(utcOffsetStr);
          }

          Loaded = true;
        }
      }
    }

    protected string StripOldDomains(string refCkVal)
    {
      if (!String.IsNullOrEmpty(refCkVal))
      {
        StringBuilder newRefCkValBuilder = new StringBuilder();
        string[] refCkValSplit = refCkVal.Split(',');

        for (int i = 0; i < refCkValSplit.Length; i++)
        {
          string refCkDomain = refCkValSplit[i];

          for (int j = 0; j < ReferringDomainsToTag.Count; j++)
          {
            if (refCkDomain.Equals(ReferringDomainsToTag[j].Domain, StringComparison.OrdinalIgnoreCase))
            {
              if (newRefCkValBuilder.Length > 0)
                newRefCkValBuilder.Append(",");

              newRefCkValBuilder.Append(refCkDomain);
              break;
            }
          }
        }

        return newRefCkValBuilder.ToString();
      }

      return refCkVal;
    }
  }

  public class ReferringDomain
  {
    public string Domain { get; protected set; }
    public SortedList HitsPerDayList = new SortedList();

    public ReferringDomain(string domain)
    {
      Domain = domain;
    }

    public string GetLocalCurrDateStr()
    {
      DateTime currLocalDateTime = DateTime.UtcNow.AddHours(ReferringDomainTaggerModule.UtcOffset);
      StringBuilder currDateStrBuilder = new StringBuilder(8);

      currDateStrBuilder.Append(currLocalDateTime.Month).Append("/");
      currDateStrBuilder.Append(currLocalDateTime.Day).Append("/");
      currDateStrBuilder.Append(currLocalDateTime.Year);

      return currDateStrBuilder.ToString();
    }

    public ReferringDomainStats GetHitsDetails()
    {
      string currLocalDateStr = GetLocalCurrDateStr();
      ReferringDomainStats hitsDetails = null;
      
      HitsPerDayList.TryGetValue(currLocalDateStr, out hitsDetails);

      if (hitsDetails == null)
      {
        // New day, new hits
        hitsDetails = new ReferringDomainStats();
        HitsPerDayList.Add(currLocalDateStr, hitsDetails);
      }

      return hitsDetails;
    }

    public void IncrementAttemptsToTag()
    {
      lock (HitsPerDayList)
      {
        GetHitsDetails().AttemptsToTag++;
      }
    }

    public void IncrementPreinstalledTags()
    {
      lock (HitsPerDayList)
      {
        GetHitsDetails().PreinstalledTags++;
      }
    }
  }

  public class ReferringDomainStats
  {
    public int AttemptsToTag { get; set; }
    public int PreinstalledTags { get; set; }
  }
}

If by looking at the code you were able to figure out what it is doing or if you want to save time, then quickly move over to develop the next component for this application on the next page. Otherwise, a detailed explanation of the code follows.

Explanation

The very first line is a comment which gives credit to the developer of this application, Faisal Khan, with a link to his website, http://www.stardeveloper.com.

// Author: Faisal Khan (http://www.stardeveloper.com)

Next, we are greeted with a long list of namespaces whose classes we are going to make use of in this HTTP module. Most of these namespaces are added automatically the moment we create a new C# file in Visual Studio.

System.Collections.Generic is listed because we are going to make use of List<> collection.

System.Configuration is required because we want to access application level variables from web.config file and for that we need access to ConfigurationManager class that is present in this namespace.

System.Text is for StringBuilder class and System.Web is for all the classes like HttpRequest and HttpResponse that we are going to use to read and write cookies to the user's browser.

System.Data and System.Data.SqlClient are actually not needed since we are not going to access database in this application.

using System;
using System.Collections.Generic;
using System.Configuration;
using System.Data;
using System.Data.SqlClient;
using System.Linq;
using System.Text;
using System.Threading;
using System.Web;

Next, we declare a namespace, Stardeveloper.Modules, so that when we create ReferringDomainTaggerModule class, it does not conflict with another class of possibly the same name in your ASP.NET website. And also to put all the modules in one namespace, kind of grouping all the modules into their own namespace.

namespace Stardeveloper.Modules
{
}

Next, we are create our ReferringDomainTaggerModule class and have it implement IHttpModule interface.

For a class to work as an HTTP module in ASP.NET, it has to implement IHttpModule interface.

There are only two methods to implement in IHttpModule interface: Init() and Dispose(). By implementing this interface, our ReferringDomainTaggerModule class will be able to be deployed and run as an HTTP module on our ASP.NET website.

namespace Stardeveloper.Modules
{
  public class ReferringDomainTaggerModule : IHttpModule
  {
    public void Init(HttpApplication app)
    {
    }

    public void Dispose()
    {
    }
  }
}

Within ReferringDomainTaggerModule, we declare static variables to provide global access to certain constant and variable elements of this class.

CookieName_TaggedDomains is a constant to hold the name of the cookie we are going to read and write to the user's browser to save list of domain names he has been tagged with. By default, its name is "SD_TaggedDomains".

public const string CookieName_TaggedDomains = "SD_TaggedDomains";

public static List ReferringDomainsToTag = new List();
public static int UtcOffset = 0;
protected static bool Loaded = false;

Next, we create the constructor for this class. In the constructor we are going to call a method which will load certain values from web.config into the class-level objects we have created above.

public ReferringDomainTaggerModule()
{
  LoadReferringDomainsToTag();
}

When the HTTP module is created and is ready to serve requests, and one instance is created for each HTTP request that IIS receives, its Init() method is called to give an opportunity to the module to register for events in the HTTP request handling pipline.

This article provides good introduction to HTTP modules and HTTP request handling pipeline in ASP.NET.

In the Init() method, we register to handle the BeginRequest event. This event is raised at the beginning of the HTTP request handling pipeline. This is the first event to be raised when an HTTP request is received by IIS. We will put the logic to scan the referring link and to read and write the cookie in this event handler, as we'll see in a moment.

public void Init(HttpApplication app)
{
  app.BeginRequest += new EventHandler(app_BeginRequest);
}

Next, comes our event handler method which will be called when the BeginRequest event is raised at the start of the HTTP request handling pipeline.

protected void app_BeginRequest(object sender, EventArgs e)
{
}

The first thing we do within this method is to store references to important objects, which we'll be using later, in local variables.

The caller (or sender as it is called in .NET) of this event (BeginRequest) is HttpApplication object. We obtain a reference to this object by casting the sender object to HttpApplication using the as keyword and saving this reference in the local variable, app.

We use the reference to HttpApplication to obtain references to HttpRequest and HttpResponse objects, and save them in local variables. These objects encapsulate HTTP request and response, respectively. Using the HttpRequest object, we obtain the reference to the referring Uri (or URL as it is more commonly called) object.

The HttpRequest.UrlReferrer will return a null reference if this HTTP request did not have any referring link, that is, the user was not referred to this web page by following a link from another.

HttpApplication app = sender as HttpApplication;
HttpRequest req = app.Request;
HttpResponse res = app.Response;
Uri urlReferrer = req.UrlReferrer;

Next, we check the Loaded variable to see if the list of domains to tag has been successfully loaded into an in-memory list. If so, then we check to see if this list is empty, that is, there are no domains to tag, then we remove the "SD_TaggedDomains" cookie if there is one in the user's browser.

We remove the cookie by setting its Value to an empty string and then setting its Expires property to a date in the past. When the browser will see that there is a cookie in the HTTP response whose expiry date has already passed, that is, the cookie has expired, the browser will remove the cookie from its store.

if (Loaded)
{
  // If there is no domain to tag, we will remove the cookie if found in the user's browser
  if (ReferringDomainsToTag.Count == 0)
  {
    HttpCookie refCk = req.Cookies[CookieName_TaggedDomains];

    if (refCk != null)
    {
      res.Cookies[CookieName_TaggedDomains].Value = String.Empty;
      res.Cookies[CookieName_TaggedDomains].Expires = DateTime.Now.AddYears(-1);

      return;
    }
  }
  else
  {
	...
  }
}

Next, we check the referred Uri object to see if it is null or not by using the urlReferrer reference that we had saved earlier. If the user requested your page directly then this object reference will be null. If it is not null, that is, the user is coming from a link on another web page, then we proceed further.

We then iterate through the list of domain names that we have saved in the List object referenced by ReferringDomainTaggerModule.ReferringDomainsToTag variable. Each ReferringDomain object refers to one domain name that we have included in the web.config file to tag. As we'll see just a while later, ReferringDomain class provides a property with the name of Domain that contains the domain name to tag. During the iteration through the list of domains to tag, we compare the domain name with the referring link's domain name. If there is match, that is, the user is coming from a domain name we'll like to tag, we proceed further.

// Check to see if user is coming to this website by clicking a link from another website
if (urlReferrer != null)
{
  ReferringDomain matchedReferringDomain = null;

  // Check to see if the referring domain is one of the domains in our list to tag
  for (int i = 0; i < ReferringDomainsToTag.Count; i++)
  {
    ReferringDomain rd = ReferringDomainsToTag[i];

    if (urlReferrer.Host.IndexOf(rd.Domain, StringComparison.OrdinalIgnoreCase) != -1)
    {
      matchedReferringDomain = rd;
      break;
    }
  }

  // If we had a match
  if (matchedReferringDomain != null)
  {
    ...
  }
}

Alright, we had a match. We have a user who is coming to our website from a website whose domain name we had put in the web.config file to tag. Now we have to tag this user. We'll do that my placing a cookie on this user's browser. We check the cookies (if any) sent by this browser to our website to see if there is already a cookie with the name of "SD_TaggedDomains" in that list. If there is not, then we create a new cookie with the name of "SD_TaggedDomains". We set it's Value to the domain this user is to be tagged to, that is, the domain he was referred from, to our website. We set this cookie's expiry date far in the future by setting its Expires property to a date 10 years in the future. When this cookie is ready to be sent to the user's browser, we add it to the outgoing HTTP response stream.

To display some stats to the admin, that is, you, we call the IncrementAttemptsToTag() method of the matched ReferringDomain object. This method will internally increment the number of attempts that were made to send the cookie to the user's browser once there was a confirmed match. This internal log will be read by the ReferringDomainTaggerModuleStats.aspx page to display these stats in the form of a table. Later on that.

HttpCookie refCk = req.Cookies[CookieName_TaggedDomains];

if (refCk == null)
{
  // Create and place a new cookie
  refCk = new HttpCookie(CookieName_TaggedDomains, matchedReferringDomain.Domain);
  refCk.Expires = DateTime.Now.AddYears(10);

  res.Cookies.Add(refCk);

  matchedReferringDomain.IncrementAttemptsToTag();
}
else
{
  ...
}

If the user already had the cookie, "SD_TaggedDomains", then we check to see if this user is already tagged for this referring domain, that is, the user previously visited our website from this domain name and was tagged (a cookie was placed). If the user is already tagged, we log it by calling the IncrementPreinstalledTags() method of the ReferringDomain that was matched.

If the user had the cookie but was not tagged for this referring domain, we append the value of this domain name to the cookie's Value. We call StripOldDomains() method of this HTTP module to remove any stale domain names that are still residing in this cookie. You'll see the code for this method in a moment.

Finally, we update the cookie and set its Expires property to a date well advance in the future. And we are done processing.

string refCkVal = refCk.Value;

if (refCkVal.IndexOf(matchedReferringDomain.Domain, StringComparison.OrdinalIgnoreCase) != -1)
{
  matchedReferringDomain.IncrementPreinstalledTags();
}
else
{
  refCkVal += "," + matchedReferringDomain.Domain;

  refCkVal = StripOldDomains(refCkVal);
  matchedReferringDomain.IncrementAttemptsToTag();
}

  res.Cookies[CookieName_TaggedDomains].Value = refCkVal;
  res.Cookies[CookieName_TaggedDomains].Expires = DateTime.Now.AddYears(10);

There is nothing that we would want to do when this instance of ReferringDomainTaggerModule HTTP module is about to be disposed. So, we leave the body of this method, empty.

public void Dispose()
{
}

Now, we'll code the body of the LoadReferringDomainsToTag() method. This method is called from within the constructor of ReferringDomainTaggerModule class. The purpose of this method is to:

Retrieve the list of domain names to tag from web.config file and to add these domain names in the ReferringDomainTaggerModule.ReferringDomainsToTag list.
Retrieve UTC offset from web.config file and save it in the ReferringDomainTaggerModule.UtcOffset variable.

The first thing we do when we enter in this method is to check the Loaded variable to see if we have done the loading of domain names and UtcOffset already or not. If not, then we proceed further.

We lock the ReferringDomainsToTag class to prevent two threads simultaneously updating the shared variables.

The comma-separated list of domain names (for example, "facebook.com,twitter.com,dzone.com") is retrieved from the web.config file. It is split and each domain name is placed in a ReferringDomain object. Each ReferringDomain object is then added to the ReferringDomainTaggerModule.ReferringDomainsToTag list.

The UtcOffset is retrieved and converted into an integer. It is then saved in the ReferringDomainTaggerModule.UtcOffset variable.

Lastly, we set the Loaded variable to true to prevent any repeat retrieval and addition of domain names and UtcOffset from the web.config file until the application is restarted, and release the lock to ReferringDomainTaggerModule class.

protected void LoadReferringDomainsToTag()
{
  if (Loaded == false)
  {
    lock (ReferringDomainsToTag)
    {
      // Retrieving the list of domains to tag
      string retrievedList = ConfigurationManager.AppSettings["SD_ReferringDomainTaggerModule_ReferringDomainsToTag"];

      if (!String.IsNullOrEmpty(retrievedList))
      {
        ReferringDomainsToTag.Clear();

        string[] retrievedListSplit = retrievedList.Split(',');

        for (int i = 0; i < retrievedListSplit.Length; i++)
        {
          string referringDomain = retrievedListSplit[i].Trim();

          if (referringDomain != String.Empty)
            ReferringDomainsToTag.Add(new ReferringDomain(referringDomain));
        }

        // Retrieving UTC offset
        string utcOffsetStr = ConfigurationManager.AppSettings["SD_ReferringDomainTaggerModule_UtcOffset"];

        if (!String.IsNullOrEmpty(utcOffsetStr))
          UtcOffset = Convert.ToInt32(utcOffsetStr);
      }

      Loaded = true;
    }
  }
}

StripOldDomains() is a handy method to remove any domain names that have been removed from the list in web.config file but are still present in the cookie.

We split the list of domain names in the cookie's Value and match each of them with list of domain names to match to, present in our web.config file. If there is a match we append that domain name in our local StringBuilder object. This StringBuilder object represents the new list of domain names that this cookie is allowed to have.

If there is no match, then this means that the domain name in the cookie is old, then we don't append it in the StringBuilder object. Once the iteration through the list is complete, we return the new list of domain names for this cookie.

protected string StripOldDomains(string refCkVal)
{
  if (!String.IsNullOrEmpty(refCkVal))
  {
    StringBuilder newRefCkValBuilder = new StringBuilder();
    string[] refCkValSplit = refCkVal.Split(',');

    for (int i = 0; i < refCkValSplit.Length; i++)
    {
      string refCkDomain = refCkValSplit[i];

      for (int j = 0; j < ReferringDomainsToTag.Count; j++)
      {
        if (refCkDomain.Equals(ReferringDomainsToTag[j].Domain, StringComparison.OrdinalIgnoreCase))
        {
          if (newRefCkValBuilder.Length > 0)
            newRefCkValBuilder.Append(",");

          newRefCkValBuilder.Append(refCkDomain);
          break;
        }
      }
    }

    return newRefCkValBuilder.ToString();
  }

  return refCkVal;
}

Each domain name that you place in the list of domain names to tag in the web.config file, is at the time of loading placed in a separate instance of ReferringDomain class. This class not only represents a referring domain, but it contains an internal sorted list of dates (per day) and hits (attempts made at setting a cookie in the browser and if the cookie was found preinstalled in the user's browser).

public class ReferringDomain
{
  public string Domain { get; protected set; }
  public SortedList HitsPerDayList = new SortedList();

  public ReferringDomain(string domain)
  {
    Domain = domain;
  }

  public string GetLocalCurrDateStr()
  {
    DateTime currLocalDateTime = DateTime.UtcNow.AddHours(ReferringDomainTaggerModule.UtcOffset);
    StringBuilder currDateStrBuilder = new StringBuilder(8);

    currDateStrBuilder.Append(currLocalDateTime.Month).Append("/");
    currDateStrBuilder.Append(currLocalDateTime.Day).Append("/");
    currDateStrBuilder.Append(currLocalDateTime.Year);

    return currDateStrBuilder.ToString();
  }

  public ReferringDomainStats GetHitsDetails()
  {
    string currLocalDateStr = GetLocalCurrDateStr();
    ReferringDomainStats hitsDetails = null;
      
    HitsPerDayList.TryGetValue(currLocalDateStr, out hitsDetails);

    if (hitsDetails == null)
    {
      // New day, new hits
      hitsDetails = new ReferringDomainStats();
      HitsPerDayList.Add(currLocalDateStr, hitsDetails);
    }

    return hitsDetails;
  }

  public void IncrementAttemptsToTag()
  {
    lock (HitsPerDayList)
    {
      GetHitsDetails().AttemptsToTag++;
    }
  }

  public void IncrementPreinstalledTags()
  {
    lock (HitsPerDayList)
    {
      GetHitsDetails().PreinstalledTags++;
    }
  }
}

The hits are saved by ReferringDomain in an internal list of ReferringDomainStats objects.

public class ReferringDomainStats
{
  public int AttemptsToTag { get; set; }
  public int PreinstalledTags { get; set; }
}

Where to place this file?

As explained before, you have to save Stardeveloper.Modules.ReferringDomainTaggerModule.cs file in the /App_Code folder of your ASP.NET website.

The job is not complete and this module will not start to tag users until you add it in your web.config file and add two application level variables as well. We'll look that in next few pages.

Next, we'll move on to the next component of this Referring Domain Tagger Application which happens to be the ReferringDomainTaggerModuleStats.aspx ASP.NET page.

ASP.NET application to tag and identify users from Facebook, Twitter, Dzone and more

Faisal Khan — Mon, 06 Dec 2010 08:13:00 GMT

Users visit a website either directly by typing its address in the URL bar of the browser or when they visit a link to that website from another website or search engine. It is possible to tag a user visiting your website following a link on another popular website like Facebook or Twitter, by placing a small cookie on his browser. That cookie can be used to identify that user when he browses your website or visits it later. You can use the tag on this user's browser to display site specific content like displaying the 'Like' button to Facebook users and 'Retweet' button to Twitter users. You can display Dzone's voting button to its users, not just for the link they visited to but for other articles and content on your website which they'll vote up if they like. Dzone users can even submit a link on your website to their network if they find it interesting and if it is not already there. Dzone's voting button will make it easier for them to do that. This application can prove quite helpful if you want to increase the appeal and traffic of your website across social networks and popular websites.

If you have not followed us from the start, we are developing an ASP.NET web application to tag users from certain websites and to display them relevant content by using custom web server controls. This is a slick and neatly developed application and the time you spend learning how it works, how to setup, and how to run the application on your website can reward you big time in the long run. You may want to read the introduction to learn why there is a need for this application, how it works, how to add and remove referring domains to be tagged, how to remove the tags if you'd want, and what is needed to run this application.

Live demonstration of a working example of this application

While you will come to see a live demonstration of this application working on Stardeveloper.com, later, I am going to show a couple of images from an article on this website. In the following images you can see that a Dzone voting button gets shown only to a user who visited this website from Dzone.com:

No Dzone voting button for other users of Stardeveloper.com
(click the image to view in full size)

Dzone voting button being shown to a visitor from Dzone.com
(click the image to view in full size)

We'll now take a look at a short introduction to understand the design and structure of this application which we'll call 'Referring Domain Tagger Application' from here onwards.

Introduction to Referring Domain Tagger Application

It is a small plug and play ASP.NET web application. Apart from minor updates to your web.config file, there are no changes required to be made to your existing ASP.NET website. So you should be able to plug this application in, without much hassle. Following is the list of components this application is made of:

ReferringDomainTaggerModule
This is a small HTTP module whose function is to scan the referring links and to match their domain names with the list of domain names we have provided in the web.config file. If their is a match then this module will place a cookie, "SD_TaggedDomains", as a tag in the user's browser. This cookie will be used later by the custom web server control to identify the tag and display specific content for that, like displaying the 'Like' button for Facebook users as we have talked previously.
ReferringDomainTaggerModuleStats.aspx
It is an ASP.NET page that will display stats for the cookies placed (or attempts made to place the cookie since we don't know if the browser is accepting cookies or not). The HTTP module above will log stats for each domain we are attempting to tag, in its memory. This ASP.NET page will scan that log and display the stats to you, the admin. Remember: the stats are refreshed each time the ASP.NET application is restarted because the log is maintained in memory and not in the database.

Following image shows sample stats as they will be displayed by this ASP.NET page:

Sample stats from ReferringDomainTaggerModuleStats.aspx
(click the image to view in full size)
TaggedDomainMatcherControl
It is a custom web server control that we will develop later. The only custom attribute this control will have is DomainToMatch which takes, as its name implies, the domain name to match. If the user is already tagged with that domain name by the HTTP module, that is, was referred from that domain, the user will be displayed the content enclosed by this control.
```
	Hi Dzone user!
```
Above code shows what it will take to make use of TaggedDomainMatcherControl and how to specify a domain name in the DomainToMatch attribute. The message, "Hi Dzone user!", will only be shown to users who were referred from Dzone.com or who have in their cookie, "SD_TaggedDomains", the domain name "dzone.com".
Web.config
The last component, or shall I say step, is making a few changes in web.config file. There are just two key/value pairs to be added in the application settings. After that, we'll configure the above HTTP module and we are all done.

How this application works?

The absolutely key component of this application is the ReferringDomainTaggerModule which is an HTTP module that does all the work of reading and writing cookies in the user's browser, that is, tagging him with the domain that referred him to our website. If you only install this HTTP module, the users will begin to get tagged. That's it. No other component or file is needed to tag or untag the users.

Practically speaking though, you'll want to make some use of those tags, right? This is where the second most important component of this application comes in, and this is TaggedDomainMatcherControl. It is a custom web server control that makes it easy to add site or domain specific content in your ASP.NET pages with just a couple of lines of code, as we saw above. If you have this control in your website, using this application becomes a breeze. No other component is required to run or make use of this application.

The ASP.NET page to display stats is purely optional. It only displays what the HTTP module logs in its memory. You will like to have this ASP.NET page some where protected in your admin only folder. This ASP.NET page will display (not change any settings) the stats of each domain on a per day basis. Whenever you restart your ASP.NET application or update your web.config file, the changes will be lost and the stats will start to get recorded from zero. You'll be fine if you just want to delete this file all together, it is entirely up to you.

There is no other file to create, code to develop or to changes to make in your existing ASP.NET pages. An application level variable to tell the HTTP module which referring domains to tag and another one to configure UTC offset to correctly log dates per your UTC offset and not your web server's, is what you'll have to add to your web.config file. Then configure HTTP module and you are all done. It is that simple!

Start developing!

Now that we know what tagging is all about, what our application consists of and how it all works, let us open our editors and code the first component, that is, Stardeveloper.Modules.ReferringDomainTaggerModule.cs file for our HTTP module. Move to the next page please!