Stardeveloper

SPF Record Syntax

Faisal Khan — Fri, 26 Feb 2010 06:51:00 GMT

Sender Policy Framework prevents fraudulent abuse of your domain name in emails sent by others claiming to be sent from your domain name. This cuts down the spam which was originating from spammers who were using your domain name to bypass filters on other users' email addresses.

A domain name which does not have an SPF record set in its DNS records is open to spammers to use the address of this domain name in the 'From' field of their emails. They can then send thousands of these unsolicited emails to end users. And by using the repute of that domain name, can entice users to click on links which lead to their products, and to websites that fraudulently seek confidential and sensitive information (phishing attacks).

It is thus very important that you understand what is Sender Policy Framework (SPF) and how it works.

SPF works by making you explicitly declare a list of hosts and IP addresses from which legitimate emails can originate for your domain name. This is called creating a white list. You can also create a black list by explicitly declaring hosts and IP addresses which cannot send emails for your domain name. Generally, only a white list is created. Any other host or IP address not present in the white list is set to fail or soft fail, as the case may be.

What should be included in an SPF record?

An SPF record should comprehensively list all the hosts and IP addresses from which your website or domain name sends emails. This generally means an A record (IP address of your website), SMTP Servers, IP addresses from your company's network from which an email may originate, and any third party mailers, e.g., Google Apps, BlackBerry Internet Service, etc.

When you have created such a list, you should use SPF record syntax to publish this list in the DNS records of your domain name.

Introduction to SPF record syntax

SPF record syntax follows standard DNS syntax to declare a policy using a list of mechanisms. A mechanism is any rule you specify as part of your SPF record to allow or disallow a host or IP address range. You can specify zero or more mechanisms, each delimited by white space. All of these mechanisms constitute a policy for your domain name. An SPF record always begins with v=spf1 and ends with ~all, as shown below:

"v=spf1 ~all"

All DNS records including SPF are enclosed in double quotes. v=spf1 tells the SPF record validator that this is an SPF record and that it belongs to SPF version 1.

For practical purposes, always start from an SPF record like above, and then add mechanisms, relating to your hosts and IP addresses, one by one.

Quantifiers

Before we delve ourselves into mechanisms, let me touch a bit about quantifiers. If you remember the discussion above, I talked about white listing or black listing hosts and IP addresses. To allow (white list) a host or IP address, we append a '+' letter at its beginning e.g., +a. To disallow (black list) a host or IP address range, we append a '-' letter at its beginning e.g., -a. An a mechanism in these examples references the A record for this domain name. Between allowing and disallowing are two other quantifiers; '~' and '?'. An '~' quantifier (like the one at the beginning of all mechanism in our first example of an SPF record) means "soft fail". Soft fail means to accept the emails but put them in the 'Spam' folder and not the user's inbox. Soft fail is most often used in combination with all mechanism. An all mechanism is very powerful as it catches all the hosts and IP addresses that have not been explicitly declared in the preceding mechanisms. Finally, a '?' quantifier tells the SPF record validator to make this mechanism "neutral". A neutral mechanism is generally interpreted as allowance for that mechanism's host or IP address.

For practical purposes, I'd suggest that you limit yourself to '+' and '-' to allow and disallow any host or IP address range, and use '~' for the all mechanism.

Here is the recap of quantifiers you can use with the mechanisms in your SPF record:

'+' - pass
'?' - neutral (generally means pass)
'~' - soft fail (accept the email but put it in 'Spam' folder)
'-' - fail

It is not necessary to append '+' (pass) quantifier with mechanisms as it is assumed that a mechanism with no explicit quantifier has a '+' letter prepended. For example, following two SPF records mean one and the same thing:

"v=spf1 a ~all"

"v=spf1 +a ~all"

Mechanisms

A mechanism (rule) allows you to declare a single IPv4 or IPv6 IP address or an IP address range, a host, an SPF record from another host, an a record, an mx (Mail Exchanger) record, and a ptr record.

Following is the list of all mechanisms available:

ip4
ip6
a
mx
include
ptr
exists
all

We will look at all of them, one by one.

ip4 mechanism

An ip4 mechanism is used to include an IPv4 address or network range in an SPF record. A network range is specified by first declaring the IPv4 network address and then after a slash ('/'), is declared the length of the network range. All ip4 mechanisms start with "ip4:".

For example, following SPF record declares a single IPv4 address:

"v=spf1 ip4:127.0.0.1 ~all"

Following SPF record declares an ip4 mechanism to include all IP addresses in the 127.0.0.1-127.0.0.255 range:

"v=spf1 ip4:127.0.0.1/8 ~all"

Following SPF record declares an ip4 mechanism to include all IP addresses in the 127.0.0.1-127.0.255.255 range:

"v=spf1 ip4:127.0.0.1/16 ~all"

Since no quantifier was prepended with the ip4 mechanisms in the examples above; a '+' quantifier is assumed. This means that any email originating from these IP addresses should be allowed to pass successfully.

ip6 mechanism

An ip6 mechanism is used to include an IPv6 address or network range in an SPF record. All ip6 mechanisms start with "ip6:".

Following SPF record declares a single IPv6 address:

"v=spf1 ip6:2001:db8:85a3::8a2e:370:7334 ~all"

Following SPF record declares an ip6 mechanism to include all IPv6 addresses in the 2001:db8:85a3::8a2e:0000:0000-2001:db8:85a3::8a2e:FFFF:FFFF range:

"v=spf1 ip6:2001:db8:85a3::8a2e:370:7334/96 ~all"

a mechanism

An a mechanism allows us to reference (and include) the IP address(es) of the current domain for which this SPF record exists. Every domain name almost always has an A record which resolves to the IP address it is hosted at. Larger websites have more than one IP address that an A record can resolve to. By including and allowing an a record in the SPF record of your domain name, you are basically allowing any email originating from your domain name to pass.

For example, following SPF record includes an A record to allow any emails originating from the IP address of this domain name to pass:

"v=spf1 a ~all"

If you have another domain name and want to include its A record so that any email originating from the IP address of that domain name is also passed for your domain name, then use this syntax:

"v=spf1 a:anotherdomain.com ~all"

mx mechanism

An mx mechanism allows us to include the MX (Mail Exchanger) servers of this domain name in the SPF record. An MX record in the DNS records of a domain name points to one or more mail servers that are entitled to receive emails for that domain name. For example, if a person sends an email to an address at Stardeveloper.com, he will most probably send it via an SMTP relay server. That SMTP relay server then fetches the MX record of Stardeveloper.com and resolves it to an IP address. Then it connects to that IP address on the port 25 to deliver that email for Stardeveloper.com. As the admin for Stardeveloper.com, I can change the MX records and point them to any mail server of my choosing which I have configured to receive emails for my domain name.

Now that we know what an MX record is, let us see how we declare an mx mechanism to allow all mail servers who are registered to receive emails for our domain names to also be given a pass so that if any outgoing email originates from them, it will be passed.

Following SPF record includes an mx mechanism to allow any emails originating from the MX servers of this domain name to pass:

"v=spf1 mx ~all"

You can also include MX records for another domain name like the syntax we saw earlier for A record:

"v=spf1 mx:anotherdomain.com ~all"

include mechanism

An include mechanism is used to include a policy (SPF record) from another host or domain name. You will use this if your domain's emails also go through 3rd party SMTP servers over which you have no control. The 3rd party has its own policy (SPF record) for their SMTP servers/domain names. By including their SPF record in the SPF record of your domain name, you are allowing all emails to pass which are allowed to pass by that domain name. In other words, all emails originating from 3rd party's SMTP servers will be passed.

Following SPF record includes a policy (SPF record) from a third party's domain name:

"v=spf1 include:anotherdomain.com ~all"

Following is a real World example of an include mechanism to allow emails going through Google's SMTP servers to be passed:

"v=spf1 include:aspmx.googlemail.com ~all"

If you plan to use Google Apps with your domain name, then you must include this mechanism in the SPF record of your domain name: include:aspmx.googlemail.com. Learn more about how to setup SPF records for Google Apps.

ptr mechanism

A ptr mechanism allows you to include a PTR (pointer) record for a domain name. A PTR record is used in reverse DNS lookup. If your domain name has A and PTR records set correctly, an A record will resolve to an IP address, and that IP address can be reverse looked up back to the original domain name using PTR record.

Following SPF record uses a ptr mechanism to include a PTR record for this domain name:

"v=spf1 ptr ~all"

To include a PTR record of another domain name, you will use this syntax:

"v=spf1 ptr:anotherdomain.com ~all"

It is recommended that you do not unnecessarily use a ptr mechanism unless you have to because it results in unnecessary DNS lookups.

exists mechanism

An exists mechanism is used for checking if an A record exists for a given domain name. And if it exists, the email is allowed to pass.

It does not matter if the IP address of that domain name matches the originating IP address for this email or not. What matters is that, an A record should exist and that's all.

Following SPF record includes an exists mechanism for anotherdomain.com:

"v=spf1 exists:anotherdomain.com ~all"

Personally, I am not sure why if an A record exists for a domain name, you will allow this email to pass. But then, there must be some reason why this mechanism exists, no?

all mechanism

An all mechanism is used to declare a rule for hosts and IP addresses that have not matched with the preceding mechanisms. All SPF records almost always have an all mechanism declared at the end of the SPF record.

What you should do is to declare one or more mechanisms that if matched, will allow the email to pass (white list). At the end you should declare an all mechanism which should be quantified with an '~' to soft fail the rest.

Following SPF record declares an all mechanism to soft fail all emails i.e., no legitimate emails can originate for this domain name:

"v=spf1 ~all"

Following SPF record will fail all emails originating for this domain name:

"v=spf1 -all"

Following SPF record will pass all emails with this domain name in the 'From' address of the email, no matter from where it originted:

"v=spf1 all"

The example above to allow all emails to pass is for demonstration purposes only. You obviously will not be using this in real practice.

SPF record generating tips

First create a white list of hosts and IP addresses from which legitimate emails can originate for your domain name. You can always look at the example SPF record of Stardeveloper.com to have a look.

To create an SPF record for this list, start with a simple SPF record which soft fails every email like this:

"v=spf1 ~all"

Then go through the white list you created, one by one, and using the mechanisms' syntax we learned above, add the mechanisms to your SPF record.

My advice is to always include the A record in the SPF record for your domain name. Rest, you can decide.

Conclusion

In this article, we learned the SPF record syntax. We learned about quantifiers and mechanisms which collectively formulate a policy for a domain name. I hope the information in this article will be sufficient to help you setup correct SPF record for your domain name.

Good luck!

What is Sender Policy Framework?

Faisal Khan — Mon, 22 Feb 2010 09:19:00 GMT

SPF stands for 'Sender Policy Framework' which is an open standard to prevent the abuse of your domain name by someone else pretending to be you by using your email address or domain name in the 'From' field of his email.

How many times do you get bounced email back in your inbox for emails containing your email address in the 'From' field, selling products or containing links to indecent websites? You never sent these emails. Yet someone has picked your email or a fake email using your website's domain name to send spam to thousands of others users using your email address to bypass the filters on their inboxes and to use your repute and recognition to make the end-user give undeserved attention to their spam links.

How does SPF work?

SPF works by making use of a list of IP addresses that you specifically allow from which a legitimate email can originate for your domain name. For your website you have to make a list of IP addresses (or host names) from which legitimate emails can be sent for your domain name. This list should contain the IP addresses of your website (the A records) and the IP addresses of your SMTP servers (all of them). If you are using BlackBerry to send emails for your website or company, you will have to include their IP addresses in this list as well.

Now it is not easy to know the IP addresses of all servers from where the legitimate emails can originate for your domain name, you can make use of SPF record syntax to create a policy to include IP addresses by using host names and domain names instead of plain IP addresses in your SPF record. You can also include other domains' SPF records. The cool thing with SPF's syntax is that with very little effort you can comprehensively include all the servers without needing to hard code any single IP address at all.

Once a list of IP addresses and host names has been setup for your domain name, then the next time an authorized person tries to send an email using the 'From' address of your domain name e.g., fakedname@yourwebsite.com, the MX (Mail Exchanger) server receiving this email will lookup SPF record for your domain by checking the DNS records of your domain name. When it finds an SPF record, it will go through the list of IP addresses and host names, one by one, to see if this email is originating from the white list of IP addresses allowed to send email on your behalf or not. Since the originating email in this case did not originate from an authorized IP address, the receiving mail server will silently place this email in the 'Spam' folder, and not the user's 'Inbox'.

This is how SPF prevents misuse of your domain name by unauthorized persons.

This of course implies that you have to have your own domain name, for example, yourwebsite.com, and an email address like yourname@yourwebsite.com. If you are using Gmail or Yahoo and have email address like yourname@gmail.com, then you do not need to setup an SPF record; because it is already set by those email providers.

An example SPF record

The SPF record for Stardeveloper looks like this:

"v=spf1 a include:aspmx.googlemail.com include:srs.bis.eu.blackberry.com ~all"

Explanation

v=spf1

An SPF record always begings with v=spf1. This tells the SPF record validator that this SPF record's syntax belongs to version 1 of Sender Policy Framework.

Next, the SPF record contains an a which means an 'A' record for the domain name. What this means is that legitimate email can originate for Stardeveloper.com from the IP address it is hosted at. Note: the IP address has not been hard coded for the domain name, instead an a record has been specified using SPF record syntax. If Stardeveloper is relocated to another IP address, the 'A' record will automatically point to the new IP address without requiring any change in the SPF record for this domain name.

include:aspmx.googlemail.com

Since, Stardeveloper is using Google Apps for email hosting, I have included Google Apps' SPF record as part of the SPF record of Stardeveloper.com to make sure that any email that is sent through Google's SMTP servers is marked as legitimate.

include:srs.bis.eu.blackberry.com

I am using BlackBerry Bold 9700 as my mobile device. I use it to send and receive emails. To be able to mark all emails sent through my BlackBerry device that make use of BlackBerry's SMTP servers, I have included their SPF records from their domain name.

If you are based in North America, then you should change the 'eu' with 'na' to include BlackBerry's North American SMTP servers, that is, include:srs.bis.na.blackberry.com .

~all

The final mechanism beginning with a tilde '~' suggests a 'soft fail' for emails not being sent from authorized IP address. Soft fail means that emails originating from unauthorized IP addresses will be received by mail servers (MX) but will be placed in 'Spam' folders and not users' inbox. If this syntax had been -all, it would have meant 'hard fail'. Hard fail means to reject all emails originating from unlisted IP addresses straight away. Since Google recommends a soft fail based SPF syntax, Stardeveloper uses this syntax and so should you (unless of course if you know what you are doing).

How to check the SPF record for a domain name?

You can use the SPF record checking tool here. Just enter the domain name of your website (or any other to just have a look how they have setup their SPF records), and submit the form.

Should you have an SPF record for your website?

It is absolutely important for you to have an SPF record for your domain name. You must prevent the abuse of your domain name. You must not let your Uncle get an email from an unauthorized person claiming as you and suggesting your Uncle to visit his spam website. And this becomes even more important if you have a repute online.

Conclusion

In this article, we learned what is an SPF record and how it works. We looked at a sample SPF record and learned how simple the SPF record syntax can be to set one up for one's website.

Convert IP Address to Long

Faisal Khan — Sun, 27 Dec 2009 08:48:00 GMT

An IP address is a numerical address that is assigned to your computer the moment it connects to a network. Whenever you access a website on internet, a request is sent from your computer containing IP address of your computer, the address to which the website's data should be sent back to. This IP address can be used for logging, tracking stats, allowing (white lists) or blocking (black lists) content, and for a variety of other reasons by the website administrator.

ASP.NET makes it very easy to develop HTTP Modules and database driven web applications to analyze and store these IP addresses in the database. A convenient property made available to all ASP.NET applications as part of an HTTP request is HttpRequest.UserHostAddress property which will return a string representation of the user's IP address.

Do not store IP Addresses as strings

The problem here is that to analyze or store this IP address as string is not the most efficient or recommended method. Not only does it take more space (as much as 15 bytes) to store data, sorting and indexing this data is inefficient.

It takes 15 bytes (varchar(15)) to store an IP address like "255.255.255.255".
Sorting is inefficient because the values will be sorted as string literals.
It is not possible to find an IP address e.g., "255.255.255.127" in a block of IP addresses e.g., "255.255.255.0" - "255.255.255.255". You will have to use inefficient SQL queries like:
```
"SELECT * FROM [IPAddresses] WHERE [IPAddress] LIKE '255.255.255.%'"
```

So, what should you do?

Solution: Convert IP Addresses to Long values

The solution is to convert an IP address into an integer.. In most programming languages, the integer is declared as long to provide 8 bytes of space to the variable. In C#, an 8 byte long value is declared using long keyword. In T-SQL (i.e., for SQL Server), an 8 byte long value is declared as bigint. Here is why storing IP addresses as long (or bigint in T-SQL) is more efficient:

It takes only 8 bytes to store an IP address as long. An IP address like "255.255.255.255" is stored as 4294967295. The code to do this conversion, both in C# and T-SQL, will be looked upon later in this tutorial.
Sorting is extremely efficient as it is very easy to do the sorting on numerical data like 4294967295 than string literals like "255.255.255.255".
You can very easily find if a given IP address exists in a block of IP addresses using the query:
```
"SELECT * FROM [IPAddresses] WHERE
	[IPAddress] BETWEEN [IPAddressBlock1] AND [IPAddressBlock2]"
```

Now, we will look at the code to convert a string literal IP address (like "255.255.255.255") to a numerical value (like 4294967295) and back.

Converting IP Addresses to Long Values and Back in C#

To convert a string IP address to a long value, we will use the ConvertIPToLong() method as shown below. We first convert the IP address string literal to System.Net.IPAddress. Then we convert this IP address to an array of 4 bytes by splitting this IP address using the dot (".") letter. It then converts the 4 bytes array to a single long value.

/// <summary>
/// Author: Faisal Khan (http://www.stardeveloper.com)
/// </summary>
public static long ConvertIPToLong(string ipAddress)
{
	System.Net.IPAddress ip;

	if (System.Net.IPAddress.TryParse(ipAddress, out ip))
	{
		byte[] bytes = ip.GetAddressBytes();

		return (long)((bytes[0] << 24) | (bytes[1] << 16) |
			(bytes[2] << 8) | bytes[3]);
	}
	else
		return 0;
}

Now to convert an IP address long value back to its string representation, we will use the ConvertLongToIP() method as shown below:

/// <summary>
/// Author: Faisal Khan (http://www.stardeveloper.com)
/// </summary>
public static string ConvertLongToIP(long ipLong)
{
	StringBuilder b = new StringBuilder();
	long tempLong, temp;

	tempLong = ipLong;
	temp = tempLong / (256 * 256 * 256);
	tempLong = tempLong - (temp * 256 * 256 * 256);
	b.Append(Convert.ToString(temp)).Append(".");
	temp = tempLong / (256 * 256);
	tempLong = tempLong - (temp * 256 * 256);
	b.Append(Convert.ToString(temp)).Append(".");
	temp = tempLong / 256;
	tempLong = tempLong - (temp * 256);
	b.Append(Convert.ToString(temp)).Append(".");
	temp = tempLong;
	tempLong = tempLong - temp;
	b.Append(Convert.ToString(temp));

	return b.ToString().ToLower();
}

Converting IP Addresses to Bigint Values and Back in T-SQL

We will use ConvertIPToLong() function to convert varchar(15) based IP address to a bigint value as shown below:

' Author: Faisal Khan (http://www.stardeveloper.com)
CREATE FUNCTION [dbo].[ConvertIPToLong](@IP varchar(15))
RETURNS bigint
AS
BEGIN
	DECLARE @Long bigint
	SET @Long = CONVERT(bigint, PARSENAME(@IP, 4)) * 256 * 256 * 256 +
		CONVERT(bigint, PARSENAME(@IP, 3)) * 256 * 256 +
		CONVERT(bigint, PARSENAME(@IP, 2)) * 256 +
		CONVERT(bigint, PARSENAME(@IP, 1))

	RETURN (@Long)
END

Now, to convert this bigint value back to its varchar(15) representation, we will use the ConvertLongToIP() function as shown below:

' Author: Faisal Khan (http://www.stardeveloper.com)
CREATE FUNCTION [dbo].[ConvertLongToIP](@Long bigint)
RETURNS varchar(15)
AS
BEGIN
	DECLARE @IP varchar(15)
	DECLARE @TempLong bigint
	DECLARE @Temp bigint

	SET @TempLong = @Long
	SET @Temp = @TempLong / (256 * 256 * 256)
	SET @TempLong = @TempLong - (@Temp * 256 * 256 * 256)
	SET @IP = CONVERT(varchar(3), @Temp) + '.'
	SET @Temp = @TempLong / (256 * 256)
	SET @TempLong = @TempLong - (@Temp * 256 * 256)
	SET @IP = @IP + CONVERT(varchar(3), @Temp) + '.'
	SET @Temp = @TempLong / 256
	SET @TempLong = @TempLong - (@Temp * 256)
	SET @IP = @IP + CONVERT(varchar(3), @Temp) + '.'
	SET @Temp = @TempLong
	SET @TempLong = @TempLong - @Temp
	SET @IP = @IP + CONVERT(varchar(3), @Temp)

	RETURN (@IP)
END

After you have created these functions in your database, you can run the following SQL code to test these functions:

SELECT [dbo].ConvertIPToLong('255.255.255.255'), [dbo].ConvertLongToIP(4294967295)

You should create these functions in your C# libraries and in SQL Server databases and use them whenever you want to store or use an IP address in your code or database.

Final Words

I have very succinctly described the code to convert an IP address from a string value to long and back. I hope this code will be useful to you in your web and database application development. Good luck!

Introduction to Developing HTTP Modules in ASP.NET

Faisal Khan — Sat, 18 Jul 2009 09:00:00 GMT

Introduction
In ASP.NET, an HTTP Module is a class that implements the IHttpModule interface. By doing that it can listen to and handle events in the web request handling pipeline. They are the equivalent of 'ISAPI Filters' in IIS and 'Filters' in JSP. They are used for multiple purposes from authenticating a request to modifying the response and logging the request. In this tutorial, we will learn:

Common uses of HTTP Modules
Web Request Handling Pipeline
Structure of an HTTP Module
Creating your first HTTP Module

Common Uses of HTTP Modules
Following is a noncomprehensive list of common uses of HTTP Modules:

Authentication - To authenticate a user. For example, you can place a cookie in the user's browser once he has logged in. Next time when the user visits the website, his session will have timed out. An HTTP Module can fetch the cookie and log the user back in by retrieving and matching the cookie string against user data in the database, without the user having to enter his username/password on the website. This can provide a smooth experience to the user who will never come to know how he has been logged back in automatically.
Authorization - To protect access to restricted resources on the website. An HTTP Module can authenticate the user accessing the resource and if the user is not listed in the roles allowed to access this resource, then the user can be redirected back to a "login" page.
Protection against Denial of Service Attacks - An HTTP Module can maintain an internal list of IP addresses accessing the website along with the hits the website is receiving from each of them. When an IP address is noted to be sending too many requests out of proportion to the rest of the IP addresses, that IP address can be temporarily blocked from accessing the website.
HTTP Compression - To compress the text output of web pages using Gzip and Deflate algorithms. Compression can not only substantially reduce the bandwidth usage of a website, it can also make the pages appear faster in the users' browsers because of smaller sizes of compressed pages that are sent over the internet.
Encryption - To encrypt an outgoing response into encrypted text.
Caching - To cache commonly accessed content into an in-memory cache. This can be applied to static as well as dynamic generated content. Again an internal list can be used to gather statistics first as to which content is accessed more often than others. Then the popular content can be cached.
Collecting User Information - To get the user IP address and match it against an internal database of IP addresses along with names of countries. The HTTP Module can then pass the fetched country name (or country code) silently to the request handlers which can then display it as a country flag to the user. This is useful for not only providing customized experience to the user, you can also use it to display different content (e.g., ads) to users from different countries.
Modifying Headers - To modify the outgoing headers. For example, you can use it to tell the client browser when the requested resource was last modified and so on. You can send the headers for static content to be cached in the client browser and intermediate proxy servers, more often than for example dynamic content.
Formatting Output - To transform XML files with XSL stylesheets on the fly. So when the user accesses an XML file on your website directly, he/she will be served an HTML transformation of the XML file, displaying the content in a more pleasant fashion.
Replacing Content - To use regular expressions to parse and replace content in the response generated by the request handler. For example, to highlight words in the outgoing HTML content matching keywords that a user entered in the search engine e.g., Google, to visit your website. This will improve the experience of a person on your website who entered some keywords in Google and found your website among the ones that Google displayed. Now when he visited your website he sees his keywords highlighted within the text of that page.
Logging - To log requests in a file or a database. Some examples:-
- To log hits that your website is getting every hour. An ASP.NET page can be used to display these hits in a chart to you providing you with a quick snapshot of the activity on your website.
- To scan the Status Code e.g., 500, generated by a resource downstream, and log it in the database. An ASP.NET page again can be used to view in the form of a chart, when and how many 500 HTTP errors were generated on the website and by which resource.
- To log bandwidth that your website is using, day in and day out.

Note: What you can do with HTTP Modules is only limited by your imagination. Soon when you are up to speed with HTTP Modules, you will be developing ones for needs you had never thought you can be developing for, before.

On the next page, we will look at the events in the 'Web Request Handling Pipeline'.

Web Request Handling Pipeline
In ASP.NET, there is a series of events that is raised before a request is handed over to a request handler to execute. And on its way back to the client, another series of events is raised. This model is what I refer to as the 'Web Request Handling Pipeline'. The list of these events, in the order that they are fired from the moment a request is received to the moment it is disposed, is given below with brief description:

BeginRequest - Fired as the first event when ASP.NET responds to a request.
AuthenticateRequest - Occurs when the identity of the user is being established.
PostAuthenticateRequest - Occurs when the identity of the user has been established.
AuthorizeRequest - Occurs when the request is being authorized for this user.
PostAuthorizeRequest - Occurs when the request has been authorized for this user.
ResolveRequestCache - Occurs when the request is being examined for the fact that the requested resource be sent directly from the cache or not.
PostResolveRequestCache - Occurs when the requested resource is being sent directly from the cache and the request handler will not be called to process this request.
MapRequestHandler - Occurs when the request is being mapped to an appropriate request handler. This event is only raised in IIS 7.0 when it is run in integrated mode with .NET Framework 3.0 or above.
PostMapRequestHandler - Occurs when appropriate request handler has been mapped with the request.
AcquireRequestState - Occurs when state associated with the request is being acquired.
PostAcquireRequestState - Occurs when state associated with the request has been acquired.
PreRequestHandlerExecute - Occurs just before request handler (e.g., an ASP.NET page) is handed over control for this request. In the tutorial on "Enabling Gzip and Deflate HTTP Compression in ASP.NET pages", this event is handled to enable HTTP Compression in ASP.NET pages.

Note: The Request Handler is now handed over the request.

PostRequestHandlerExecute - Occurs right after the request handler has been executed.
ReleaseRequestState - Occurs when the state (e.g., session state associated with the request) associated with the request is being released.
PostReleaseRequestState - Occurs when the state associated with the request has been released.
UpdateRequestCache - Occurs when request handler has executed the request. This event provides opportunity to HTTP Modules that want to cache the response generated by this request so that subesquent requests can be handled directly from the cache.
PostUpdateRequestCache - Occurs when HTTP Modules that wanted to cache the response generated by this request have finished doing so.
LogRequest - Occurs when ASP.NET is logging this request. This event is supported by IIS 7.0 in integrated mode with .NET Framework 3.0 or above.
PostLogRequest - Occurs when ASP.NET has finished logging this request. Again, this event is only supported in IIS 7.0 in integrated mode with .NET Framework 3.0 or above.
EndRequest - Finally, this is the last event in the chain of events in web request handling pipeline.

Note: Above events are associated with HttpApplication object, the reference of which is provided to all HTTP Modules when they are initialized so that they can subscribe to the events of their interest.

We will now learn what an HTTP Module looks like in the code and how it can subscribe to and handle these events.

Structure of an HTTP Module
An HTTP Module is an ordinary class that implements IHttpModule interface.

IHttpModule
It is defined in the System.Web namespace. This interface has just two methods to be implemented as described below:

Init(HttpApplication application) - This method is called by ASP.NET to initialize the HTTP Module. In this method, the HTTP Module can use the supplied argument of type, HttpApplication, to subscribe to events it wishes to handle.
Dispose() - This method is called by ASP.NET just before it wishes to dispose the given instance of an HTTP Module.

HttpApplication Events
The HttpApplication object passed as the argument in the Init() method of the IHttpModule interface can be used by the HTTP Module to subscribe to events of its choice. The list of the events available to the HTTP Module has already been given above.

Note: You should only handle the event(s) that is/are most appropriate for the task that you want to achieve with your HTTP Module.

Now that you have seen IHttpModule interface and are aware of the events that are made available to HTTP Modules, we will look at the code of an empty HTTP Module and later add some code to handle an event from the web request handling pipeline.

Creating your first HTTP Module
Following code shows how a class can implement the IHttpModule interface:

using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Web;

namespace Stardeveloper.Modules
{
	public class SampleModule : IHttpModule
	{
		public void Init(HttpApplication application)
		{
		}

		public void Dispose()
		{
		}
	}
}

That wasn't too much of a code, was it? A class with the name of SampleModule declared in the Stardeveloper.Modules namespace implements the IHttpModule interface. An empty implementation of the methods is shown above.

To handle events in the web request handling pipeline, it subscribes to them as follows:

using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Web;

namespace Stardeveloper.Modules
{
	public class SampleModule : IHttpModule
	{
		public void Init(HttpApplication application)
		{
			application.PreRequestHandlerExecute +=
				new EventHandler(application_PreRequestHandlerExecute);
			application.PostRequestHandlerExecute +=
				new EventHandler(application_PostRequestHandlerExecute);
		}

		protected void application_PreRequestHandlerExecute(object sender,
			EventArgs e)
		{
			HttpApplication application = sender as HttpApplication;
			application.Response.Write("Welcome!");
		}
		
		protected void application_PostRequestHandlerExecute(object sender,
			EventArgs e)
		{
			HttpApplication application = sender as HttpApplication;
			application.Response.Write("Bye!");
		}

		public void Dispose()
		{
		}
	}
}

Same class now uses C#'s syntax of subscribing to events to subscribe to two events; PreRequestHandlerExecute and PostRequestHandlerExecute. In the body of the event handlers, it casts the reference from the sender object to HttpApplication object and uses its Response property (which is of type HttpResponse) to write some text in the response being sent to the client browser.

To install this module in your ASP.NET web application, place this code in SampleModule.cs file and put it in the /App_Code folder of your ASP.NET web application. Then edit your web.config file to add following line within the "httpModules" section:

<configuration>
	<system.web>
		<httpModules>
			<add name="SampleModule" type="Stardeveloper.Modules.SampleModule"/>
		</httpModules>
	</system.web>
</configuration>

Now, when you access any ASP.NET page within your ASP.NET web application, you will see all the text wrapped between "Welcome!" and "Bye!".

Summary
In this tutorial, we were introduced with the concept of events in the web request handling pipeline. We also got a good idea of what HTTP Modules can do. We then looked at the IHttpModule interface and HttpApplication object. Finally, we created an empty HTTP Module, made it handle a couple of events from the web request handling pipeline, configure it in the web.config file, and display some text to the user.

In the next tutorial, we will look at a more real-World example of an HTTP Module in action.

Displaying Emails from Mailbox on Gmail using POP3 and ASP.NET

Faisal Khan — Fri, 10 Jul 2009 05:52:00 GMT

Introduction
In this tutorial, we will create the missing ASP.NET page, that was left in the previous tutorial (Connecting to Mailbox on Gmail and Fetching List of Emails using POP3 and ASP.NET), that takes an individual email's sort number on the POP3 server to retrieve and display it to the user after properly decoding content-transfer-encodings like base64 and quoted-printable. You will learn how to access individual message parts in MIME encoded emails including fetching and enlisting attachments.

Note: It is mandatory that you read the previous tutorial (Connecting to Mailbox on Gmail and Fetching List of Emails using POP3 and ASP.NET) to download and place the essential Pop3.cs file in the /App_Code folder of your ASP.NET web application. Without this code, you will not be able to connect and retrieve emails using POP3.

In the previous tutorial, we limited ourselves to only listing the emails in our mailbox on Gmail. That in itself was quite a bit of a job since we had to develop all the baseline code that connects, sends commands, receives data, parses and instantiates objects. The classes that we created like Pop3Client, Email, and MessagePart made the task a lot more easier for us. Only few lines of code were needed to retrieve the list of emails in the mailbox and display it on the ASP.NET page. We will build on that base in this tutorial.

If you are looking for a production ready, stable, easy to use ASP.NET Newsletter application, then look no further: Faisal Khan (the author of this tutorial) has created a great Newsletter Application for you to download and start using today. It is easy to install, lets your users subscribe using a subscription form, provides a simple administration interface to create and send newsletters, handles the task of sending newsletters in the background using thread queues on the server, and lets users unsubscribe if they wish so.

Following is the screen shot of the ASP.NET page that we developed in the previous tutorial:

If you followed previous tutorial and created Pop3Client.aspx ASP.NET page that is listing the emails as shown in screen shot above, you'd have found out that the subject of the email is displayed as a link to DisplayPop3Email.aspx ASP.NET page. That ASP.NET page is what we will create in this tutorial.

Following screen shots shows DisplayPop3Email.aspx ASP.NET page displaying a plain text email:

Following screen shot shows DisplayPop3Email.aspx displaying an HTML email:

Following screen shot shows DisplayPop3Email.aspx listing attachments:

Now that you know what DisplayPop3Email.aspx ASP.NET page that we will create in this tutorial will do, we will now go straight into its code.

DisplayPop3Email.aspx
Open Notepad and copy following text in it. Then save it as "DisplayPop3Email.aspx" in your ASP.NET web application where you saved Pop3Client.aspx ASP.NET page from the previous tutorial:

<%@ Page Language="C#" %>

<%@ Import Namespace="System.Collections.Generic" %>
<%@ Import Namespace="Stardeveloper.Pop3" %>

<script runat="server">
	public const string Host = "pop.gmail.com";
	public const int Port = 995;
	public const string Email = "youremail@gmail.com";
	public const string Password = "yourpassword";

	protected static Regex CharsetRegex =
		new Regex("charset=\"?(?<charset>[^\\s\"]+)\"?",
		RegexOptions.IgnoreCase | RegexOptions.Compiled);
	protected static Regex QuotedPrintableRegex =
		new Regex("=(?<hexchars>[0-9a-fA-F]{2,2})",
		RegexOptions.IgnoreCase | RegexOptions.Compiled);
	protected static Regex UrlRegex =
		new Regex("(?<url>https?://[^\\s\"]+)",
		RegexOptions.IgnoreCase | RegexOptions.Compiled);
	protected static Regex FilenameRegex =
		new Regex("filename=\"?(?<filename>[^\\s\"]+)\"?",
		RegexOptions.IgnoreCase| RegexOptions.Compiled);
	protected static Regex NameRegex =
		new Regex("name=\"?(?<filename>[^\\s\"]+)\"?",
		RegexOptions.IgnoreCase | RegexOptions.Compiled);

	protected void Page_Load(object source, EventArgs e)
	{
		/* Setting email id */
		int emailId = -1;

		if (Request.QueryString["emailId"] == null)
		{
			Response.Redirect("Pop3Client.aspx");
			Response.Flush();
			Response.End();
		}
		else
			emailId = Convert.ToInt32(
				Request.QueryString["emailId"]);

		/* Creating Pop3Client object and accessing email data */
		Email email = null;
		List<MessagePart> msgParts = null;

		using (Pop3Client client = new Pop3Client(Host, Port,
			Email, Password, true))
		{
			client.Connect();

			// Fetching email headers
			email = client.FetchEmail(emailId);
			// Fetching email body
			msgParts = client.FetchMessageParts(emailId);
		}

		if (email == null || msgParts == null)
		{
			Response.Redirect("Pop3Client.aspx");
			Response.Flush();
			Response.End();
		}

		/* Selecting a message part to display to the user */
		MessagePart preferredMsgPart = FindMessagePart(msgParts, "text/html");

		if (preferredMsgPart == null)
			preferredMsgPart = FindMessagePart(msgParts, "text/plain");
		else if (preferredMsgPart == null && msgParts.Count > 0)
			preferredMsgPart = msgParts[0];

		string contentType, charset, contentTransferEncoding, body = null;

		if (preferredMsgPart != null)
		{
			contentType = preferredMsgPart.Headers["Content-Type"];
			charset = "us-ascii"; // default charset
			contentTransferEncoding =
				preferredMsgPart.Headers["Content-Transfer-Encoding"];

			Match m = CharsetRegex.Match(contentType);

			if (m.Success)
				charset = m.Groups["charset"].Value;

			HeadersLiteral.Text = contentType != null ? "Content-Type: " +
				contentType + "<br />" : string.Empty;
			HeadersLiteral.Text += contentTransferEncoding != null ?
				"Content-Transfer-Encoding: " +
				contentTransferEncoding : string.Empty;

			/* Decoding base64 and quoted-printable encoded messages */
			if (contentTransferEncoding != null)
			{
				if (contentTransferEncoding.ToLower() == "base64")
					body = DecodeBase64String(charset,
						preferredMsgPart.MessageText);
				else if (contentTransferEncoding.ToLower() ==
						"quoted-printable")
					body = DecodeQuotedPrintableString(
						preferredMsgPart.MessageText);
				else
					body = preferredMsgPart.MessageText;
			}
			else
				body = preferredMsgPart.MessageText;
		}

		/* Displaying information to the user */
		EmailIdLiteral.Text = Convert.ToString(emailId);
		DateLiteral.Text = email.UtcDateTime.ToString(); ;
		FromLiteral.Text = email.From;
		SubjectLiteral.Text = email.Subject;
		BodyLiteral.Text = preferredMsgPart != null ? 
			(preferredMsgPart.Headers["Content-Type"]
				.IndexOf("text/plain") != -1 ?
			"<pre>" + FormatUrls(body) + "</pre>" : body) : null;

		ListAttachments(msgParts);
	}

The code continues on the next page.

The code continues from the previous page.

	protected Decoder GetDecoder(string charset)
	{
		Decoder decoder;

		switch (charset.ToLower())
		{
			case "utf-7":
				decoder = Encoding.UTF7.GetDecoder();
				break;
			case "utf-8":
				decoder = Encoding.UTF8.GetDecoder();
				break;
			case "us-ascii":
				decoder = Encoding.ASCII.GetDecoder();
				break;
			case "iso-8859-1":
				decoder = Encoding.ASCII.GetDecoder();
				break;
			default:
				decoder = Encoding.ASCII.GetDecoder();
				break;
		}

		return decoder;
	}

	protected string DecodeBase64String(string charset, string encodedString)
	{
		Decoder decoder = GetDecoder(charset);

		byte[] buffer = Convert.FromBase64String(encodedString);
		char[] chararr = new char[decoder.GetCharCount(buffer,
			0, buffer.Length)];

		decoder.GetChars(buffer, 0, buffer.Length, chararr, 0);

		return new string(chararr);
	}

	protected string DecodeQuotedPrintableString(string encodedString)
	{
		StringBuilder b = new StringBuilder();
		int startIndx = 0;

		MatchCollection matches = QuotedPrintableRegex.Matches(encodedString);

		for (int i = 0; i < matches.Count; i++)
		{
			Match m = matches[i];
			string hexchars = m.Groups["hexchars"].Value;
			int charcode = Convert.ToInt32(hexchars, 16);
			char c = (char)charcode;

			if (m.Index > 0)
				b.Append(encodedString.Substring(startIndx,
					(m.Index - startIndx)));

			b.Append(c);

			startIndx = m.Index + 3;
		}
		
		if (startIndx < encodedString.Length)
			b.Append(encodedString.Substring(startIndx));

		return Regex.Replace(b.ToString(), "=\r\n", "");
	}

	protected void ListAttachments(List<MessagePart> msgParts)
	{
		bool attachmentsFound = false;
		StringBuilder b = new StringBuilder();
		b.Append("<ol>");

		foreach (MessagePart p in msgParts)
		{
			string contentType = p.Headers["Content-Type"];
			string contentDisposition = p.Headers["Content-Disposition"];

			Match m;

			if (contentDisposition != null)
			{
				m = FilenameRegex.Match(contentDisposition);

				if (m.Success)
				{
					attachmentsFound = true;
					b.Append("<li>").
						Append(m.Groups["filename"].
							Value).Append("</li>");
				}
			}
			else if (contentType != null)
			{
				m = NameRegex.Match(contentType);

				if (m.Success)
				{
					attachmentsFound = true;
					b.Append("<li>").
						Append(m.Groups["filename"].
							Value).Append("</li>");
				}
			}
		}
			
		b.Append("</ol>");

		if (attachmentsFound)
			AttachmentsLiteral.Text = b.ToString();
		else
			AttachementsRow.Visible = false;
	}

	protected MessagePart FindMessagePart(List<MessagePart> msgParts,
		string contentType)
	{
		foreach (MessagePart p in msgParts)
			if (p.ContentType != null &&
					p.ContentType.IndexOf(contentType) != -1)
				return p;

		return null;
	}

	protected string FormatUrls(string plainText)
	{
		string replacementLink = "<a href=\"${url}\">${url}</a>";

		return UrlRegex.Replace(plainText, replacementLink);
	}
</script>

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
	"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<html xmlns="http://www.w3.org/1999/xhtml">
<head runat="server">
    <title></title>
    <style type="text/css">
		.emails-table { width: 600px; border: solid 1px #444444; }
		.emails-table-header { font-family: "Trebuchet MS"; font-size: 9pt;
			background-color: #0099B9; font-weight: bold; color: white;
			text-align: center; border: solid 1px #444444; }
		.emails-table-header-cell { font-family: "Georgia"; font-size: 9pt;
			font-weight: bold; border: solid 1px #666666; padding: 6px; }
		.emails-table-cell { font-family: "Georgia"; font-size: 9pt;
			border: solid 1px #666666; padding: 6px; }
		.emails-table-footer { border: solid 1px #666666; padding: 3px;
			width: 50%; }
		.email-datetime { float: right; color: #666666; }
		
		a { font-family: "Lucida Sans Unicode", "Trebuchet MS"; font-size: 9pt;
			color: #005B7F; }
		a:hover { color:red; }
		pre { font-family: "Georgia"; font-size: 9pt; }
    </style>
</head>
<body>
    <form id="form1" runat="server">
    <asp:Literal ID="DebugLiteral" runat="server" />
    
    <table class="emails-table">
    <tr>
		<td class="emails-table-header" colspan="2">
		Email #<asp:Literal ID="EmailIdLiteral" runat="server" /></td>
    </tr>
    <tr>
		<td class="emails-table-header-cell">Date &amp; Time</td>
		<td class="emails-table-cell">
			<asp:Literal ID="DateLiteral" runat="server" /></td>
    </tr>
    <tr>
		<td class="emails-table-header-cell">From</td>
		<td class="emails-table-cell">
			<asp:Literal ID="FromLiteral" runat="server" /></td>
    </tr>
    <tr>
		<td class="emails-table-header-cell">Subject</td>
		<td class="emails-table-cell">
			<asp:Literal ID="SubjectLiteral" runat="server" /></td>
    </tr>
    <tr id="AttachementsRow" runat="server">
		<td class="emails-table-header-cell">Attachments</td>
		<td class="emails-table-cell">
			<asp:Literal ID="AttachmentsLiteral" runat="server" /></td>
    </tr>
     <tr>
		<td class="emails-table-cell" colspan="2">
			<asp:Literal ID="HeadersLiteral" runat="server" /></td>
    </tr>
    <tr>
		<td class="emails-table-cell" colspan="2">
			<asp:Literal ID="BodyLiteral" runat="server" /></td>
    </tr>
    </table>
    </form>
</body>
</html>

Explanation
We will now dissect the code, bit by bit.

The first thing we do is to use a Page level declaration to tell the ASP.NET compiler that all the server-side code encountered in this document should, by default, be assumed to be of the language, C#.

<%@ Page Language="C#" %>

Next, we import the necessary namespaces.

Note: We are importing the namespace Stardeveloper.Pop3 to make use of the code that we created in the previous tutorial (Connecting to Mailbox on Gmail and Fetching List of Emails using POP3 and ASP.NET). This will make available classes like Pop3Client, Email and MessagePart to us.

<%@ Import Namespace="System.Collections.Generic" %>
<%@ Import Namespace="Stardeveloper.Pop3" %>

All the server-side code is encapsulated between script tags.

<script runat="server">
	...
</script>

The host, port, email, and password arguments required in the constructor of Pop3Client class will be set using the following four constants.

public const string Host = "pop.gmail.com";
public const int Port = 995;
public const string Email = "youremail@gmail.com";
public const string Password = "yourpassword";

Regular expressions used in the code later have been created as static objects. The RegexOptions.Compiled bit is also set in the constructor of all the Regex objects so that they are compiled the first time they are used and from there onwards, running these regular expressions will be super fast.

protected static Regex CharsetRegex =
	new Regex("charset=\"?(?<charset>[^\\s\"]+)\"?",
	RegexOptions.IgnoreCase | RegexOptions.Compiled);
protected static Regex QuotedPrintableRegex =
	new Regex("=(?<hexchars>[0-9a-fA-F]{2,2})",
	RegexOptions.IgnoreCase | RegexOptions.Compiled);
protected static Regex UrlRegex =
	new Regex("(?<url>https?://[^\\s\"]+)",
	RegexOptions.IgnoreCase | RegexOptions.Compiled);
protected static Regex FilenameRegex =
	new Regex("filename=\"?(?<filename>[^\\s\"]+)\"?",
	RegexOptions.IgnoreCase| RegexOptions.Compiled);
protected static Regex NameRegex =
	new Regex("name=\"?(?<filename>[^\\s\"]+)\"?",
	RegexOptions.IgnoreCase | RegexOptions.Compiled);

Now we will dig into the Page_Load() method which is called with every request our DisplayPop3Email.aspx receives.

protected void Page_Load(object source, EventArgs e)
{
	...
}

We retrieve the emailId value from the querystring and save it in a variable of the same name.

Note: emailId is the sort number of the email in the list of emails on the POP3 server. You can determine the number of emails by sending the STAT command. For instance, you have 324 emails in your POP3 server. In those 324 emails, the email #324 is the latest email that your mailbox has received and email #1 is oldest email among all your emails in your mailbox. This is how emails are sorted by the POP3 server. Now if you want to retrieve the email #324, you should call the DisplayPop3Email.aspx ASP.NET page with the querystring value of the variable emailId set to 324 i.e., DisplayPop3Email.aspx?emailId=324. I hope this wasn't too difficult for you to understand.

/* Setting email id */
int emailId = -1;

if (Request.QueryString["emailId"] == null)
{
	Response.Redirect("Pop3Client.aspx");
	Response.Flush();
	Response.End();
}
else
	emailId = Convert.ToInt32(
		Request.QueryString["emailId"]);

Once we have created the Pop3Client object, we quickly call its Connect() method to connect with the POP3 server. Once there, we call FetchEmail() and FetchMessageParts() methods to retrieve the full email, including all headers and its body.

/* Creating Pop3Client object and accessing email data */
Email email = null;
List<MessagePart> msgParts = null;

using (Pop3Client client = new Pop3Client(Host, Port,
	Email, Password, true))
{
	client.Connect();

	// Fetching email headers
	email = client.FetchEmail(emailId);
	// Fetching email body
	msgParts = client.FetchMessageParts(emailId);
}

If the email's sort number was invalid e.g., the POP3 server has 324 emails and you accessed email #325, null references will be created by the code segment above. In which case we redirect the user to the ASP.NET page listing all the emails.

if (email == null || msgParts == null)
{
	Response.Redirect("Pop3Client.aspx");
	Response.Flush();
	Response.End();
}

We then try to iterate through all the MessagePart objects retrieved from the email and find the one most appropriate to display on the page. We begin by finding an HTML version of the email. If we cannot find it, we search for a plain text version of the email. If there is still no one found, we select the first MessagePart object we encounter in the list.

/* Selecting a message part to display to the user */
MessagePart preferredMsgPart = FindMessagePart(msgParts, "text/html");

if (preferredMsgPart == null)
	preferredMsgPart = FindMessagePart(msgParts, "text/plain");
else if (preferredMsgPart == null && msgParts.Count > 0)
	preferredMsgPart = msgParts[0];

One we have the MessagePart object we want to display on the page, we use its content-type and content-transfer-encoding headers to determine if the message body needs to be decoded first. If the message's body was encoded using base64 or quoted-printable transfer encodings, we decode the content first and then display it to the user.

string contentType, charset, contentTransferEncoding, body = null;

if (preferredMsgPart != null)
{
	contentType = preferredMsgPart.Headers["Content-Type"];
	charset = "us-ascii"; // default charset
	contentTransferEncoding =
		preferredMsgPart.Headers["Content-Transfer-Encoding"];

	Match m = CharsetRegex.Match(contentType);

	if (m.Success)
		charset = m.Groups["charset"].Value;

	HeadersLiteral.Text = contentType != null ? "Content-Type: " +
		contentType + "<br />" : string.Empty;
	HeadersLiteral.Text += contentTransferEncoding != null ?
		"Content-Transfer-Encoding: " +
		contentTransferEncoding : string.Empty;

	/* Decoding base64 and quoted-printable encoded messages */
	if (contentTransferEncoding != null)
	{
		if (contentTransferEncoding.ToLower() == "base64")
			body = DecodeBase64String(charset,
				preferredMsgPart.MessageText);
		else if (contentTransferEncoding.ToLower() ==
				"quoted-printable")
			body = DecodeQuotedPrintableString(
				preferredMsgPart.MessageText);
		else
			body = preferredMsgPart.MessageText;
	}
	else
		body = preferredMsgPart.MessageText;
}

We set a few literals with information from the email to display on the page.

/* Displaying information to the user */
EmailIdLiteral.Text = Convert.ToString(emailId);
DateLiteral.Text = email.UtcDateTime.ToString(); ;
FromLiteral.Text = email.From;
SubjectLiteral.Text = email.Subject;
BodyLiteral.Text = preferredMsgPart != null ? 
	(preferredMsgPart.Headers["Content-Type"]
		.IndexOf("text/plain") != -1 ?
	"<pre>" + FormatUrls(body) + "</pre>" : body) : null;

ListAttachments(msgParts);

GetDecoder() returns the appropriate decoder to decode the message text depending on the charset.

protected Decoder GetDecoder(string charset)
{
	Decoder decoder;

	switch (charset.ToLower())
	{
		case "utf-7":
			decoder = Encoding.UTF7.GetDecoder();
			break;
		case "utf-8":
			decoder = Encoding.UTF8.GetDecoder();
			break;
		case "us-ascii":
			decoder = Encoding.ASCII.GetDecoder();
			break;
		case "iso-8859-1":
			decoder = Encoding.ASCII.GetDecoder();
			break;
		default:
			decoder = Encoding.ASCII.GetDecoder();
			break;
	}

	return decoder;
}

DecodeBase64String() decodes a base64 encoded message and returns the decoded text.

Note: Most of the base64 encodings that you will encounter in a MIME encoded email will be for binary attachments that have been base64 encoded so that they can be attached as text with an email. But when when I went about to view emails on my email account, I encountered normal plain text emails that were base64 encoded. So I had to write a base64 decoder to decode and properly present them on the ASP.NET page.

protected string DecodeBase64String(string charset, string encodedString)
{
	Decoder decoder = GetDecoder(charset);

	byte[] buffer = Convert.FromBase64String(encodedString);
	char[] chararr = new char[decoder.GetCharCount(buffer,
		0, buffer.Length)];

	decoder.GetChars(buffer, 0, buffer.Length, chararr, 0);

	return new string(chararr);
}

DecodeQuotedPrintableString() method decodes a message whose body has been encoded with quoted-printable encoding.

protected string DecodeQuotedPrintableString(string encodedString)
{
	StringBuilder b = new StringBuilder();
	int startIndx = 0;

	MatchCollection matches = QuotedPrintableRegex.Matches(encodedString);

	for (int i = 0; i < matches.Count; i++)
	{
		Match m = matches[i];
		string hexchars = m.Groups["hexchars"].Value;
		int charcode = Convert.ToInt32(hexchars, 16);
		char c = (char)charcode;

		if (m.Index > 0)
			b.Append(encodedString.Substring(startIndx,
				(m.Index - startIndx)));

		b.Append(c);

		startIndx = m.Index + 3;
	}
	
	if (startIndx < encodedString.Length)
		b.Append(encodedString.Substring(startIndx));

	return Regex.Replace(b.ToString(), "=\r\n", "");
}

ListAttachments() method scans all the MessagePart objects to determine which of these is an attachment. It then displays the list of attachments on the page.

protected void ListAttachments(List<MessagePart> msgParts)
{
	bool attachmentsFound = false;
	StringBuilder b = new StringBuilder();
	b.Append("<ol>");

	foreach (MessagePart p in msgParts)
	{
		string contentType = p.Headers["Content-Type"];
		string contentDisposition = p.Headers["Content-Disposition"];

		Match m;

		if (contentDisposition != null)
		{
			m = FilenameRegex.Match(contentDisposition);

			if (m.Success)
			{
				attachmentsFound = true;
				b.Append("<li>").
					Append(m.Groups["filename"].
						Value).Append("</li>");
			}
		}
		else if (contentType != null)
		{
			m = NameRegex.Match(contentType);

			if (m.Success)
			{
				attachmentsFound = true;
				b.Append("<li>").
					Append(m.Groups["filename"].
						Value).Append("</li>");
			}
		}
	}
		
	b.Append("</ol>");

	if (attachmentsFound)
		AttachmentsLiteral.Text = b.ToString();
	else
		AttachementsRow.Visible = false;
}

FindMessagePart() scans all the MessagePart objects to find the one whose content-type matches the contentType argument.

protected MessagePart FindMessagePart(List<MessagePart> msgParts,
	string contentType)
{
	foreach (MessagePart p in msgParts)
		if (p.ContentType != null &&
				p.ContentType.IndexOf(contentType) != -1)
			return p;

	return null;
}

FormatUrls() formats URL strings for display on the page.

protected string FormatUrls(string plainText)
{
	string replacementLink = "<a href=\"${url}\">${url}</a>";

	return UrlRegex.Replace(plainText, replacementLink);
}

Now that we have looked at the code, we will run this ASP.NET page.

Running DisplayPop3Email.aspx
I placed Pop3.cs in the /App_Code folder, Pop3Client.aspx and DisplayPop3Email.aspx in the root folder (/). Then I started the ASP.NET web server and accessed Pop3Client.aspx in the browser. From the emails' list that appeared, I clicked one email's subject and this brought me to the DisplayPop3Email.aspx ASP.NET page that we just created. Following screen shot shows the result of this request:

Summary
In this tutorial, we created an ASP.NET page that takes an email's sort number on the POP3 server and connects to, retrieves, and displays that email on the page. This ASP.NET page was the missing page that was left in the previous tutorial (Connecting to Mailbox on Gmail and Fetching List of Emails using POP3 and ASP.NET).

I hope you can expand and modify the source code of this application to suit the needs of your application or if you are a beginner, learn the network programming aspects of it.

Connecting to Mailbox on Gmail and Fetching List of Emails using POP3 and ASP.NET

Faisal Khan — Wed, 08 Jul 2009 10:13:00 GMT

Introduction
In this tutorial, we will learn about Post Office Protocol (POP3). It is a protocol (a set of commands) used by an email client to connect to and retrieve email(s) from the mailbox on the remote server. We will develop the code that connects to the POP3 server and sends commands to retrieve the list of emails on the server, and then later, using an ASP.NET page, displays the list to the user. To demonstrate the code, we will be connecting to Gmail's POP3 server using SSL (Secure Sockets Layer).

To connect to your own POP3 mail server, you can either use the secure connection (SSL) if your mail server supports it or connect in unsecure mode. The code is simple to understand and by looking at it you will learn a lot about how network programming is done in the .NET environment.

Note: In this tutorial, we will limit ourselves to retrieving and displaying a list of emails from user's mailbox which the user can browse through using 'Previous Page' and 'Next Page' links. In the next tutorial (Displaying Emails from Mailbox on Gmail using POP3 and ASP.NET), we will learn how to retrieve and view an individual email.

Following screen shot shows an ASP.NET page listing emails in my Gmail mailbox:

Basic POP3 Commands
Following is a list of commonly used POP3 commands:

USER - Takes one argument i.e., the email address of the user trying to connect to his/her mailbox. Example usage:
```
USER youremail@xyz.com
```
PASS - Takes one argument i.e., the password of the user. Example usage:
```
PASS yourpassword
```
STAT - Returns the number of emails in the mailbox and the number of bytes all the emails are taking on the server. Example usage:
```
STAT
```
TOP - Takes two arguments i.e., the sort number of the email on the server and the number of lines of text to retrieve from the body of the email. Example usage:
```
TOP 1 10
```
RETR - Takes one argument i.e., the sort number of the email on the server and returns all the headers and lines from the body of the email. Example usage:
```
RETR 1
```
DELE - Takes one argument i.e., the sort number of the email on the server and deletes it. Example usage:
```
DELE 1
```
RSET - Resets any DELE commands given above. The emails marked to be deleted by DELE command are unmarked. Example usage:
```
RSET
```
QUIT - Closes the user session with the server. Example usage:
```
QUIT
```
Note: In the code that we will develop, we will not close the session with the remote server using QUIT command. Because if we have issued RETR command to retrieve any emails from the POP3 server, issuing QUIT commands gives the wrong impression to the server that those emails have been downloaded safely on our system so the server can remove them from its list. Since we do not want to inadvertently remove or delete any emails from the POP3 server, we will not issue QUIT command and simply close the connection.

Developing the Code
In this tutorial, we will develop C# classes that encapsulate all the code that is necessary to connect to and retrieve emails from Gmail's mailbox. We will place these classes in "Pop3.cs" C# source file. These classes will provide an easy to use interface to access the mailbox. An ASP.NET page with the name of "Pop3Client.aspx" will use these classes to list emails in the mailbox.

Files that we will develop in this tutorial:

Pop3.cs
Pop3Client.aspx

i. Pop3.cs
All right guys, open Notepad and copy following code in it. Then save this new document as Pop3.cs in the /App_Code folder of your ASP.NET web application:

/* Author: Faisal Khan */
/* © Stardeveloper.com */

using System;
using System.Collections.Generic;
using System.Collections.Specialized;
using System.Configuration;
using System.IO;
using System.Linq;
using System.Net;
using System.Net.Security;
using System.Net.Sockets;
using System.Text;
using System.Text.RegularExpressions;
using System.Web;
using System.Xml;

namespace Stardeveloper.Pop3
{
  public class Pop3Client : IDisposable
  {
    public string Host { get; protected set; }
    public int Port { get; protected set; }
    public string Email { get; protected set; }
    public string Password { get; protected set; }
    public bool IsSecure { get; protected set; }

    public TcpClient Client { get; protected set; }
    public Stream ClientStream { get; protected set; }

    public StreamWriter Writer { get; protected set; }
    public StreamReader Reader { get; protected set; }

    private bool disposed = false;

    public Pop3Client(string host, int port, string email,
      string password)
      : this(host, port, email, password, false)
    {
    }

    public Pop3Client(string host, int port, string email,
      string password, bool secure)
    {
      Host = host;
      Port = port;
      Email = email;
      Password = password;
      IsSecure = secure;
    }

    public void Connect()
    {
      if (Client == null)
        Client = new TcpClient();

      if (!Client.Connected)
        Client.Connect(Host, Port);

      if (IsSecure)
      {
        SslStream secureStream =
          new SslStream(Client.GetStream());
        secureStream.AuthenticateAsClient(Host);

        ClientStream = secureStream;
        secureStream = null;
      }
      else
        ClientStream = Client.GetStream();

      Writer = new StreamWriter(ClientStream);
      Reader = new StreamReader(ClientStream);

      ReadLine();
      Login();
    }

    public int GetEmailCount()
    {
      int count = 0;
      string response = SendCommand("STAT");

      if (IsResponseOk(response))
      {
        string[] arr = response.Substring(4).Split(' ');
        count = Convert.ToInt32(arr[0]);
      }
      else
        count = -1;

      return count;
    }

    public Email FetchEmail(int emailId)
    {
      if (IsResponseOk(SendCommand("TOP " + emailId + " 0")))
        return new Email(ReadLines());
      else
        return null;
    }

    public List<Email> FetchEmailList(int start, int count)
    {
      List<Email> emails = new List<Email>(count);

      for (int i = start; i < (start + count); i++)
      {
        Email email = FetchEmail(i);

        if (email != null)
          emails.Add(email);
      }

      return emails;
    }

    public List<MessagePart> FetchMessageParts(int emailId)
    {
      if (IsResponseOk(SendCommand("RETR " + emailId)))
        return Util.ParseMessageParts(ReadLines());

      return null;
    }

The code continues on the next page.

The code is continuing from the previous page.

public List<MessagePart> FetchMessageParts(int emailId)
    {
      if (IsResponseOk(SendCommand("RETR " + emailId)))
        return Util.ParseMessageParts(ReadLines());

      return null;
    }

    public void Close()
    {
      if (Client != null)
      {
        if (Client.Connected)
          Logout();

        Client.Close();
        Client = null;
      }

      if (ClientStream != null)
      {
        ClientStream.Close();
        ClientStream = null;
      }

      if (Writer != null)
      {
        Writer.Close();
        Writer = null;
      }

      if (Reader != null)
      {
        Reader.Close();
        Reader = null;
      }

      disposed = true;
    }

    public void Dispose()
    {
      if (!disposed)
        Close();
    }

    protected void Login()
    {
      if (!IsResponseOk(SendCommand("USER " + Email)) ||
        !IsResponseOk(SendCommand("PASS " + Password)))
        throw new Exception("User/password not accepted");
    }

    protected void Logout()
    {
      SendCommand("RSET");
      //SendCommand("QUIT");
    }

    protected string SendCommand(string cmdtext)
    {
      Writer.WriteLine(cmdtext);
      Writer.Flush();

      return ReadLine();
    }

    protected string ReadLine()
    {
      return Reader.ReadLine() + "\r\n";
    }

    protected string ReadLines()
    {
      StringBuilder b = new StringBuilder();

      while (true)
      {
        string temp = ReadLine();

        if (temp == ".\r\n" || temp.IndexOf("-ERR") != -1)
          break;

        b.Append(temp);
      }

      return b.ToString();
    }

    protected static bool IsResponseOk(string response)
    {
      if (response.StartsWith("+OK"))
        return true;
      if (response.StartsWith("-ERR"))
        return false;

      throw new Exception("Cannot understand server response: " +
        response);
    }
  }

  public class Email
  {
    public NameValueCollection Headers { get; protected set; }

    public string ContentType { get; protected set; }
    public DateTime UtcDateTime { get; protected set; }
    public string From { get; protected set; }
    public string To { get; protected set; }
    public string Subject { get; protected set; }

    public Email(string emailText)
    {
      Headers = Util.ParseHeaders(emailText);

      ContentType = Headers["Content-Type"];
      From = Headers["From"];
      To = Headers["To"];
      Subject = Headers["Subject"];

      if (Headers["Date"] != null)
        try
        {
        UtcDateTime =
          Util.ConvertStrToUtcDateTime(Headers["Date"]);
        }
        catch (FormatException)
        {
          UtcDateTime = DateTime.MinValue;
        }
      else
        UtcDateTime = DateTime.MinValue;
    }
  }

  public class MessagePart
  {
    public NameValueCollection Headers { get; protected set; }

    public string ContentType { get; protected set; }
    public string MessageText { get; protected set; }

    public MessagePart(NameValueCollection headers, string messageText)
    {
      Headers = headers;
      ContentType = Headers["Content-Type"];
      MessageText = messageText;
    }
  }

  public class Util
  {
    protected static Regex BoundaryRegex =
      new Regex("Content-Type: multipart(?:/\\S+;)" +
      "\\s+[^\r\n]*boundary=\"?(?<boundary>" +
      "[^\"\r\n]+)\"?\r\n", RegexOptions.IgnoreCase |
      RegexOptions.Compiled);
    protected static Regex UtcDateTimeRegex = new Regex(
      @"^(?:\w+,\s+)?(?<day>\d+)\s+(?<month>\w+)\s+(?<year>\d+)\s+(?<hour>\d{1,2})" +
      @":(?<minute>\d{1,2}):(?<second>\d{1,2})\s+(?<offsetsign>\-|\+)(?<offsethours>" +
      @"\d{2,2})(?<offsetminutes>\d{2,2})(?:.*)$",
      RegexOptions.IgnoreCase | RegexOptions.Compiled);

    public static NameValueCollection ParseHeaders(string headerText)
    {
      NameValueCollection headers = new NameValueCollection();
      StringReader reader = new StringReader(headerText);

      string line;
      string headerName = null, headerValue;
      int colonIndx;

      while ((line = reader.ReadLine()) != null)
      {
        if (line == "")
          break;

        if (Char.IsLetterOrDigit(line[0]) && (colonIndx = line.IndexOf(':')) != -1)
        {
          headerName = line.Substring(0, colonIndx);
          headerValue = line.Substring(colonIndx + 1).Trim();

          headers.Add(headerName, headerValue);
        }
        else if (headerName != null)
          headers[headerName] += " " + line.Trim();
        else
          throw new FormatException("Could not parse headers");
      }

      return headers;
    }

    public static List<MessagePart> ParseMessageParts(string emailText)
    {
      List<MessagePart> messageParts = new List<MessagePart>();
      int newLinesIndx = emailText.IndexOf("\r\n\r\n");

      Match m = BoundaryRegex.Match(emailText);

      if (m.Index < emailText.IndexOf("\r\n\r\n") && m.Success)
      {
        string boundary = m.Groups["boundary"].Value;
        string startingBoundary = "\r\n--" + boundary;

        int startingBoundaryIndx = -1;

        while (true)
        {
          if (startingBoundaryIndx == -1)
            startingBoundaryIndx = emailText.IndexOf(startingBoundary);

          if (startingBoundaryIndx != -1)
          {
            int nextBoundaryIndx = emailText.IndexOf(startingBoundary,
              startingBoundaryIndx + startingBoundary.Length);

            if (nextBoundaryIndx != -1 && nextBoundaryIndx != startingBoundaryIndx)
            {
              string multipartMsg = emailText.Substring(startingBoundaryIndx +
                startingBoundary.Length,
                (nextBoundaryIndx - startingBoundaryIndx - startingBoundary.Length));

              int headersIndx = multipartMsg.IndexOf("\r\n\r\n");

              if (headersIndx == -1)
                throw new FormatException("Incompatible multipart message format");

              string bodyText = multipartMsg.Substring(headersIndx).Trim();

              NameValueCollection headers = Util.ParseHeaders(multipartMsg.Trim());
              messageParts.Add(new MessagePart(headers, bodyText));
            }
            else
              break;

            startingBoundaryIndx = nextBoundaryIndx;
          }
          else
            break;
        }

        if (newLinesIndx != -1)
        {
          string emailBodyText = emailText.Substring(newLinesIndx + 1);
        }
      }
      else
      {
        int headersIndx = emailText.IndexOf("\r\n\r\n");

        if (headersIndx == -1)
          throw new FormatException("Incompatible multipart message format");

        string bodyText = emailText.Substring(headersIndx).Trim();

        NameValueCollection headers = Util.ParseHeaders(emailText);
        messageParts.Add(new MessagePart(headers, bodyText));
      }

      return messageParts;
    }

The code continues on the next page.

The code is continuing from the previous page.

public static DateTime ConvertStrToUtcDateTime(string str)
    {
      Match m = UtcDateTimeRegex.Match(str);

      int day, month, year, hour, min, sec;

      if (m.Success)
      {
        day = Convert.ToInt32(m.Groups["day"].Value);
        year = Convert.ToInt32(m.Groups["year"].Value);
        hour = Convert.ToInt32(m.Groups["hour"].Value);
        min = Convert.ToInt32(m.Groups["minute"].Value);
        sec = Convert.ToInt32(m.Groups["second"].Value);

        switch (m.Groups["month"].Value)
        {
          case "Jan":
            month = 1;
            break;
          case "Feb":
            month = 2;
            break;
          case "Mar":
            month = 3;
            break;
          case "Apr":
            month = 4;
            break;
          case "May":
            month = 5;
            break;
          case "Jun":
            month = 6;
            break;
          case "Jul":
            month = 7;
            break;
          case "Aug":
            month = 8;
            break;
          case "Sep":
            month = 9;
            break;
          case "Oct":
            month = 10;
            break;
          case "Nov":
            month = 11;
            break;
          case "Dec":
            month = 12;
            break;
          default:
            throw new FormatException("Unknown month.");
        }

        string offsetSign = m.Groups["offsetsign"].Value;
        int offsetHours = Convert.ToInt32(m.Groups["offsethours"].Value);
        int offsetMinutes = Convert.ToInt32(m.Groups["offsetminutes"].Value);

        DateTime dt = new DateTime(year, month, day, hour, min, sec);

        if (offsetSign == "+")
        {
          dt.AddHours(offsetHours);
          dt.AddMinutes(offsetMinutes);
        }
        else if (offsetSign == "-")
        {
          dt.AddHours(-offsetHours);
          dt.AddMinutes(-offsetMinutes);
        }

        return dt;
      }

      throw new FormatException("Incompatible date/time string format");
    }
  }
}

Explanation
Pop3.cs defines Stardeveloper.Pop3 namespace, which contains following 4 classes:

Pop3Client - This class uses classes in the System.Net and sub-namespaces to connect with the Pop3 server and send commands. It retrieves the data, and uses the Util class to create Email and MessagePart objects from the received data.
Email - It represents an Email message on the server. It provides you access to the "From", "To", "Subject", "Date" and "Content-Type" headers. It does not let you access the body of the email. The body of the email is represented by an array of MessagePart objects which are retrieved separately by calling Pop3Client.FetchMessageParts() method.
MessagePart - It represents a part of the body of the email. Since most of the emails are MIME encoded, which means that there can be more than one type (text/plain, text/html, etc.) of message part, and that there can be attached files, we represent the body of the email with an array of MessagePart objects. This class lets you access the headers and body of a message part of the email. More on it, later.
Util - This class contains 3 static methods to parse headers, message parts, and to convert UTC date/time string to DateTime object.

We will now look at the code of each of these classes in detail.

The first thing we do is to import the necessary namespaces.

using System;
using System.Collections.Generic;
using System.Collections.Specialized;
using System.Configuration;
using System.IO;
using System.Linq;
using System.Net;
using System.Net.Security;
using System.Net.Sockets;
using System.Text;
using System.Text.RegularExpressions;
using System.Web;
using System.Xml;

The Pop3Client class implements IDisposable interface.

public class Pop3Client : IDisposable
{
	...
}

Following properties provide access to the variables we set in the constructor of Pop3Client class.

public string Host { get; protected set; }
public int Port { get; protected set; }
public string Email { get; protected set; }
public string Password { get; protected set; }
public bool IsSecure { get; protected set; }

The TcpClient object that is used internally by Pop3Client class is exposed using Client property. The ClientStream property is either of the SslStream type if SSL is used to connect to the Pop3 server, or it is of NetworkStream type if the connection is unsecure.

public TcpClient Client { get; protected set; }
public Stream ClientStream { get; protected set; } // SslStream or NetworkStream

Since all POP3 commands sent by the client and responses received from the server end in the new-line characters ("\r\n"), we use StreamWriter and StreamReader classes to simplify the job of sending and receiving the data for us. You will see plenty of calls to their WriteLine() and ReadLine() methods, respectively, later.

public StreamWriter Writer { get; protected set; }
public StreamReader Reader { get; protected set; }

Lastly, a private variable is used to set the disposed state for this object. Remember, Pop3Client class implements IDisposable interface to make it easy for you to wrap access to this class in a using statement.

private bool disposed = false;

The constructor of Pop3Client class takes as its arguments the name of the host you want to connect to e.g. "pop.gmail.com", the port of the server you want to connect to e.g., 995 for Gmail, email address and password of your account on that Pop3 server. By default, the constructor will create an unsecure connection to the server. To use SSL to connect to the server, you have to set the final and optional argument to the constructor to true.

public Pop3Client(string host, int port, string email, string password)
	: this(host, port, email, password, false)
{
}

public Pop3Client(string host, int port, string email,
	string password, bool secure)
{
	Host = host;
	Port = port;
	Email = email;
	Password = password;
	IsSecure = secure;
}

The Connect() method instantiates necessary objects and connects to the POP3 server on the given host and port. If it was specified in the constructor to open a secure connection to the POP3 server, a SslStream object is created which encapsulates the NetworkStream returned from the TcpClient.GetStream() method. Lastly, StreamWriter and StreamReader objects are instantiated, and Login() method is called.

public void Connect()
{
	if (Client == null)
		Client = new TcpClient();

	if (!Client.Connected)
		Client.Connect(Host, Port);

	if (IsSecure)
	{
		SslStream secureStream =
			new SslStream(Client.GetStream());
		secureStream.AuthenticateAsClient(Host);

		ClientStream = secureStream;
		secureStream = null;
	}
	else
		ClientStream = Client.GetStream();

	Writer = new StreamWriter(ClientStream);
	Reader = new StreamReader(ClientStream);

	ReadLine();
	Login();
}

The Login() method sends the user email and password to the server. Each call to send the command with required argument is sent using SendCommand() method, which internally uses StreamWriter.WriteLine() method to send the command. The response of the server is parsed using IsResponseOk() method to determine if the response was successfull i.e., it began with an "+OK" message or if it was an error i.e., it began with an "-ERR" message. If everything went fine, the method simply returns, otherwise an Exception is thrown.

Note: All responses received from the server start with either "+OK", if +ve response has been generated on the server, or with "-ERR", if an error occurred.

protected void Login()
{
	if (!IsResponseOk(SendCommand("USER " + Email)) ||
		!IsResponseOk(SendCommand("PASS " + Password)))
		throw new Exception("User/password not accepted");
}

As explained above, the SendCommand() method uses StreamWriter.WriteLine() to send the command to the server. We use WriteLine() and not Write() because each command to the server must end with new-line characters i.e., "\r\n". Since StreamWriter internally buffers the data we want sent to the server, we call its Flush() method to make it send the command immediately. The response that is received from the server is fetched using StreamReader.ReadLine() method and is returned to the calling code.

protected string SendCommand(string cmdtext)
{
	Writer.WriteLine(cmdtext);
	Writer.Flush();

	return ReadLine();
}

As explained above, the IsResponseOk() method parses the response received from the server and checks for the "+OK" and "-ERR" strings to determine if the response was ok or if an error has been returned.

protected static bool IsResponseOk(string response)
{
	if (response.StartsWith("+OK"))
		return true;
	if (response.StartsWith("-ERR"))
		return false;

	throw new Exception("Cannot understand server response: " +
		response);
}

GetEmailCount() returns the number of emails in the mailbox. It uses "STAT" POP3 command to retrieve the number of emails. As you can see in the code below, the response of the server is parsed and the word containing number of emails is taken and converted to an integer, which is then returned to the calling code.

public int GetEmailCount()
{
	int count = 0;
	string response = SendCommand("STAT");

	if (IsResponseOk(response))
	{
		string[] arr = response.Substring(4).Split(' ');
		count = Convert.ToInt32(arr[0]);
	}
	else
		count = -1;

	return count;
}

FetchEmail() takes the sort number of the email as its argument and retrieves the headers for that email using "TOP" POP3 command. This command takes two parameters, the sort no of email and the number of lines of the body of email you want to fetch. We give this command the sort no of the email to fetch and 0 (zero) as the number of lines of text to retrieve from the body of the email. We set the second parameter to 0 because we only want to retrieve headers. The headers returned are used to instantiate Email object and that is then returned to the calling code.

public Email FetchEmail(int emailId)
{
	if (IsResponseOk(SendCommand("TOP " + emailId + " 0")))
		return new Email(ReadLines());
	else
		return null;
}

FetchEmailList() fetches the email headers, using FetchEmail() method, of given number of emails starting from given sort number. For instance, you have 300 emails in your mailbox and want to now display a list of 10 emails from 150 to 160, you should call this method as: FetchEmailList(151, 10).

 public List<Email> FetchEmailList(int start, int count)
{
	List<Email> emails = new List<Email>(count);

	for (int i = start; i < (start + count); i++)
	{
		Email email = FetchEmail(i);

		if (email != null)
			emails.Add(email);
	}

	return emails;
}

FetchMessageParts() retrieves an email from the server and then parses its body into an array of MessagePart objects. Each MessagePart object encapsulates one part of the body of the email and any attached files.

public List<MessagePart> FetchMessageParts(int emailId)
{
	if (IsResponseOk(SendCommand("RETR " + emailId)))
		return Util.ParseMessageParts(ReadLines());

	return null;
}

Close() logs out the user, closes the connection, and properly disposes all the Stream objects in use.

public void Close()
{
	if (Client != null)
	{
		if (Client.Connected)
			Logout();

		Client.Close();
		Client = null;
	}

	if (ClientStream != null)
	{
		ClientStream.Close();
		ClientStream = null;
	}

	if (Writer != null)
	{
		Writer.Close();
		Writer = null;
	}

	if (Reader != null)
	{
		Reader.Close();
		Reader = null;
	}

	disposed = true;
}

Dispose() method is implemented as part of the implementation of IDisposable interface. It calls the Close() method above if not already called.

public void Dispose()
{
	if (!disposed)
		Close();
}

Logout() method does nothing but to send the "RSET" command to the server which resets the server meaning there by that if any emails were marked as deleted, they are unmarked. Although no part in our code marks the emails to be deleted, we call it anyway.

protected void Logout()
{
	SendCommand("RSET");
}

We will now look at the code that constitutes the Email class.

public class Email
{
	...
}

An Email object represents an email on the server minus its body. It contains all the headers contained in an email. It provides access to the headers' collection using Headers property. Commonly used headers like "From", "To", "Subject", "Date", "Content-Type" can be accessed using properites like From, To, Subject, UtcDateTime, and ContentType, respectively.

Note: The purpose of properties mentioned above is only to provide easy access to header fields. For practical purposes code like Email.Headers["From"] and Email.From will yield the same value. And if any of these headers is not found in the email, the property will return null.

public NameValueCollection Headers { get; protected set; }

public string ContentType { get; protected set; }
public DateTime UtcDateTime { get; protected set; }
public string From { get; protected set; }
public string To { get; protected set; }
public string Subject { get; protected set; }

Within the constructor of Email class, we retrieve header fields as a NameValueCollection object using Util.ParseHeaders() method. Once we have the collection, we set the properties to the values retrieved from the header fields.

Unlike other properties, the property Date is not of type string. It is of type DateTime. We parse the Headers["Date"] header string and convert it in to a DateTime structure using Util.ConvertStrToUtcDateTime() method.

public Email(string emailText)
{
	Headers = Util.ParseHeaders(emailText);

	ContentType = Headers["Content-Type"];
	From = Headers["From"];
	To = Headers["To"];
	Subject = Headers["Subject"];

	if (Headers["Date"] != null)
		try
		{
			UtcDateTime = Util.ConvertStrToUtcDateTime(Headers["Date"]);
		}
		catch (FormatException)
		{
			UtcDateTime = DateTime.MinValue;
		}
	else
		UtcDateTime = DateTime.MinValue;
}

We will now look at the code of MessagePart class.

public class MessagePart
{
	...
}

A MessagePart object represents just one component of the body of an email. Remember, that most emails are MIME compatible and have multiple parts that form the body of that email.

For instance, you send an HTML email to a person. Just in case if the person is using an email client that cannot render HTML emails, either you or your email client also creates a plain text version of the email and appends it to the email. Now your email has two parts; one is of text/html content-type and the other is of text/plain content-type. The email client of the person you sent that email to will upon receiving that email try to find a plain text version of the email so that it can display that to the person without any difficulty. On searching through the MIME compatible email's message parts, it finds a version of text/plain content-type. It parses and decodes (if needed) it and displays it to that person.

Now if you had attached 2 files with that email, the total message parts would have become four:

HTML version of the email - text/html
Plain text version of the email - text/plain
Attached File #1 - base64
Attached File #2 - base64

Now upon parsing this email using the code above, you'll have an array of four MessagePart objects. I hope you understand now what MessagePart represents.

Note: base64 is a content-transfer-encoding for sending binary data like images files, in a text based email message.

It has a Headers property that makes available all the headers that this message part contains. The ContentType property is there to easily access the Headers["Content-Type"] field. The MessageText property makes available the content of the body of this message part.

public NameValueCollection Headers { get; protected set; }

public string ContentType { get; protected set; }
public string MessageText { get; protected set; }

In the constructor, we set the properties.

public MessagePart(NameValueCollection headers, string messageText)
{
	Headers = headers;
	ContentType = Headers["Content-Type"];
	MessageText = messageText;
}

We will now look at the code of Util class, which contains 3 static methods.

public class Util
{
	...
}

ParseHeaders() parses all the headers and converts them into easily accessible NameValueCollection.

public static NameValueCollection ParseHeaders(string headerText)
{
	NameValueCollection headers = new NameValueCollection();
	StringReader reader = new StringReader(headerText);

	string line;
	string headerName = null, headerValue;
	int colonIndx;

	while ((line = reader.ReadLine()) != null)
	{
		if (line == "")
			break;

		if (Char.IsLetterOrDigit(line[0]) && (colonIndx = line.IndexOf(':')) != -1)
		{
			headerName = line.Substring(0, colonIndx);
			headerValue = line.Substring(colonIndx + 1).Trim();

			headers.Add(headerName, headerValue);
		}
		else if (headerName != null)
			headers[headerName] += " " + line.Trim();
		else
			throw new FormatException("Could not parse headers");
	}

	return headers;
}

ParseMessageParts() parses the body of an email and separates the body into separate message parts using the boundary header.

public static List<MessagePart> ParseMessageParts(string emailText)
    {
      List<MessagePart> messageParts = new List<MessagePart>();
      int newLinesIndx = emailText.IndexOf("\r\n\r\n");

      Match m = BoundaryRegex.Match(emailText);

      if (m.Index < emailText.IndexOf("\r\n\r\n") && m.Success)
      {
        string boundary = m.Groups["boundary"].Value;
        string startingBoundary = "\r\n--" + boundary;

        int startingBoundaryIndx = -1;

        while (true)
        {
          if (startingBoundaryIndx == -1)
            startingBoundaryIndx = emailText.IndexOf(startingBoundary);

          if (startingBoundaryIndx != -1)
          {
            int nextBoundaryIndx = emailText.IndexOf(startingBoundary,
              startingBoundaryIndx + startingBoundary.Length);

            if (nextBoundaryIndx != -1 && nextBoundaryIndx != startingBoundaryIndx)
            {
              string multipartMsg = emailText.Substring(startingBoundaryIndx +
                startingBoundary.Length,
                (nextBoundaryIndx - startingBoundaryIndx - startingBoundary.Length));

              int headersIndx = multipartMsg.IndexOf("\r\n\r\n");

              if (headersIndx == -1)
                throw new FormatException("Incompatible multipart message format");

              string bodyText = multipartMsg.Substring(headersIndx).Trim();

              NameValueCollection headers = Util.ParseHeaders(multipartMsg.Trim());
              messageParts.Add(new MessagePart(headers, bodyText));
            }
            else
              break;

            startingBoundaryIndx = nextBoundaryIndx;
          }
          else
            break;
        }

        if (newLinesIndx != -1)
        {
          string emailBodyText = emailText.Substring(newLinesIndx + 1);
        }
      }
      else
      {
        int headersIndx = emailText.IndexOf("\r\n\r\n");

        if (headersIndx == -1)
          throw new FormatException("Incompatible multipart message format");

        string bodyText = emailText.Substring(headersIndx).Trim();

        NameValueCollection headers = Util.ParseHeaders(emailText);
        messageParts.Add(new MessagePart(headers, bodyText));
      }

      return messageParts;
    }

ConvertStrToUtcDateTime() parses a date & time string and converts it into a DateTime structure. It makes use of a regular expression to parse individual components from the date & time string.

public static DateTime ConvertStrToUtcDateTime(string str)
{
	Match m = UtcDateTimeRegex.Match(str);

	int day, month, year, hour, min, sec;

	if (m.Success)
	{
		day = Convert.ToInt32(m.Groups["day"].Value);
		year = Convert.ToInt32(m.Groups["year"].Value);
		hour = Convert.ToInt32(m.Groups["hour"].Value);
		min = Convert.ToInt32(m.Groups["minute"].Value);
		sec = Convert.ToInt32(m.Groups["second"].Value);

		switch (m.Groups["month"].Value)
		{
			case "Jan":
				month = 1;
				break;
			case "Feb":
				month = 2;
				break;
			case "Mar":
				month = 3;
				break;
			case "Apr":
				month = 4;
				break;
			case "May":
				month = 5;
				break;
			case "Jun":
				month = 6;
				break;
			case "Jul":
				month = 7;
				break;
			case "Aug":
				month = 8;
				break;
			case "Sep":
				month = 9;
				break;
			case "Oct":
				month = 10;
				break;
			case "Nov":
				month = 11;
				break;
			case "Dec":
				month = 12;
				break;
			default:
				throw new FormatException("Unknown month.");
		}

		string offsetSign = m.Groups["offsetsign"].Value;
		int offsetHours = Convert.ToInt32(m.Groups["offsethours"].Value);
		int offsetMinutes = Convert.ToInt32(m.Groups["offsetminutes"].Value);

		DateTime dt = new DateTime(year, month, day, hour, min, sec);

		if (offsetSign == "+")
		{
			dt.AddHours(offsetHours);
			dt.AddMinutes(offsetMinutes);
		}
		else if (offsetSign == "-")
		{
			dt.AddHours(-offsetHours);
			dt.AddMinutes(-offsetMinutes);
		}

		return dt;
	}

	throw new FormatException("Incompatible date/time string format");
}

Now that we have created the necessary classes, we will create the front-end ASP.NET page to display the list of emails to the user.

ii. Pop3Client.aspx
Open Notepad and copy following text in it, and then save it as "Pop3Client.aspx" in your ASP.NET web application:

<%@ Page Language="C#" %>

<%@ Import Namespace="System.Collections.Generic" %>
<%@ Import Namespace="Stardeveloper.Pop3" %>

<script runat="server">
	public const string Host = "pop.gmail.com";
	public const int Port = 995;
	public const string Email = "youremail@gmail.com";
	public const string Password = "yourpassword";

	public const int NoOfEmailsPerPage = 5;
	public const string SelfLink =
		"<a href=\"Pop3Client.aspx?page={0}\">{1}</a>";
	public const string DisplayEmailLink =
		"<a href=\"DisplayPop3Email.aspx?emailId={0}\">{1}</a>";

	protected void Page_Load(object source, EventArgs e)
	{
		/* Setting current page */
		int page = 1;

		if (Request.QueryString["page"] == null)
		{
			Response.Redirect("Pop3Client.aspx?page=1");
			Response.Flush();
			Response.End();
		}
		else
			page = Convert.ToInt32(Request.QueryString["page"]);

		/* Creating Pop3Client object and accessing email data */
		int totalEmails;
		List<Email> emails;
		string emailAddress;

		using(Pop3Client client = new Pop3Client(Host, Port, Email, Password, true))
		{
			emailAddress = client.Email;

			client.Connect();

			totalEmails = client.GetEmailCount();
			emails = client.FetchEmailList(((page - 1) *
				NoOfEmailsPerPage) + 1, NoOfEmailsPerPage);
		}

		/* Computing total pages */
		int totalPages;
		int mod = totalEmails % NoOfEmailsPerPage;

		if (mod == 0)
			totalPages = totalEmails / NoOfEmailsPerPage;
		else
			totalPages = ((totalEmails - mod) / NoOfEmailsPerPage) + 1;

		for (int i = 0; i < emails.Count; i++)
		{
			Email email = emails[i];

			/* Appending each email to an existing table */
			int emailId = ((page - 1) * NoOfEmailsPerPage) + i + 1;

			TableCell noCell = new TableCell();
			noCell.CssClass = "emails-table-cell";
			noCell.Text = Convert.ToString(emailId);
			
			TableCell fromCell = new TableCell();
			fromCell.CssClass = "emails-table-cell";
			fromCell.Text = email.From;
			
			TableCell subjectCell = new TableCell();
			subjectCell.CssClass = "emails-table-cell";
			subjectCell.Style["width"] = "300px";
			subjectCell.Text = String.Format(DisplayEmailLink,
				emailId, email.Subject);

			//TableCell contentTypeCell = new TableCell();
			//contentTypeCell.CssClass = "emails-table-cell";
			//contentTypeCell.Text = email.ContentType;
			
			TableCell dateCell = new TableCell();
			dateCell.CssClass = "emails-table-cell";
			
			if (email.UtcDateTime != DateTime.MinValue)
				dateCell.Text = email.UtcDateTime.ToString();

			TableRow emailRow = new TableRow();
			emailRow.Cells.Add(noCell);
			emailRow.Cells.Add(fromCell);
			emailRow.Cells.Add(subjectCell);
			//emailRow.Cells.Add(contentTypeCell);
			emailRow.Cells.Add(dateCell);

			EmailsTable.Rows.AddAt(2 + i, emailRow);
		}

		/* Creating links to previous and next pages */
		if (totalPages > 1)
		{
			if (page > 1)
				PreviousPageLiteral.Text =
					String.Format(SelfLink, page - 1, "Previous Page");

			if (page > 0 && page < totalPages)
				NextPageLiteral.Text =
					String.Format(SelfLink, page + 1, "Next Page");
		}

		/* Displaying information to the user */
		EmailFromLiteral.Text =
			Convert.ToString(((page - 1) * NoOfEmailsPerPage) + 1);
		EmailToLiteral.Text = Convert.ToString(page * NoOfEmailsPerPage);
		EmailTotalLiteral.Text = Convert.ToString(totalEmails);
		EmailLiteral.Text = emailAddress;
	}
</script>

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
	"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<html xmlns="http://www.w3.org/1999/xhtml">
<head runat="server">
    <title></title>
    <style type="text/css">
		.emails-table { width: 600px; border: solid 1px #444444; }
		.emails-table-header { font-family: "Trebuchet MS"; font-size: 9pt;
			background-color: #0099B9; color: white;
			border: solid 1px #444444; }
		.emails-table-header-cell { font-family: "Georgia"; font-size: 9pt;
			font-weight: bold; border: solid 1px #666666; padding: 6px; }
		.emails-table-cell { font-family: "Georgia"; font-size: 9pt;
			border: solid 1px #666666; padding: 6px; }
		.emails-table-footer { border: solid 1px #666666; padding: 3px;
			width: 50%; }
		.email-datetime { float: right; color: #666666; }
		
		a { font-family: "Lucida Sans Unicode", "Trebuchet MS"; font-size: 9pt;
			color: #005B7F; }
		a:hover { color:red; }
    </style>
</head>
<body>
    <form id="form1" runat="server">
    <div>
		<asp:Table ID="EmailsTable" CssClass="emails-table" runat="server">
		<asp:TableHeaderRow>
			<asp:TableHeaderCell CssClass="emails-table-header"
				ColumnSpan="4">Listing emails 
			<asp:Literal ID="EmailFromLiteral" runat="server" />-
			<asp:Literal ID="EmailToLiteral" runat="server" /> of 
			<asp:Literal ID="EmailTotalLiteral" runat="server" /> for 
			<asp:Literal ID="EmailLiteral" runat="server" />
			</asp:TableHeaderCell>
		</asp:TableHeaderRow>
		<asp:TableRow>
			<asp:TableCell CssClass="emails-table-header-cell">
				#</asp:TableCell>
			<asp:TableCell CssClass="emails-table-header-cell">
				From</asp:TableCell>
			<asp:TableCell CssClass="emails-table-header-cell">
				Subject</asp:TableCell>
			<asp:TableCell CssClass="emails-table-header-cell">
				Date &amp; Time</asp:TableCell>
		</asp:TableRow>
		<asp:TableFooterRow>
			<asp:TableCell CssClass="emails-table-footer" ColumnSpan="4">
			<asp:Table ID="FooterTable" Width="100%"
				BorderWidth="0" runat="server">
			<asp:TableRow>
			<asp:TableCell>
				<asp:Literal ID="PreviousPageLiteral" runat="server" />
			</asp:TableCell>
			<asp:TableCell HorizontalAlign="Right">
				<asp:Literal ID="NextPageLiteral" runat="server" />
			</asp:TableCell>
			</asp:TableRow>
			</asp:Table>
			</asp:TableCell>
		</asp:TableFooterRow>
		</asp:Table>
    </div>
    </form>
</body>
</html>

Explanation
This ASP.NET page is responsible for using the Pop3Client class we developed earlier to display list of emails in Gmail's mailbox, page by page. The user will be able to browse all the pages using 'Next Page' and 'Previous Page' links.

The first thing we do is to declare a Page level directive and tell the ASP.NET compiler that the language we are going to use on this page is "C#".

<%@ Page Language="C#" %>

Next, we import the necessary namespaces.

<%@ Import Namespace="System.Collections.Generic" %>
<%@ Import Namespace="Stardeveloper.Pop3" %>

The server-side code is encapsulated between the <script></script> tags.

<script runat="server">
	...
</script>

We declare a few constants. The first four are required to properly connect and login to the Gmail's POP3 server. The Host and Port constants can be changed to point to your email provider's POP3 server, if it is not Gmail. For the purpose of demonstration, we will connect to Gmail's POP3 server. The Email and Password constants should be changed to your email address and password.

NoOfEmailsPerPage constant is used to declare how many emails we want to show per page. You can change it to 10 or 20 to suit the needs of your web application. SelfLink and DisplayLink constants contain strings for formatting links and to display the full email in the "DisplayPop3Email.aspx" ASP.NET page, which we will create in the next tutorial.

public const string Host = "pop.gmail.com";
public const int Port = 995;
public const string Email = "youremail@gmail.com";
public const string Password = "yourpassword";

public const int NoOfEmailsPerPage = 5;
public const string SelfLink =
	"<a href=\"Pop3Client.aspx?page={0}\">{1}</a>";
public const string DisplayEmailLink =
	"<a href=\"DisplayPop3Email.aspx?emailId={0}\">{1}</a>";

Now we will dig into the all important code in the Page_Load() method.

protected void Page_Load(object source, EventArgs e)
{
	...
}

If you access Pop3Client.aspx directly, it will redirect you to Pop3Client.aspx?page=1. The current page is retrieved and saved in the page variable.

/* Setting current page */
int page = 1;

if (Request.QueryString["page"] == null)
{
	Response.Redirect("Pop3Client.aspx?page=1");
	Response.Flush();
	Response.End();
}
else
	page = Convert.ToInt32(Request.QueryString["page"]);

Now we create the Pop3Client object. We create it within the using statement so that we don't explicitly have to call the Pop3Client.Close() method. Gorgeous!

We call the GetEmailCount() method to get the number of emails in the mailbox. Next, we retrieve the list of emails for the current page.

Note: The number of emails in the list will be a maximum of NoOfEmailsPerPage i.e., 5. We do not retrieve all the emails in the mailbox. We retrieve only those that are required to be displayed on the current page.

/* Creating Pop3Client object and accessing email data */
int totalEmails;
List<Email> emails;
string emailAddress;

using(Pop3Client client = new Pop3Client(Host, Port, Email, Password, true))
{
	emailAddress = client.Email;

	client.Connect();

	totalEmails = client.GetEmailCount();
	emails = client.FetchEmailList(((page - 1) *
		NoOfEmailsPerPage) + 1, NoOfEmailsPerPage);
}

Next, we compute the total number of pages the user can browse through.

/* Computing total pages */
int totalPages;
int mod = totalEmails % NoOfEmailsPerPage;

if (mod == 0)
	totalPages = totalEmails / NoOfEmailsPerPage;
else
	totalPages = ((totalEmails - mod) / NoOfEmailsPerPage) + 1;

Next, we iterate through the retrieved list of emails, one by one, and append their information to an HTML table which we will display to the user.

for (int i = 0; i < emails.Count; i++)
{
	Email email = emails[i];

	/* Appending each email to an existing table */
	int emailId = ((page - 1) * NoOfEmailsPerPage) + i + 1;

	TableCell noCell = new TableCell();
	noCell.CssClass = "emails-table-cell";
	noCell.Text = Convert.ToString(emailId);
	
	TableCell fromCell = new TableCell();
	fromCell.CssClass = "emails-table-cell";
	fromCell.Text = email.From;
	
	TableCell subjectCell = new TableCell();
	subjectCell.CssClass = "emails-table-cell";
	subjectCell.Style["width"] = "300px";
	subjectCell.Text = String.Format(DisplayEmailLink,
		emailId, email.Subject);

	//TableCell contentTypeCell = new TableCell();
	//contentTypeCell.CssClass = "emails-table-cell";
	//contentTypeCell.Text = email.ContentType;
	
	TableCell dateCell = new TableCell();
	dateCell.CssClass = "emails-table-cell";
	
	if (email.UtcDateTime != DateTime.MinValue)
		dateCell.Text = email.UtcDateTime.ToString();

	TableRow emailRow = new TableRow();
	emailRow.Cells.Add(noCell);
	emailRow.Cells.Add(fromCell);
	emailRow.Cells.Add(subjectCell);
	//emailRow.Cells.Add(contentTypeCell);
	emailRow.Cells.Add(dateCell);

	EmailsTable.Rows.AddAt(2 + i, emailRow);
}

Next, we create 'Previous Page' and 'Next Page' links.

/* Creating links to previous and next pages */
if (totalPages > 1)
{
	if (page > 1)
		PreviousPageLiteral.Text =
			String.Format(SelfLink, page - 1, "Previous Page");

	if (page > 0 && page < totalPages)
		NextPageLiteral.Text =
			String.Format(SelfLink, page + 1, "Next Page");
}

Lastly, we set a few literals to display on the page.

/* Displaying information to the user */
EmailFromLiteral.Text =
	Convert.ToString(((page - 1) * NoOfEmailsPerPage) + 1);
EmailToLiteral.Text = Convert.ToString(page * NoOfEmailsPerPage);
EmailTotalLiteral.Text = Convert.ToString(totalEmails);
EmailLiteral.Text = emailAddress;

We are now all set to run this page on our ASP.NET web server. But before that, there is one final step left, and that is to enable POP3 access for our mailbox on Gmail.

Enabling POP3 access on Gmail
Follow these steps to enable POP3 access for your email address on Gmail:

Login to your account at Gmail.
Click the "Settings" link on the top right.
From the 'Settings' panel, click the "Forwarding and POP/IMAP" tab.
In the 'POP Download' section, select the option "Enable POP for all mail (even mail that's already been downloaded)"
Make sure 'When messages are accessed with POP' is set to "keep Gmail's copy in the Inbox". This is always a safe bet to keep your emails in the Gmail's inbox and not have them deleted when you retrieve them from Gmail's POP3 server.
Now click 'Save Changes' button to save the changes.

Running the Pop3Client.aspx Page
After placing "Pop3.cs" in the /App_Code sub-folder and Pop3Client.aspx in the / folder of my ASP.NET web application, I ran the Pop3Client.aspx page and got following result:

I hope you were also able to successfully connect to your email's mailbox on the remote server and were able to view the list of emails in it.

Note: It is recommended that you study the code in full to understand how we are connecting to, sending and receiving data over the network. If your POP3 server doesn't support secure access over SSL, you should set the secure argument in the constructor of Pop3Client class to false.

Summary
In this tutorial, we developed code to connect to Gmail's secure POP3 server using SSL. We learned about TcpClient, NetworkStream, SslStream, StreamWriter and StreamReader classes. Using these classes we created our own set of top level classes to make the task of communicating with the server, easier. An ASP.NET page in the end, using the code that we developed, successfully displayed the list of emails retrieved from the server.

In the next tutorial (Displaying Emails from Mailbox on Gmail using POP3 and ASP.NET), we will create an ASP.NET page that will retrieve the actual email from the POP3 server and display it to the user.

Displaying Twitter Updates in a Page by Page fashion with ASP.NET

Faisal Khan — Sat, 20 Jun 2009 06:49:00 GMT

Introduction
In this tutorial, we will develop a technique to retrieve and display a page full of Twitter updates to the user, giving him the ability to move to the next page. The user will be able to browse all the pages using 'Previous Page' and 'Next Page' links. Our technique will be highly efficient as we will be retrieving only enough number of Twitter updates from a given screen name, to display on a single page. Only when the user browses to the next page will another set of Twitter updates be retrieved. At the same time we will cache the results, page by page, giving seamless and smooth performance to the end user.

The code in this tutorial makes use of the code we developed in the previous tutorial: Retrieving Twitter Updates with ASP.NET. We will only be updating TwitterClient.cs a little bit to enable retrieval of Twitter updates page by page. Rest of the files will remain the same. We will develop an ASP.NET page to demonstrate the technique.

I advise that you read this series of tutorials from the beginning and create/place the files as advised in the previous tutorial.

This tutorial is 3rd in the series of tutorials regarding Twitter and ASP.NET:

Introduction to Twitter for ASP.NET Developers
Retrieving Twitter Updates with ASP.NET
Displaying Twitter Updates in a Page by Page fashion with ASP.NET

Demonstration of Twitter Updates being Displayed Page by Page
In this tutorial, we will develop an ASP.NET page that retrieves 5 status updates for a given screen name and displays them on a page. When 'Next Page' link is clicked, next 5 status updates are retrieved and displayed to the user.

Note: 5 is an arbitrary number. You can modify the code to display any number of status updates per page.

Following screen shots show the ASP.NET page displaying Twitter status updates in a page by page fashion:

Let us now go ahead and develop the necessary code.

Developing the Code
We need to update TwitterClient.cs file to accomodate the retrieval of Twitter status updates page by page.

Note: It is absolutely necessary to create and place the files as described in Retrieving Twitter Updates with ASP.NET tutorial. Without which you will not be able to make the changes as described below.

Updated TwitterClient.cs
You should replace the code in TwitterClient.cs with the code given below. The updated sections of the code have been highlighted.

using System;
using System.Collections.Generic;
using System.Configuration;
using System.IO;
using System.Linq;
using System.Net;
using System.Text;
using System.Text.RegularExpressions;
using System.Web;
using System.Xml;
using Stardeveloper.Twitter;

namespace Stardeveloper.Twitter
{
	public class TwitterClient
	{
	public const string StatusesUserTimelineStr =
		"http://twitter.com/statuses/user_timeline.xml?" +
		"screen_name={0}&count={1}&page={2}";

	public TwitterClient()
	{
	}

	protected virtual string FetchXmlFromUrl(string url)
	{
		using (WebClient client = new WebClient())
		using (Stream s = client.OpenRead(url))
		using (StreamReader sr = new StreamReader(s))
		{
			return sr.ReadToEnd();
		}
	}

The code continues on the next page.

protected virtual List<Status>
		CreateStatusObjectsFromXml(string xml, int count)
	{
		List<Status> statuses = new List<Status>(count);

		XmlDocument xmldoc = new XmlDocument();
		xmldoc.LoadXml(xml);

		foreach (XmlNode status in xmldoc.DocumentElement)
		{
		// created_at
		string value = status.SelectSingleNode("created_at").InnerXml;
		DateTime statusCreatedAt = ConvertStrToUtcDateTime(value);

		// id
		value = status.SelectSingleNode("id").InnerXml;
		long statusId = Convert.ToInt64(value);

		// text
		string text = status.SelectSingleNode("text").InnerXml;

		// source
		string source = status.SelectSingleNode("source").InnerXml;

		// truncated
		value = status.SelectSingleNode("truncated").InnerXml;
		bool truncated = Convert.ToBoolean(value);

		// in_reply_to_status_id
		value = status.SelectSingleNode("in_reply_to_status_id").InnerXml;
		long inReplyToStatusId = value != "" ? Convert.ToInt64(value) : 0;

		// in_reply_to_user_id
		value = status.SelectSingleNode("in_reply_to_user_id").InnerXml;
		long inReplyToUserId = value != "" ? Convert.ToInt64(value) : 0;

		// favorited
		value = status.SelectSingleNode("favorited").InnerXml;
		bool favorited = Convert.ToBoolean(value);

		// in_reply_to_screen_name
		string inReplyToScreenName =
			status.SelectSingleNode("in_reply_to_screen_name").InnerXml;

		/* Getting into user element */
		XmlNode user = status.SelectSingleNode("user");

		// id
		value = user.SelectSingleNode("id").InnerXml;
		long userId = Convert.ToInt64(value);

		// name
		string name = user.SelectSingleNode("name").InnerXml;

		// screen_name
		string screenName = user.SelectSingleNode("screen_name").InnerXml;

		// location
		string location = user.SelectSingleNode("location").InnerXml;

		// description
		string description = user.SelectSingleNode("description").InnerXml;

		// profile_image_url
		string profileImageUrl =
			user.SelectSingleNode("profile_image_url").InnerXml;

		// url
		string url = user.SelectSingleNode("url").InnerXml;

		// protected
		value = user.SelectSingleNode("protected").InnerXml;
		bool protectedstr = Convert.ToBoolean(value);

		// followers_count
		value = user.SelectSingleNode("followers_count").InnerXml;
		long followersCount = Convert.ToInt64(value);

		// profile_background_color
		string profileBackgroundColor =
			user.SelectSingleNode("profile_background_color").InnerXml;

		// profile_text_color
		string profileTextColor =
			user.SelectSingleNode("profile_text_color").InnerXml;

		// profile_link_color
		string profileLinkColor =
			user.SelectSingleNode("profile_link_color").InnerXml;

		// profile_sidebar_fill_color
		string profileSidebarFillColor =
			user.SelectSingleNode("profile_sidebar_fill_color")
				.InnerXml;

		// profile_sidebar_border_color
		string profileSidebarBorderColor =
			user.SelectSingleNode("profile_sidebar_border_color")
				.InnerXml;

		// friends_count
		value = user.SelectSingleNode("friends_count").InnerXml;
		long friendsCount = Convert.ToInt64(value);

		// created_at
		value = user.SelectSingleNode("created_at").InnerXml;
		DateTime userCreatedAt = TwitterClient.ConvertStrToUtcDateTime(value);

		// favourites_count
		value = user.SelectSingleNode("favourites_count").InnerXml;
		long favouritesCount = Convert.ToInt64(value);

		// utc_offset
		value = user.SelectSingleNode("utc_offset").InnerXml;
		int utcOffset = Convert.ToInt32(value);

		// time_zone
		string timeZone = user.SelectSingleNode("time_zone").InnerXml;

		// profile_background_image_url
		string profileBackgroundImageUrl =
			user.SelectSingleNode("profile_background_image_url").InnerXml;

		// profile_background_tile
		value = user.SelectSingleNode("profile_background_tile").InnerXml;
		bool profileBackgroundTile = Convert.ToBoolean(value);

		// statuses_count
		value = user.SelectSingleNode("statuses_count").InnerXml;
		long statusesCount = Convert.ToInt64(value);

		// notifications
		string notifications = user.SelectSingleNode("notifications").InnerXml;

		// following
		string following = user.SelectSingleNode("following").InnerXml;

		/* Creating User object */
		User userobj = new User(userId, name, screenName, location,
			description, profileImageUrl, url, protectedstr,
			followersCount, profileBackgroundColor, profileTextColor,
			profileLinkColor, profileSidebarFillColor,
			profileSidebarBorderColor, friendsCount, userCreatedAt,
			favouritesCount, utcOffset, timeZone,
			profileBackgroundImageUrl, profileBackgroundTile,
			statusesCount, notifications, following);

		Status statusobj = new Status(statusCreatedAt, statusId,
			text, source, truncated, inReplyToStatusId, inReplyToUserId,
			favorited, inReplyToScreenName, userobj);

		statuses.Add(statusobj);
		}

		return statuses;
	}

	public Status GetLatestStatusUpdate(string screenName)
	{
		List<Status> statuses = GetStatusUpdates(screenName, 1);

		return statuses[0];
	}

	public List<Status> GetStatusUpdates(string screenName, int count)
	{
		return GetStatusUpdates(screenName, count, 1);
	}

	public List<Status> GetStatusUpdates(string screenName, int count,
		int page)
	{
		string url = String.Format(StatusesUserTimelineStr,
			screenName, count, page);
		string xml = FetchXmlFromUrl(url);

		return CreateStatusObjectsFromXml(xml, count);
	}

	public static DateTime ConvertStrToUtcDateTime(string str)
	{
		string[] arr = str.Split(' ');

		int day, month, year, hour, min, sec;

		switch (arr[1])
		{
			case "Jan":
				month = 1;
				break;
			case "Feb":
				month = 2;
				break;
			case "Mar":
				month = 3;
				break;
			case "Apr":
				month = 4;
				break;
			case "May":
				month = 5;
				break;
			case "Jun":
				month = 6;
				break;
			case "Jul":
				month = 7;
				break;
			case "Aug":
				month = 8;
				break;
			case "Sep":
				month = 9;
				break;
			case "Oct":
				month = 10;
				break;
			case "Nov":
				month = 11;
				break;
			case "Dec":
				month = 12;
				break;
			default:
				throw new FormatException("Unknown month.");
		}

		day = Convert.ToInt32(arr[2]);

		string[] arr1 = arr[3].Split(':');

		hour = Convert.ToInt32(arr1[0]);
		min = Convert.ToInt32(arr1[1]);
		sec = Convert.ToInt32(arr1[2]);

		string offsetSign = arr[4].Substring(0, 1);
		int offsetHours = Convert.ToInt32(arr[4].Substring(1, 2));
		int offsetMinutes = Convert.ToInt32(arr[4].Substring(3, 2));

		year = Convert.ToInt32(arr[5]);

		DateTime dt = new DateTime(year, month, day, hour, min, sec);

		if (offsetSign == "+")
		{
			dt.AddHours(offsetHours);
			dt.AddMinutes(offsetMinutes);
		}
		else if (offsetSign == "-")
		{
			dt.AddHours(-offsetHours);
			dt.AddMinutes(-offsetMinutes);
		}

		return dt;
	}
	}
}

Code is explained on the next page.

Explanation
The URL to retrieve Twitter status updates remains the same but we add an extra querystring parameter to tell Twitter which page we want to retrieve. The screen_name should be set to the screen name of the user whose status updates we want to retrieve. The count parameter should be set to a numeric value of number of status updates you want to show per page. The page parameter should be set to a numeric value of the page whose status updates you want to retrieve, beginning from 1. You don't have to manually set these values. It is only for you to understand what is going on behind the scenes.

public const string StatusesUserTimelineStr =
	"http://twitter.com/statuses/user_timeline.xml?" +
	"screen_name={0}&count={1}&page={2}";

Next we add another GetStatusUpdates() method with the extra option of specifying the page you want to retrieve status updates from.

public List<Status> GetStatusUpdates(string screenName, int count,
	int page)
{
	string url = String.Format(StatusesUserTimelineStr,
		screenName, count, page);
	string xml = FetchXmlFromUrl(url);

	return CreateStatusObjectsFromXml(xml, count);
}

This is it as far as the changes in TwitterClient.cs are concerned.

TwitterAllUpdates.aspx
The logic to pull the the right page and display the status updates properly is handled by this ASP.NET page. Open notepad and copy following code in it and then save it as "TwitterAllUpdates.aspx" in the main folder of your ASP.NET web application:

<%@ Page Language="C#" %>

<%@ OutputCache VaryByParam="page" Duration="1200" %>

<%@ Import Namespace="System.Collections.Generic" %>
<%@ Import Namespace="Stardeveloper.Twitter" %>

<script runat="server">
public const int StatusUpdatesPerPage = 5;
public const string ScreenName = "Stardeveloper";
public const string LinkText =
	"<a href=\"TwitterAllUpdates.aspx?page={0}\">{1}</a>";

protected void Page_Load(object source, EventArgs e)
{
	/* Setting current page and total pages */
	int page = 1;
	int totalPages = -1;

	if (Request.QueryString["page"] == null)
		Response.Redirect("TwitterAllUpdates.aspx?page=1");
	else
		page = Convert.ToInt32(Request.QueryString["page"]);
	
	/* Retrieving status updates in real-time */
	TwitterClient client = new TwitterClient();
	List<Status> statuses = client.GetStatusUpdates(ScreenName,
		StatusUpdatesPerPage, page);

	foreach (Status status in statuses)
	{
		/* Appending each status to existing table */
		TableCell statusCell = new TableCell();
		statusCell.ColumnSpan = 2;
		statusCell.CssClass = "status-updates-table-cell";

		StringBuilder statusBuilder = new StringBuilder();
		statusBuilder.Append(status.FormattedText);
		statusBuilder.Append("<div class=\"status-datetime\">");
		statusBuilder.Append(status.CreatedAt.ToShortDateString());
		statusBuilder.Append(" ");
		statusBuilder.Append(status.CreatedAt.ToShortTimeString());
		statusBuilder.Append("</div>");
		
		statusCell.Text = statusBuilder.ToString();

		TableRow statusRow = new TableRow();
		statusRow.Cells.Add(statusCell);

		StatusUpdatesTable.Rows.Add(statusRow);

		/* Computing total possible pages */
		if (totalPages == -1)
		{
			int statusesCount =
				Convert.ToInt32(status.User.StatusesCount);
			int mod = statusesCount % StatusUpdatesPerPage;
			
			if (mod == 0)
				totalPages = statusesCount / StatusUpdatesPerPage;
			else
				totalPages = ((statusesCount - mod) /
					StatusUpdatesPerPage) + 1;
		}
	}

	/* Creating links to previous and next pages */
	if (totalPages > 1)
	{
		TableCell prevPageCell = new TableCell();
		TableCell nextPageCell = new TableCell();

		prevPageCell.CssClass = "status-updates-table-footer";
		nextPageCell.CssClass = "status-updates-table-footer";
		nextPageCell.Style["text-align"] = "right";

		if (page > 1)
			prevPageCell.Text = String.Format(LinkText, page - 1,
				"Previous Page");

		if (page > 0 && page < totalPages)
			nextPageCell.Text = String.Format(LinkText, page + 1,
				"Next Page");

		TableFooterRow footerRow = new TableFooterRow();
		footerRow.Cells.Add(prevPageCell);
		footerRow.Cells.Add(nextPageCell);

		StatusUpdatesTable.Rows.Add(footerRow);
	}

	/* Setting some values to display to the user */
	ScreenNameLiteral.Text = ScreenName;
	PageLiteral.Text = Convert.ToString(page);
	TotalPagesLiteral.Text = Convert.ToString(totalPages);
}
</script>

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
	"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<html xmlns="http://www.w3.org/1999/xhtml">
<head runat="server">
    <title></title>
    <style type="text/css">
		.status-updates-table { width: 400px; border: solid 1px #444444; }
		.status-updates-table-header { font-family: "Trebuchet MS";
			font-size: 9pt; background-color: #0099B9; color: white;
			border: solid 1px #444444; }
		.status-updates-table-cell { font-family: "Georgia";
			font-size: 9pt; border: solid 1px #666666; padding: 6px; }
		.status-updates-table-footer { border: solid 1px #666666;
			padding: 3px; width: 50%; }
		.status-datetime { float: right; color: #666666; }
		
		a { font-family: "Lucida Sans Unicode",
			"Trebuchet MS"; font-size: 9pt;
			color: #005B7F; }
		a:hover { color:red; }
    </style>
</head>
<body>
    <form id="form1" runat="server">
	<asp:Table ID="StatusUpdatesTable"
		CssClass="status-updates-table" runat="server">
	<asp:TableHeaderRow>
		<asp:TableHeaderCell CssClass="status-updates-table-header"
		ColumnSpan="2">
		<asp:Literal ID="ScreenNameLiteral" runat="server" /> Updates (page 
		<asp:Literal ID="PageLiteral" runat="server" /> of 
		<asp:Literal ID="TotalPagesLiteral" runat="server" />)
		</asp:TableHeaderCell>
	</asp:TableHeaderRow>
	</asp:Table>
    </form>
</body>
</html>

Explanation
The first Page level directive tells ASP.NET compiler that the default language in this page is going to be C#.

<%@ Page Language="C#" %>

Next, an OutputCache directive tells the ASP.NET runtime to cache the output of this page for 20 minutes (20 min x 60 sec = 1200 sec). The VaryByParam attribute is set to "page" which makes sure that a new copy of the output of this page will be cached for each variation of "page" querystring parameter.

<%@ OutputCache VaryByParam="page" Duration="1200" %>

Necessary namespaces are imported. System.Collections.Generic namespace is necessary because we want to use generic List<T> class. Stardeveloper.Twitter namespace is necessary because we want to make use of TwitterClient, Status, and User classes, we developed earlier.

<%@ Import Namespace="System.Collections.Generic" %>
<%@ Import Namespace="Stardeveloper.Twitter" %>

A server-side script tag tells the ASP.NET compiler that this segment is going to contain server-side code.

<script runat="server">
	...
</script>

We define 3 page level constant variables. StatusUpdatesPerPage variable is set to the number of status updates we want to display per page. You can change this value to suit the need of your web application. ScreenName variable is set to the screen name of the user whose status updates you want to retrieve from Twitter. LinkText contains string which contains placeholders which will be set by the code later. This variable will be used to create 'Previous Page' and 'Next Page' links, in the table, under the status updates, later.

public const int StatusUpdatesPerPage = 5;
public const string ScreenName = "Stardeveloper";
public const string LinkText =
	"<a href=\"TwitterAllUpdates.aspx?page={0}\">{1}</a>";

Next, we place all of our important code in the Page_Load() method.

protected void Page_Load(object source, EventArgs e)
{
	...
}

Within the Page_Load() method, we start by declaring two important variables. page should be set to the page whose status updates we want to retrieve from Twitter and display to the user. totalPages should be set to the total number of pages that are available for this screen name. If a user has 100 status updates at Twitter and we want to show 20 updates per page, then totalPages should be set to 5.

If the querystring parameter, "page", is not set, the we set the querystring parameter and redirect the user to the same page. Otherwise, we fetch and set the page variable's value to it.

/* Setting current page and total pages */
int page = 1;
int totalPages = -1;

if (Request.QueryString["page"] == null)
	Response.Redirect("TwitterAllUpdates.aspx?page=1");
else
	page = Convert.ToInt32(Request.QueryString["page"]);

We then create TwitterClient object and retrieve given status updates for a given page from Twitter. Once we have the list of status updates as Status objects, we iterate through them, one by one, and populate our HTML table with its values.

/* Retrieving status updates in real-time */
TwitterClient client = new TwitterClient();
List<Status> statuses = client.GetStatusUpdates(ScreenName,
	StatusUpdatesPerPage, page);

foreach (Status status in statuses)
{
	...
}

For each Status object, we fetch formatted text by accessing the Status.FormattedText property and append to it date & time of the status update. This text is then appended as an HTML row to an HTML table we have already placed as a user control in our ASP.NET page.

/* Appending each status to existing table */
TableCell statusCell = new TableCell();
statusCell.ColumnSpan = 2;
statusCell.CssClass = "status-updates-table-cell";

StringBuilder statusBuilder = new StringBuilder();
statusBuilder.Append(status.FormattedText);
statusBuilder.Append("<div class=\"status-datetime\">");
statusBuilder.Append(status.CreatedAt.ToShortDateString());
statusBuilder.Append(" ");
statusBuilder.Append(status.CreatedAt.ToShortTimeString());
statusBuilder.Append("</div>");

statusCell.Text = statusBuilder.ToString();

TableRow statusRow = new TableRow();
statusRow.Cells.Add(statusCell);

StatusUpdatesTable.Rows.Add(statusRow);

Last bit of code checks and sets the totalPages variable to total number of pages for this screen name using given status updates per page.

/* Computing total possible pages */
if (totalPages == -1)
{
	int statusesCount =
		Convert.ToInt32(status.User.StatusesCount);
	int mod = statusesCount % StatusUpdatesPerPage;
	
	if (mod == 0)
		totalPages = statusesCount / StatusUpdatesPerPage;
	else
		totalPages = ((statusesCount - mod) /
			StatusUpdatesPerPage) + 1;
}

Once all Status objects' formatted text has been appended to the HTML table in our ASP.NET page, we append 'Previous Page' and 'Next Page' links to the HTML table using the logic given below:

/* Creating links to previous and next pages */
if (totalPages > 1)
{
	TableCell prevPageCell = new TableCell();
	TableCell nextPageCell = new TableCell();

	prevPageCell.CssClass = "status-updates-table-footer";
	nextPageCell.CssClass = "status-updates-table-footer";
	nextPageCell.Style["text-align"] = "right";

	if (page > 1)
		prevPageCell.Text = String.Format(LinkText, page - 1,
			"Previous Page");

	if (page > 0 && page < totalPages)
		nextPageCell.Text = String.Format(LinkText, page + 1,
			"Next Page");

	TableFooterRow footerRow = new TableFooterRow();
	footerRow.Cells.Add(prevPageCell);
	footerRow.Cells.Add(nextPageCell);

	StatusUpdatesTable.Rows.Add(footerRow);
}

Lastly, 3 Literal controls' values are set. These values will be displayed to the user.

/* Setting some values to display to the user */
ScreenNameLiteral.Text = ScreenName;
PageLiteral.Text = Convert.ToString(page);
TotalPagesLiteral.Text = Convert.ToString(totalPages);

StatusUpdatesTable is the id of the HTML table that will display the Twitter status updates to the user in our ASP.NET page.

<asp:Table ID="StatusUpdatesTable"
	CssClass="status-updates-table" runat="server">
<asp:TableHeaderRow>
	<asp:TableHeaderCell CssClass="status-updates-table-header"
	ColumnSpan="2">
	<asp:Literal ID="ScreenNameLiteral" runat="server" /> Updates (page 
	<asp:Literal ID="PageLiteral" runat="server" /> of 
	<asp:Literal ID="TotalPagesLiteral" runat="server" />)
	</asp:TableHeaderCell>
</asp:TableHeaderRow>
</asp:Table>

When this ASP.NET page was run on my computer, it produced following output:

Summary
In this tutorial, we updated the code to add paging ability in our ASP.NET web application. We demonstrated the code by developing an ASP.NET page which displayed Twitter status updates, page by page.

In the next tutorial, we will learn how to display "followers' count" on our website for a given screen name.

Retrieving Twitter Updates with ASP.NET

Faisal Khan — Wed, 17 Jun 2009 07:33:00 GMT

Introduction
In this tutorial, we will learn how to retrieve Twitter status updates using Twitter API and ASP.NET. We will develop code that encapsulates calls to Twitter API and provides easy to use .NET classes to retrieve Twitter status updates, programmatically. We will demonstrate the code by developing an ASP.NET control that uses these .NET classes to retrieve and display Twitter status updates on our website.

This tutorial is 2nd in the series of tutorials regarding Twitter and ASP.NET:

Introduction to Twitter for ASP.NET Developers
Retrieving Twitter Updates with ASP.NET
Displaying Twitter Updates in a Page by Page fashion with ASP.NET

Things to know about Twitter API
Before getting started, it is prudent to know a few things about Twitter API.

Twitter API is entirely HTTP based. This means that all you need to do to retrieve or post status updates is to make simple HTTP GET and POST calls, respectively, over the internet.

Twitter API conforms to REST (Representational State Transfer) style of software architecture. You can use URL like http://twitter.com/statuses/user_timeline.format to retrieve status updates of a user. You can choose the desired format of the response by specifying the format in the URL. For instance, to retrieve the status updates in XML, the complete URL will be http://twitter.com/statuses/user_timeline.xml. You have the option of choosing the format between XML, JSON, RSS and Atom.

Twitter API is rate limited. You can only make 100 calls per hour. If you find that this limit is insufficient for the needs of your application, you can request "whitelisting" of your account. To do this you have to submit a request a manually using Whitelisting request form.

You can learn more about the Twitter API on Twitter API Wiki. A good "Getting Started" guide for developers can be found on this link: Things Every Developer Should Know.

Retrieving Twitter Status Updates
The URL to retrieve the status updates of a user on Twitter is: http://twitter.com/statuses/user_timeline.xml. The documentation for what values this URL accepts as querystring parameters and what this URL returns, can be found on Twitter REST API Method: statuses user_timeline.

For the purpose of this tutorial, there are only two querystring parameters of value to us. The first is "screen_name". Its value is the "Screen Name" (not the username) of the user whose status updates you want to retrieve. For instance, to retrieve the status updates of Stardeveloper, we will be specifying "stardeveloper" as the screen_name.

Note: "Screen Name" is the name after http://twitter.com/, in the Twitter URL that you use to view status updates of a user. For example, in the URL http://twitter.com/stardeveloper, stardeveloper is the screen name.

The second paramater is "count", which accepts a numeric value stating how many status updates do we want to retrieve. For instance, if you want only to retrieve the latest status update, set its value to '1'.

To view in your browser what XML string Twitter API will return for this URL, if we set the screen_name to "stardeveloper" and count to 1, click this link: http://twitter.com/statuses/user_timeline.xml?screen_name=stardeveloper&count=1.

In my browser, it returned following XML string:

<?xml version="1.0" encoding="UTF-8"?>
<statuses type="array">
<status>
  <created_at>Sat Jun 13 19:58:29 +0000 2009</created_at>
  <id>2157106641</id>
  <text>New tutorial published: Introduction to Twitter for 
	ASP.NET Developers http://bit.ly/17mAhX</text>
  <source>web</source>
  <truncated>false</truncated>

  <in_reply_to_status_id></in_reply_to_status_id>
  <in_reply_to_user_id></in_reply_to_user_id>
  <favorited>false</favorited>
  <in_reply_to_screen_name></in_reply_to_screen_name>
  <user>
    <id>43659543</id>
    <name>Faisal Khan</name>

    <screen_name>Stardeveloper</screen_name>
    <location></location>
    <description>Chief author and developer at 
		Stardeveloper.com.</description>
    <profile_image_url>http://s3.amazonaws.com/
		twitter_production/profile_images/241256611/
		udh_normal.png</profile_image_url>
    <url>http://www.stardeveloper.com</url>
    <protected>false</protected>

    <followers_count>3</followers_count>
    <profile_background_color>0099B9</profile_background_color>
    <profile_text_color>444444</profile_text_color>
    <profile_link_color>006699</profile_link_color>
    <profile_sidebar_fill_color>F5F1E5</profile_sidebar_fill_color>
    <profile_sidebar_border_color>E2F0F4</profile_sidebar_border_color>

    <friends_count>0</friends_count>
    <created_at>Sun May 31 07:44:58 +0000 2009</created_at>
    <favourites_count>0</favourites_count>
    <utc_offset>18000</utc_offset>
    <time_zone>Islamabad</time_zone>
    <profile_background_image_url>http://static.twitter.com/
		images/themes/theme1/bg.gif</profile_background_image_url>

    <profile_background_tile>false</profile_background_tile>
    <statuses_count>7</statuses_count>
    <notifications></notifications>
    <verified_profile>false</verified_profile>
    <following></following>
  </user>
</status>

In the XML string that was returned, we can clearly see the latest status update at Stardeveloper.com. Twitter also returns lot more other information as XML elements.

Now that we know what URL to send the HTTP GET request to from our code, what values to set in the querystring parameters, and what XML string to expect to be returned; we will be developing the code to do all of this for us programmatically and to display the latest status update on our website using an ASP.NET control which we will develop later.

Code to Retrieve the Twitter Update
The code that we will develop will do the job of retrieving status updates at Twitter in following steps:

Fetch the XML string from the URL, http://twitter.com/statuses/user_timeline.xml?screen_name=stardeveloper&count=1 using System.Net.WebClient class. There will the option of sepcifying screen_name and count querystring parameters, so that you can retrieve the status updates of the user of your choice. We will encapsulate all of this code in TwitterClient class.
Develop separate classes to represent status and user elements in the XML file. The Status and User classes' properties will be named according to the names of their sub-elements in the XML string. For instance, to retrieve the status update from the XML string you will have to access the property of Status object as: Status.Text.
Convert the XML string to a list of Status objects. Each Status object will have a property with the name of User which you can use to access the sub-elements of the user element in the XML string. For instance to retrieve the name of the user element, the code to access is: Status.User.Name.
Once we have the code that fetches the XML string and converts it into easily accessible objects, we will create an ASP.NET control with the name of "TwitterControl" to display this status update on our website. This control will cache the result in memory for 20 mins. We do this so that we only have to access Twitter API not more than 3 times per hour. You will be provided the option of changing the cache duration to suit the needs of your web application.

We should now get started with developing the code. We will first develop User and Status classes, because these classes are needed by the TwitterClient class which we will develop later.

User.cs
Open notepad and copy following text in it. Then save it as "User.cs" in the /App_Code/ sub-folder of your ASP.NET web application.

using System;
using System.Collections.Generic;
using System.Configuration;
using System.IO;
using System.Linq;
using System.Net;
using System.Text;
using System.Text.RegularExpressions;
using System.Web;
using System.Xml;

namespace Stardeveloper.Twitter
{
	public class User
	{
	public long Id { get; protected set; }
	public string Name { get; protected set; }
	public string ScreenName { get; protected set; }
	public string Location { get; protected set; }
	public string Description { get; protected set; }
	public string ProfileImageUrl { get; protected set; }
	public string Url { get; protected set; }
	public bool Protected { get; protected set; }
	public long FollowersCount { get; protected set; }
	public string ProfileBackgroundColor { get; protected set; }
	public string ProfileTextColor { get; protected set; }
	public string ProfileLinkColor { get; protected set; }
	public string ProfileSidebarFillColor { get; protected set; }
	public string ProfileSidebarBorderColor { get; protected set; }
	public long FriendsCount { get; protected set; }
	public DateTime CreatedAt { get; protected set; }
	public long FavouritesCount { get; protected set; }
	public int UtcOffset { get; protected set; }
	public string TimeZone { get; protected set; }
	public string ProfileBackgroundImageUrl { get; protected set; }
	public bool ProfileBackgroundTile { get; protected set; }
	public long StatusesCount { get; protected set; }
	public string Notifications { get; protected set; }
	public string Following { get; protected set; }

	public User(long id, string name, string screenName,
		string location, string description, string profileImageUrl,
		string url, bool protectedstr, long followersCount,
		string profileBackgroundColor, string profileTextColor,
		string profileLinkColor, string profileSidebarFillColor,
		string profileSidebarBorderColor, long friendsCount,
		DateTime createdAt, long favourtiesCount, int utcOffset,
		string timeZone, string profileBackgroundImageUrl,
		bool profileBackgroundTile, long statusesCount,
		string notifications, string following)
	{
		Id = id;
		Name = name;
		ScreenName = screenName;
		Location = location;
		Description = description;
		ProfileImageUrl = profileImageUrl;
		Url = url;
		Protected = protectedstr;
		FollowersCount = followersCount;
		ProfileBackgroundColor = profileBackgroundColor;
		ProfileTextColor = profileTextColor;
		ProfileLinkColor = profileLinkColor;
		ProfileSidebarFillColor = profileSidebarFillColor;
		ProfileSidebarBorderColor = profileSidebarBorderColor;
		FriendsCount = friendsCount;
		CreatedAt = createdAt;
		FavouritesCount = favourtiesCount;
		UtcOffset = utcOffset;
		TimeZone = timeZone;
		ProfileBackgroundImageUrl = profileBackgroundImageUrl;
		ProfileBackgroundTile = profileBackgroundTile;
		StatusesCount = statusesCount;
		Notifications = notifications;
		Following = following;
	}

	public override string ToString()
	{
		return Name;
	}
	}
}

The User class does nothing more than to provide easy access to the sub-elements of user element in the XML string.

Status.cs
Open notepad and copy following text in it. Then save it as "Status.cs" in the /App_Code/ sub-folder of your ASP.NET web application.

using System;
using System.Collections.Generic;
using System.Configuration;
using System.IO;
using System.Linq;
using System.Net;
using System.Text;
using System.Text.RegularExpressions;
using System.Web;
using System.Xml;
using Stardeveloper.Twitter

namespace Stardeveloper.Twitter
{
	public class Status
	{
		protected static Regex UrlReplacement =
			new Regex(@"(https?://([\w-]+\.)+[\w-]+([^ ]*))",
			RegexOptions.Compiled);
		protected static Regex UserNameReplacement =
			new Regex(@"@([\w_]+)",
			RegexOptions.Compiled);

		public DateTime CreatedAt { get; protected set; }
		public long Id { get; protected set; }
		public string Text { get; protected set; }
		public string FormattedText { get; protected set; }
		public string Source { get; protected set; }
		public bool Truncated { get; protected set; }
		public long InReplyToStatusId { get; protected set; }
		public long InReplyToUserId { get; protected set; }
		public bool Favorited { get; protected set; }
		public string InReplyToScreenName { get; protected set; }
		public User User { get; protected set; }

		public Status(DateTime createdAt, long id, string text,
			string source, bool truncated, long inReplyToStatusId,
			long inReplyToUserId, bool favorited,
			string inReplyToScreenName, User user)
		{
			CreatedAt = createdAt;
			Id = id;
			Text = text;
			Source = source;
			Truncated = truncated;
			InReplyToStatusId = inReplyToStatusId;
			InReplyToUserId = inReplyToUserId;
			Favorited = favorited;
			InReplyToScreenName = inReplyToScreenName;
			User = user;

			FormattedText = UrlReplacement.Replace(text,
				"<a href=\"$1\">$1</a>");
			FormattedText = UserNameReplacement.Replace(FormattedText,
				"@<a href=\"http://twitter.com/$1\">$1</a>");

		}

		public override string ToString()
		{
			return Text;
		}
	}
}

Apart from providing access to the sub-elements of status element in the XML string, Status class also lets you access the text of the status update in a formatted manner by accessing the Status.FormattedText property. All the HTTP links and references to usernames are converted to clickable HTML links.

For example, following status update is in plain text:

New tutorial published: Introduction to Twitter for 
	ASP.NET Developers http://bit.ly/17mAhX

It is formatted into following string:

New tutorial published: Introduction to Twitter for 
	ASP.NET Developers http://bit.ly/17mAhX

Notice how the HTTP link was formatted to appear as a clickable link in your browser. This very useful property of Status class will prove very handy when we'd want to display this status update on our website and will need all the links and references to be properly formatted.

Status makes use of 2 regular expressions to convert the HTTP links and references to usernames, to clickable HTML links. These two regular expressions are:

For HTTP links: @"(https?://([\w-]+\.)+[\w-]+([^ ]*))"
For references to usernames: @"@([\w_]+)"

These regular expressions are statically compiled and will be executed at great speed on the server to format the text of the status update.

TwitterClient.cs
Open notepad and copy following text in it. Then save it as "TwitterClient.cs" in the /App_Code/ sub-folder of your ASP.NET web application.

using System;
using System.Collections.Generic;
using System.Configuration;
using System.IO;
using System.Linq;
using System.Net;
using System.Text;
using System.Text.RegularExpressions;
using System.Web;
using System.Xml;
using Stardeveloper.Twitter;

namespace Stardeveloper.Twitter
{
	public class TwitterClient
	{
	public const string StatusesUserTimelineStr =
		"http://twitter.com/statuses/user_timeline.xml?" +
		"screen_name={0}&count={1}";

	public TwitterClient()
	{
	}

	protected virtual string FetchXmlFromUrl(string url)
	{
		using (WebClient client = new WebClient())
		using (Stream s = client.OpenRead(url))
		using (StreamReader sr = new StreamReader(s))
		{
			return sr.ReadToEnd();
		}
	}

	protected virtual List<Status>
		CreateStatusObjectsFromXml(string xml, int count)
	{
		List<Status> statuses = new List<Status>(count);

		XmlDocument xmldoc = new XmlDocument();
		xmldoc.LoadXml(xml);

		foreach (XmlNode status in xmldoc.DocumentElement)
		{
		// created_at
		string value = status.SelectSingleNode("created_at").InnerXml;
		DateTime statusCreatedAt = ConvertStrToUtcDateTime(value);

		// id
		value = status.SelectSingleNode("id").InnerXml;
		long statusId = Convert.ToInt64(value);

		// text
		string text = status.SelectSingleNode("text").InnerXml;

		// source
		string source = status.SelectSingleNode("source").InnerXml;

		// truncated
		value = status.SelectSingleNode("truncated").InnerXml;
		bool truncated = Convert.ToBoolean(value);

		// in_reply_to_status_id
		value = status.SelectSingleNode("in_reply_to_status_id").InnerXml;
		long inReplyToStatusId = value != "" ? Convert.ToInt64(value) : 0;

		// in_reply_to_user_id
		value = status.SelectSingleNode("in_reply_to_user_id").InnerXml;
		long inReplyToUserId = value != "" ? Convert.ToInt64(value) : 0;

		// favorited
		value = status.SelectSingleNode("favorited").InnerXml;
		bool favorited = Convert.ToBoolean(value);

		// in_reply_to_screen_name
		string inReplyToScreenName =
			status.SelectSingleNode("in_reply_to_screen_name").InnerXml;

		/* Getting into user element */
		XmlNode user = status.SelectSingleNode("user");

		// id
		value = user.SelectSingleNode("id").InnerXml;
		long userId = Convert.ToInt64(value);

		// name
		string name = user.SelectSingleNode("name").InnerXml;

		// screen_name
		string screenName = user.SelectSingleNode("screen_name").InnerXml;

		// location
		string location = user.SelectSingleNode("location").InnerXml;

		// description
		string description = user.SelectSingleNode("description").InnerXml;

		// profile_image_url
		string profileImageUrl =
			user.SelectSingleNode("profile_image_url").InnerXml;

		// url
		string url = user.SelectSingleNode("url").InnerXml;

		// protected
		value = user.SelectSingleNode("protected").InnerXml;
		bool protectedstr = Convert.ToBoolean(value);

		// followers_count
		value = user.SelectSingleNode("followers_count").InnerXml;
		long followersCount = Convert.ToInt64(value);

		// profile_background_color
		string profileBackgroundColor =
			user.SelectSingleNode("profile_background_color").InnerXml;

		// profile_text_color
		string profileTextColor =
			user.SelectSingleNode("profile_text_color").InnerXml;

		// profile_link_color
		string profileLinkColor =
			user.SelectSingleNode("profile_link_color").InnerXml;

		// profile_sidebar_fill_color
		string profileSidebarFillColor =
			user.SelectSingleNode("profile_sidebar_fill_color").InnerXml;

		// profile_sidebar_border_color
		string profileSidebarBorderColor =
			user.SelectSingleNode("profile_sidebar_border_color")
				.InnerXml;

		// friends_count
		value = user.SelectSingleNode("friends_count").InnerXml;
		long friendsCount = Convert.ToInt64(value);

		// created_at
		value = user.SelectSingleNode("created_at").InnerXml;
		DateTime userCreatedAt = ConvertStrToUtcDateTime(value);

		// favourites_count
		value = user.SelectSingleNode("favourites_count").InnerXml;
		long favouritesCount = Convert.ToInt64(value);

		// utc_offset
		value = user.SelectSingleNode("utc_offset").InnerXml;
		int utcOffset = Convert.ToInt32(value);

		// time_zone
		string timeZone = user.SelectSingleNode("time_zone").InnerXml;

		// profile_background_image_url
		string profileBackgroundImageUrl =
			user.SelectSingleNode("profile_background_image_url")
				.InnerXml;

		// profile_background_tile
		value = user.SelectSingleNode("profile_background_tile").InnerXml;
		bool profileBackgroundTile = Convert.ToBoolean(value);

		// statuses_count
		value = user.SelectSingleNode("statuses_count").InnerXml;
		long statusesCount = Convert.ToInt64(value);

		// notifications
		string notifications =
			user.SelectSingleNode("notifications").InnerXml;

		// following
		string following = user.SelectSingleNode("following").InnerXml;

		/* Creating User object */
		User userobj = new User(userId, name, screenName, location,
			description, profileImageUrl, url, protectedstr,
			followersCount, profileBackgroundColor, profileTextColor,
			profileLinkColor, profileSidebarFillColor,
			profileSidebarBorderColor, friendsCount, userCreatedAt,
			favouritesCount, utcOffset, timeZone,
			profileBackgroundImageUrl, profileBackgroundTile,
			statusesCount, notifications, following);

		/* Creating Status object */
		Status statusobj = new Status(statusCreatedAt, statusId,
			text, source, truncated, inReplyToStatusId, inReplyToUserId,
			favorited, inReplyToScreenName, userobj);

		statuses.Add(statusobj);
		}

		return statuses;
	}public Status GetLatestStatusUpdate(string screenName)
	{
		List<Status> statuses = GetStatusUpdates(screenName, 1);

		return statuses[0];
	}

	public List<Status> GetStatusUpdates(string screenName, int count)
	{
		string url = String.Format(StatusesUserTimelineStr, screenName,
			count);
		string xml = FetchXmlFromUrl(url);

		return CreateStatusObjectsFromXml(xml, count);
	}

	public static DateTime ConvertStrToUtcDateTime(string str)
	{
		string[] arr = str.Split(' ');

		int day, month, year, hour, min, sec;

		switch (arr[1])
		{
			case "Jan":
				month = 1;
				break;
			case "Feb":
				month = 2;
				break;
			case "Mar":
				month = 3;
				break;
			case "Apr":
				month = 4;
				break;
			case "May":
				month = 5;
				break;
			case "Jun":
				month = 6;
				break;
			case "Jul":
				month = 7;
				break;
			case "Aug":
				month = 8;
				break;
			case "Sep":
				month = 9;
				break;
			case "Oct":
				month = 10;
				break;
			case "Nov":
				month = 11;
				break;
			case "Dec":
				month = 12;
				break;
			default:
				throw new
					FormatException("Unknown month.");
		}

		day = Convert.ToInt32(arr[2]);

		string[] arr1 = arr[3].Split(':');

		hour = Convert.ToInt32(arr1[0]);
		min = Convert.ToInt32(arr1[1]);
		sec = Convert.ToInt32(arr1[2]);

		string offsetSign = arr[4].Substring(0, 1);
		int offsetHours = Convert.ToInt32(arr[4].Substring(1, 2));
		int offsetMinutes = Convert.ToInt32(arr[4].Substring(3, 2));

		year = Convert.ToInt32(arr[5]);

		DateTime dt = new DateTime(year, month, day, hour, min, sec);

		if (offsetSign == "+")
		{
			dt.AddHours(offsetHours);
			dt.AddMinutes(offsetMinutes);
		}
		else if (offsetSign == "-")
		{
			dt.AddHours(-offsetHours);
			dt.AddMinutes(-offsetMinutes);
		}

		return dt;
	}
	}
}

Don't be deterred by the length of the code in this class. The code is very simple to understand. The actual code that does the job of fetching the XML from the URL is this:

protected virtual string FetchXmlFromUrl(string url)
{
	using (WebClient client = new WebClient())
	using (Stream s = client.OpenRead(url))
	using (StreamReader sr = new StreamReader(s))
	{
		return sr.ReadToEnd();
	}
}

We create a WebClient object and call its OpenRead() method providing reference to the string containing the URL to send the HTTP GET request to. Once we have reference to the returned Stream object, we create a new StreamReader object to read this stream from beginning to end. By calling StreamReader.ReadToEnd() method, we get the  string object returned to us by Twitter API. This string object actually is the XML string that we saw earlier in our browser. We will have to convert this raw XML string to Status and User objects to make use of it in our ASP.NET control later.

The method that does the job of converting this raw XML string into Status and User objects is CreateStatusObjectsFromXml():

protected virtual List
	CreateStatusObjectsFromXml(string xml, int count)
{
	List<Status> statuses = new List<Status>(count);
	
	...
	
	return statuses;
}

It creates a list of Status objects. Each Status object corresponds to one status element in the XML string i.e., to one status update on Twitter. Since there can be more than one status elements returned, depending on the count parameter, we create a list of Status objects to incorporate as many status elements as possible.

Now that we need to access the values of elements from the returned XML string, we create an XmlDocument object to make this task easy for us.

XmlDocument xmldoc = new XmlDocument();
xmldoc.LoadXml(xml);

foreach (XmlNode status in xmldoc.DocumentElement)
{
	...
}

As shown in the code above, we then iterate through this XML string, one node at a time, i.e., one status element at a time. In each iteration, we create a User and a Status object. We properly set the properties of these objects to the values fetched from their XML counterparts as shown below:

foreach (XmlNode status in xmldoc.DocumentElement)
{
	// created_at
	string value = status.SelectSingleNode("created_at").InnerXml;
	DateTime statusCreatedAt = ConvertStrToUtcDateTime(value);

	// id
	value = status.SelectSingleNode("id").InnerXml;
	long statusId = Convert.ToInt64(value);

	...

	/* Getting into user element */
	XmlNode user = status.SelectSingleNode("user");

	// id
	value = user.SelectSingleNode("id").InnerXml;
	long userId = Convert.ToInt64(value);

	// name
	string name = user.SelectSingleNode("name").InnerXml;

	...

	/* Creating User object */
	User userobj = new User(userId, name, screenName, location,
		description, profileImageUrl, url, protectedstr,
		followersCount, profileBackgroundColor, profileTextColor,
		profileLinkColor, profileSidebarFillColor,
		profileSidebarBorderColor, friendsCount, userCreatedAt,
		favouritesCount, utcOffset, timeZone,
		profileBackgroundImageUrl, profileBackgroundTile,
		statusesCount, notifications, following);

	/* Creating Status object */
	Status statusobj = new Status(statusCreatedAt, statusId,
		text, source, truncated, inReplyToStatusId, inReplyToUserId,
		favorited, inReplyToScreenName, userobj);

	statuses.Add(statusobj);
}

Above segment of code is a stripped down version of the code that fetches the XML values, creates new User and Status objects, and sets the properties of these objects to fetched XML values. All of this code is run, again and again, for each iteration of the status element in the returned XML string.

All of the calls to previous methods are encapsulated in a very simple to call public method of TwitterClient object; GetStatusUpdates(), which returns a list of Status objects:

public List<Status> GetStatusUpdates(string screenName, int count)
{
	string url = String.Format(StatusesUserTimelineStr, screenName,
		count);
	string xml = FetchXmlFromUrl(url);

	return CreateStatusObjectsFromXml(xml, count);
}

Another method is also provided if you only want to retrieve the latest status update:

public Status GetLatestStatusUpdate(string screenName)
{
	List<Status> statuses = GetStatusUpdates(screenName, 1);

	return statuses[0];
}

Lastly, the code in the method ConvertStrToUtcDateTime(), rips apart the date & time string in the parameter and creates a new DateTime structure and properly sets its properties to the values fetched from the date & time string. This method is very handy in converting UTC date & time strings that Twitter API returns to UTC DateTime objects that our ASP.NET code can understand.
public static DateTime ConvertStrToUtcDateTime(string str)
{
	...
}

Once we have properly placed all the 3 *.cs files and have understood the code, we should move on to create the ASP.NET control that accesses this code to display the status update on our website.

TwitterControl.ascx

Open notepad and copy following text in it. Then save it as "TwitterControl.ascx" in the main folder of your ASP.NET web application.

<%@ Control Language="C#" ClassName="TwitterControl" %>

<%@ OutputCache VaryByParam="none" Duration="1200" Shared="true" %>

<%@ Import Namespace="Stardeveloper.Twitter" %>

<script runat="server">
	protected void Page_Load(object source, EventArgs e)
	{
		TwitterClient client = new TwitterClient();
		Status status = client.GetLatestStatusUpdate("stardeveloper");

		// Setting Status Text
		TwitterLiteral.Text = status.FormattedText;
		
		// Setting Status Date
		DateTime createdAt = status.CreatedAt;
		DateTime utcNow = DateTime.UtcNow;

		if (createdAt.Year == utcNow.Year &&
			createdAt.Month == utcNow.Month &&
			createdAt.Day == utcNow.Day)
			TwitterDateLiteral.Text = "<b>Today</b>";
		else
			TwitterDateLiteral.Text = "on " +
				status.CreatedAt.ToShortDateString();
	}
</script>

<div>Stardeveloper Update</div>
<div>
	<asp:Literal ID="TwitterLiteral" runat="server" /> 
	(<asp:Literal ID="TwitterDateLiteral" runat="server" />)
</div>
<div>
	<a href="http://twitter.com/stardeveloper">See all updates</a>
</div>

The code in the TwitterControl.ascx control fetches the latest status update as a Status object and then sets the text of one of the Literal controls on the page to the value of the Status.FormattedText property. The most important lines of the code have been highlighted above.

TwitterUpdate.aspx

Open notepad and copy following text in it. Then save it as "TwitterUpdate.aspx" in the main folder of your ASP.NET web application.

<%@ Page Language="C#" %>

<%@ Register TagPrefix="sd" TagName="Twitter" Src="~/TwitterControl.ascx" %>

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
	"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<html xmlns="http://www.w3.org/1999/xhtml">
<head runat="server">
    <title></title>
    <style>
    .twitter-update { font-family: "Georgia"; font-size: 8pt;
		width: 300px; padding: 20px; }
    </style>
</head>
<body>
    <form id="form1" runat="server">
    <div class="twitter-update">
		<sd:Twitter runat="server" />
    </div>
    </form>
</body>
</html>

We created a very simple ASP.NET page that uses the control we created earlier to display the latest status update of a user (which in this case is stardeveloper).

When all files that we created above were placed at proper places and TwitterUpdate.aspx page was called, this is the result that I got in my browser:





The same code is used at Stardeveloper.com to display the latest status update. After changing the CSS formatting a little bit, this is how it looks on Stardeveloper.com:





This is it as far as the code to retrieve and display the latest status update from Twitter, was concerned.

Note: You should keep in mind that the code for all the classes that we created in this tutorial will change as we continue to add more code in the upcoming tutorials.

Summary

After introducing Twitter in the last article, we looked at the code to retrieve and display the latest status update of a user from Twitter, in this tutorial. Much of the code was quite simple. After creating the classes, all that is left to make use of them in ASP.NET controls and pages is nothing more than 2-3 lines of code.

In the next tutorial, Displaying Twitter Updates in a Page by Page fashion with ASP.NET, we will learn how to page through Twitter updates, an essential technique to learn if you are interested in displaying more than one status update to the user.

Introduction to Twitter for ASP.NET Developers

Faisal Khan — Sat, 13 Jun 2009 12:49:00 GMT

Introduction
Twitter is a free social networking service that lets its users communicate with each other through short text messages. There is a limit of 140 characters per message. Users can send messages using online website, mobile devices or by sending SMS from their cell phones. These messages, known as &apostweets' are then delivered, by a variety of means, to the users who have subscribed to receive them; known as 'followers'.

Guidelines for sending Messages
In text messages, username of the person you are replying to should start with &apos@' (at) sign e.g., @Stardeveloper. Twitter will convert these usernames in the text messages to clickable links e.g., "Hello @Stardeveloper!".

This tutorial is 1st in the series of tutorials regarding Twitter and ASP.NET:

Introduction to Twitter for ASP.NET Developers
Retrieving Twitter Updates with ASP.NET
Displaying Twitter Updates in a Page by Page fashion with ASP.NET

Connecting with your Audience
Twitter provides another opportunity for ASP.NET developers to connect with their website audience. ASP.NET developers can create accounts with screen names of their websites e.g., Stardeveloper (Faisal Khan). This account can then be used as a micro-blog for the website. Not only should the status updates include important website updates, but should also focus on the interesting things you, as a developer, are working on or can benefit your audience. A link is placed on the website linking to the Twitter profile, giving users the opportunity to view the profile, and if he/she chooses so, to start following this account.

Following image shows how a link to Stardeveloper's (Faisal Khan) profile is placed on Stardeveloper.com's home page:

Following image shows how Twitter API is being used to pull the latest update from Stardeveloper's (Faisal Khan) profile:

We will learn how to use Twitter API in ASP.NET pages in upcoming tutorials.

What is in it for ASP.NET Developers?
While much of this can be done by any developer or webmaster, doing it with ASP.NET allows us to harness the Twitter API and display those updates right on our own websites. We can even create a separate private Twitter account for things like logging website activity. Twitter API can be used to actually create status updates for website usage, errors, files missing, DOS attacks, etc. Then chosen users who work on our website can follow that account and be notified of things going on within the website. Thus separate Twitter accounts can be used for different purposes, all harnessing the Twitter API.

Following is a small list of things you can do with Twitter API in your ASP.NET web application:

Create a Twitter account with status updated related to your website and interesting things you are working on or can benefit your audience e.g., @WebsiteName.
Display status updates right on your website using .NET and the Twitter API.
Create separate Twitter account which only your selected users can follow. You can use a screen name such as @WebsiteName-Development, to share and synchronize the development you are doing on your web application.
Create separate Twitter account which only selected users can follow e.g., @WebsiteName-Log. You can use the Twitter API in your ASP.NET web application to log all the website activity in that Twitter account. You can log messages like memory cache size, articles cached, site stats, error messages, database activity, files not found, etc. Your own developers should follow this account to keep a tap on your web application.
You can use your cell phone and control your website via SMS. Here is how you can do it: Create two separate Twitter accounts e.g., @WebsiteName-Acc1 and @WebsiteName-Acc2. You should develop code in your ASP.NET applicaton to follow (listen for any messages in) @WebsiteName-Acc1 account. You should then send an SMS with text message like "STATS" to @WebsiteName-Acc1 account. Your ASP.NET code should wakeup every minute to check for new messages in this account. As soon as your SMS arrives, the code scans the text and finds the command. It then acts on the command and creates a new text message by retrieving web application stats from the database and sending this message to @WebsiteName-Acc2 Twitter account. Now you should be following this second account. As soon as ASP.NET application delivers a message to this second account, it gets delivered to your cell phone as an SMS with text like "Active Uses: 132, Total Users Today: 3,456, Page Views Today: 9,677". You can add more commands to your liking and control your ASP.NET application from your SMS enabled cell phone.

The Twitter API
Twitter offers a comprehensive, yet simple, API to developers. It is based on simple HTTP GET and POST requests. As ASP.NET developers, we can create a .NET API on top of it to make tasks such as creating and retrieving updates even simpler. On ASP.NET pages, we can then use custom controls to display things like status updates, number of followers, etc. A control that displays the latest status update from Stardeveloper has already been shown in the image above, we will study its code in next tutorial.

Tip: You can follow the Stardeveloper account on Twitter or you can subscribe to my newsletter (at the top left of this page) to be notified of the latest tutorials, articles and updates at Stardeveloper.com.

In the next tutorial, we will learn how to use Twitter API to retrieve latest status updates in our ASP.NET application. Rest assured it will be simple, yet interesting. It will give you new ideas for your ASP.NET web application. In the mean time you should create your own Twitter account, if you don't have one, as the things are only going to get more interesting from here onwards. Good luck!

Validating an XML Document using DTD in ASP.NET

Faisal Khan — Fri, 05 Jun 2009 03:36:00 GMT

Introduction
In this tutorial, we will learn how to validate an XML document using DTD. We will develop an ASP.NET page to demonstrate the code. After you are finished reading this tutorial, you will be proficient in writing C# code that validates XML documents using DTD.

Following topics will be covered in this tutorial:

Well-formedness vs Validation.
Validating XML Document using DTD.

This tutorial is 6th one in the series of tutorials about XML and ASP.NET:

Introduction to XML
Reading XML Files with ASP.NET
Introduction to XPath for ASP.NET Developers
Creating new XML Document in ASP.NET Programmatically
Introduction to DTD (Document Type Definition)
Validating an XML Document using DTD in ASP.NET

We will first refresh ourselves with basic XML concepts of well-formedness and validation, and later develop the code to validate XML document using DTD.

Well-formedness vs Validation
Well-formedness means that an XML document obeys the rules that are necessary for an XML document to be, frankly, an XML document. What this means is that there should be an XML declaration, there should be a root element, the name of element in opening tag must match with the name in the closing tag, and so on. We covered a good part of it in the first article on introduction to XML.

Validation on the other hand is an entirely different thing. An XML document is valid only if it conforms to the constraints declared in DTD. Put it simply, if the structure of an XML document is exactly as it is described in DTD, then that XML document is valid (in reference to that DTD), otherwise it is not.

Validating XML Document using DTD
We will use the sample XML document and DTD file from the article on Document Type Definition. We will then write code to validate that XML document using that DTD.

Sample XML Document
Following is the XML document that we will be using for demonstration:

<?xml version="1.0" encoding="UTF-8"?>

<!DOCTYPE article SYSTEM "article.dtd">

<article>
	<title>Sample XML Document.</title>
	<author email="hidden@xyz.com">Faisal Khan</author>
	<body>This is a sample XML Document.</body>
</article>

Copy above code in a new text file and save it as "article.xml" in the /App_Data folder of your ASP.NET application.

Sample DTD File
Following is the DTD file that we will be using for demonstration:

<!ELEMENT article (title, author?, body)>
<!ELEMENT title (#PCDATA)>
<!ELEMENT author (#PCDATA)> 
<!ELEMENT body (#PCDATA)>

<!ATTLIST author email CDATA #REQUIRED>

Copy above code in a new text file and save it as "article.dtd" in the /App_Data folder of your ASP.NET application.

If you look at the XML document and DTD file closely, you'll see that the XML document conforms to the constraints in the DTD file, and is thus, a valid XML document with respect to that DTD file. To do this manual verfication programmatically, we'll have to write code. Luckily, in .NET, writing code that validates an XML document is quite simple.

ASP.NET Page that Validates an XML Document using DTD
Copy following code in a new text file and save it as "ValidateXml.aspx" in your ASP.NET application:

<%@ Page Language="C#" %>

<%@ Import Namespace="System.Xml" %>
<%@ Import Namespace="System.Xml.Schema" %>

<script runat="server">
	private bool valid = true;
	private StringBuilder msgs = new StringBuilder();

	protected void Page_Load(object source, EventArgs e)
	{
		string fullPathToXmlFile = Server.MapPath("~/App_Data/article.xml");

		XmlReaderSettings settings = new XmlReaderSettings();
		settings.ProhibitDtd = false;
		settings.ValidationType = ValidationType.DTD;
		settings.ValidationEventHandler +=
			new ValidationEventHandler(ValidationHandler);

		XmlReader reader = XmlReader.Create(fullPathToXmlFile, settings);

		while (reader.Read())
			;

		reader.Close();
		reader = null;

		if (valid == true)
			StatusLiteral.Text = "Is Valid";
		else
			StatusLiteral.Text = "Is NOT Valid";

		MessagesLiteral.Text = msgs.ToString();
	}

	protected void ValidationHandler(object sender, ValidationEventArgs args)
	{
		valid = false;

		msgs.Append(args.Message);
		msgs.Append("<br />");
	}
</script>

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
	"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<html xmlns="http://www.w3.org/1999/xhtml">
<head runat="server">
    <title></title>
    <style type="text/css">
    body { font-family: Verdana; font-size: 9pt; }
    pre { font-family: Lucida Console; font-size: 8pt; }
    </style>
</head>
<body>
    <form id="form1" runat="server">
    <div><asp:Label ID="StatusLiteral" runat="server" /></div>
    <pre><asp:Literal ID="MessagesLiteral" runat="server" /></pre>
    </form>
</body>
</html>

Explanation
Since our code is going to be in the language, C#, we declare it as the first line in our ASP.NET page.

<%@ Page Language="C#" %>

Next, we import the namespaces that we will be using in validating the XML document.

<%@ Import Namespace="System.Xml" %>
<%@ Import Namespace="System.Xml.Schema" %>

We place our code in script tags.

<script runat="server">
	...
</script>

Before we look at the code, we should see what server side controls we have laid in-between the HTML to display status messages to the user. StatusLiteral will display the success or failure of the validation process in one sentence. MessagesLiteral will display all the failure messages, if any, to the user in detail.

<asp:Label ID="StatusLiteral" runat="server" />
<asp:Literal ID="MessagesLiteral" runat="server" />

Now, we will delve ourselves into the real code that does the validation. We are using two instance level variables to hold the data for this request. The first variable, valid, will be set to false by a segment of code later if the XML document turns out to be invalid. The second variable, msgs, will be appended error messages one after the other (again, if any), and displayed to the user in the end.

private bool valid = true;
private StringBuilder msgs = new StringBuilder();

We have placed our code in the Load event of the Page object. This way, every time the page loads, our code will be run.

protected void Page_Load(object source, EventArgs e)
{
	..
}

The first thing we do is to get the full physical path to the 'article.xml' file, using Server.MapPath() method.

string fullPathToXmlFile = Server.MapPath("~/App_Data/article.xml");

Note: There is no direct reference to the 'article.dtd' file in the code. It will be loaded by the objects we create later, automatically from the DOCTYPE declaration given in the 'article.xml' file.

Next, we create an instance of XmlReaderSettings object. This object contains the settings we want to use later when are about to the read the XML document. This object is required to create the XmlReader object later. In the settings, we set its properties so that it knows we want to use DTD to validate an XML document. We also set a ValidationEventHanlder method. We do this so that every time an error is thrown if the XML document contains invalid segments, we can set the valid property to false and append the error message to the msgs variable.

XmlReaderSettings settings = new XmlReaderSettings();
settings.ProhibitDtd = false;
settings.ValidationType = ValidationType.DTD;
settings.ValidationEventHandler +=
	new ValidationEventHandler(ValidationHandler);

The working of this event handler has already been explained.

protected void ValidationHandler(object sender, ValidationEventArgs args)
{
	valid = false;

	msgs.Append(args.Message);
	msgs.Append("<br />");
}

It is this XmlReader object which we create that does the actual job of validating the XML document against that DTD. We create it by calling a static method of XmlReader object and passing as its arguments, the reference to full physical path of the XML document and a reference to the XmlReaderSettings object which we created earlier.

XmlReader reader = XmlReader.Create(fullPathToXmlFile, settings);

Once we have created the XmlReader object, we enter a while loop to repeatidly call its Read() method. In each iteration, the XmlReader object examines each node of the XML document. Any error messages that generated during the call to this method are handled by our event handler method we set earlier and are saved to be displayed to the user, later.

while (reader.Read())
	;

Once all the nodes have been examined, we close the XmlReader object.

reader.Close();
reader = null;

We set the text to be displayed to the user depending if the valid variable is true or false. This variable will be false if even a single error was encountered during validating the XML document.

if (valid == true)
	StatusLiteral.Text = "Is Valid";
else
	StatusLiteral.Text = "Is NOT Valid";

Error messages, if any, are set to the appropriate control, to be displayed to the user.

MessagesLiteral.Text = msgs.ToString();

This is pretty much it. The code is extremely simple and easy to understand and use. When this ASP.NET page was run with the XML document and DTD file as given in this tutorial above, it produced following result:

This XML document was a valid XML document to begin with, and we validated it against that DTD, programmatically. Now let us see what happens if we change the XML document a bit, by adding one extra element, so that the XML document becomes non-conforming. After adding one extra element, the XML document looks like this:

<?xml version="1.0" encoding="UTF-8"?>

<!DOCTYPE article SYSTEM "article.dtd">

<article>
	<title>Sample XML Document.</title>
	<author email="hidden@xyz.com">Faisal Khan</author>
	<body>This is a sample XML Document.</body>
	<test>Hello</test>
</article>

Now, when I accessed 'ValidateXml.aspx' from my browser, it produced following result:

You can see that the ASP.NET page was able to successfully identify the document as invalid. The detailed error message shows that an extra 'test' element was found in the XML document which was not declared in the DTD.

Summary
In this tutorial, we learned the code to programmatically validate an XML document with DTD.

Stardeveloper

SPF Record Syntax

What should be included in an SPF record?

Introduction to SPF record syntax

Quantifiers

Mechanisms

ip4 mechanism

ip6 mechanism

a mechanism

mx mechanism

include mechanism

ptr mechanism

exists mechanism

all mechanism

SPF record generating tips

Conclusion

What is Sender Policy Framework?

How does SPF work?

An example SPF record

Explanation

How to check the SPF record for a domain name?

Should you have an SPF record for your website?

Conclusion

Read Next

Convert IP Address to Long

Do not store IP Addresses as strings

Solution: Convert IP Addresses to Long values

Converting IP Addresses to Long Values and Back in C#

Converting IP Addresses to Bigint Values and Back in T-SQL

Final Words

Introduction to Developing HTTP Modules in ASP.NET

Displaying Emails from Mailbox on Gmail using POP3 and ASP.NET

Connecting to Mailbox on Gmail and Fetching List of Emails using POP3 and ASP.NET

Displaying Twitter Updates in a Page by Page fashion with ASP.NET

Retrieving Twitter Updates with ASP.NET

Introduction to Twitter for ASP.NET Developers

Validating an XML Document using DTD in ASP.NET