Stardeveloper

Real-World example of load time improvement using HTTP static resource caching

Faisal Khan — Thu, 02 Sep 2010 11:28:00 GMT

If you read the article about using Expires and Max-Age headers to improve website load times, then you must be wondering what if any improvement this may have in the real-World. In this article, I am going to provide you with statistics of the load time of a sample page on Stardeveloper.com, and how using the HTTP static resource caching technique, discussed in that article, resulted in load time improvement by an order of magnitude. So keep on reading!

Tools required to measure the load time of any web page

The first thing I want to share is the tool I use to measure the load time of a web page. The tool is named Fiddler and it works on Windows. It is obviously free to use. I use browsers like Safari, Firefox and Google's Chrome, and Fiddler works with all of them. To get started, download and install Fiddler from the link on their website. Then open any browser and start browsing the internet the way you normally do. Then when you switch the window back to Fiddler, you will see the details of the parent request and the child requests attached with every request you send using the browser. You can view details of a single page, like its request and response headers, load time, bytes sent and received, etc. or select a parent page, right-click, and select the child requests attached to that request, and then you can view the timeline for all the child requests. It is a very useful program and you will love it the moment you start using it.

Load time improvements

The sample page whose load time I am going to share in this article is actually the article itself about using Expires and Max-Age headers to turn on static resource caching in ASP.NET. I am going to first share with you the load time and bytes sent/received when that page was accessed the first time after clearing the browser's cache. The IIS server that Stardeveloper.com runs on will send a Max-Age header for every static resource requested for that page with an expiry date set to one year in the future. This makes sure that the browser will cache the static resources like CSS, JavaScript and images on its hard disk. Then I will share you the stats when the same page was accessed the second time. And you will see how much improvement in load time it resulted in and how wonderful a small change like enabling static resource caching using web.config file really is in ASP.NET.

Load time of the sample page the first time it was requested

The first time I requested the sample page, the HTTP request resulted in 20 child requests once that page began to load, obviously to load all the CSS, JavaScript and image files. It took a total of 12.8 seconds. Total bytes received were ~217 KB.

Load time of the sample page the second time it was requested

The second time when I requested the same page, the HTTP request resulted in only 7 child requests (13 less than the first time). The time it took to load the full page was only just a little over 5 seconds (approx. 7 seconds less than the first time). Total bytes received were also less by a good order of magnitude, they were only about 45 KB (compared to 217 KB the first time).

What if you had not enabled static resource caching in IIS?

In that case when the page was requested a second time, the browser would have sent If-Modified-Since request for every static resource (CSS, JavaScript and images) it encounterd on the web page. This would have inflated the request count from 7 to 20. The server in return would have sent a 304 'Document not modified' header back saying that the documents have not been modified sine they were requested the first time. While this may not have inflated the bytes received too much, but this overall process would have resulted in at least 13 extra round trips to and from the server which would have slowed the loading of the web page.

Conclusion

It will take just a few minutes to setup static resource caching using Expires or Max-Age headers as described in the previous article using just the web.config file if you are using ASP.NET. If you weren't convinced when you read that article the first time, I hope this article will convince you enough to make the necessary changes. When you have made the changes, you can confirm the improvement in load time using Fiddler. Some times 20% of small changes can make 80% of the difference in load time and this technique is one of those 20% changes. Good luck!

Setting Expires and Max-Age Headers for static resources in ASP.NET

Faisal Khan — Tue, 31 Aug 2010 12:41:00 GMT

As developers, we are constantly striving to improve the performance of our web applications. While we can do everything on the server-side from improved database design and access code to caching using built-in ASP.NET Cache object and OutputCache directives in the user controls and ASP.NET pages, tests conducted show that it does not result in more than 5-12% improvement in the load time of a web page. The majority of the time to load a web page is taken by static resources like CSS, JavaScript and image files. You can use browser add-ons like Page Speed and YSlow in Firefox browser, or a stand-alone application like Fiddler, to measure the load time of your website. In this article, we will focus on one simple yet very effective technique to speed up the load time of your website, and that is to use couple of handly HTTP headers to tell client browser or the intermediary proxy server to cache static resources from our web server. It will not take more than a few minutes for you to set them so please just focus a little on what is to follow as this will have good effect on the performance of your website.

What is Caching and why does it matter?

When a browser sends a request for a web page, that web page sends HTML markup back to the client browser. That HTML markup among the actual textual content, also contains markup to load accompanying CSS, JavaScript and image files. A very simple HTML page can make a browser go fetch, each time it is requested to load, all the heavy CSS, JavaScript and images stuff. Since the browser knows that many of these resources are static and may not have changed between different requests, the next time it is confronted with a request to load these resources, by the same or a different page on the same server, it checks with the server using If-Modified-Since HTTP header to make sure to only load these resources if they have changed between requests. This is highly efficient stuff and is almost always going-on on your ASP.NET website whether you know it or not. So how can you improve it further?

Introduction to Expires and Max-Age headers

You can use Expires or Max-Age HTTP headers to tell the browser the first time it requests a static resource, like CSS, JavaScript or image file, to cache it for at least like 7 days, a month or an year. Most experts recommend sending an expiry date for your static content to an year in the future. According to w3.org, an expiry date for static content must not be set to more than one year in the future. When you set an expiry date using Expires or Max-Age headers, the differences between the two we'll look in a moment, the browser will cache these resources on the client's hard disk and will not even send a If-Modified-Since header to the server and will simply load the static files from the browser's local cache right away. This saves at least one extra round trip of request/response from and to the server for every static resource. So if there are 12 CSS, JavaScript and image files, this saves 12 round trips. To the end-user, it appears as if the website is very fast and the document has loaded instantaneously. This speeds up the overall load time of the whole website, quite a bit.

How to make changes to static files whose expiry dates have been set to far in the future?

One thing to mention here is that although setting expiry date of the static resources to a far date in the future is very handy, you should keep in mind that if you want to change the static file on the server and want all the user's to load the newer and updated CSS, JavaScript or image file, then you should simply use a new name for that static file. For instance, if you had an image with the name of logo.png in your website to display the logo of your website, and now you want to update or change it to a new one, then simply create a new file and save it as logo-1.png and edit the HTML markup in your ASP.NET web application to refer to the new logo image file. Simple technique!

How to set Expires and Max-Age headers in ASP.NET?

The discussion so far about caching of static resources and HTTP headers applies equally to all web servers and is not specific to IIS. But what we will learn now is very specific to ASP.NET. We will now learn how to set Expires and/or Max-Age headers for static resources on our ASP.NET websites and web applications using no programming at all. We will simply add a few lines in web.config file of our ASP.NET web application and we'll be done.

Setting Max-Age header

To set a Max-Age header for all static resources on your web server, add following lines in your ASP.NET web application's web.config file:

</system.webServer>
	<staticContent>
		<clientCache cacheControlMode="UseMaxAge" cacheControlMaxAge="365.00:00:00"/>
	</staticContent>
</system.webServer>

The above markup tells the IIS to send Max-Age HTTP header when any static resource (CSS, JavaScript, images, etc.) is requested from the server. To use Max-Age header, you have to set the cacheControlMode attribute to "UseMaxAge". The value of this header is a relative timespan value set to a far date and time in the future. In the example markup given above, we have set the expiry date and time to 365 days in the future. Had you wanted to just set the expiry date to, let's say, 7 days, you would have set the cacheControlMaxAge attribute to "7.00:00:00". The format for this attribute is d.hh.mm:ss where d is day(s), hh is hours, mm is minutes and ss is seconds.

Setting Expires header

To set an Expires header for all static resources on your web server, add following lines in your ASP.NET web application's web.config file:

</system.webServer>
	<staticContent>
		<clientCache cacheControlMode="UseExpires" httpExpires="Thu, 01 Sep 2011 12:00:00 UTC"/>
	</staticContent>
</system.webServer>

To use Expires header, you have to set the cacheControlMode attribute to "UseExpires". The value of the Expires header is a fixed date and time in the future, unlike Max-Age in which the value is a relative date and time (timespan) in the future. You also have to be cautious in correctly entering the date and time value for the Expires header as given the in the sample above. Any mistake in the format, and your Expires header will be rendered useless.

Which is better? Expires or Max-Age?

Expires was specified in HTTP 1.0 specification as compared to Max-Age, which was introduced in the early HTTP 1.1 specification. The value of the Expires header has to be in a very specific date and time format, any error in which will make your resources non-cacheable. The Max-Age header's value when sent to the browser is in seconds, the chances of any error happening in which is quite less.

Since you can specify only one of the two headers in your web.config file, I'd suggest going with the Max-Age header because of the flexibility it offers in setting a relative timespan from the present date to a date in the future. You can basically set and forget, as compared to the case with Expires header, whose value you will have to remember to update at least once every year. And if you set both headers programmatically from within your code, know that the value of Max-Age header will take precedence over Expires header. So, something to keep in mind there as well.

Now that you are up-to-speed in setting up HTTP cache headers for your static resources, on the next page, I will provide you real-World statistics of using this technique to improve load time of a web page. You will be surprised to see the results.

Setting up SPF Record for BlackBerry Internet Service

Faisal Khan — Tue, 27 Apr 2010 10:18:00 GMT

It is imperative for a person who owns a domain name or runs a website to understand what is SPF (Sender Policy Framework) and how it works to provide control to the webmaster to explicitly specify which mail servers can send emails on the behalf of this domain name. Once, you have understood Sender Policy Framework, you should familiarize yourself with SPF record syntax. Learning its syntax will allow you to setup an SPF record for your website.

In this article, we will talk about a specific problem that many people who use popular BlackBerry devices may encounter when they have specified SPF records for their domain names. Every BlackBerry user uses either BIS (BlackBerry Internet Service) or BES (BlackBerry Enterprise Server) to setup one or more email addresses with his BlackBerry device. BES is beyond the scope of this article. For personal users who own domain names and have setup SPF records for their websites and have also setup their domains' email addresses with their BlackBerry devices, this article is for them.

Sending Emails with BlackBerry

When you setup your email address with BlackBerry Internet Service (BIS) and also have valid SPF records for your website's domain name, problem arises when you send email from your BlackBerry device. That email will most probably end up being sent from BlackBerry's SMTP servers. And since you did not include BlackBerry's SMTP servers in your domain name's SPF records, it becomes uncertain if the email will make it to the destination mail box.

This of course implies that every email that you send from your BlackBerry device is important to you. And since you are using BlackBerry, it is fair to assume that it is the case.

So, what is the solution?

Including BlackBerry SMTP Servers in your domain's SPF Records

The solution is to enlist or include all BlackBerry SMTP servers in the SPF record of your domain name if they are not already there.

This sounds fairly simple but it is not. We are not provided a comprehensive list of all SMTP servers that BlackBerry uses to send your emails. So what to do? The solution that I have come up after doing my bit of research online is to include SMTP servers from North America (NA) and Europe (EU) used by BlackBerry. There are 2 domain names to include in your SPF records as given below:

BIS NA (North America) BlackBerry

For North American SMTP servers, use the following include directive to include all SMTP servers:

include:srs.bis.na.blackberry.com

BIS EU (Europe) BlackBerry

For European SMTP servers, use the following include directive to include all SMTP servers:

include:srs.bis.eu.blackberry.com

Final Sample SPF Record

Here is a sample SPF record for a domain that has included BlackBerry SMTP servers from both NA and EU:

"v=spf1 a include:srs.bis.na.blackberry.com include:srs.bis.eu.blackberry.com ~all"

If you find it hard to understand the SPF syntax given above, please follow the tutorial on SPF record syntax to learn more about it.

I am not aware of any SMTP servers for BlackBerry out of North America and Europe. If you think you can improve the SPF record given above to more comprehensively include all BlackBerry SMTP servers, then feel free to share the information with me. I will keep this article updated with the latest information that I have on this topic.

Conclusion

In this article we learned how to setup SPF records to include BlackBerry SMTP servers from both North America and Europe so that the emails that are sent from our BlackBerry devices will be delivered to the destination mail box.

SPF Record Syntax

Faisal Khan — Fri, 26 Feb 2010 06:51:00 GMT

Sender Policy Framework prevents fraudulent abuse of your domain name in emails sent by others claiming to be sent from your domain name. This cuts down the spam which was originating from spammers who were using your domain name to bypass filters on other users' email addresses.

A domain name which does not have an SPF record set in its DNS records is open to spammers to use the address of this domain name in the 'From' field of their emails. They can then send thousands of these unsolicited emails to end users. And by using the repute of that domain name, can entice users to click on links which lead to their products, and to websites that fraudulently seek confidential and sensitive information (phishing attacks).

It is thus very important that you understand what is Sender Policy Framework (SPF) and how it works.

SPF works by making you explicitly declare a list of hosts and IP addresses from which legitimate emails can originate for your domain name. This is called creating a white list. You can also create a black list by explicitly declaring hosts and IP addresses which cannot send emails for your domain name. Generally, only a white list is created. Any other host or IP address not present in the white list is set to fail or soft fail, as the case may be.

What should be included in an SPF record?

An SPF record should comprehensively list all the hosts and IP addresses from which your website or domain name sends emails. This generally means an A record (IP address of your website), SMTP Servers, IP addresses from your company's network from which an email may originate, and any third party mailers, e.g., Google Apps, BlackBerry Internet Service, etc.

When you have created such a list, you should use SPF record syntax to publish this list in the DNS records of your domain name.

Introduction to SPF record syntax

SPF record syntax follows standard DNS syntax to declare a policy using a list of mechanisms. A mechanism is any rule you specify as part of your SPF record to allow or disallow a host or IP address range. You can specify zero or more mechanisms, each delimited by white space. All of these mechanisms constitute a policy for your domain name. An SPF record always begins with v=spf1 and ends with ~all, as shown below:

"v=spf1 ~all"

All DNS records including SPF are enclosed in double quotes. v=spf1 tells the SPF record validator that this is an SPF record and that it belongs to SPF version 1.

For practical purposes, always start from an SPF record like above, and then add mechanisms, relating to your hosts and IP addresses, one by one.

Quantifiers

Before we delve ourselves into mechanisms, let me touch a bit about quantifiers. If you remember the discussion above, I talked about white listing or black listing hosts and IP addresses. To allow (white list) a host or IP address, we append a '+' letter at its beginning e.g., +a. To disallow (black list) a host or IP address range, we append a '-' letter at its beginning e.g., -a. An a mechanism in these examples references the A record for this domain name. Between allowing and disallowing are two other quantifiers; '~' and '?'. An '~' quantifier (like the one at the beginning of all mechanism in our first example of an SPF record) means "soft fail". Soft fail means to accept the emails but put them in the 'Spam' folder and not the user's inbox. Soft fail is most often used in combination with all mechanism. An all mechanism is very powerful as it catches all the hosts and IP addresses that have not been explicitly declared in the preceding mechanisms. Finally, a '?' quantifier tells the SPF record validator to make this mechanism "neutral". A neutral mechanism is generally interpreted as allowance for that mechanism's host or IP address.

For practical purposes, I'd suggest that you limit yourself to '+' and '-' to allow and disallow any host or IP address range, and use '~' for the all mechanism.

Here is the recap of quantifiers you can use with the mechanisms in your SPF record:

'+' - pass
'?' - neutral (generally means pass)
'~' - soft fail (accept the email but put it in 'Spam' folder)
'-' - fail

It is not necessary to append '+' (pass) quantifier with mechanisms as it is assumed that a mechanism with no explicit quantifier has a '+' letter prepended. For example, following two SPF records mean one and the same thing:

"v=spf1 a ~all"

"v=spf1 +a ~all"

Mechanisms

A mechanism (rule) allows you to declare a single IPv4 or IPv6 IP address or an IP address range, a host, an SPF record from another host, an a record, an mx (Mail Exchanger) record, and a ptr record.

Following is the list of all mechanisms available:

ip4
ip6
a
mx
include
ptr
exists
all

We will look at all of them, one by one.

ip4 mechanism

An ip4 mechanism is used to include an IPv4 address or network range in an SPF record. A network range is specified by first declaring the IPv4 network address and then after a slash ('/'), is declared the length of the network range. All ip4 mechanisms start with "ip4:".

For example, following SPF record declares a single IPv4 address:

"v=spf1 ip4:127.0.0.1 ~all"

Following SPF record declares an ip4 mechanism to include all IP addresses in the 127.0.0.1-127.0.0.255 range:

"v=spf1 ip4:127.0.0.1/8 ~all"

Following SPF record declares an ip4 mechanism to include all IP addresses in the 127.0.0.1-127.0.255.255 range:

"v=spf1 ip4:127.0.0.1/16 ~all"

Since no quantifier was prepended with the ip4 mechanisms in the examples above; a '+' quantifier is assumed. This means that any email originating from these IP addresses should be allowed to pass successfully.

ip6 mechanism

An ip6 mechanism is used to include an IPv6 address or network range in an SPF record. All ip6 mechanisms start with "ip6:".

Following SPF record declares a single IPv6 address:

"v=spf1 ip6:2001:db8:85a3::8a2e:370:7334 ~all"

Following SPF record declares an ip6 mechanism to include all IPv6 addresses in the 2001:db8:85a3::8a2e:0000:0000-2001:db8:85a3::8a2e:FFFF:FFFF range:

"v=spf1 ip6:2001:db8:85a3::8a2e:370:7334/96 ~all"

a mechanism

An a mechanism allows us to reference (and include) the IP address(es) of the current domain for which this SPF record exists. Every domain name almost always has an A record which resolves to the IP address it is hosted at. Larger websites have more than one IP address that an A record can resolve to. By including and allowing an a record in the SPF record of your domain name, you are basically allowing any email originating from your domain name to pass.

For example, following SPF record includes an A record to allow any emails originating from the IP address of this domain name to pass:

"v=spf1 a ~all"

If you have another domain name and want to include its A record so that any email originating from the IP address of that domain name is also passed for your domain name, then use this syntax:

"v=spf1 a:anotherdomain.com ~all"

mx mechanism

An mx mechanism allows us to include the MX (Mail Exchanger) servers of this domain name in the SPF record. An MX record in the DNS records of a domain name points to one or more mail servers that are entitled to receive emails for that domain name. For example, if a person sends an email to an address at Stardeveloper.com, he will most probably send it via an SMTP relay server. That SMTP relay server then fetches the MX record of Stardeveloper.com and resolves it to an IP address. Then it connects to that IP address on the port 25 to deliver that email for Stardeveloper.com. As the admin for Stardeveloper.com, I can change the MX records and point them to any mail server of my choosing which I have configured to receive emails for my domain name.

Now that we know what an MX record is, let us see how we declare an mx mechanism to allow all mail servers who are registered to receive emails for our domain names to also be given a pass so that if any outgoing email originates from them, it will be passed.

Following SPF record includes an mx mechanism to allow any emails originating from the MX servers of this domain name to pass:

"v=spf1 mx ~all"

You can also include MX records for another domain name like the syntax we saw earlier for A record:

"v=spf1 mx:anotherdomain.com ~all"

include mechanism

An include mechanism is used to include a policy (SPF record) from another host or domain name. You will use this if your domain's emails also go through 3rd party SMTP servers over which you have no control. The 3rd party has its own policy (SPF record) for their SMTP servers/domain names. By including their SPF record in the SPF record of your domain name, you are allowing all emails to pass which are allowed to pass by that domain name. In other words, all emails originating from 3rd party's SMTP servers will be passed.

Following SPF record includes a policy (SPF record) from a third party's domain name:

"v=spf1 include:anotherdomain.com ~all"

Following is a real World example of an include mechanism to allow emails going through Google's SMTP servers to be passed:

"v=spf1 include:aspmx.googlemail.com ~all"

If you plan to use Google Apps with your domain name, then you must include this mechanism in the SPF record of your domain name: include:aspmx.googlemail.com. Learn more about how to setup SPF records for Google Apps.

ptr mechanism

A ptr mechanism allows you to include a PTR (pointer) record for a domain name. A PTR record is used in reverse DNS lookup. If your domain name has A and PTR records set correctly, an A record will resolve to an IP address, and that IP address can be reverse looked up back to the original domain name using PTR record.

Following SPF record uses a ptr mechanism to include a PTR record for this domain name:

"v=spf1 ptr ~all"

To include a PTR record of another domain name, you will use this syntax:

"v=spf1 ptr:anotherdomain.com ~all"

It is recommended that you do not unnecessarily use a ptr mechanism unless you have to because it results in unnecessary DNS lookups.

exists mechanism

An exists mechanism is used for checking if an A record exists for a given domain name. And if it exists, the email is allowed to pass.

It does not matter if the IP address of that domain name matches the originating IP address for this email or not. What matters is that, an A record should exist and that's all.

Following SPF record includes an exists mechanism for anotherdomain.com:

"v=spf1 exists:anotherdomain.com ~all"

Personally, I am not sure why if an A record exists for a domain name, you will allow this email to pass. But then, there must be some reason why this mechanism exists, no?

all mechanism

An all mechanism is used to declare a rule for hosts and IP addresses that have not matched with the preceding mechanisms. All SPF records almost always have an all mechanism declared at the end of the SPF record.

What you should do is to declare one or more mechanisms that if matched, will allow the email to pass (white list). At the end you should declare an all mechanism which should be quantified with an '~' to soft fail the rest.

Following SPF record declares an all mechanism to soft fail all emails i.e., no legitimate emails can originate for this domain name:

"v=spf1 ~all"

Following SPF record will fail all emails originating for this domain name:

"v=spf1 -all"

Following SPF record will pass all emails with this domain name in the 'From' address of the email, no matter from where it originted:

"v=spf1 all"

The example above to allow all emails to pass is for demonstration purposes only. You obviously will not be using this in real practice.

SPF record generating tips

First create a white list of hosts and IP addresses from which legitimate emails can originate for your domain name. You can always look at the example SPF record of Stardeveloper.com to have a look.

To create an SPF record for this list, start with a simple SPF record which soft fails every email like this:

"v=spf1 ~all"

Then go through the white list you created, one by one, and using the mechanisms' syntax we learned above, add the mechanisms to your SPF record.

My advice is to always include the A record in the SPF record for your domain name. Rest, you can decide.

Conclusion

In this article, we learned the SPF record syntax. We learned about quantifiers and mechanisms which collectively formulate a policy for a domain name. I hope the information in this article will be sufficient to help you setup correct SPF record for your domain name.

Good luck!

What is Sender Policy Framework?

Faisal Khan — Mon, 22 Feb 2010 09:19:00 GMT

SPF stands for 'Sender Policy Framework' which is an open standard to prevent the abuse of your domain name by someone else pretending to be you by using your email address or domain name in the 'From' field of his email.

How many times do you get bounced email back in your inbox for emails containing your email address in the 'From' field, selling products or containing links to indecent websites? You never sent these emails. Yet someone has picked your email or a fake email using your website's domain name to send spam to thousands of others users using your email address to bypass the filters on their inboxes and to use your repute and recognition to make the end-user give undeserved attention to their spam links.

How does SPF work?

SPF works by making use of a list of IP addresses that you specifically allow from which a legitimate email can originate for your domain name. For your website you have to make a list of IP addresses (or host names) from which legitimate emails can be sent for your domain name. This list should contain the IP addresses of your website (the A records) and the IP addresses of your SMTP servers (all of them). If you are using BlackBerry to send emails for your website or company, you will have to include their IP addresses in this list as well.

Now it is not easy to know the IP addresses of all servers from where the legitimate emails can originate for your domain name, you can make use of SPF record syntax to create a policy to include IP addresses by using host names and domain names instead of plain IP addresses in your SPF record. You can also include other domains' SPF records. The cool thing with SPF's syntax is that with very little effort you can comprehensively include all the servers without needing to hard code any single IP address at all.

Once a list of IP addresses and host names has been setup for your domain name, then the next time an authorized person tries to send an email using the 'From' address of your domain name e.g., fakedname@yourwebsite.com, the MX (Mail Exchanger) server receiving this email will lookup SPF record for your domain by checking the DNS records of your domain name. When it finds an SPF record, it will go through the list of IP addresses and host names, one by one, to see if this email is originating from the white list of IP addresses allowed to send email on your behalf or not. Since the originating email in this case did not originate from an authorized IP address, the receiving mail server will silently place this email in the 'Spam' folder, and not the user's 'Inbox'.

This is how SPF prevents misuse of your domain name by unauthorized persons.

This of course implies that you have to have your own domain name, for example, yourwebsite.com, and an email address like yourname@yourwebsite.com. If you are using Gmail or Yahoo and have email address like yourname@gmail.com, then you do not need to setup an SPF record; because it is already set by those email providers.

An example SPF record

The SPF record for Stardeveloper looks like this:

"v=spf1 a include:aspmx.googlemail.com include:srs.bis.eu.blackberry.com ~all"

Explanation

v=spf1

An SPF record always begings with v=spf1. This tells the SPF record validator that this SPF record's syntax belongs to version 1 of Sender Policy Framework.

Next, the SPF record contains an a which means an 'A' record for the domain name. What this means is that legitimate email can originate for Stardeveloper.com from the IP address it is hosted at. Note: the IP address has not been hard coded for the domain name, instead an a record has been specified using SPF record syntax. If Stardeveloper is relocated to another IP address, the 'A' record will automatically point to the new IP address without requiring any change in the SPF record for this domain name.

include:aspmx.googlemail.com

Since, Stardeveloper is using Google Apps for email hosting, I have included Google Apps' SPF record as part of the SPF record of Stardeveloper.com to make sure that any email that is sent through Google's SMTP servers is marked as legitimate.

include:srs.bis.eu.blackberry.com

I am using BlackBerry Bold 9700 as my mobile device. I use it to send and receive emails. To be able to mark all emails sent through my BlackBerry device that make use of BlackBerry's SMTP servers, I have included their SPF records from their domain name.

If you are based in North America, then you should change the 'eu' with 'na' to include BlackBerry's North American SMTP servers, that is, include:srs.bis.na.blackberry.com .

~all

The final mechanism beginning with a tilde '~' suggests a 'soft fail' for emails not being sent from authorized IP address. Soft fail means that emails originating from unauthorized IP addresses will be received by mail servers (MX) but will be placed in 'Spam' folders and not users' inbox. If this syntax had been -all, it would have meant 'hard fail'. Hard fail means to reject all emails originating from unlisted IP addresses straight away. Since Google recommends a soft fail based SPF syntax, Stardeveloper uses this syntax and so should you (unless of course if you know what you are doing).

How to check the SPF record for a domain name?

You can use the SPF record checking tool here. Just enter the domain name of your website (or any other to just have a look how they have setup their SPF records), and submit the form.

Should you have an SPF record for your website?

It is absolutely important for you to have an SPF record for your domain name. You must prevent the abuse of your domain name. You must not let your Uncle get an email from an unauthorized person claiming as you and suggesting your Uncle to visit his spam website. And this becomes even more important if you have a repute online.

Conclusion

In this article, we learned what is an SPF record and how it works. We looked at a sample SPF record and learned how simple the SPF record syntax can be to set one up for one's website.

Convert IP Address to Long

Faisal Khan — Sun, 27 Dec 2009 08:48:00 GMT

An IP address is a numerical address that is assigned to your computer the moment it connects to a network. Whenever you access a website on internet, a request is sent from your computer containing IP address of your computer, the address to which the website's data should be sent back to. This IP address can be used for logging, tracking stats, allowing (white lists) or blocking (black lists) content, and for a variety of other reasons by the website administrator.

ASP.NET makes it very easy to develop HTTP Modules and database driven web applications to analyze and store these IP addresses in the database. A convenient property made available to all ASP.NET applications as part of an HTTP request is HttpRequest.UserHostAddress property which will return a string representation of the user's IP address.

Do not store IP Addresses as strings

The problem here is that to analyze or store this IP address as string is not the most efficient or recommended method. Not only does it take more space (as much as 15 bytes) to store data, sorting and indexing this data is inefficient.

It takes 15 bytes (varchar(15)) to store an IP address like "255.255.255.255".
Sorting is inefficient because the values will be sorted as string literals.
It is not possible to find an IP address e.g., "255.255.255.127" in a block of IP addresses e.g., "255.255.255.0" - "255.255.255.255". You will have to use inefficient SQL queries like:
```
"SELECT * FROM [IPAddresses] WHERE [IPAddress] LIKE '255.255.255.%'"
```

So, what should you do?

Solution: Convert IP Addresses to Long values

The solution is to convert an IP address into an integer.. In most programming languages, the integer is declared as long to provide 8 bytes of space to the variable. In C#, an 8 byte long value is declared using long keyword. In T-SQL (i.e., for SQL Server), an 8 byte long value is declared as bigint. Here is why storing IP addresses as long (or bigint in T-SQL) is more efficient:

It takes only 8 bytes to store an IP address as long. An IP address like "255.255.255.255" is stored as 4294967295. The code to do this conversion, both in C# and T-SQL, will be looked upon later in this tutorial.
Sorting is extremely efficient as it is very easy to do the sorting on numerical data like 4294967295 than string literals like "255.255.255.255".
You can very easily find if a given IP address exists in a block of IP addresses using the query:
```
"SELECT * FROM [IPAddresses] WHERE
	[IPAddress] BETWEEN [IPAddressBlock1] AND [IPAddressBlock2]"
```

Now, we will look at the code to convert a string literal IP address (like "255.255.255.255") to a numerical value (like 4294967295) and back.

Converting IP Addresses to Long Values and Back in C#

To convert a string IP address to a long value, we will use the ConvertIPToLong() method as shown below. We first convert the IP address string literal to System.Net.IPAddress. Then we convert this IP address to an array of 4 bytes by splitting this IP address using the dot (".") letter. It then converts the 4 bytes array to a single long value.

/// <summary>
/// Author: Faisal Khan (http://www.stardeveloper.com)
/// </summary>
public static long ConvertIPToLong(string ipAddress)
{
	System.Net.IPAddress ip;

	if (System.Net.IPAddress.TryParse(ipAddress, out ip))
	{
		byte[] bytes = ip.GetAddressBytes();

		return (long)((bytes[0] << 24) | (bytes[1] << 16) |
			(bytes[2] << 8) | bytes[3]);
	}
	else
		return 0;
}

Now to convert an IP address long value back to its string representation, we will use the ConvertLongToIP() method as shown below:

/// <summary>
/// Author: Faisal Khan (http://www.stardeveloper.com)
/// </summary>
public static string ConvertLongToIP(long ipLong)
{
	StringBuilder b = new StringBuilder();
	long tempLong, temp;

	tempLong = ipLong;
	temp = tempLong / (256 * 256 * 256);
	tempLong = tempLong - (temp * 256 * 256 * 256);
	b.Append(Convert.ToString(temp)).Append(".");
	temp = tempLong / (256 * 256);
	tempLong = tempLong - (temp * 256 * 256);
	b.Append(Convert.ToString(temp)).Append(".");
	temp = tempLong / 256;
	tempLong = tempLong - (temp * 256);
	b.Append(Convert.ToString(temp)).Append(".");
	temp = tempLong;
	tempLong = tempLong - temp;
	b.Append(Convert.ToString(temp));

	return b.ToString().ToLower();
}

Converting IP Addresses to Bigint Values and Back in T-SQL

We will use ConvertIPToLong() function to convert varchar(15) based IP address to a bigint value as shown below:

' Author: Faisal Khan (http://www.stardeveloper.com)
CREATE FUNCTION [dbo].[ConvertIPToLong](@IP varchar(15))
RETURNS bigint
AS
BEGIN
	DECLARE @Long bigint
	SET @Long = CONVERT(bigint, PARSENAME(@IP, 4)) * 256 * 256 * 256 +
		CONVERT(bigint, PARSENAME(@IP, 3)) * 256 * 256 +
		CONVERT(bigint, PARSENAME(@IP, 2)) * 256 +
		CONVERT(bigint, PARSENAME(@IP, 1))

	RETURN (@Long)
END

Now, to convert this bigint value back to its varchar(15) representation, we will use the ConvertLongToIP() function as shown below:

' Author: Faisal Khan (http://www.stardeveloper.com)
CREATE FUNCTION [dbo].[ConvertLongToIP](@Long bigint)
RETURNS varchar(15)
AS
BEGIN
	DECLARE @IP varchar(15)
	DECLARE @TempLong bigint
	DECLARE @Temp bigint

	SET @TempLong = @Long
	SET @Temp = @TempLong / (256 * 256 * 256)
	SET @TempLong = @TempLong - (@Temp * 256 * 256 * 256)
	SET @IP = CONVERT(varchar(3), @Temp) + '.'
	SET @Temp = @TempLong / (256 * 256)
	SET @TempLong = @TempLong - (@Temp * 256 * 256)
	SET @IP = @IP + CONVERT(varchar(3), @Temp) + '.'
	SET @Temp = @TempLong / 256
	SET @TempLong = @TempLong - (@Temp * 256)
	SET @IP = @IP + CONVERT(varchar(3), @Temp) + '.'
	SET @Temp = @TempLong
	SET @TempLong = @TempLong - @Temp
	SET @IP = @IP + CONVERT(varchar(3), @Temp)

	RETURN (@IP)
END

After you have created these functions in your database, you can run the following SQL code to test these functions:

SELECT [dbo].ConvertIPToLong('255.255.255.255'), [dbo].ConvertLongToIP(4294967295)

You should create these functions in your C# libraries and in SQL Server databases and use them whenever you want to store or use an IP address in your code or database.

Final Words

I have very succinctly described the code to convert an IP address from a string value to long and back. I hope this code will be useful to you in your web and database application development. Good luck!

Introduction to Developing HTTP Modules in ASP.NET

Faisal Khan — Sat, 18 Jul 2009 09:00:00 GMT

Introduction
In ASP.NET, an HTTP Module is a class that implements the IHttpModule interface. By doing that it can listen to and handle events in the web request handling pipeline. They are the equivalent of 'ISAPI Filters' in IIS and 'Filters' in JSP. They are used for multiple purposes from authenticating a request to modifying the response and logging the request. In this tutorial, we will learn:

Common uses of HTTP Modules
Web Request Handling Pipeline
Structure of an HTTP Module
Creating your first HTTP Module

Common Uses of HTTP Modules
Following is a noncomprehensive list of common uses of HTTP Modules:

Authentication - To authenticate a user. For example, you can place a cookie in the user's browser once he has logged in. Next time when the user visits the website, his session will have timed out. An HTTP Module can fetch the cookie and log the user back in by retrieving and matching the cookie string against user data in the database, without the user having to enter his username/password on the website. This can provide a smooth experience to the user who will never come to know how he has been logged back in automatically.
Authorization - To protect access to restricted resources on the website. An HTTP Module can authenticate the user accessing the resource and if the user is not listed in the roles allowed to access this resource, then the user can be redirected back to a "login" page.
Protection against Denial of Service Attacks - An HTTP Module can maintain an internal list of IP addresses accessing the website along with the hits the website is receiving from each of them. When an IP address is noted to be sending too many requests out of proportion to the rest of the IP addresses, that IP address can be temporarily blocked from accessing the website.
HTTP Compression - To compress the text output of web pages using Gzip and Deflate algorithms. Compression can not only substantially reduce the bandwidth usage of a website, it can also make the pages appear faster in the users' browsers because of smaller sizes of compressed pages that are sent over the internet.
Encryption - To encrypt an outgoing response into encrypted text.
Caching - To cache commonly accessed content into an in-memory cache. This can be applied to static as well as dynamic generated content. Again an internal list can be used to gather statistics first as to which content is accessed more often than others. Then the popular content can be cached.
Collecting User Information - To get the user IP address and match it against an internal database of IP addresses along with names of countries. The HTTP Module can then pass the fetched country name (or country code) silently to the request handlers which can then display it as a country flag to the user. This is useful for not only providing customized experience to the user, you can also use it to display different content (e.g., ads) to users from different countries.
Modifying Headers - To modify the outgoing headers. For example, you can use it to tell the client browser when the requested resource was last modified and so on. You can send the headers for static content to be cached in the client browser and intermediate proxy servers, more often than for example dynamic content.
Formatting Output - To transform XML files with XSL stylesheets on the fly. So when the user accesses an XML file on your website directly, he/she will be served an HTML transformation of the XML file, displaying the content in a more pleasant fashion.
Replacing Content - To use regular expressions to parse and replace content in the response generated by the request handler. For example, to highlight words in the outgoing HTML content matching keywords that a user entered in the search engine e.g., Google, to visit your website. This will improve the experience of a person on your website who entered some keywords in Google and found your website among the ones that Google displayed. Now when he visited your website he sees his keywords highlighted within the text of that page.
Logging - To log requests in a file or a database. Some examples:-
- To log hits that your website is getting every hour. An ASP.NET page can be used to display these hits in a chart to you providing you with a quick snapshot of the activity on your website.
- To scan the Status Code e.g., 500, generated by a resource downstream, and log it in the database. An ASP.NET page again can be used to view in the form of a chart, when and how many 500 HTTP errors were generated on the website and by which resource.
- To log bandwidth that your website is using, day in and day out.

Note: What you can do with HTTP Modules is only limited by your imagination. Soon when you are up to speed with HTTP Modules, you will be developing ones for needs you had never thought you can be developing for, before.

On the next page, we will look at the events in the 'Web Request Handling Pipeline'.

Web Request Handling Pipeline
In ASP.NET, there is a series of events that is raised before a request is handed over to a request handler to execute. And on its way back to the client, another series of events is raised. This model is what I refer to as the 'Web Request Handling Pipeline'. The list of these events, in the order that they are fired from the moment a request is received to the moment it is disposed, is given below with brief description:

BeginRequest - Fired as the first event when ASP.NET responds to a request.
AuthenticateRequest - Occurs when the identity of the user is being established.
PostAuthenticateRequest - Occurs when the identity of the user has been established.
AuthorizeRequest - Occurs when the request is being authorized for this user.
PostAuthorizeRequest - Occurs when the request has been authorized for this user.
ResolveRequestCache - Occurs when the request is being examined for the fact that the requested resource be sent directly from the cache or not.
PostResolveRequestCache - Occurs when the requested resource is being sent directly from the cache and the request handler will not be called to process this request.
MapRequestHandler - Occurs when the request is being mapped to an appropriate request handler. This event is only raised in IIS 7.0 when it is run in integrated mode with .NET Framework 3.0 or above.
PostMapRequestHandler - Occurs when appropriate request handler has been mapped with the request.
AcquireRequestState - Occurs when state associated with the request is being acquired.
PostAcquireRequestState - Occurs when state associated with the request has been acquired.
PreRequestHandlerExecute - Occurs just before request handler (e.g., an ASP.NET page) is handed over control for this request. In the tutorial on "Enabling Gzip and Deflate HTTP Compression in ASP.NET pages", this event is handled to enable HTTP Compression in ASP.NET pages.

Note: The Request Handler is now handed over the request.

PostRequestHandlerExecute - Occurs right after the request handler has been executed.
ReleaseRequestState - Occurs when the state (e.g., session state associated with the request) associated with the request is being released.
PostReleaseRequestState - Occurs when the state associated with the request has been released.
UpdateRequestCache - Occurs when request handler has executed the request. This event provides opportunity to HTTP Modules that want to cache the response generated by this request so that subesquent requests can be handled directly from the cache.
PostUpdateRequestCache - Occurs when HTTP Modules that wanted to cache the response generated by this request have finished doing so.
LogRequest - Occurs when ASP.NET is logging this request. This event is supported by IIS 7.0 in integrated mode with .NET Framework 3.0 or above.
PostLogRequest - Occurs when ASP.NET has finished logging this request. Again, this event is only supported in IIS 7.0 in integrated mode with .NET Framework 3.0 or above.
EndRequest - Finally, this is the last event in the chain of events in web request handling pipeline.

Note: Above events are associated with HttpApplication object, the reference of which is provided to all HTTP Modules when they are initialized so that they can subscribe to the events of their interest.

We will now learn what an HTTP Module looks like in the code and how it can subscribe to and handle these events.

Structure of an HTTP Module
An HTTP Module is an ordinary class that implements IHttpModule interface.

IHttpModule
It is defined in the System.Web namespace. This interface has just two methods to be implemented as described below:

Init(HttpApplication application) - This method is called by ASP.NET to initialize the HTTP Module. In this method, the HTTP Module can use the supplied argument of type, HttpApplication, to subscribe to events it wishes to handle.
Dispose() - This method is called by ASP.NET just before it wishes to dispose the given instance of an HTTP Module.

HttpApplication Events
The HttpApplication object passed as the argument in the Init() method of the IHttpModule interface can be used by the HTTP Module to subscribe to events of its choice. The list of the events available to the HTTP Module has already been given above.

Note: You should only handle the event(s) that is/are most appropriate for the task that you want to achieve with your HTTP Module.

Now that you have seen IHttpModule interface and are aware of the events that are made available to HTTP Modules, we will look at the code of an empty HTTP Module and later add some code to handle an event from the web request handling pipeline.

Creating your first HTTP Module
Following code shows how a class can implement the IHttpModule interface:

using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Web;

namespace Stardeveloper.Modules
{
	public class SampleModule : IHttpModule
	{
		public void Init(HttpApplication application)
		{
		}

		public void Dispose()
		{
		}
	}
}

That wasn't too much of a code, was it? A class with the name of SampleModule declared in the Stardeveloper.Modules namespace implements the IHttpModule interface. An empty implementation of the methods is shown above.

To handle events in the web request handling pipeline, it subscribes to them as follows:

using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Web;

namespace Stardeveloper.Modules
{
	public class SampleModule : IHttpModule
	{
		public void Init(HttpApplication application)
		{
			application.PreRequestHandlerExecute +=
				new EventHandler(application_PreRequestHandlerExecute);
			application.PostRequestHandlerExecute +=
				new EventHandler(application_PostRequestHandlerExecute);
		}

		protected void application_PreRequestHandlerExecute(object sender,
			EventArgs e)
		{
			HttpApplication application = sender as HttpApplication;
			application.Response.Write("Welcome!");
		}
		
		protected void application_PostRequestHandlerExecute(object sender,
			EventArgs e)
		{
			HttpApplication application = sender as HttpApplication;
			application.Response.Write("Bye!");
		}

		public void Dispose()
		{
		}
	}
}

Same class now uses C#'s syntax of subscribing to events to subscribe to two events; PreRequestHandlerExecute and PostRequestHandlerExecute. In the body of the event handlers, it casts the reference from the sender object to HttpApplication object and uses its Response property (which is of type HttpResponse) to write some text in the response being sent to the client browser.

To install this module in your ASP.NET web application, place this code in SampleModule.cs file and put it in the /App_Code folder of your ASP.NET web application. Then edit your web.config file to add following line within the "httpModules" section:

<configuration>
	<system.web>
		<httpModules>
			<add name="SampleModule" type="Stardeveloper.Modules.SampleModule"/>
		</httpModules>
	</system.web>
</configuration>

Now, when you access any ASP.NET page within your ASP.NET web application, you will see all the text wrapped between "Welcome!" and "Bye!".

Summary
In this tutorial, we were introduced with the concept of events in the web request handling pipeline. We also got a good idea of what HTTP Modules can do. We then looked at the IHttpModule interface and HttpApplication object. Finally, we created an empty HTTP Module, made it handle a couple of events from the web request handling pipeline, configure it in the web.config file, and display some text to the user.

In the next tutorial, we will look at a more real-World example of an HTTP Module in action.

Displaying Emails from Mailbox on Gmail using POP3 and ASP.NET

Faisal Khan — Fri, 10 Jul 2009 05:52:00 GMT

Introduction
In this tutorial, we will create the missing ASP.NET page, that was left in the previous tutorial (Connecting to Mailbox on Gmail and Fetching List of Emails using POP3 and ASP.NET), that takes an individual email's sort number on the POP3 server to retrieve and display it to the user after properly decoding content-transfer-encodings like base64 and quoted-printable. You will learn how to access individual message parts in MIME encoded emails including fetching and enlisting attachments.

Note: It is mandatory that you read the previous tutorial (Connecting to Mailbox on Gmail and Fetching List of Emails using POP3 and ASP.NET) to download and place the essential Pop3.cs file in the /App_Code folder of your ASP.NET web application. Without this code, you will not be able to connect and retrieve emails using POP3.

In the previous tutorial, we limited ourselves to only listing the emails in our mailbox on Gmail. That in itself was quite a bit of a job since we had to develop all the baseline code that connects, sends commands, receives data, parses and instantiates objects. The classes that we created like Pop3Client, Email, and MessagePart made the task a lot more easier for us. Only few lines of code were needed to retrieve the list of emails in the mailbox and display it on the ASP.NET page. We will build on that base in this tutorial.

If you are looking for a production ready, stable, easy to use ASP.NET Newsletter application, then look no further: Faisal Khan (the author of this tutorial) has created a great Newsletter Application for you to download and start using today. It is easy to install, lets your users subscribe using a subscription form, provides a simple administration interface to create and send newsletters, handles the task of sending newsletters in the background using thread queues on the server, and lets users unsubscribe if they wish so.

Following is the screen shot of the ASP.NET page that we developed in the previous tutorial:

If you followed previous tutorial and created Pop3Client.aspx ASP.NET page that is listing the emails as shown in screen shot above, you'd have found out that the subject of the email is displayed as a link to DisplayPop3Email.aspx ASP.NET page. That ASP.NET page is what we will create in this tutorial.

Following screen shots shows DisplayPop3Email.aspx ASP.NET page displaying a plain text email:

Following screen shot shows DisplayPop3Email.aspx displaying an HTML email:

Following screen shot shows DisplayPop3Email.aspx listing attachments:

Now that you know what DisplayPop3Email.aspx ASP.NET page that we will create in this tutorial will do, we will now go straight into its code.

DisplayPop3Email.aspx
Open Notepad and copy following text in it. Then save it as "DisplayPop3Email.aspx" in your ASP.NET web application where you saved Pop3Client.aspx ASP.NET page from the previous tutorial:

<%@ Page Language="C#" %>

<%@ Import Namespace="System.Collections.Generic" %>
<%@ Import Namespace="Stardeveloper.Pop3" %>

<script runat="server">
	public const string Host = "pop.gmail.com";
	public const int Port = 995;
	public const string Email = "youremail@gmail.com";
	public const string Password = "yourpassword";

	protected static Regex CharsetRegex =
		new Regex("charset=\"?(?<charset>[^\\s\"]+)\"?",
		RegexOptions.IgnoreCase | RegexOptions.Compiled);
	protected static Regex QuotedPrintableRegex =
		new Regex("=(?<hexchars>[0-9a-fA-F]{2,2})",
		RegexOptions.IgnoreCase | RegexOptions.Compiled);
	protected static Regex UrlRegex =
		new Regex("(?<url>https?://[^\\s\"]+)",
		RegexOptions.IgnoreCase | RegexOptions.Compiled);
	protected static Regex FilenameRegex =
		new Regex("filename=\"?(?<filename>[^\\s\"]+)\"?",
		RegexOptions.IgnoreCase| RegexOptions.Compiled);
	protected static Regex NameRegex =
		new Regex("name=\"?(?<filename>[^\\s\"]+)\"?",
		RegexOptions.IgnoreCase | RegexOptions.Compiled);

	protected void Page_Load(object source, EventArgs e)
	{
		/* Setting email id */
		int emailId = -1;

		if (Request.QueryString["emailId"] == null)
		{
			Response.Redirect("Pop3Client.aspx");
			Response.Flush();
			Response.End();
		}
		else
			emailId = Convert.ToInt32(
				Request.QueryString["emailId"]);

		/* Creating Pop3Client object and accessing email data */
		Email email = null;
		List<MessagePart> msgParts = null;

		using (Pop3Client client = new Pop3Client(Host, Port,
			Email, Password, true))
		{
			client.Connect();

			// Fetching email headers
			email = client.FetchEmail(emailId);
			// Fetching email body
			msgParts = client.FetchMessageParts(emailId);
		}

		if (email == null || msgParts == null)
		{
			Response.Redirect("Pop3Client.aspx");
			Response.Flush();
			Response.End();
		}

		/* Selecting a message part to display to the user */
		MessagePart preferredMsgPart = FindMessagePart(msgParts, "text/html");

		if (preferredMsgPart == null)
			preferredMsgPart = FindMessagePart(msgParts, "text/plain");
		else if (preferredMsgPart == null && msgParts.Count > 0)
			preferredMsgPart = msgParts[0];

		string contentType, charset, contentTransferEncoding, body = null;

		if (preferredMsgPart != null)
		{
			contentType = preferredMsgPart.Headers["Content-Type"];
			charset = "us-ascii"; // default charset
			contentTransferEncoding =
				preferredMsgPart.Headers["Content-Transfer-Encoding"];

			Match m = CharsetRegex.Match(contentType);

			if (m.Success)
				charset = m.Groups["charset"].Value;

			HeadersLiteral.Text = contentType != null ? "Content-Type: " +
				contentType + "<br />" : string.Empty;
			HeadersLiteral.Text += contentTransferEncoding != null ?
				"Content-Transfer-Encoding: " +
				contentTransferEncoding : string.Empty;

			/* Decoding base64 and quoted-printable encoded messages */
			if (contentTransferEncoding != null)
			{
				if (contentTransferEncoding.ToLower() == "base64")
					body = DecodeBase64String(charset,
						preferredMsgPart.MessageText);
				else if (contentTransferEncoding.ToLower() ==
						"quoted-printable")
					body = DecodeQuotedPrintableString(
						preferredMsgPart.MessageText);
				else
					body = preferredMsgPart.MessageText;
			}
			else
				body = preferredMsgPart.MessageText;
		}

		/* Displaying information to the user */
		EmailIdLiteral.Text = Convert.ToString(emailId);
		DateLiteral.Text = email.UtcDateTime.ToString(); ;
		FromLiteral.Text = email.From;
		SubjectLiteral.Text = email.Subject;
		BodyLiteral.Text = preferredMsgPart != null ? 
			(preferredMsgPart.Headers["Content-Type"]
				.IndexOf("text/plain") != -1 ?
			"<pre>" + FormatUrls(body) + "</pre>" : body) : null;

		ListAttachments(msgParts);
	}

The code continues on the next page.

The code continues from the previous page.

	protected Decoder GetDecoder(string charset)
	{
		Decoder decoder;

		switch (charset.ToLower())
		{
			case "utf-7":
				decoder = Encoding.UTF7.GetDecoder();
				break;
			case "utf-8":
				decoder = Encoding.UTF8.GetDecoder();
				break;
			case "us-ascii":
				decoder = Encoding.ASCII.GetDecoder();
				break;
			case "iso-8859-1":
				decoder = Encoding.ASCII.GetDecoder();
				break;
			default:
				decoder = Encoding.ASCII.GetDecoder();
				break;
		}

		return decoder;
	}

	protected string DecodeBase64String(string charset, string encodedString)
	{
		Decoder decoder = GetDecoder(charset);

		byte[] buffer = Convert.FromBase64String(encodedString);
		char[] chararr = new char[decoder.GetCharCount(buffer,
			0, buffer.Length)];

		decoder.GetChars(buffer, 0, buffer.Length, chararr, 0);

		return new string(chararr);
	}

	protected string DecodeQuotedPrintableString(string encodedString)
	{
		StringBuilder b = new StringBuilder();
		int startIndx = 0;

		MatchCollection matches = QuotedPrintableRegex.Matches(encodedString);

		for (int i = 0; i < matches.Count; i++)
		{
			Match m = matches[i];
			string hexchars = m.Groups["hexchars"].Value;
			int charcode = Convert.ToInt32(hexchars, 16);
			char c = (char)charcode;

			if (m.Index > 0)
				b.Append(encodedString.Substring(startIndx,
					(m.Index - startIndx)));

			b.Append(c);

			startIndx = m.Index + 3;
		}
		
		if (startIndx < encodedString.Length)
			b.Append(encodedString.Substring(startIndx));

		return Regex.Replace(b.ToString(), "=\r\n", "");
	}

	protected void ListAttachments(List<MessagePart> msgParts)
	{
		bool attachmentsFound = false;
		StringBuilder b = new StringBuilder();
		b.Append("<ol>");

		foreach (MessagePart p in msgParts)
		{
			string contentType = p.Headers["Content-Type"];
			string contentDisposition = p.Headers["Content-Disposition"];

			Match m;

			if (contentDisposition != null)
			{
				m = FilenameRegex.Match(contentDisposition);

				if (m.Success)
				{
					attachmentsFound = true;
					b.Append("<li>").
						Append(m.Groups["filename"].
							Value).Append("</li>");
				}
			}
			else if (contentType != null)
			{
				m = NameRegex.Match(contentType);

				if (m.Success)
				{
					attachmentsFound = true;
					b.Append("<li>").
						Append(m.Groups["filename"].
							Value).Append("</li>");
				}
			}
		}
			
		b.Append("</ol>");

		if (attachmentsFound)
			AttachmentsLiteral.Text = b.ToString();
		else
			AttachementsRow.Visible = false;
	}

	protected MessagePart FindMessagePart(List<MessagePart> msgParts,
		string contentType)
	{
		foreach (MessagePart p in msgParts)
			if (p.ContentType != null &&
					p.ContentType.IndexOf(contentType) != -1)
				return p;

		return null;
	}

	protected string FormatUrls(string plainText)
	{
		string replacementLink = "<a href=\"${url}\">${url}</a>";

		return UrlRegex.Replace(plainText, replacementLink);
	}
</script>

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
	"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<html xmlns="http://www.w3.org/1999/xhtml">
<head runat="server">
    <title></title>
    <style type="text/css">
		.emails-table { width: 600px; border: solid 1px #444444; }
		.emails-table-header { font-family: "Trebuchet MS"; font-size: 9pt;
			background-color: #0099B9; font-weight: bold; color: white;
			text-align: center; border: solid 1px #444444; }
		.emails-table-header-cell { font-family: "Georgia"; font-size: 9pt;
			font-weight: bold; border: solid 1px #666666; padding: 6px; }
		.emails-table-cell { font-family: "Georgia"; font-size: 9pt;
			border: solid 1px #666666; padding: 6px; }
		.emails-table-footer { border: solid 1px #666666; padding: 3px;
			width: 50%; }
		.email-datetime { float: right; color: #666666; }
		
		a { font-family: "Lucida Sans Unicode", "Trebuchet MS"; font-size: 9pt;
			color: #005B7F; }
		a:hover { color:red; }
		pre { font-family: "Georgia"; font-size: 9pt; }
    </style>
</head>
<body>
    <form id="form1" runat="server">
    <asp:Literal ID="DebugLiteral" runat="server" />
    
    <table class="emails-table">
    <tr>
		<td class="emails-table-header" colspan="2">
		Email #<asp:Literal ID="EmailIdLiteral" runat="server" /></td>
    </tr>
    <tr>
		<td class="emails-table-header-cell">Date &amp; Time</td>
		<td class="emails-table-cell">
			<asp:Literal ID="DateLiteral" runat="server" /></td>
    </tr>
    <tr>
		<td class="emails-table-header-cell">From</td>
		<td class="emails-table-cell">
			<asp:Literal ID="FromLiteral" runat="server" /></td>
    </tr>
    <tr>
		<td class="emails-table-header-cell">Subject</td>
		<td class="emails-table-cell">
			<asp:Literal ID="SubjectLiteral" runat="server" /></td>
    </tr>
    <tr id="AttachementsRow" runat="server">
		<td class="emails-table-header-cell">Attachments</td>
		<td class="emails-table-cell">
			<asp:Literal ID="AttachmentsLiteral" runat="server" /></td>
    </tr>
     <tr>
		<td class="emails-table-cell" colspan="2">
			<asp:Literal ID="HeadersLiteral" runat="server" /></td>
    </tr>
    <tr>
		<td class="emails-table-cell" colspan="2">
			<asp:Literal ID="BodyLiteral" runat="server" /></td>
    </tr>
    </table>
    </form>
</body>
</html>

Explanation
We will now dissect the code, bit by bit.

The first thing we do is to use a Page level declaration to tell the ASP.NET compiler that all the server-side code encountered in this document should, by default, be assumed to be of the language, C#.

<%@ Page Language="C#" %>

Next, we import the necessary namespaces.

Note: We are importing the namespace Stardeveloper.Pop3 to make use of the code that we created in the previous tutorial (Connecting to Mailbox on Gmail and Fetching List of Emails using POP3 and ASP.NET). This will make available classes like Pop3Client, Email and MessagePart to us.

<%@ Import Namespace="System.Collections.Generic" %>
<%@ Import Namespace="Stardeveloper.Pop3" %>

All the server-side code is encapsulated between script tags.

<script runat="server">
	...
</script>

The host, port, email, and password arguments required in the constructor of Pop3Client class will be set using the following four constants.

public const string Host = "pop.gmail.com";
public const int Port = 995;
public const string Email = "youremail@gmail.com";
public const string Password = "yourpassword";

Regular expressions used in the code later have been created as static objects. The RegexOptions.Compiled bit is also set in the constructor of all the Regex objects so that they are compiled the first time they are used and from there onwards, running these regular expressions will be super fast.

protected static Regex CharsetRegex =
	new Regex("charset=\"?(?<charset>[^\\s\"]+)\"?",
	RegexOptions.IgnoreCase | RegexOptions.Compiled);
protected static Regex QuotedPrintableRegex =
	new Regex("=(?<hexchars>[0-9a-fA-F]{2,2})",
	RegexOptions.IgnoreCase | RegexOptions.Compiled);
protected static Regex UrlRegex =
	new Regex("(?<url>https?://[^\\s\"]+)",
	RegexOptions.IgnoreCase | RegexOptions.Compiled);
protected static Regex FilenameRegex =
	new Regex("filename=\"?(?<filename>[^\\s\"]+)\"?",
	RegexOptions.IgnoreCase| RegexOptions.Compiled);
protected static Regex NameRegex =
	new Regex("name=\"?(?<filename>[^\\s\"]+)\"?",
	RegexOptions.IgnoreCase | RegexOptions.Compiled);

Now we will dig into the Page_Load() method which is called with every request our DisplayPop3Email.aspx receives.

protected void Page_Load(object source, EventArgs e)
{
	...
}

We retrieve the emailId value from the querystring and save it in a variable of the same name.

Note: emailId is the sort number of the email in the list of emails on the POP3 server. You can determine the number of emails by sending the STAT command. For instance, you have 324 emails in your POP3 server. In those 324 emails, the email #324 is the latest email that your mailbox has received and email #1 is oldest email among all your emails in your mailbox. This is how emails are sorted by the POP3 server. Now if you want to retrieve the email #324, you should call the DisplayPop3Email.aspx ASP.NET page with the querystring value of the variable emailId set to 324 i.e., DisplayPop3Email.aspx?emailId=324. I hope this wasn't too difficult for you to understand.

/* Setting email id */
int emailId = -1;

if (Request.QueryString["emailId"] == null)
{
	Response.Redirect("Pop3Client.aspx");
	Response.Flush();
	Response.End();
}
else
	emailId = Convert.ToInt32(
		Request.QueryString["emailId"]);

Once we have created the Pop3Client object, we quickly call its Connect() method to connect with the POP3 server. Once there, we call FetchEmail() and FetchMessageParts() methods to retrieve the full email, including all headers and its body.

/* Creating Pop3Client object and accessing email data */
Email email = null;
List<MessagePart> msgParts = null;

using (Pop3Client client = new Pop3Client(Host, Port,
	Email, Password, true))
{
	client.Connect();

	// Fetching email headers
	email = client.FetchEmail(emailId);
	// Fetching email body
	msgParts = client.FetchMessageParts(emailId);
}

If the email's sort number was invalid e.g., the POP3 server has 324 emails and you accessed email #325, null references will be created by the code segment above. In which case we redirect the user to the ASP.NET page listing all the emails.

if (email == null || msgParts == null)
{
	Response.Redirect("Pop3Client.aspx");
	Response.Flush();
	Response.End();
}

We then try to iterate through all the MessagePart objects retrieved from the email and find the one most appropriate to display on the page. We begin by finding an HTML version of the email. If we cannot find it, we search for a plain text version of the email. If there is still no one found, we select the first MessagePart object we encounter in the list.

/* Selecting a message part to display to the user */
MessagePart preferredMsgPart = FindMessagePart(msgParts, "text/html");

if (preferredMsgPart == null)
	preferredMsgPart = FindMessagePart(msgParts, "text/plain");
else if (preferredMsgPart == null && msgParts.Count > 0)
	preferredMsgPart = msgParts[0];

One we have the MessagePart object we want to display on the page, we use its content-type and content-transfer-encoding headers to determine if the message body needs to be decoded first. If the message's body was encoded using base64 or quoted-printable transfer encodings, we decode the content first and then display it to the user.

string contentType, charset, contentTransferEncoding, body = null;

if (preferredMsgPart != null)
{
	contentType = preferredMsgPart.Headers["Content-Type"];
	charset = "us-ascii"; // default charset
	contentTransferEncoding =
		preferredMsgPart.Headers["Content-Transfer-Encoding"];

	Match m = CharsetRegex.Match(contentType);

	if (m.Success)
		charset = m.Groups["charset"].Value;

	HeadersLiteral.Text = contentType != null ? "Content-Type: " +
		contentType + "<br />" : string.Empty;
	HeadersLiteral.Text += contentTransferEncoding != null ?
		"Content-Transfer-Encoding: " +
		contentTransferEncoding : string.Empty;

	/* Decoding base64 and quoted-printable encoded messages */
	if (contentTransferEncoding != null)
	{
		if (contentTransferEncoding.ToLower() == "base64")
			body = DecodeBase64String(charset,
				preferredMsgPart.MessageText);
		else if (contentTransferEncoding.ToLower() ==
				"quoted-printable")
			body = DecodeQuotedPrintableString(
				preferredMsgPart.MessageText);
		else
			body = preferredMsgPart.MessageText;
	}
	else
		body = preferredMsgPart.MessageText;
}

We set a few literals with information from the email to display on the page.

/* Displaying information to the user */
EmailIdLiteral.Text = Convert.ToString(emailId);
DateLiteral.Text = email.UtcDateTime.ToString(); ;
FromLiteral.Text = email.From;
SubjectLiteral.Text = email.Subject;
BodyLiteral.Text = preferredMsgPart != null ? 
	(preferredMsgPart.Headers["Content-Type"]
		.IndexOf("text/plain") != -1 ?
	"<pre>" + FormatUrls(body) + "</pre>" : body) : null;

ListAttachments(msgParts);

GetDecoder() returns the appropriate decoder to decode the message text depending on the charset.

protected Decoder GetDecoder(string charset)
{
	Decoder decoder;

	switch (charset.ToLower())
	{
		case "utf-7":
			decoder = Encoding.UTF7.GetDecoder();
			break;
		case "utf-8":
			decoder = Encoding.UTF8.GetDecoder();
			break;
		case "us-ascii":
			decoder = Encoding.ASCII.GetDecoder();
			break;
		case "iso-8859-1":
			decoder = Encoding.ASCII.GetDecoder();
			break;
		default:
			decoder = Encoding.ASCII.GetDecoder();
			break;
	}

	return decoder;
}

DecodeBase64String() decodes a base64 encoded message and returns the decoded text.

Note: Most of the base64 encodings that you will encounter in a MIME encoded email will be for binary attachments that have been base64 encoded so that they can be attached as text with an email. But when when I went about to view emails on my email account, I encountered normal plain text emails that were base64 encoded. So I had to write a base64 decoder to decode and properly present them on the ASP.NET page.

protected string DecodeBase64String(string charset, string encodedString)
{
	Decoder decoder = GetDecoder(charset);

	byte[] buffer = Convert.FromBase64String(encodedString);
	char[] chararr = new char[decoder.GetCharCount(buffer,
		0, buffer.Length)];

	decoder.GetChars(buffer, 0, buffer.Length, chararr, 0);

	return new string(chararr);
}

DecodeQuotedPrintableString() method decodes a message whose body has been encoded with quoted-printable encoding.

protected string DecodeQuotedPrintableString(string encodedString)
{
	StringBuilder b = new StringBuilder();
	int startIndx = 0;

	MatchCollection matches = QuotedPrintableRegex.Matches(encodedString);

	for (int i = 0; i < matches.Count; i++)
	{
		Match m = matches[i];
		string hexchars = m.Groups["hexchars"].Value;
		int charcode = Convert.ToInt32(hexchars, 16);
		char c = (char)charcode;

		if (m.Index > 0)
			b.Append(encodedString.Substring(startIndx,
				(m.Index - startIndx)));

		b.Append(c);

		startIndx = m.Index + 3;
	}
	
	if (startIndx < encodedString.Length)
		b.Append(encodedString.Substring(startIndx));

	return Regex.Replace(b.ToString(), "=\r\n", "");
}

ListAttachments() method scans all the MessagePart objects to determine which of these is an attachment. It then displays the list of attachments on the page.

protected void ListAttachments(List<MessagePart> msgParts)
{
	bool attachmentsFound = false;
	StringBuilder b = new StringBuilder();
	b.Append("<ol>");

	foreach (MessagePart p in msgParts)
	{
		string contentType = p.Headers["Content-Type"];
		string contentDisposition = p.Headers["Content-Disposition"];

		Match m;

		if (contentDisposition != null)
		{
			m = FilenameRegex.Match(contentDisposition);

			if (m.Success)
			{
				attachmentsFound = true;
				b.Append("<li>").
					Append(m.Groups["filename"].
						Value).Append("</li>");
			}
		}
		else if (contentType != null)
		{
			m = NameRegex.Match(contentType);

			if (m.Success)
			{
				attachmentsFound = true;
				b.Append("<li>").
					Append(m.Groups["filename"].
						Value).Append("</li>");
			}
		}
	}
		
	b.Append("</ol>");

	if (attachmentsFound)
		AttachmentsLiteral.Text = b.ToString();
	else
		AttachementsRow.Visible = false;
}

FindMessagePart() scans all the MessagePart objects to find the one whose content-type matches the contentType argument.

protected MessagePart FindMessagePart(List<MessagePart> msgParts,
	string contentType)
{
	foreach (MessagePart p in msgParts)
		if (p.ContentType != null &&
				p.ContentType.IndexOf(contentType) != -1)
			return p;

	return null;
}

FormatUrls() formats URL strings for display on the page.

protected string FormatUrls(string plainText)
{
	string replacementLink = "<a href=\"${url}\">${url}</a>";

	return UrlRegex.Replace(plainText, replacementLink);
}

Now that we have looked at the code, we will run this ASP.NET page.

Running DisplayPop3Email.aspx
I placed Pop3.cs in the /App_Code folder, Pop3Client.aspx and DisplayPop3Email.aspx in the root folder (/). Then I started the ASP.NET web server and accessed Pop3Client.aspx in the browser. From the emails' list that appeared, I clicked one email's subject and this brought me to the DisplayPop3Email.aspx ASP.NET page that we just created. Following screen shot shows the result of this request:

Summary
In this tutorial, we created an ASP.NET page that takes an email's sort number on the POP3 server and connects to, retrieves, and displays that email on the page. This ASP.NET page was the missing page that was left in the previous tutorial (Connecting to Mailbox on Gmail and Fetching List of Emails using POP3 and ASP.NET).

I hope you can expand and modify the source code of this application to suit the needs of your application or if you are a beginner, learn the network programming aspects of it.

Connecting to Mailbox on Gmail and Fetching List of Emails using POP3 and ASP.NET

Faisal Khan — Wed, 08 Jul 2009 10:13:00 GMT

Introduction
In this tutorial, we will learn about Post Office Protocol (POP3). It is a protocol (a set of commands) used by an email client to connect to and retrieve email(s) from the mailbox on the remote server. We will develop the code that connects to the POP3 server and sends commands to retrieve the list of emails on the server, and then later, using an ASP.NET page, displays the list to the user. To demonstrate the code, we will be connecting to Gmail's POP3 server using SSL (Secure Sockets Layer).

To connect to your own POP3 mail server, you can either use the secure connection (SSL) if your mail server supports it or connect in unsecure mode. The code is simple to understand and by looking at it you will learn a lot about how network programming is done in the .NET environment.

Note: In this tutorial, we will limit ourselves to retrieving and displaying a list of emails from user's mailbox which the user can browse through using 'Previous Page' and 'Next Page' links. In the next tutorial (Displaying Emails from Mailbox on Gmail using POP3 and ASP.NET), we will learn how to retrieve and view an individual email.

Following screen shot shows an ASP.NET page listing emails in my Gmail mailbox:

Basic POP3 Commands
Following is a list of commonly used POP3 commands:

USER - Takes one argument i.e., the email address of the user trying to connect to his/her mailbox. Example usage:
```
USER youremail@xyz.com
```
PASS - Takes one argument i.e., the password of the user. Example usage:
```
PASS yourpassword
```
STAT - Returns the number of emails in the mailbox and the number of bytes all the emails are taking on the server. Example usage:
```
STAT
```
TOP - Takes two arguments i.e., the sort number of the email on the server and the number of lines of text to retrieve from the body of the email. Example usage:
```
TOP 1 10
```
RETR - Takes one argument i.e., the sort number of the email on the server and returns all the headers and lines from the body of the email. Example usage:
```
RETR 1
```
DELE - Takes one argument i.e., the sort number of the email on the server and deletes it. Example usage:
```
DELE 1
```
RSET - Resets any DELE commands given above. The emails marked to be deleted by DELE command are unmarked. Example usage:
```
RSET
```
QUIT - Closes the user session with the server. Example usage:
```
QUIT
```
Note: In the code that we will develop, we will not close the session with the remote server using QUIT command. Because if we have issued RETR command to retrieve any emails from the POP3 server, issuing QUIT commands gives the wrong impression to the server that those emails have been downloaded safely on our system so the server can remove them from its list. Since we do not want to inadvertently remove or delete any emails from the POP3 server, we will not issue QUIT command and simply close the connection.

Developing the Code
In this tutorial, we will develop C# classes that encapsulate all the code that is necessary to connect to and retrieve emails from Gmail's mailbox. We will place these classes in "Pop3.cs" C# source file. These classes will provide an easy to use interface to access the mailbox. An ASP.NET page with the name of "Pop3Client.aspx" will use these classes to list emails in the mailbox.

Files that we will develop in this tutorial:

Pop3.cs
Pop3Client.aspx

i. Pop3.cs
All right guys, open Notepad and copy following code in it. Then save this new document as Pop3.cs in the /App_Code folder of your ASP.NET web application:

/* Author: Faisal Khan */
/* © Stardeveloper.com */

using System;
using System.Collections.Generic;
using System.Collections.Specialized;
using System.Configuration;
using System.IO;
using System.Linq;
using System.Net;
using System.Net.Security;
using System.Net.Sockets;
using System.Text;
using System.Text.RegularExpressions;
using System.Web;
using System.Xml;

namespace Stardeveloper.Pop3
{
  public class Pop3Client : IDisposable
  {
    public string Host { get; protected set; }
    public int Port { get; protected set; }
    public string Email { get; protected set; }
    public string Password { get; protected set; }
    public bool IsSecure { get; protected set; }

    public TcpClient Client { get; protected set; }
    public Stream ClientStream { get; protected set; }

    public StreamWriter Writer { get; protected set; }
    public StreamReader Reader { get; protected set; }

    private bool disposed = false;

    public Pop3Client(string host, int port, string email,
      string password)
      : this(host, port, email, password, false)
    {
    }

    public Pop3Client(string host, int port, string email,
      string password, bool secure)
    {
      Host = host;
      Port = port;
      Email = email;
      Password = password;
      IsSecure = secure;
    }

    public void Connect()
    {
      if (Client == null)
        Client = new TcpClient();

      if (!Client.Connected)
        Client.Connect(Host, Port);

      if (IsSecure)
      {
        SslStream secureStream =
          new SslStream(Client.GetStream());
        secureStream.AuthenticateAsClient(Host);

        ClientStream = secureStream;
        secureStream = null;
      }
      else
        ClientStream = Client.GetStream();

      Writer = new StreamWriter(ClientStream);
      Reader = new StreamReader(ClientStream);

      ReadLine();
      Login();
    }

    public int GetEmailCount()
    {
      int count = 0;
      string response = SendCommand("STAT");

      if (IsResponseOk(response))
      {
        string[] arr = response.Substring(4).Split(' ');
        count = Convert.ToInt32(arr[0]);
      }
      else
        count = -1;

      return count;
    }

    public Email FetchEmail(int emailId)
    {
      if (IsResponseOk(SendCommand("TOP " + emailId + " 0")))
        return new Email(ReadLines());
      else
        return null;
    }

    public List<Email> FetchEmailList(int start, int count)
    {
      List<Email> emails = new List<Email>(count);

      for (int i = start; i < (start + count); i++)
      {
        Email email = FetchEmail(i);

        if (email != null)
          emails.Add(email);
      }

      return emails;
    }

    public List<MessagePart> FetchMessageParts(int emailId)
    {
      if (IsResponseOk(SendCommand("RETR " + emailId)))
        return Util.ParseMessageParts(ReadLines());

      return null;
    }

The code continues on the next page.

The code is continuing from the previous page.

public List<MessagePart> FetchMessageParts(int emailId)
    {
      if (IsResponseOk(SendCommand("RETR " + emailId)))
        return Util.ParseMessageParts(ReadLines());

      return null;
    }

    public void Close()
    {
      if (Client != null)
      {
        if (Client.Connected)
          Logout();

        Client.Close();
        Client = null;
      }

      if (ClientStream != null)
      {
        ClientStream.Close();
        ClientStream = null;
      }

      if (Writer != null)
      {
        Writer.Close();
        Writer = null;
      }

      if (Reader != null)
      {
        Reader.Close();
        Reader = null;
      }

      disposed = true;
    }

    public void Dispose()
    {
      if (!disposed)
        Close();
    }

    protected void Login()
    {
      if (!IsResponseOk(SendCommand("USER " + Email)) ||
        !IsResponseOk(SendCommand("PASS " + Password)))
        throw new Exception("User/password not accepted");
    }

    protected void Logout()
    {
      SendCommand("RSET");
      //SendCommand("QUIT");
    }

    protected string SendCommand(string cmdtext)
    {
      Writer.WriteLine(cmdtext);
      Writer.Flush();

      return ReadLine();
    }

    protected string ReadLine()
    {
      return Reader.ReadLine() + "\r\n";
    }

    protected string ReadLines()
    {
      StringBuilder b = new StringBuilder();

      while (true)
      {
        string temp = ReadLine();

        if (temp == ".\r\n" || temp.IndexOf("-ERR") != -1)
          break;

        b.Append(temp);
      }

      return b.ToString();
    }

    protected static bool IsResponseOk(string response)
    {
      if (response.StartsWith("+OK"))
        return true;
      if (response.StartsWith("-ERR"))
        return false;

      throw new Exception("Cannot understand server response: " +
        response);
    }
  }

  public class Email
  {
    public NameValueCollection Headers { get; protected set; }

    public string ContentType { get; protected set; }
    public DateTime UtcDateTime { get; protected set; }
    public string From { get; protected set; }
    public string To { get; protected set; }
    public string Subject { get; protected set; }

    public Email(string emailText)
    {
      Headers = Util.ParseHeaders(emailText);

      ContentType = Headers["Content-Type"];
      From = Headers["From"];
      To = Headers["To"];
      Subject = Headers["Subject"];

      if (Headers["Date"] != null)
        try
        {
        UtcDateTime =
          Util.ConvertStrToUtcDateTime(Headers["Date"]);
        }
        catch (FormatException)
        {
          UtcDateTime = DateTime.MinValue;
        }
      else
        UtcDateTime = DateTime.MinValue;
    }
  }

  public class MessagePart
  {
    public NameValueCollection Headers { get; protected set; }

    public string ContentType { get; protected set; }
    public string MessageText { get; protected set; }

    public MessagePart(NameValueCollection headers, string messageText)
    {
      Headers = headers;
      ContentType = Headers["Content-Type"];
      MessageText = messageText;
    }
  }

  public class Util
  {
    protected static Regex BoundaryRegex =
      new Regex("Content-Type: multipart(?:/\\S+;)" +
      "\\s+[^\r\n]*boundary=\"?(?<boundary>" +
      "[^\"\r\n]+)\"?\r\n", RegexOptions.IgnoreCase |
      RegexOptions.Compiled);
    protected static Regex UtcDateTimeRegex = new Regex(
      @"^(?:\w+,\s+)?(?<day>\d+)\s+(?<month>\w+)\s+(?<year>\d+)\s+(?<hour>\d{1,2})" +
      @":(?<minute>\d{1,2}):(?<second>\d{1,2})\s+(?<offsetsign>\-|\+)(?<offsethours>" +
      @"\d{2,2})(?<offsetminutes>\d{2,2})(?:.*)$",
      RegexOptions.IgnoreCase | RegexOptions.Compiled);

    public static NameValueCollection ParseHeaders(string headerText)
    {
      NameValueCollection headers = new NameValueCollection();
      StringReader reader = new StringReader(headerText);

      string line;
      string headerName = null, headerValue;
      int colonIndx;

      while ((line = reader.ReadLine()) != null)
      {
        if (line == "")
          break;

        if (Char.IsLetterOrDigit(line[0]) && (colonIndx = line.IndexOf(':')) != -1)
        {
          headerName = line.Substring(0, colonIndx);
          headerValue = line.Substring(colonIndx + 1).Trim();

          headers.Add(headerName, headerValue);
        }
        else if (headerName != null)
          headers[headerName] += " " + line.Trim();
        else
          throw new FormatException("Could not parse headers");
      }

      return headers;
    }

    public static List<MessagePart> ParseMessageParts(string emailText)
    {
      List<MessagePart> messageParts = new List<MessagePart>();
      int newLinesIndx = emailText.IndexOf("\r\n\r\n");

      Match m = BoundaryRegex.Match(emailText);

      if (m.Index < emailText.IndexOf("\r\n\r\n") && m.Success)
      {
        string boundary = m.Groups["boundary"].Value;
        string startingBoundary = "\r\n--" + boundary;

        int startingBoundaryIndx = -1;

        while (true)
        {
          if (startingBoundaryIndx == -1)
            startingBoundaryIndx = emailText.IndexOf(startingBoundary);

          if (startingBoundaryIndx != -1)
          {
            int nextBoundaryIndx = emailText.IndexOf(startingBoundary,
              startingBoundaryIndx + startingBoundary.Length);

            if (nextBoundaryIndx != -1 && nextBoundaryIndx != startingBoundaryIndx)
            {
              string multipartMsg = emailText.Substring(startingBoundaryIndx +
                startingBoundary.Length,
                (nextBoundaryIndx - startingBoundaryIndx - startingBoundary.Length));

              int headersIndx = multipartMsg.IndexOf("\r\n\r\n");

              if (headersIndx == -1)
                throw new FormatException("Incompatible multipart message format");

              string bodyText = multipartMsg.Substring(headersIndx).Trim();

              NameValueCollection headers = Util.ParseHeaders(multipartMsg.Trim());
              messageParts.Add(new MessagePart(headers, bodyText));
            }
            else
              break;

            startingBoundaryIndx = nextBoundaryIndx;
          }
          else
            break;
        }

        if (newLinesIndx != -1)
        {
          string emailBodyText = emailText.Substring(newLinesIndx + 1);
        }
      }
      else
      {
        int headersIndx = emailText.IndexOf("\r\n\r\n");

        if (headersIndx == -1)
          throw new FormatException("Incompatible multipart message format");

        string bodyText = emailText.Substring(headersIndx).Trim();

        NameValueCollection headers = Util.ParseHeaders(emailText);
        messageParts.Add(new MessagePart(headers, bodyText));
      }

      return messageParts;
    }

The code continues on the next page.

The code is continuing from the previous page.

public static DateTime ConvertStrToUtcDateTime(string str)
    {
      Match m = UtcDateTimeRegex.Match(str);

      int day, month, year, hour, min, sec;

      if (m.Success)
      {
        day = Convert.ToInt32(m.Groups["day"].Value);
        year = Convert.ToInt32(m.Groups["year"].Value);
        hour = Convert.ToInt32(m.Groups["hour"].Value);
        min = Convert.ToInt32(m.Groups["minute"].Value);
        sec = Convert.ToInt32(m.Groups["second"].Value);

        switch (m.Groups["month"].Value)
        {
          case "Jan":
            month = 1;
            break;
          case "Feb":
            month = 2;
            break;
          case "Mar":
            month = 3;
            break;
          case "Apr":
            month = 4;
            break;
          case "May":
            month = 5;
            break;
          case "Jun":
            month = 6;
            break;
          case "Jul":
            month = 7;
            break;
          case "Aug":
            month = 8;
            break;
          case "Sep":
            month = 9;
            break;
          case "Oct":
            month = 10;
            break;
          case "Nov":
            month = 11;
            break;
          case "Dec":
            month = 12;
            break;
          default:
            throw new FormatException("Unknown month.");
        }

        string offsetSign = m.Groups["offsetsign"].Value;
        int offsetHours = Convert.ToInt32(m.Groups["offsethours"].Value);
        int offsetMinutes = Convert.ToInt32(m.Groups["offsetminutes"].Value);

        DateTime dt = new DateTime(year, month, day, hour, min, sec);

        if (offsetSign == "+")
        {
          dt.AddHours(offsetHours);
          dt.AddMinutes(offsetMinutes);
        }
        else if (offsetSign == "-")
        {
          dt.AddHours(-offsetHours);
          dt.AddMinutes(-offsetMinutes);
        }

        return dt;
      }

      throw new FormatException("Incompatible date/time string format");
    }
  }
}

Explanation
Pop3.cs defines Stardeveloper.Pop3 namespace, which contains following 4 classes:

Pop3Client - This class uses classes in the System.Net and sub-namespaces to connect with the Pop3 server and send commands. It retrieves the data, and uses the Util class to create Email and MessagePart objects from the received data.
Email - It represents an Email message on the server. It provides you access to the "From", "To", "Subject", "Date" and "Content-Type" headers. It does not let you access the body of the email. The body of the email is represented by an array of MessagePart objects which are retrieved separately by calling Pop3Client.FetchMessageParts() method.
MessagePart - It represents a part of the body of the email. Since most of the emails are MIME encoded, which means that there can be more than one type (text/plain, text/html, etc.) of message part, and that there can be attached files, we represent the body of the email with an array of MessagePart objects. This class lets you access the headers and body of a message part of the email. More on it, later.
Util - This class contains 3 static methods to parse headers, message parts, and to convert UTC date/time string to DateTime object.

We will now look at the code of each of these classes in detail.

The first thing we do is to import the necessary namespaces.

using System;
using System.Collections.Generic;
using System.Collections.Specialized;
using System.Configuration;
using System.IO;
using System.Linq;
using System.Net;
using System.Net.Security;
using System.Net.Sockets;
using System.Text;
using System.Text.RegularExpressions;
using System.Web;
using System.Xml;

The Pop3Client class implements IDisposable interface.

public class Pop3Client : IDisposable
{
	...
}

Following properties provide access to the variables we set in the constructor of Pop3Client class.

public string Host { get; protected set; }
public int Port { get; protected set; }
public string Email { get; protected set; }
public string Password { get; protected set; }
public bool IsSecure { get; protected set; }

The TcpClient object that is used internally by Pop3Client class is exposed using Client property. The ClientStream property is either of the SslStream type if SSL is used to connect to the Pop3 server, or it is of NetworkStream type if the connection is unsecure.

public TcpClient Client { get; protected set; }
public Stream ClientStream { get; protected set; } // SslStream or NetworkStream

Since all POP3 commands sent by the client and responses received from the server end in the new-line characters ("\r\n"), we use StreamWriter and StreamReader classes to simplify the job of sending and receiving the data for us. You will see plenty of calls to their WriteLine() and ReadLine() methods, respectively, later.

public StreamWriter Writer { get; protected set; }
public StreamReader Reader { get; protected set; }

Lastly, a private variable is used to set the disposed state for this object. Remember, Pop3Client class implements IDisposable interface to make it easy for you to wrap access to this class in a using statement.

private bool disposed = false;

The constructor of Pop3Client class takes as its arguments the name of the host you want to connect to e.g. "pop.gmail.com", the port of the server you want to connect to e.g., 995 for Gmail, email address and password of your account on that Pop3 server. By default, the constructor will create an unsecure connection to the server. To use SSL to connect to the server, you have to set the final and optional argument to the constructor to true.

public Pop3Client(string host, int port, string email, string password)
	: this(host, port, email, password, false)
{
}

public Pop3Client(string host, int port, string email,
	string password, bool secure)
{
	Host = host;
	Port = port;
	Email = email;
	Password = password;
	IsSecure = secure;
}

The Connect() method instantiates necessary objects and connects to the POP3 server on the given host and port. If it was specified in the constructor to open a secure connection to the POP3 server, a SslStream object is created which encapsulates the NetworkStream returned from the TcpClient.GetStream() method. Lastly, StreamWriter and StreamReader objects are instantiated, and Login() method is called.

public void Connect()
{
	if (Client == null)
		Client = new TcpClient();

	if (!Client.Connected)
		Client.Connect(Host, Port);

	if (IsSecure)
	{
		SslStream secureStream =
			new SslStream(Client.GetStream());
		secureStream.AuthenticateAsClient(Host);

		ClientStream = secureStream;
		secureStream = null;
	}
	else
		ClientStream = Client.GetStream();

	Writer = new StreamWriter(ClientStream);
	Reader = new StreamReader(ClientStream);

	ReadLine();
	Login();
}

The Login() method sends the user email and password to the server. Each call to send the command with required argument is sent using SendCommand() method, which internally uses StreamWriter.WriteLine() method to send the command. The response of the server is parsed using IsResponseOk() method to determine if the response was successfull i.e., it began with an "+OK" message or if it was an error i.e., it began with an "-ERR" message. If everything went fine, the method simply returns, otherwise an Exception is thrown.

Note: All responses received from the server start with either "+OK", if +ve response has been generated on the server, or with "-ERR", if an error occurred.

protected void Login()
{
	if (!IsResponseOk(SendCommand("USER " + Email)) ||
		!IsResponseOk(SendCommand("PASS " + Password)))
		throw new Exception("User/password not accepted");
}

As explained above, the SendCommand() method uses StreamWriter.WriteLine() to send the command to the server. We use WriteLine() and not Write() because each command to the server must end with new-line characters i.e., "\r\n". Since StreamWriter internally buffers the data we want sent to the server, we call its Flush() method to make it send the command immediately. The response that is received from the server is fetched using StreamReader.ReadLine() method and is returned to the calling code.

protected string SendCommand(string cmdtext)
{
	Writer.WriteLine(cmdtext);
	Writer.Flush();

	return ReadLine();
}

As explained above, the IsResponseOk() method parses the response received from the server and checks for the "+OK" and "-ERR" strings to determine if the response was ok or if an error has been returned.

protected static bool IsResponseOk(string response)
{
	if (response.StartsWith("+OK"))
		return true;
	if (response.StartsWith("-ERR"))
		return false;

	throw new Exception("Cannot understand server response: " +
		response);
}

GetEmailCount() returns the number of emails in the mailbox. It uses "STAT" POP3 command to retrieve the number of emails. As you can see in the code below, the response of the server is parsed and the word containing number of emails is taken and converted to an integer, which is then returned to the calling code.

public int GetEmailCount()
{
	int count = 0;
	string response = SendCommand("STAT");

	if (IsResponseOk(response))
	{
		string[] arr = response.Substring(4).Split(' ');
		count = Convert.ToInt32(arr[0]);
	}
	else
		count = -1;

	return count;
}

FetchEmail() takes the sort number of the email as its argument and retrieves the headers for that email using "TOP" POP3 command. This command takes two parameters, the sort no of email and the number of lines of the body of email you want to fetch. We give this command the sort no of the email to fetch and 0 (zero) as the number of lines of text to retrieve from the body of the email. We set the second parameter to 0 because we only want to retrieve headers. The headers returned are used to instantiate Email object and that is then returned to the calling code.

public Email FetchEmail(int emailId)
{
	if (IsResponseOk(SendCommand("TOP " + emailId + " 0")))
		return new Email(ReadLines());
	else
		return null;
}

FetchEmailList() fetches the email headers, using FetchEmail() method, of given number of emails starting from given sort number. For instance, you have 300 emails in your mailbox and want to now display a list of 10 emails from 150 to 160, you should call this method as: FetchEmailList(151, 10).

 public List<Email> FetchEmailList(int start, int count)
{
	List<Email> emails = new List<Email>(count);

	for (int i = start; i < (start + count); i++)
	{
		Email email = FetchEmail(i);

		if (email != null)
			emails.Add(email);
	}

	return emails;
}

FetchMessageParts() retrieves an email from the server and then parses its body into an array of MessagePart objects. Each MessagePart object encapsulates one part of the body of the email and any attached files.

public List<MessagePart> FetchMessageParts(int emailId)
{
	if (IsResponseOk(SendCommand("RETR " + emailId)))
		return Util.ParseMessageParts(ReadLines());

	return null;
}

Close() logs out the user, closes the connection, and properly disposes all the Stream objects in use.

public void Close()
{
	if (Client != null)
	{
		if (Client.Connected)
			Logout();

		Client.Close();
		Client = null;
	}

	if (ClientStream != null)
	{
		ClientStream.Close();
		ClientStream = null;
	}

	if (Writer != null)
	{
		Writer.Close();
		Writer = null;
	}

	if (Reader != null)
	{
		Reader.Close();
		Reader = null;
	}

	disposed = true;
}

Dispose() method is implemented as part of the implementation of IDisposable interface. It calls the Close() method above if not already called.

public void Dispose()
{
	if (!disposed)
		Close();
}

Logout() method does nothing but to send the "RSET" command to the server which resets the server meaning there by that if any emails were marked as deleted, they are unmarked. Although no part in our code marks the emails to be deleted, we call it anyway.

protected void Logout()
{
	SendCommand("RSET");
}

We will now look at the code that constitutes the Email class.

public class Email
{
	...
}

An Email object represents an email on the server minus its body. It contains all the headers contained in an email. It provides access to the headers' collection using Headers property. Commonly used headers like "From", "To", "Subject", "Date", "Content-Type" can be accessed using properites like From, To, Subject, UtcDateTime, and ContentType, respectively.

Note: The purpose of properties mentioned above is only to provide easy access to header fields. For practical purposes code like Email.Headers["From"] and Email.From will yield the same value. And if any of these headers is not found in the email, the property will return null.

public NameValueCollection Headers { get; protected set; }

public string ContentType { get; protected set; }
public DateTime UtcDateTime { get; protected set; }
public string From { get; protected set; }
public string To { get; protected set; }
public string Subject { get; protected set; }

Within the constructor of Email class, we retrieve header fields as a NameValueCollection object using Util.ParseHeaders() method. Once we have the collection, we set the properties to the values retrieved from the header fields.

Unlike other properties, the property Date is not of type string. It is of type DateTime. We parse the Headers["Date"] header string and convert it in to a DateTime structure using Util.ConvertStrToUtcDateTime() method.

public Email(string emailText)
{
	Headers = Util.ParseHeaders(emailText);

	ContentType = Headers["Content-Type"];
	From = Headers["From"];
	To = Headers["To"];
	Subject = Headers["Subject"];

	if (Headers["Date"] != null)
		try
		{
			UtcDateTime = Util.ConvertStrToUtcDateTime(Headers["Date"]);
		}
		catch (FormatException)
		{
			UtcDateTime = DateTime.MinValue;
		}
	else
		UtcDateTime = DateTime.MinValue;
}

We will now look at the code of MessagePart class.

public class MessagePart
{
	...
}

A MessagePart object represents just one component of the body of an email. Remember, that most emails are MIME compatible and have multiple parts that form the body of that email.

For instance, you send an HTML email to a person. Just in case if the person is using an email client that cannot render HTML emails, either you or your email client also creates a plain text version of the email and appends it to the email. Now your email has two parts; one is of text/html content-type and the other is of text/plain content-type. The email client of the person you sent that email to will upon receiving that email try to find a plain text version of the email so that it can display that to the person without any difficulty. On searching through the MIME compatible email's message parts, it finds a version of text/plain content-type. It parses and decodes (if needed) it and displays it to that person.

Now if you had attached 2 files with that email, the total message parts would have become four:

HTML version of the email - text/html
Plain text version of the email - text/plain
Attached File #1 - base64
Attached File #2 - base64

Now upon parsing this email using the code above, you'll have an array of four MessagePart objects. I hope you understand now what MessagePart represents.

Note: base64 is a content-transfer-encoding for sending binary data like images files, in a text based email message.

It has a Headers property that makes available all the headers that this message part contains. The ContentType property is there to easily access the Headers["Content-Type"] field. The MessageText property makes available the content of the body of this message part.

public NameValueCollection Headers { get; protected set; }

public string ContentType { get; protected set; }
public string MessageText { get; protected set; }

In the constructor, we set the properties.

public MessagePart(NameValueCollection headers, string messageText)
{
	Headers = headers;
	ContentType = Headers["Content-Type"];
	MessageText = messageText;
}

We will now look at the code of Util class, which contains 3 static methods.

public class Util
{
	...
}

ParseHeaders() parses all the headers and converts them into easily accessible NameValueCollection.

public static NameValueCollection ParseHeaders(string headerText)
{
	NameValueCollection headers = new NameValueCollection();
	StringReader reader = new StringReader(headerText);

	string line;
	string headerName = null, headerValue;
	int colonIndx;

	while ((line = reader.ReadLine()) != null)
	{
		if (line == "")
			break;

		if (Char.IsLetterOrDigit(line[0]) && (colonIndx = line.IndexOf(':')) != -1)
		{
			headerName = line.Substring(0, colonIndx);
			headerValue = line.Substring(colonIndx + 1).Trim();

			headers.Add(headerName, headerValue);
		}
		else if (headerName != null)
			headers[headerName] += " " + line.Trim();
		else
			throw new FormatException("Could not parse headers");
	}

	return headers;
}

ParseMessageParts() parses the body of an email and separates the body into separate message parts using the boundary header.

public static List<MessagePart> ParseMessageParts(string emailText)
    {
      List<MessagePart> messageParts = new List<MessagePart>();
      int newLinesIndx = emailText.IndexOf("\r\n\r\n");

      Match m = BoundaryRegex.Match(emailText);

      if (m.Index < emailText.IndexOf("\r\n\r\n") && m.Success)
      {
        string boundary = m.Groups["boundary"].Value;
        string startingBoundary = "\r\n--" + boundary;

        int startingBoundaryIndx = -1;

        while (true)
        {
          if (startingBoundaryIndx == -1)
            startingBoundaryIndx = emailText.IndexOf(startingBoundary);

          if (startingBoundaryIndx != -1)
          {
            int nextBoundaryIndx = emailText.IndexOf(startingBoundary,
              startingBoundaryIndx + startingBoundary.Length);

            if (nextBoundaryIndx != -1 && nextBoundaryIndx != startingBoundaryIndx)
            {
              string multipartMsg = emailText.Substring(startingBoundaryIndx +
                startingBoundary.Length,
                (nextBoundaryIndx - startingBoundaryIndx - startingBoundary.Length));

              int headersIndx = multipartMsg.IndexOf("\r\n\r\n");

              if (headersIndx == -1)
                throw new FormatException("Incompatible multipart message format");

              string bodyText = multipartMsg.Substring(headersIndx).Trim();

              NameValueCollection headers = Util.ParseHeaders(multipartMsg.Trim());
              messageParts.Add(new MessagePart(headers, bodyText));
            }
            else
              break;

            startingBoundaryIndx = nextBoundaryIndx;
          }
          else
            break;
        }

        if (newLinesIndx != -1)
        {
          string emailBodyText = emailText.Substring(newLinesIndx + 1);
        }
      }
      else
      {
        int headersIndx = emailText.IndexOf("\r\n\r\n");

        if (headersIndx == -1)
          throw new FormatException("Incompatible multipart message format");

        string bodyText = emailText.Substring(headersIndx).Trim();

        NameValueCollection headers = Util.ParseHeaders(emailText);
        messageParts.Add(new MessagePart(headers, bodyText));
      }

      return messageParts;
    }

ConvertStrToUtcDateTime() parses a date & time string and converts it into a DateTime structure. It makes use of a regular expression to parse individual components from the date & time string.

public static DateTime ConvertStrToUtcDateTime(string str)
{
	Match m = UtcDateTimeRegex.Match(str);

	int day, month, year, hour, min, sec;

	if (m.Success)
	{
		day = Convert.ToInt32(m.Groups["day"].Value);
		year = Convert.ToInt32(m.Groups["year"].Value);
		hour = Convert.ToInt32(m.Groups["hour"].Value);
		min = Convert.ToInt32(m.Groups["minute"].Value);
		sec = Convert.ToInt32(m.Groups["second"].Value);

		switch (m.Groups["month"].Value)
		{
			case "Jan":
				month = 1;
				break;
			case "Feb":
				month = 2;
				break;
			case "Mar":
				month = 3;
				break;
			case "Apr":
				month = 4;
				break;
			case "May":
				month = 5;
				break;
			case "Jun":
				month = 6;
				break;
			case "Jul":
				month = 7;
				break;
			case "Aug":
				month = 8;
				break;
			case "Sep":
				month = 9;
				break;
			case "Oct":
				month = 10;
				break;
			case "Nov":
				month = 11;
				break;
			case "Dec":
				month = 12;
				break;
			default:
				throw new FormatException("Unknown month.");
		}

		string offsetSign = m.Groups["offsetsign"].Value;
		int offsetHours = Convert.ToInt32(m.Groups["offsethours"].Value);
		int offsetMinutes = Convert.ToInt32(m.Groups["offsetminutes"].Value);

		DateTime dt = new DateTime(year, month, day, hour, min, sec);

		if (offsetSign == "+")
		{
			dt.AddHours(offsetHours);
			dt.AddMinutes(offsetMinutes);
		}
		else if (offsetSign == "-")
		{
			dt.AddHours(-offsetHours);
			dt.AddMinutes(-offsetMinutes);
		}

		return dt;
	}

	throw new FormatException("Incompatible date/time string format");
}

Now that we have created the necessary classes, we will create the front-end ASP.NET page to display the list of emails to the user.

ii. Pop3Client.aspx
Open Notepad and copy following text in it, and then save it as "Pop3Client.aspx" in your ASP.NET web application:

<%@ Page Language="C#" %>

<%@ Import Namespace="System.Collections.Generic" %>
<%@ Import Namespace="Stardeveloper.Pop3" %>

<script runat="server">
	public const string Host = "pop.gmail.com";
	public const int Port = 995;
	public const string Email = "youremail@gmail.com";
	public const string Password = "yourpassword";

	public const int NoOfEmailsPerPage = 5;
	public const string SelfLink =
		"<a href=\"Pop3Client.aspx?page={0}\">{1}</a>";
	public const string DisplayEmailLink =
		"<a href=\"DisplayPop3Email.aspx?emailId={0}\">{1}</a>";

	protected void Page_Load(object source, EventArgs e)
	{
		/* Setting current page */
		int page = 1;

		if (Request.QueryString["page"] == null)
		{
			Response.Redirect("Pop3Client.aspx?page=1");
			Response.Flush();
			Response.End();
		}
		else
			page = Convert.ToInt32(Request.QueryString["page"]);

		/* Creating Pop3Client object and accessing email data */
		int totalEmails;
		List<Email> emails;
		string emailAddress;

		using(Pop3Client client = new Pop3Client(Host, Port, Email, Password, true))
		{
			emailAddress = client.Email;

			client.Connect();

			totalEmails = client.GetEmailCount();
			emails = client.FetchEmailList(((page - 1) *
				NoOfEmailsPerPage) + 1, NoOfEmailsPerPage);
		}

		/* Computing total pages */
		int totalPages;
		int mod = totalEmails % NoOfEmailsPerPage;

		if (mod == 0)
			totalPages = totalEmails / NoOfEmailsPerPage;
		else
			totalPages = ((totalEmails - mod) / NoOfEmailsPerPage) + 1;

		for (int i = 0; i < emails.Count; i++)
		{
			Email email = emails[i];

			/* Appending each email to an existing table */
			int emailId = ((page - 1) * NoOfEmailsPerPage) + i + 1;

			TableCell noCell = new TableCell();
			noCell.CssClass = "emails-table-cell";
			noCell.Text = Convert.ToString(emailId);
			
			TableCell fromCell = new TableCell();
			fromCell.CssClass = "emails-table-cell";
			fromCell.Text = email.From;
			
			TableCell subjectCell = new TableCell();
			subjectCell.CssClass = "emails-table-cell";
			subjectCell.Style["width"] = "300px";
			subjectCell.Text = String.Format(DisplayEmailLink,
				emailId, email.Subject);

			//TableCell contentTypeCell = new TableCell();
			//contentTypeCell.CssClass = "emails-table-cell";
			//contentTypeCell.Text = email.ContentType;
			
			TableCell dateCell = new TableCell();
			dateCell.CssClass = "emails-table-cell";
			
			if (email.UtcDateTime != DateTime.MinValue)
				dateCell.Text = email.UtcDateTime.ToString();

			TableRow emailRow = new TableRow();
			emailRow.Cells.Add(noCell);
			emailRow.Cells.Add(fromCell);
			emailRow.Cells.Add(subjectCell);
			//emailRow.Cells.Add(contentTypeCell);
			emailRow.Cells.Add(dateCell);

			EmailsTable.Rows.AddAt(2 + i, emailRow);
		}

		/* Creating links to previous and next pages */
		if (totalPages > 1)
		{
			if (page > 1)
				PreviousPageLiteral.Text =
					String.Format(SelfLink, page - 1, "Previous Page");

			if (page > 0 && page < totalPages)
				NextPageLiteral.Text =
					String.Format(SelfLink, page + 1, "Next Page");
		}

		/* Displaying information to the user */
		EmailFromLiteral.Text =
			Convert.ToString(((page - 1) * NoOfEmailsPerPage) + 1);
		EmailToLiteral.Text = Convert.ToString(page * NoOfEmailsPerPage);
		EmailTotalLiteral.Text = Convert.ToString(totalEmails);
		EmailLiteral.Text = emailAddress;
	}
</script>

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
	"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<html xmlns="http://www.w3.org/1999/xhtml">
<head runat="server">
    <title></title>
    <style type="text/css">
		.emails-table { width: 600px; border: solid 1px #444444; }
		.emails-table-header { font-family: "Trebuchet MS"; font-size: 9pt;
			background-color: #0099B9; color: white;
			border: solid 1px #444444; }
		.emails-table-header-cell { font-family: "Georgia"; font-size: 9pt;
			font-weight: bold; border: solid 1px #666666; padding: 6px; }
		.emails-table-cell { font-family: "Georgia"; font-size: 9pt;
			border: solid 1px #666666; padding: 6px; }
		.emails-table-footer { border: solid 1px #666666; padding: 3px;
			width: 50%; }
		.email-datetime { float: right; color: #666666; }
		
		a { font-family: "Lucida Sans Unicode", "Trebuchet MS"; font-size: 9pt;
			color: #005B7F; }
		a:hover { color:red; }
    </style>
</head>
<body>
    <form id="form1" runat="server">
    <div>
		<asp:Table ID="EmailsTable" CssClass="emails-table" runat="server">
		<asp:TableHeaderRow>
			<asp:TableHeaderCell CssClass="emails-table-header"
				ColumnSpan="4">Listing emails 
			<asp:Literal ID="EmailFromLiteral" runat="server" />-
			<asp:Literal ID="EmailToLiteral" runat="server" /> of 
			<asp:Literal ID="EmailTotalLiteral" runat="server" /> for 
			<asp:Literal ID="EmailLiteral" runat="server" />
			</asp:TableHeaderCell>
		</asp:TableHeaderRow>
		<asp:TableRow>
			<asp:TableCell CssClass="emails-table-header-cell">
				#</asp:TableCell>
			<asp:TableCell CssClass="emails-table-header-cell">
				From</asp:TableCell>
			<asp:TableCell CssClass="emails-table-header-cell">
				Subject</asp:TableCell>
			<asp:TableCell CssClass="emails-table-header-cell">
				Date &amp; Time</asp:TableCell>
		</asp:TableRow>
		<asp:TableFooterRow>
			<asp:TableCell CssClass="emails-table-footer" ColumnSpan="4">
			<asp:Table ID="FooterTable" Width="100%"
				BorderWidth="0" runat="server">
			<asp:TableRow>
			<asp:TableCell>
				<asp:Literal ID="PreviousPageLiteral" runat="server" />
			</asp:TableCell>
			<asp:TableCell HorizontalAlign="Right">
				<asp:Literal ID="NextPageLiteral" runat="server" />
			</asp:TableCell>
			</asp:TableRow>
			</asp:Table>
			</asp:TableCell>
		</asp:TableFooterRow>
		</asp:Table>
    </div>
    </form>
</body>
</html>

Explanation
This ASP.NET page is responsible for using the Pop3Client class we developed earlier to display list of emails in Gmail's mailbox, page by page. The user will be able to browse all the pages using 'Next Page' and 'Previous Page' links.

The first thing we do is to declare a Page level directive and tell the ASP.NET compiler that the language we are going to use on this page is "C#".

<%@ Page Language="C#" %>

Next, we import the necessary namespaces.

<%@ Import Namespace="System.Collections.Generic" %>
<%@ Import Namespace="Stardeveloper.Pop3" %>

The server-side code is encapsulated between the <script></script> tags.

<script runat="server">
	...
</script>

We declare a few constants. The first four are required to properly connect and login to the Gmail's POP3 server. The Host and Port constants can be changed to point to your email provider's POP3 server, if it is not Gmail. For the purpose of demonstration, we will connect to Gmail's POP3 server. The Email and Password constants should be changed to your email address and password.

NoOfEmailsPerPage constant is used to declare how many emails we want to show per page. You can change it to 10 or 20 to suit the needs of your web application. SelfLink and DisplayLink constants contain strings for formatting links and to display the full email in the "DisplayPop3Email.aspx" ASP.NET page, which we will create in the next tutorial.

public const string Host = "pop.gmail.com";
public const int Port = 995;
public const string Email = "youremail@gmail.com";
public const string Password = "yourpassword";

public const int NoOfEmailsPerPage = 5;
public const string SelfLink =
	"<a href=\"Pop3Client.aspx?page={0}\">{1}</a>";
public const string DisplayEmailLink =
	"<a href=\"DisplayPop3Email.aspx?emailId={0}\">{1}</a>";

Now we will dig into the all important code in the Page_Load() method.

protected void Page_Load(object source, EventArgs e)
{
	...
}

If you access Pop3Client.aspx directly, it will redirect you to Pop3Client.aspx?page=1. The current page is retrieved and saved in the page variable.

/* Setting current page */
int page = 1;

if (Request.QueryString["page"] == null)
{
	Response.Redirect("Pop3Client.aspx?page=1");
	Response.Flush();
	Response.End();
}
else
	page = Convert.ToInt32(Request.QueryString["page"]);

Now we create the Pop3Client object. We create it within the using statement so that we don't explicitly have to call the Pop3Client.Close() method. Gorgeous!

We call the GetEmailCount() method to get the number of emails in the mailbox. Next, we retrieve the list of emails for the current page.

Note: The number of emails in the list will be a maximum of NoOfEmailsPerPage i.e., 5. We do not retrieve all the emails in the mailbox. We retrieve only those that are required to be displayed on the current page.

/* Creating Pop3Client object and accessing email data */
int totalEmails;
List<Email> emails;
string emailAddress;

using(Pop3Client client = new Pop3Client(Host, Port, Email, Password, true))
{
	emailAddress = client.Email;

	client.Connect();

	totalEmails = client.GetEmailCount();
	emails = client.FetchEmailList(((page - 1) *
		NoOfEmailsPerPage) + 1, NoOfEmailsPerPage);
}

Next, we compute the total number of pages the user can browse through.

/* Computing total pages */
int totalPages;
int mod = totalEmails % NoOfEmailsPerPage;

if (mod == 0)
	totalPages = totalEmails / NoOfEmailsPerPage;
else
	totalPages = ((totalEmails - mod) / NoOfEmailsPerPage) + 1;

Next, we iterate through the retrieved list of emails, one by one, and append their information to an HTML table which we will display to the user.

for (int i = 0; i < emails.Count; i++)
{
	Email email = emails[i];

	/* Appending each email to an existing table */
	int emailId = ((page - 1) * NoOfEmailsPerPage) + i + 1;

	TableCell noCell = new TableCell();
	noCell.CssClass = "emails-table-cell";
	noCell.Text = Convert.ToString(emailId);
	
	TableCell fromCell = new TableCell();
	fromCell.CssClass = "emails-table-cell";
	fromCell.Text = email.From;
	
	TableCell subjectCell = new TableCell();
	subjectCell.CssClass = "emails-table-cell";
	subjectCell.Style["width"] = "300px";
	subjectCell.Text = String.Format(DisplayEmailLink,
		emailId, email.Subject);

	//TableCell contentTypeCell = new TableCell();
	//contentTypeCell.CssClass = "emails-table-cell";
	//contentTypeCell.Text = email.ContentType;
	
	TableCell dateCell = new TableCell();
	dateCell.CssClass = "emails-table-cell";
	
	if (email.UtcDateTime != DateTime.MinValue)
		dateCell.Text = email.UtcDateTime.ToString();

	TableRow emailRow = new TableRow();
	emailRow.Cells.Add(noCell);
	emailRow.Cells.Add(fromCell);
	emailRow.Cells.Add(subjectCell);
	//emailRow.Cells.Add(contentTypeCell);
	emailRow.Cells.Add(dateCell);

	EmailsTable.Rows.AddAt(2 + i, emailRow);
}

Next, we create 'Previous Page' and 'Next Page' links.

/* Creating links to previous and next pages */
if (totalPages > 1)
{
	if (page > 1)
		PreviousPageLiteral.Text =
			String.Format(SelfLink, page - 1, "Previous Page");

	if (page > 0 && page < totalPages)
		NextPageLiteral.Text =
			String.Format(SelfLink, page + 1, "Next Page");
}

Lastly, we set a few literals to display on the page.

/* Displaying information to the user */
EmailFromLiteral.Text =
	Convert.ToString(((page - 1) * NoOfEmailsPerPage) + 1);
EmailToLiteral.Text = Convert.ToString(page * NoOfEmailsPerPage);
EmailTotalLiteral.Text = Convert.ToString(totalEmails);
EmailLiteral.Text = emailAddress;

We are now all set to run this page on our ASP.NET web server. But before that, there is one final step left, and that is to enable POP3 access for our mailbox on Gmail.

Enabling POP3 access on Gmail
Follow these steps to enable POP3 access for your email address on Gmail:

Login to your account at Gmail.
Click the "Settings" link on the top right.
From the 'Settings' panel, click the "Forwarding and POP/IMAP" tab.
In the 'POP Download' section, select the option "Enable POP for all mail (even mail that's already been downloaded)"
Make sure 'When messages are accessed with POP' is set to "keep Gmail's copy in the Inbox". This is always a safe bet to keep your emails in the Gmail's inbox and not have them deleted when you retrieve them from Gmail's POP3 server.
Now click 'Save Changes' button to save the changes.

Running the Pop3Client.aspx Page
After placing "Pop3.cs" in the /App_Code sub-folder and Pop3Client.aspx in the / folder of my ASP.NET web application, I ran the Pop3Client.aspx page and got following result:

I hope you were also able to successfully connect to your email's mailbox on the remote server and were able to view the list of emails in it.

Note: It is recommended that you study the code in full to understand how we are connecting to, sending and receiving data over the network. If your POP3 server doesn't support secure access over SSL, you should set the secure argument in the constructor of Pop3Client class to false.

Summary
In this tutorial, we developed code to connect to Gmail's secure POP3 server using SSL. We learned about TcpClient, NetworkStream, SslStream, StreamWriter and StreamReader classes. Using these classes we created our own set of top level classes to make the task of communicating with the server, easier. An ASP.NET page in the end, using the code that we developed, successfully displayed the list of emails retrieved from the server.

In the next tutorial (Displaying Emails from Mailbox on Gmail using POP3 and ASP.NET), we will create an ASP.NET page that will retrieve the actual email from the POP3 server and display it to the user.

Displaying Twitter Updates in a Page by Page fashion with ASP.NET

Faisal Khan — Sat, 20 Jun 2009 06:49:00 GMT

Introduction
In this tutorial, we will develop a technique to retrieve and display a page full of Twitter updates to the user, giving him the ability to move to the next page. The user will be able to browse all the pages using 'Previous Page' and 'Next Page' links. Our technique will be highly efficient as we will be retrieving only enough number of Twitter updates from a given screen name, to display on a single page. Only when the user browses to the next page will another set of Twitter updates be retrieved. At the same time we will cache the results, page by page, giving seamless and smooth performance to the end user.

The code in this tutorial makes use of the code we developed in the previous tutorial: Retrieving Twitter Updates with ASP.NET. We will only be updating TwitterClient.cs a little bit to enable retrieval of Twitter updates page by page. Rest of the files will remain the same. We will develop an ASP.NET page to demonstrate the technique.

I advise that you read this series of tutorials from the beginning and create/place the files as advised in the previous tutorial.

This tutorial is 3rd in the series of tutorials regarding Twitter and ASP.NET:

Introduction to Twitter for ASP.NET Developers
Retrieving Twitter Updates with ASP.NET
Displaying Twitter Updates in a Page by Page fashion with ASP.NET

Demonstration of Twitter Updates being Displayed Page by Page
In this tutorial, we will develop an ASP.NET page that retrieves 5 status updates for a given screen name and displays them on a page. When 'Next Page' link is clicked, next 5 status updates are retrieved and displayed to the user.

Note: 5 is an arbitrary number. You can modify the code to display any number of status updates per page.

Following screen shots show the ASP.NET page displaying Twitter status updates in a page by page fashion:

Let us now go ahead and develop the necessary code.

Developing the Code
We need to update TwitterClient.cs file to accomodate the retrieval of Twitter status updates page by page.

Note: It is absolutely necessary to create and place the files as described in Retrieving Twitter Updates with ASP.NET tutorial. Without which you will not be able to make the changes as described below.

Updated TwitterClient.cs
You should replace the code in TwitterClient.cs with the code given below. The updated sections of the code have been highlighted.

using System;
using System.Collections.Generic;
using System.Configuration;
using System.IO;
using System.Linq;
using System.Net;
using System.Text;
using System.Text.RegularExpressions;
using System.Web;
using System.Xml;
using Stardeveloper.Twitter;

namespace Stardeveloper.Twitter
{
	public class TwitterClient
	{
	public const string StatusesUserTimelineStr =
		"http://twitter.com/statuses/user_timeline.xml?" +
		"screen_name={0}&count={1}&page={2}";

	public TwitterClient()
	{
	}

	protected virtual string FetchXmlFromUrl(string url)
	{
		using (WebClient client = new WebClient())
		using (Stream s = client.OpenRead(url))
		using (StreamReader sr = new StreamReader(s))
		{
			return sr.ReadToEnd();
		}
	}

The code continues on the next page.

protected virtual List<Status>
		CreateStatusObjectsFromXml(string xml, int count)
	{
		List<Status> statuses = new List<Status>(count);

		XmlDocument xmldoc = new XmlDocument();
		xmldoc.LoadXml(xml);

		foreach (XmlNode status in xmldoc.DocumentElement)
		{
		// created_at
		string value = status.SelectSingleNode("created_at").InnerXml;
		DateTime statusCreatedAt = ConvertStrToUtcDateTime(value);

		// id
		value = status.SelectSingleNode("id").InnerXml;
		long statusId = Convert.ToInt64(value);

		// text
		string text = status.SelectSingleNode("text").InnerXml;

		// source
		string source = status.SelectSingleNode("source").InnerXml;

		// truncated
		value = status.SelectSingleNode("truncated").InnerXml;
		bool truncated = Convert.ToBoolean(value);

		// in_reply_to_status_id
		value = status.SelectSingleNode("in_reply_to_status_id").InnerXml;
		long inReplyToStatusId = value != "" ? Convert.ToInt64(value) : 0;

		// in_reply_to_user_id
		value = status.SelectSingleNode("in_reply_to_user_id").InnerXml;
		long inReplyToUserId = value != "" ? Convert.ToInt64(value) : 0;

		// favorited
		value = status.SelectSingleNode("favorited").InnerXml;
		bool favorited = Convert.ToBoolean(value);

		// in_reply_to_screen_name
		string inReplyToScreenName =
			status.SelectSingleNode("in_reply_to_screen_name").InnerXml;

		/* Getting into user element */
		XmlNode user = status.SelectSingleNode("user");

		// id
		value = user.SelectSingleNode("id").InnerXml;
		long userId = Convert.ToInt64(value);

		// name
		string name = user.SelectSingleNode("name").InnerXml;

		// screen_name
		string screenName = user.SelectSingleNode("screen_name").InnerXml;

		// location
		string location = user.SelectSingleNode("location").InnerXml;

		// description
		string description = user.SelectSingleNode("description").InnerXml;

		// profile_image_url
		string profileImageUrl =
			user.SelectSingleNode("profile_image_url").InnerXml;

		// url
		string url = user.SelectSingleNode("url").InnerXml;

		// protected
		value = user.SelectSingleNode("protected").InnerXml;
		bool protectedstr = Convert.ToBoolean(value);

		// followers_count
		value = user.SelectSingleNode("followers_count").InnerXml;
		long followersCount = Convert.ToInt64(value);

		// profile_background_color
		string profileBackgroundColor =
			user.SelectSingleNode("profile_background_color").InnerXml;

		// profile_text_color
		string profileTextColor =
			user.SelectSingleNode("profile_text_color").InnerXml;

		// profile_link_color
		string profileLinkColor =
			user.SelectSingleNode("profile_link_color").InnerXml;

		// profile_sidebar_fill_color
		string profileSidebarFillColor =
			user.SelectSingleNode("profile_sidebar_fill_color")
				.InnerXml;

		// profile_sidebar_border_color
		string profileSidebarBorderColor =
			user.SelectSingleNode("profile_sidebar_border_color")
				.InnerXml;

		// friends_count
		value = user.SelectSingleNode("friends_count").InnerXml;
		long friendsCount = Convert.ToInt64(value);

		// created_at
		value = user.SelectSingleNode("created_at").InnerXml;
		DateTime userCreatedAt = TwitterClient.ConvertStrToUtcDateTime(value);

		// favourites_count
		value = user.SelectSingleNode("favourites_count").InnerXml;
		long favouritesCount = Convert.ToInt64(value);

		// utc_offset
		value = user.SelectSingleNode("utc_offset").InnerXml;
		int utcOffset = Convert.ToInt32(value);

		// time_zone
		string timeZone = user.SelectSingleNode("time_zone").InnerXml;

		// profile_background_image_url
		string profileBackgroundImageUrl =
			user.SelectSingleNode("profile_background_image_url").InnerXml;

		// profile_background_tile
		value = user.SelectSingleNode("profile_background_tile").InnerXml;
		bool profileBackgroundTile = Convert.ToBoolean(value);

		// statuses_count
		value = user.SelectSingleNode("statuses_count").InnerXml;
		long statusesCount = Convert.ToInt64(value);

		// notifications
		string notifications = user.SelectSingleNode("notifications").InnerXml;

		// following
		string following = user.SelectSingleNode("following").InnerXml;

		/* Creating User object */
		User userobj = new User(userId, name, screenName, location,
			description, profileImageUrl, url, protectedstr,
			followersCount, profileBackgroundColor, profileTextColor,
			profileLinkColor, profileSidebarFillColor,
			profileSidebarBorderColor, friendsCount, userCreatedAt,
			favouritesCount, utcOffset, timeZone,
			profileBackgroundImageUrl, profileBackgroundTile,
			statusesCount, notifications, following);

		Status statusobj = new Status(statusCreatedAt, statusId,
			text, source, truncated, inReplyToStatusId, inReplyToUserId,
			favorited, inReplyToScreenName, userobj);

		statuses.Add(statusobj);
		}

		return statuses;
	}

	public Status GetLatestStatusUpdate(string screenName)
	{
		List<Status> statuses = GetStatusUpdates(screenName, 1);

		return statuses[0];
	}

	public List<Status> GetStatusUpdates(string screenName, int count)
	{
		return GetStatusUpdates(screenName, count, 1);
	}

	public List<Status> GetStatusUpdates(string screenName, int count,
		int page)
	{
		string url = String.Format(StatusesUserTimelineStr,
			screenName, count, page);
		string xml = FetchXmlFromUrl(url);

		return CreateStatusObjectsFromXml(xml, count);
	}

	public static DateTime ConvertStrToUtcDateTime(string str)
	{
		string[] arr = str.Split(' ');

		int day, month, year, hour, min, sec;

		switch (arr[1])
		{
			case "Jan":
				month = 1;
				break;
			case "Feb":
				month = 2;
				break;
			case "Mar":
				month = 3;
				break;
			case "Apr":
				month = 4;
				break;
			case "May":
				month = 5;
				break;
			case "Jun":
				month = 6;
				break;
			case "Jul":
				month = 7;
				break;
			case "Aug":
				month = 8;
				break;
			case "Sep":
				month = 9;
				break;
			case "Oct":
				month = 10;
				break;
			case "Nov":
				month = 11;
				break;
			case "Dec":
				month = 12;
				break;
			default:
				throw new FormatException("Unknown month.");
		}

		day = Convert.ToInt32(arr[2]);

		string[] arr1 = arr[3].Split(':');

		hour = Convert.ToInt32(arr1[0]);
		min = Convert.ToInt32(arr1[1]);
		sec = Convert.ToInt32(arr1[2]);

		string offsetSign = arr[4].Substring(0, 1);
		int offsetHours = Convert.ToInt32(arr[4].Substring(1, 2));
		int offsetMinutes = Convert.ToInt32(arr[4].Substring(3, 2));

		year = Convert.ToInt32(arr[5]);

		DateTime dt = new DateTime(year, month, day, hour, min, sec);

		if (offsetSign == "+")
		{
			dt.AddHours(offsetHours);
			dt.AddMinutes(offsetMinutes);
		}
		else if (offsetSign == "-")
		{
			dt.AddHours(-offsetHours);
			dt.AddMinutes(-offsetMinutes);
		}

		return dt;
	}
	}
}

Code is explained on the next page.

Explanation
The URL to retrieve Twitter status updates remains the same but we add an extra querystring parameter to tell Twitter which page we want to retrieve. The screen_name should be set to the screen name of the user whose status updates we want to retrieve. The count parameter should be set to a numeric value of number of status updates you want to show per page. The page parameter should be set to a numeric value of the page whose status updates you want to retrieve, beginning from 1. You don't have to manually set these values. It is only for you to understand what is going on behind the scenes.

public const string StatusesUserTimelineStr =
	"http://twitter.com/statuses/user_timeline.xml?" +
	"screen_name={0}&count={1}&page={2}";

Next we add another GetStatusUpdates() method with the extra option of specifying the page you want to retrieve status updates from.

public List<Status> GetStatusUpdates(string screenName, int count,
	int page)
{
	string url = String.Format(StatusesUserTimelineStr,
		screenName, count, page);
	string xml = FetchXmlFromUrl(url);

	return CreateStatusObjectsFromXml(xml, count);
}

This is it as far as the changes in TwitterClient.cs are concerned.

TwitterAllUpdates.aspx
The logic to pull the the right page and display the status updates properly is handled by this ASP.NET page. Open notepad and copy following code in it and then save it as "TwitterAllUpdates.aspx" in the main folder of your ASP.NET web application:

<%@ Page Language="C#" %>

<%@ OutputCache VaryByParam="page" Duration="1200" %>

<%@ Import Namespace="System.Collections.Generic" %>
<%@ Import Namespace="Stardeveloper.Twitter" %>

<script runat="server">
public const int StatusUpdatesPerPage = 5;
public const string ScreenName = "Stardeveloper";
public const string LinkText =
	"<a href=\"TwitterAllUpdates.aspx?page={0}\">{1}</a>";

protected void Page_Load(object source, EventArgs e)
{
	/* Setting current page and total pages */
	int page = 1;
	int totalPages = -1;

	if (Request.QueryString["page"] == null)
		Response.Redirect("TwitterAllUpdates.aspx?page=1");
	else
		page = Convert.ToInt32(Request.QueryString["page"]);
	
	/* Retrieving status updates in real-time */
	TwitterClient client = new TwitterClient();
	List<Status> statuses = client.GetStatusUpdates(ScreenName,
		StatusUpdatesPerPage, page);

	foreach (Status status in statuses)
	{
		/* Appending each status to existing table */
		TableCell statusCell = new TableCell();
		statusCell.ColumnSpan = 2;
		statusCell.CssClass = "status-updates-table-cell";

		StringBuilder statusBuilder = new StringBuilder();
		statusBuilder.Append(status.FormattedText);
		statusBuilder.Append("<div class=\"status-datetime\">");
		statusBuilder.Append(status.CreatedAt.ToShortDateString());
		statusBuilder.Append(" ");
		statusBuilder.Append(status.CreatedAt.ToShortTimeString());
		statusBuilder.Append("</div>");
		
		statusCell.Text = statusBuilder.ToString();

		TableRow statusRow = new TableRow();
		statusRow.Cells.Add(statusCell);

		StatusUpdatesTable.Rows.Add(statusRow);

		/* Computing total possible pages */
		if (totalPages == -1)
		{
			int statusesCount =
				Convert.ToInt32(status.User.StatusesCount);
			int mod = statusesCount % StatusUpdatesPerPage;
			
			if (mod == 0)
				totalPages = statusesCount / StatusUpdatesPerPage;
			else
				totalPages = ((statusesCount - mod) /
					StatusUpdatesPerPage) + 1;
		}
	}

	/* Creating links to previous and next pages */
	if (totalPages > 1)
	{
		TableCell prevPageCell = new TableCell();
		TableCell nextPageCell = new TableCell();

		prevPageCell.CssClass = "status-updates-table-footer";
		nextPageCell.CssClass = "status-updates-table-footer";
		nextPageCell.Style["text-align"] = "right";

		if (page > 1)
			prevPageCell.Text = String.Format(LinkText, page - 1,
				"Previous Page");

		if (page > 0 && page < totalPages)
			nextPageCell.Text = String.Format(LinkText, page + 1,
				"Next Page");

		TableFooterRow footerRow = new TableFooterRow();
		footerRow.Cells.Add(prevPageCell);
		footerRow.Cells.Add(nextPageCell);

		StatusUpdatesTable.Rows.Add(footerRow);
	}

	/* Setting some values to display to the user */
	ScreenNameLiteral.Text = ScreenName;
	PageLiteral.Text = Convert.ToString(page);
	TotalPagesLiteral.Text = Convert.ToString(totalPages);
}
</script>

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
	"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<html xmlns="http://www.w3.org/1999/xhtml">
<head runat="server">
    <title></title>
    <style type="text/css">
		.status-updates-table { width: 400px; border: solid 1px #444444; }
		.status-updates-table-header { font-family: "Trebuchet MS";
			font-size: 9pt; background-color: #0099B9; color: white;
			border: solid 1px #444444; }
		.status-updates-table-cell { font-family: "Georgia";
			font-size: 9pt; border: solid 1px #666666; padding: 6px; }
		.status-updates-table-footer { border: solid 1px #666666;
			padding: 3px; width: 50%; }
		.status-datetime { float: right; color: #666666; }
		
		a { font-family: "Lucida Sans Unicode",
			"Trebuchet MS"; font-size: 9pt;
			color: #005B7F; }
		a:hover { color:red; }
    </style>
</head>
<body>
    <form id="form1" runat="server">
	<asp:Table ID="StatusUpdatesTable"
		CssClass="status-updates-table" runat="server">
	<asp:TableHeaderRow>
		<asp:TableHeaderCell CssClass="status-updates-table-header"
		ColumnSpan="2">
		<asp:Literal ID="ScreenNameLiteral" runat="server" /> Updates (page 
		<asp:Literal ID="PageLiteral" runat="server" /> of 
		<asp:Literal ID="TotalPagesLiteral" runat="server" />)
		</asp:TableHeaderCell>
	</asp:TableHeaderRow>
	</asp:Table>
    </form>
</body>
</html>

Explanation
The first Page level directive tells ASP.NET compiler that the default language in this page is going to be C#.

<%@ Page Language="C#" %>

Next, an OutputCache directive tells the ASP.NET runtime to cache the output of this page for 20 minutes (20 min x 60 sec = 1200 sec). The VaryByParam attribute is set to "page" which makes sure that a new copy of the output of this page will be cached for each variation of "page" querystring parameter.

<%@ OutputCache VaryByParam="page" Duration="1200" %>

Necessary namespaces are imported. System.Collections.Generic namespace is necessary because we want to use generic List<T> class. Stardeveloper.Twitter namespace is necessary because we want to make use of TwitterClient, Status, and User classes, we developed earlier.

<%@ Import Namespace="System.Collections.Generic" %>
<%@ Import Namespace="Stardeveloper.Twitter" %>

A server-side script tag tells the ASP.NET compiler that this segment is going to contain server-side code.

<script runat="server">
	...
</script>

We define 3 page level constant variables. StatusUpdatesPerPage variable is set to the number of status updates we want to display per page. You can change this value to suit the need of your web application. ScreenName variable is set to the screen name of the user whose status updates you want to retrieve from Twitter. LinkText contains string which contains placeholders which will be set by the code later. This variable will be used to create 'Previous Page' and 'Next Page' links, in the table, under the status updates, later.

public const int StatusUpdatesPerPage = 5;
public const string ScreenName = "Stardeveloper";
public const string LinkText =
	"<a href=\"TwitterAllUpdates.aspx?page={0}\">{1}</a>";

Next, we place all of our important code in the Page_Load() method.

protected void Page_Load(object source, EventArgs e)
{
	...
}

Within the Page_Load() method, we start by declaring two important variables. page should be set to the page whose status updates we want to retrieve from Twitter and display to the user. totalPages should be set to the total number of pages that are available for this screen name. If a user has 100 status updates at Twitter and we want to show 20 updates per page, then totalPages should be set to 5.

If the querystring parameter, "page", is not set, the we set the querystring parameter and redirect the user to the same page. Otherwise, we fetch and set the page variable's value to it.

/* Setting current page and total pages */
int page = 1;
int totalPages = -1;

if (Request.QueryString["page"] == null)
	Response.Redirect("TwitterAllUpdates.aspx?page=1");
else
	page = Convert.ToInt32(Request.QueryString["page"]);

We then create TwitterClient object and retrieve given status updates for a given page from Twitter. Once we have the list of status updates as Status objects, we iterate through them, one by one, and populate our HTML table with its values.

/* Retrieving status updates in real-time */
TwitterClient client = new TwitterClient();
List<Status> statuses = client.GetStatusUpdates(ScreenName,
	StatusUpdatesPerPage, page);

foreach (Status status in statuses)
{
	...
}

For each Status object, we fetch formatted text by accessing the Status.FormattedText property and append to it date & time of the status update. This text is then appended as an HTML row to an HTML table we have already placed as a user control in our ASP.NET page.

/* Appending each status to existing table */
TableCell statusCell = new TableCell();
statusCell.ColumnSpan = 2;
statusCell.CssClass = "status-updates-table-cell";

StringBuilder statusBuilder = new StringBuilder();
statusBuilder.Append(status.FormattedText);
statusBuilder.Append("<div class=\"status-datetime\">");
statusBuilder.Append(status.CreatedAt.ToShortDateString());
statusBuilder.Append(" ");
statusBuilder.Append(status.CreatedAt.ToShortTimeString());
statusBuilder.Append("</div>");

statusCell.Text = statusBuilder.ToString();

TableRow statusRow = new TableRow();
statusRow.Cells.Add(statusCell);

StatusUpdatesTable.Rows.Add(statusRow);

Last bit of code checks and sets the totalPages variable to total number of pages for this screen name using given status updates per page.

/* Computing total possible pages */
if (totalPages == -1)
{
	int statusesCount =
		Convert.ToInt32(status.User.StatusesCount);
	int mod = statusesCount % StatusUpdatesPerPage;
	
	if (mod == 0)
		totalPages = statusesCount / StatusUpdatesPerPage;
	else
		totalPages = ((statusesCount - mod) /
			StatusUpdatesPerPage) + 1;
}

Once all Status objects' formatted text has been appended to the HTML table in our ASP.NET page, we append 'Previous Page' and 'Next Page' links to the HTML table using the logic given below:

/* Creating links to previous and next pages */
if (totalPages > 1)
{
	TableCell prevPageCell = new TableCell();
	TableCell nextPageCell = new TableCell();

	prevPageCell.CssClass = "status-updates-table-footer";
	nextPageCell.CssClass = "status-updates-table-footer";
	nextPageCell.Style["text-align"] = "right";

	if (page > 1)
		prevPageCell.Text = String.Format(LinkText, page - 1,
			"Previous Page");

	if (page > 0 && page < totalPages)
		nextPageCell.Text = String.Format(LinkText, page + 1,
			"Next Page");

	TableFooterRow footerRow = new TableFooterRow();
	footerRow.Cells.Add(prevPageCell);
	footerRow.Cells.Add(nextPageCell);

	StatusUpdatesTable.Rows.Add(footerRow);
}

Lastly, 3 Literal controls' values are set. These values will be displayed to the user.

/* Setting some values to display to the user */
ScreenNameLiteral.Text = ScreenName;
PageLiteral.Text = Convert.ToString(page);
TotalPagesLiteral.Text = Convert.ToString(totalPages);

StatusUpdatesTable is the id of the HTML table that will display the Twitter status updates to the user in our ASP.NET page.

<asp:Table ID="StatusUpdatesTable"
	CssClass="status-updates-table" runat="server">
<asp:TableHeaderRow>
	<asp:TableHeaderCell CssClass="status-updates-table-header"
	ColumnSpan="2">
	<asp:Literal ID="ScreenNameLiteral" runat="server" /> Updates (page 
	<asp:Literal ID="PageLiteral" runat="server" /> of 
	<asp:Literal ID="TotalPagesLiteral" runat="server" />)
	</asp:TableHeaderCell>
</asp:TableHeaderRow>
</asp:Table>

When this ASP.NET page was run on my computer, it produced following output:

Summary
In this tutorial, we updated the code to add paging ability in our ASP.NET web application. We demonstrated the code by developing an ASP.NET page which displayed Twitter status updates, page by page.

In the next tutorial, we will learn how to display "followers' count" on our website for a given screen name.

Stardeveloper

Real-World example of load time improvement using HTTP static resource caching

Tools required to measure the load time of any web page

Load time improvements

Load time of the sample page the first time it was requested

Load time of the sample page the second time it was requested

What if you had not enabled static resource caching in IIS?

Conclusion

Setting Expires and Max-Age Headers for static resources in ASP.NET

What is Caching and why does it matter?

Introduction to Expires and Max-Age headers

How to make changes to static files whose expiry dates have been set to far in the future?

How to set Expires and Max-Age headers in ASP.NET?

Setting Max-Age header

Setting Expires header

Which is better? Expires or Max-Age?

Setting up SPF Record for BlackBerry Internet Service

Sending Emails with BlackBerry

Including BlackBerry SMTP Servers in your domain's SPF Records

BIS NA (North America) BlackBerry

BIS EU (Europe) BlackBerry

Final Sample SPF Record

Conclusion

SPF Record Syntax

What should be included in an SPF record?

Introduction to SPF record syntax

Quantifiers

Mechanisms

ip4 mechanism

ip6 mechanism

a mechanism

mx mechanism

include mechanism

ptr mechanism

exists mechanism

all mechanism

SPF record generating tips

Conclusion

What is Sender Policy Framework?

How does SPF work?

An example SPF record

Explanation

How to check the SPF record for a domain name?

Should you have an SPF record for your website?

Conclusion

Read Next

Convert IP Address to Long

Do not store IP Addresses as strings

Solution: Convert IP Addresses to Long values

Converting IP Addresses to Long Values and Back in C#

Converting IP Addresses to Bigint Values and Back in T-SQL

Final Words

Introduction to Developing HTTP Modules in ASP.NET

Displaying Emails from Mailbox on Gmail using POP3 and ASP.NET

Connecting to Mailbox on Gmail and Fetching List of Emails using POP3 and ASP.NET

Displaying Twitter Updates in a Page by Page fashion with ASP.NET