Making Authenticated HTTP Requests from an ASP.NET Page

Performing HTTP requests from a web page - a task commonly referred to as "screen scraping" - involves server-side code issuing an HTTP request to some other Web site, retrieving the returned results, and processing these results in some manner. For example, screen scraping is oftentimes used to grab data from another site, such as scraping the HTML from a Yahoo! Finance page to grab the current stock price for a particular stock symbol. Performing simple HTTP requests in ASP.NET requires just a few lines of code, thanks to the WebClient class. This class, found in the System.Net namespace, provides a small number of properties and methods useful for making simple HTTP requests. A previous 4Guys article of mine, Screen Scrapes in ASP.NET, illustrates how to use the WebClient class from an ASP.NET page.

RssFeed, a custom, compiled ASP.NET server control I created for displaying RSS feeds in an ASP.NET page, uses programmatic HTTP requests to grab the syndicated content from a specified URL. Recently a user of RssFeed asked me if RssFeed provided support for RSS feeds that required authentication. That is, this user wanted to display the contents of an RSS feed that was only accessible by providing authentication information, such as a username and password. While RssFeed itself didn't provide this functionality, the underlying classes used by RssFeed to access the remote RSS feed do, so I added this functionality. (For more information on displaying RSS content in your ASP.NET website, be sure to read A Custom ASP.NET Server Control for Displaying RSS Feeds.)

In this article we will first discuss the common authentication protocols used by web servers and then look at how to make programmatic HTTP requests to a resource that requires authentication. Read on to learn more!
Read More >

Read More...

Comments

Post Your Comment:

Your Name:*

e-mail ID:(required for notification)*

Image Verification: 

 

 Subscribe