Export (0) Print
Expand All
Expand Minimize
4 out of 237 rated this helpful - Rate this topic

Fun with IXMLHttpRequest and RSS

 

Dare Obasanjo
Microsoft Corporation

May 15, 2005

Summary: Dare Obasanjo shows how to create a Web page that can be used to browse various photo albums on MSN Spaces using IXMLHttpRequest and the RSS feeds provided by MSN Spaces and MSN Search. (11 printed pages)

Introduction

Recently there's been a lot of buzz about using Javascript and IXMLHttpRequest in Web applications, generated in large part by the recent article Ajax: A New Approach to Web Applications by Jesse James Garrett, which coined the term Asynchronous Javascript and XML (AJAX). The renaissance of the technique formerly known as remote scripting is primarily due to a number of high profile uses of this technique by Google Web sites, such as Gmail, Google Suggest, and Google Maps. Given the renewed interest in using XML in the browser, I decided to try my hand at building a Web page that utilized AJAX.

Around the same time I decided to try my hand at building an AJAX application, I came across an article by Daniel Steinberg entitled Bosworth's Web of Data. This article described a recent keynote by Adam Bosworth at the MySQL Users Conference 2005 where he predicted that RSS 2.0 and Atom would become the lingua franca for information interchange on the Web. RSS first gained popularity as a way to syndicate blog posts and news sites, but has turned out to be a lot more versatile than that. Sites like Feedster and Amazon's OpenSearch technology show that you can use RSS as a mechanism for providing search results and integrating search engines respectively. Podcasting shows that you can use RSS to syndicate digital media content instead of just plain old text or HTML. With Amazon's syndicated feeds you can keep abreast of when new CDs, books, and more are released. The list of innovative uses of RSS goes on.

In this month's article, I decided to combine both trends—the renewed interest in IXMLHttpRequest and the rise of RSS as a general data format for the Web. I looked around at work to see what kind of interesting application I could build with the various RSS feeds produced by MSN properties and was quite surprised by the results. Read on for the details.

Overview of IXMLHttpRequest

The XMLHttp COM object which implements the IXMLHttpRequest interface was introduced in MSXML 2.0 as a way for client computers to send HTTP requests and retrieve XML data from a server. The typical usage pattern is to call the open method, set any custom header information through the setRequestHeader method, call the send method, and finally check one of the four different response properties (responseBody, responseStream, responseText, responseXML) for the returned content. The following JScript code sample shows the typical usage of the XMLHTTP object.

var xmlhttp = new ActiveXObject("MSXML2.XMLHTTP.3.0");
xmlhttp.open("GET", "http://www.example.com/books.xml", false);
xmlhttp.send();
alert(xmlhttp.responseText);

In certain cases an application might want to utilize the asynchronous loading characteristics of the XMLHTTP object. Asynchronous loading should be used when the application wants the XML file to be downloaded in the background while other tasks are being performed. The typical usage pattern is to call the open method, set any custom header information using the setRequestHeader method, specify a callback function to be invoked when the download state changes using the onreadystatechange property, and finally calling the send method. Each time the state of the Web connection changes, the callback method specified in the onreadystatechange property is called. The callback method should test whether the readyState property of the XMLHTTP object has the value COMPLETED (4), which indicates that all the data has been downloaded. When this is the case, the XML returned by the server can be processed.

The following JScript code sample shows the typical usage of the XMLHTTP object in asynchronous scenarios.

var xmlhttp=null;

function PostOrder(xmldoc)
{
  var xmlhttp = new ActiveXObject("MSXML2.XMLHTTP.3.0");
  xmlhttp.Open("GET", "http://www.example.com/books.xml", true); 
  xmlhttp.onreadystatechange= HandleStateChange;
  xmlhttp.Send(xmldoc);
}

function HandleStateChange()
{
  if (xmlhttp.readyState == 4)
  {
    alert("Result = " + xmlhttp.responseXML.xml);
  }
}

Using the MSN Search RSS Feeds to Locate Random Spaces

The Web page I decided to build was one which enables people to browse various photo albums on MSN Spaces by entering a specific URL or by requesting that the photos on a randomly selected space be shown. The Web page is available at http://www.25hoursaday.com/spaces/photobrowser.html.

Note   The PhotoBrowser application accesses information from multiple Internet domains. Depending on your Internet Explorer security settings, cross-domain data access may be set to prompt or disable. Please see the MSDN documentation on XML Client Security for more details on XML security in Internet Explorer.

The input form on the page has two options. The user can enter the name of a space which is then passed to the getPhotos()function, or the user can request that a photo be selected from a randomly selected space using the getRandomSpace()function. The HTML for the input form is shown below.

<form>
      http://spaces.msn.com/members/ 
      <input type="text" name="spacename" value="mike">
      <input type="button" value="Browse Photos" onClick="getPhotos(this.form.spacename.value)">
      <input type="button" value="Find Random Space" onClick="getRandomSpace()">
</form>

The trickiest part of writing the code for displaying photos from random spaces was to figure out how to get a list of spaces with photo albums in the first place. I settled on issuing the query "site:spaces.msn.com photo album" on MSN Search and requesting the results as an RSS feed.

When the page is first loaded, the setStuffUp() function initiates the slide show and fetches the RSS feed from MSN Search by calling the getRssFeed() function.

function setStuffUp() {
  runSlideShow()
  getRssFeed(
'http://search.msn.com/results.aspx?q=site%3aspaces.msn.com+photo+album&format=rss&count=100')
}

The getRssFeed()function is a helper function that creates the XMLHttp object, specifies the name of the call back that should be invoked asynchronously for processing the XML results, and then sends the request.

var req 
function getRssFeed(url) {
    req = new ActiveXObject("MSXML2.XMLHTTP.3.0");
    req.onreadystatechange = processRssFeed;
    req.open("GET", url, true);
    req.send(); 
}

The processRssFeed() function determines if the feed is from MSN Search or from MSN Spaces and then calls the appropriate function to handle the feed.

function processRssFeed() {
  // only if req shows "complete"
  if (req.readyState == 4) {
    // only if "OK"
    if (req.status == 200) {
      var doc  = req.responseXML;
      if(doc.documentElement.getAttribute('xmlns:msn')!= null) {
        displayPhotos(doc)
      } else {
        loadRandomSpaces(doc)
      }
    } else {      
      alert("There was a problem retrieving the RSS feed:\n" +
 req.statusText + " " + req.status + ":" + req.responseXML.xml);
    }
  }
}

The processRssFeed() function tests that the attempt to download the RSS feed was successful. Upon confirming that the download has been completed, the resulting XML is obtained as an instance of the IXMLDOMDocument. Because this method is used to parse both the RSS results from MSN Spaces and from MSN Search, a test for the existence of a namespace declaration for the "http://schemas.microsoft.com/msn/spaces/2005/rss" namespace is used to determine what kind of feed is being processed. If the RSS feed is from MSN Search, then the XML document is passed to the loadRandomSpaces() method.

In the loadRandomSpaces() function, the value of the link element of every RSS item in the resulting feed is obtained using the following XPath query; "//item/link". This is followed by a loop that extracts the name of the space from the URLs returned by the search query and then places them in the randomSpaces variable. Once this task is completed, the randomSpacesLoaded variable is set to true.

var randomSpaces = new Array();
var randomSpacesLoaded = false;

function loadRandomSpaces(xmlDoc){
  var spaceNames = xmlDoc.selectNodes("//item/link");
  var spacesUrl  = "http://spaces.msn.com/members/";
  var str = '';
  var index;

  for (var i=0; i < spaceNames.length; i++) {
    str = spaceNames(i).text.substring(spacesUrl.length);
    index = str.indexOf("/"); 
    if(index != -1){
      str = str.substring(0, index);
    }
    randomSpaces.push(str);
  }
  randomSpacesLoaded = true;
}

To select a random space, the user has to click a button that invokes the getRandomSpace() function. The function picks a space name at random from the randomSpaces variable and then invokes the getRssFeed() function with the URL to the RSS feed to the randomly chosen space.

function getRandomSpace(){

  if(randomSpacesLoaded){

   var randIndex = Math.round(Math.random()* (randomSpaces.length - 1));
 
   document.forms[0].spacename.value = randomSpaces[randIndex]
   document.getElementById('photoAlbumsDiv').innerText = 'Downloading photos...'
   getRssFeed('http://spaces.msn.com/members/' + randomSpaces[randIndex] + '/feed.rss');  

  }else{
     setTimeout('getRandomSpaces()', 3000)
  }

 }

Retrieving Photo Albums from an MSN Spaces RSS Feed

Every public MSN Space can provide an RSS feed that contains the most recent blog entries, lists, and photo albums posted to the space. To distinguish between the various item types in the RSS feed, an extension element named msn:type from the "http://schemas.microsoft.com/msn/spaces/2005/rss" namespace is used. The valid values for the msn:type element are photoalbum, blogentry, list, musiclist, booklist, and bloglist. For this application I targeted items within the RSS feed for which the value of the msn:type element was "photoalbum".

When the user clicks the Browse Photos button, the getPhotos()function is invoked with the name of the space in the input box as its parameter. The getPhotos() function tests that a space name was specified then replaces the existing photos being displayed on the page (if any) with the text "Downloading photos...". Finally, the function invokes the getRssFeed()function with the URL to the RSS feed for the specified space as input.

function getPhotos(spaceName){
   if (spaceName == ""){
    alert("Please provide the name of an MSN Space or click the 'Find 
       Random Space' button to choose one at random")
    return
  }
  document.getElementById('photoAlbumsDiv').innerText = 'Downloading photos...'
  getRssFeed('http://spaces.msn.com/members/' + spaceName + '/feed.rss');  
}

As mentioned earlier, the getRssFeed()function is a helper method that creates a Web request and specifies the processRssFeed() function as the callback to be invoked once downloading is complete. The processRssFeed() function tests whether the RSS feed is from MSN Spaces and invokes the displayPhotos() function if this is the case.

After the processRssFeed() function tests that the attempt to download the RSS feed from the space is successful, the resulting XML is obtained as an instance of the IXMLDOMDocument. Because this method is used to parse both the RSS results from MSN Spaces and from MSN Search, a test for the existence of a namespace declaration for the "http://schemas.microsoft.com/msn/spaces/2005/rss" namespace is used to determine what kind of feed is being processed. If the RSS feed is from MSN Spaces, then the XML document is passed to the displayPhotos() method.

function displayPhotos(xmlDoc){
    //select all photo album items in RSS feed
  xmlDoc.setProperty("SelectionNamespaces", 
     "xmlns:msn='http://schemas.microsoft.com/msn/spaces/2005/rss'")

  var photoAlbums = xmlDoc.selectNodes("//item[msn:type='photoalbum']");

  //clear existing contents of page
  var photoDiv = document.getElementById('photoAlbumsDiv');
  photoDiv.innerText = '';
  var newContent = '';
  photoAlbumArray = new Array();
  photoAlbumIndexes = new Array();
  var albumImages;
  //create temporary HTML DOM node for holding photo album content from feed
  var tempNode =  document.createElement('temp');
  //display photo albums
  for (var i=0; i < photoAlbums.length; i++) {
    newContent += renderPhoto(i, photoAlbums(i), tempNode);
  }

  if(photoAlbums.length > 0) {
    newContent += '<p>The images shown above are a subset of the photos in each photo album.' + 
                  'To view the entire photo album click the link to the album.</p>';
  } else {
    newContent = '<p>No recent photo albums were found for this user</p>';
  }

  photoDiv.innerHTML = newContent;  
} 

In the displayPhotos() function, the first step is to retrieve all the RSS items representing a photo album by using the XPath query "//item[msn:type='photoalbum']". The next step is to set up the data structures needed for the slide transitions. The photoAlbumArray variable is an array of arrays, where each sub-array contains the URLs to the images in a particular photo album. The photoAlbumIndexes variable is an array that contains the indexes for the images currently being displayed from the corresponding arrays in the photoAlbumArray variable. This is followed by a loop where each RSS item representing a photo album is processed through the renderPhoto() function.

var photoAlbumArray;
var photoAlbumIndexes;

function renderPhoto(i, node, tempNode) {
  var newContent = '';

  newContent += '<a href=' + node.selectSingleNode('link').text + '>' + 
      node.selectSingleNode('title').text + '</a><br>';

  //get all img elements from description and place in temporary DHTML node 
  //to make them easier to manipulate
  tempNode.innerHTML = node.selectSingleNode('description').text;
  var photos = tempNode.getElementsByTagName('img');

  //prepare data structures and position indexes for slideshow
  photoAlbumArray[i] = new Array();
  photoAlbumIndexes[i] = 0;

  //store image links in array so we can reference them for slideshow later
  for(var j = 0; j < photos.length; j++){
    var imgsrc = photos[j].getAttribute('src')
    if(imgsrc.indexOf('storage.msn.com') != -1)
      photoAlbumArray[i].push(imgsrc)   
  }

  newContent += '<img src=' + photoAlbumArray[i][0]  + '/><br>';
  
  return newContent;
}

In the renderPhoto() function, HTML is constructed to display the title of the photo album, as well as display the first image in the album. Also, all image URLs in the album are placed in an array within the photoAlbumArray variable. Secondly, the corresponding array index within the photoAlbumIndexes variable is initialized with the value 0 to indicate that the first image in the current array within the photoAlbumArray is being displayed. After all the RSS items have been processed, the newly constructed HTML replaces the current text on the Web page in the displayPhotos() function.

The runSlideShow() function is the method that manages the slideshow transitions for the various photo albums on the page. It is included below for completeness.

function runSlideShow(){
  for(var i = 0; i < document.images.length; i++){
    document.images[i].style.filter="blendTrans(duration=2)"
    document.images[i].style.filter="blendTrans(duration=crossFadeDuration)"
    document.images[i].filters.blendTrans.Apply()      
        
    var k = photoAlbumIndexes[i]
    document.images[i].src = photoAlbumArray[i][k]   
    document.images[i].filters.blendTrans.Play()
    k = k + 1
    if (k == photoAlbumArray[i].length) k=0
    photoAlbumIndexes[i] = k
  }
  setTimeout('runSlideShow()', 5000)
}

Other Interesting Uses of MSN Search RSS Feeds

The fact that search results from MSN Search can be consumed as RSS opens up a lot of possibilities for applications that want to interact with the search engine. The PhotoBrowser application shows one way that the MSN Search RSS feeds can be used outside a traditional RSS reader. There are many other interesting uses that can be explored.

A few months ago Robert Scoble wrote a post titled Yahoo announces API for its search engine where he asked the following:

"Seriously. Blogs are increasing noise to lots of searches. We already have good engines that let you search blogs (Feedster, Pubsub, Newsgator, Technorati, and Bloglines all are letting you search blogs). What about an engine that lets you search everything BUT blogs? Where's that?
Is Yahoo's API good enough to do that? It doesn't look like it. It looks like Yahoo just gave us an API to embed its search engine into our applications. Sigh. That's not what I want. OK, MSN, your turn. Are you gonna really give us an API that'll let us build a custom search engine and let us have access to the variables that determine the result set?"

The first question Robert asks is hard, but you can take shortcuts to get approximate results. How do you determine what a blog is? Do you simply exclude all results from LiveJournal, Blogspot, and MSN Spaces? That would exclude millions of blogs, but it wouldn't catch the various blogs on self-hosted domains. Of course, you could get even trickier by always asking to exclude pages that match certain words like "DasBlog", "Movable Type", or "WordPress", which would probably take out another large chunk. By then the search results would probably be as blog-free as you can get without resorting to expensive matching techniques. For icing on the cake it would probably be useful to also be able to skew results by popularity or freshness.

The second question Scoble asks is whether there is a search engine that gives you an API that can do all this stuff. Well, MSN Search gives you RSS feeds that as we've seen from this article can sometimes be the only API your Web site needs. More importantly, as pointed out in a recent blog post by Andy Edmonds entitled Search Builder Revealed, you can control how variables, such as popularity or freshness, affect search results. For example,

  1. Search results for "star wars revenge of the sith" sorted by popularity
  2. Search results for "star wars revenge of the sith" sorted by freshness

One could probably write a first cut at the search engine Robert is asking for using the MSN Search RSS feeds in about an hour or so. In a day, it could be made to be quite polished with most of the work being in the user interface. Yet another coding project for a rainy day. If you end up building this application, send me an e-mail and I'll link to your work in my next article.

Note   The RSS feeds provided by MSN Search and MSN Spaces are only meant for personal, non-commercial use. Any other usage requires express written permission from the Microsoft Corporation. In addition, the structure of the contents of the feeds may change over time as improvements or modifications are made to both services. In the event of such changes applications that depend on their structure, such as the photo browser page referenced by this article, may cease to work properly.

Conclusion

Thanks for sticking with me while I described my first AJAX application. Although this technique is quite old in Web years and was originally pioneered by applications such as Outlook Web Access, it has gained renewed interest among Internet developers.

While writing this article I began to get a new appreciation for the amount of opportunity for data integration and manipulation that exists now that there are so many rich sources of information exposed as RSS. XML syndication has definitely grown to be more than just about syndicating content from blogs and news sites. As exciting as things are now, it's definitely just the beginning. I can't wait to see where this ride ends.

Acknowledgements

I'd like to thank Derek Denny-Brown, Adam Wiener, and Alex Krawarik for their ideas and feedback while writing this article.

Dare Obasanjo is a program manager on the MSN Communication Services Platform team. He brings his love of solving problems with XML to building the server infrastructure utilized by the MSN Messenger, MSN Hotmail, and MSN Spaces teams.

Feel free to post any questions or comments about this article on the Extreme XML message board on GotDotNet.

Did you find this helpful?
(1500 characters remaining)
Thank you for your feedback
Show:
© 2014 Microsoft. All rights reserved.