RSS Support in Longhorn
Summary: This paper describes RSS in the context of a vision for how Web content is and will be consumed, explores opportunities enabled by incorporating RSS functionality into applications, and outlines the Windows platform services that make this easy. (13 printed pages)
With increasing adoption of RSS, the concept of "subscribing" to content on the Web is becoming more prevalent. As this idea continues to gain in popularity, users will be able to subscribe not only to blogs and news, but also to content feeds (audio, video, documents, photos, calendar events, and so on) and lists (play lists, book lists, link lists, blogrolls, wish lists, top-10 lists, to-do lists, and so on). The next version of Windows, code-named "Longhorn," introduces rich support for RSS in the platform.
As the Web increases in content and usage, the way that users interact with it has become increasingly sophisticated.
Browsing has always been the centerpiece of the Web experience. Typing in URLs and following links remains the primary way to find and consume information.
Increasingly, users have learned to rely on sophisticated search engines to help them navigate the abundance of content and information that the Web has to offer. For many, search has become the primary way to begin navigating the Web.
Search is the first step on the road of harnessing the power of computers to do work on the user's behalf.
More recently, smart software is enabling an increasingly sophisticated and powerful model of interacting with the Web: After finding interesting information, users are using RSS technologies to "subscribe" to the content, ensuring that the latest changes or updates to the content are delivered to them, instead of having to return to the site to check it again.
This model of subscribing to content is pervasive, both within the technical community, and in other areas. Once users get used to it, they want it everywhere. In a similar way, users of a PVR (Personal Video Recording) device for TV will often claim that they'll "never watch TV the same way again."
The widespread usage of RSS heralds a shift to the new model of subscription. One estimate holds that there are over 60 million blogs worldwide as of April 2005 (including 14 million hosted by MSN Spaces), a majority of which are available via RSS (or similar formats). Every major online news site syndicates their news stories via an RSS feed, and several other sites are making their content syndicated via RSS. The impact of RSS cannot be denied.
Today, RSS is primarily used for news sites and blogs, and increasingly for audio-based serialized content—but it has the potential both to have broader reach and to more deeply integrate the information it delivers across applications of various kinds.
About the Format
To understand a little more about how (and why) RSS works, we must take a look at the format. RSS, as we use it in this document, refers not to a single format (such as RSS 2.0), but to the general concept of feeds of syndicated content. It should be considered to cover all feed formats that meet the basic criteria of updateable collections of items.
The most popular format, and the standard format used in most of the descriptions in this document, is RSS 2.0, authored by Dave Winer (see references). RSS 2.0, like most of the other RSS feed formats, is a simple XML-based format originally intended for syndicating or summarizing content that is available on a Web site.
An RSS feed typically contains the most recent items published on that Web site. For example, a news Web site may make summaries of stories available via an RSS feed on its Web site. A blogger may make available an RSS feed of the most recent articles he has posted to his blog.
RSS 2.0 also includes a simple extension mechanism that allows publishers and clients to define additional elements to include in a feed. RSS 2.0 also introduced the concept of enclosures—a lightweight way of attaching a file to an item within a feed.
One reason behind the success of RSS is both its simplicity and breadth. RSS 2.0 has a simple vocabulary consisting of just a handful of XML elements. These elements are good at representing everything from software updates to news articles to blog posts. This flexibility has made it possible for a diverse collection of sites to adopt RSS for various needs.
The Longhorn platform support for RSS focuses on enabling developers to build applications that support the core RSS scenarios described above, as well as more advanced applications of the RSS specification. Longhorn also leverages the RSS extension model for enabling new scenarios.
The RSS platform support in Longhorn has three parts:
- Common RSS Feed List gives application developers access to the list of feeds to which the user is subscribed.
- Common RSS Data Store is a common data store that provides a single location where applications can access any content that has been downloaded to the PC via RSS—including text, pictures, audio, calendar events, documents, and just about anything else. All applications will have access to this content for creating rich user experiences.
- RSS Platform Sync Engine automatically downloads data and files (enclosures) for use by any application. It is designed to be as efficient as possible when downloading this information, using idle network bandwidth whenever possible, in order to limit the impact on the user's Internet experience. Developers can use the platform to get RSS data without having to manage details like synchronization schedules or subscriptions.
Discovery and Subscribe
While browsing the Web, users will be able to easily discover RSS feeds through an illuminated icon, as well as read the feed while still in the Web browser. Additionally, users will be able to subscribe to an RSS feed as easily as they now add a Web page as a favorite. Once a user chooses to subscribe to a feed, the fact that the user has subscribed is available to any interested application.
This feature is enabled by the Common RSS Feed List. This core feature of Windows maintains a common list of the user's subscriptions across all applications. This allows the user to subscribe to a feed once and all RSS-enabled applications can access the common list to view the user's subscriptions.
For instance, a contact management application can automatically expose a subscribed feed containing people information as a new group in the application. Similarly, any application can add new feeds to the feed list.
Enabling New Scenarios with Enclosures
In addition to the common feed list, the Longhorn platform provides powerful platform services to enable application developers to easily harness the capabilities of RSS in their applications. In particular, RSS allows the publisher to attach files (known as enclosures) to each item in the feed that can be consumed by RSS-enabled applications.
For example, a business user about to attend a conference could subscribe to the conference's event calendar. He can then use a Longhorn RSS-enabled calendar application to view the events in the RSS feed from within his calendar app.
Enabling New Scenarios for Lists
Many collections of interesting content are actually not time-ordered; they are sorted by such properties as priority, cost, or simple ranking. In fact, one can observe these lists everywhere in one's daily life: to-do lists, shopping lists, sales reports. On the Web, one can readily observe lists of this type: Top 10 songs from a music site, a wish list from an online retailer, or a user's ranking of their favorite restaurants. In many cases, one can imagine wanting to subscribe to these lists and be notified when the content is updated. In fact, some of these lists are already being published as RSS.
However, in practice, RSS is not particularly well suited for describing lists of ordered content. In part this is because clients do not understand that the feed being received represents a complete, ordered list, and process it as if it were a time-ordered feed, resulting in incorrect behavior.
Longhorn includes support for the Simple List Extensions—a set of enhancements to RSS developed by Microsoft to help Web sites publish lists of content that users can subscribe to.
The Simple List Extensions expands the scope of RSS in two significant ways:
- The extensions enable RSS to capture information critical to representing lists, such as the ordering of items. By using these extensions, applications can recognize that a feed is more than just a feed—it is a list and the client should apply different rules with respect to changes (such as an item that was moved up or down or was removed from the list altogether).
- The extensions allow publishers to embed useful information about the list itself. For example, an online retailer can supply additional information about each item in a wish list, such as price, sales rank, average customer rating, and type of merchandise. The extensions provide a general mechanism for declaring that these additional properties are useful pivots for sorting and ordering. This allows the user to sort their friend's wish list by the sales rank of the item or to find the most popular item on the list.
Longhorn's support for the Simple List Extensions enables users to subscribe to lists and enables the publishers to provide guidance to the clients about how their lists should be manipulated. These extensions make these interesting and useful lists much more accessible to users.
Web sites that create lists using the Simple List Extensions will benefit as well; for example, a music site will be able to deliver a daily top-ten–most-popular-tracks list to drive increased sales.
See the References section for the Microsoft Simple List Extensions 1.0 specification.
The RSS support in Longhorn opens up limitless array of opportunities for publishers and application developers to create new experiences for end-users or to integrate updatable content delivered via RSS into their applications.
The diagram below shows how the existing RSS scenarios (blogs, news, and podcasting) can be expanded in interesting new dimensions with the addition of the concept of list feeds (supported by the Simple List Extensions), and by clearly defining how the existing RSS extension mechanism can be used to extend the definition of an item within an RSS feed.
Figure 1. Feed and lists in RSS
The left-most column covers scenarios based on time-sorted feeds. The RSS support in Longhorn gives developers the ability to enable any of these scenarios in their applications.
One scenario might involve interacting with a feed that contains calendar information. For example, a user wants to attend a business conference:
- On the conference's Web site, the conference organizer has made available a feed of calendar data. Whenever there are updates to the overall schedule, the publisher simply updates the feed.
- The user can subscribe to the schedule feed. Their calendar program can automatically show them the events provided by the conference organizer.
- If the location for a sessions changes, it's automatically updated in the user's calendar the next time she connects to the Internet.
In another example, a school might publish their calendar of events on the school's Web site:
- Parent meetings, plays, and other activities can appear as events in this feed.
- Parents can subscribe to the calendar and have the events automatically appear on their calendars as soon as the school schedules them (and therefore not having to rely on the school contacting them, or the child informing them).
- The school might even allow feeds tailored to individual students, so that the information in the parent's calendar includes the student's after-school activity schedule allowing the parent to know what time the child is finished each day, so that he can pick them up.
Another scenario might involve a digital photo management application. For example, a couple having their first child might want to share the pictures of their baby with their family in other countries:
- The couple creates a photo blog and posts pictures of their baby at various stages of its growth.
- A family member in another country can subscribe to a photo blog and have pictures of the new baby automatically downloaded. Any time new pictures are posted, they are automatically viewable in a photo library or a screen saver, whichever way the user wants, without needing to open a browser to check the site to see if new photos have been downloaded.
- The family members could also subscribe to other people's photo blogs, and see all of the photos from all of their friends and family together.
There are a number of interesting new scenarios that involve lists of items that are not time-ordered. Looking at the right side of the diagram above, we can see several examples—many of them paralleling similar examples for time-ordered feeds.
One example of the use of a feed might be to allow users to subscribe to wish lists.
- A daughter in another city creates a wish list on Amazon.com and updates it with new content every few weeks.
- Her mother wants to know when her daughter wants something new, in case she wants to get it. She also wants quick access to the list, for special occasions.
- The mother is able to go to the Amazon.com Web site and subscribe to her daughter's wish list.
- Whenever her daughter adds a new item, she is notified of the addition, and she can tell which one of the items is the new one.
- When she views the list, she will be able see her daughter's priority order, and she can sort by cost, category, or anything else about the items that Amazon.com makes available for her.
In another case, a user might want to be kept up-to-date with the top-10 singles on a music site.
- The music site would publish a feed of the top-10 singles using the Simple List Extensions, which enables them to give hints to the client as to how to show the list correctly.
- The user subscribes to the list, and is able to—at any time—view the ordered list of top-10 singles.
- Whenever there is a change in the list, he can be notified. The client will be able to show the correct order at all times, and can even give him some filtering controls so that he filters, for example, so that he sees the Rock songs only.
- If the user chooses to buy a song, links in the item can take him straight to a site where he can purchase it immediately.
The RSS support in the Longhorn platform provides a number of useful platform services:
- The Common RSS Feed List
- The Common RSS Data Store, with an object model to access it
- The RSS Platform Sync Engine, which automatically downloads feed content and files (enclosures) from RSS feed
The following architecture diagram shows how the RSS platform elements fit in with applications and publishers.
Figure 2. Architecture overview
The Longhorn RSS Object Model
One of the benefits of the RSS support in the Longhorn Platform is that it enables applications to take advantage of RSS through a rich object model. An application need not ever worry about XML or formats. If, however, the application developers want access to the XML version of the item or feed, it can always retrieve it through the object model.
The following diagram lays out the important pieces of the RSS object model.
Figure 3. RSS object model
The object model mirrors the major components of an RSS feed: Feed, Item, and Enclosure.
The object model is structured around a hierarchical feed list. Developers and users have a choice about whether or not they wish to leverage the hierarchy or simply use it as a flat list.
The top-level object is the Feeds object. It's the entry point into the object model. The Feeds object exposes the root folder for subscriptions.
Each Folder object contains folder properties, a collection of feeds, and a collection of subfolders along with folder properties.
The Feed object contains a collection of items as well as feed and channel properties. It is also possible to get at the actual feed XML for applications that want to perform operations on the XML instead of using the feed's properties.
The Item object contains an enclosure as well as item properties. It is also possible to get at the actual item XML for such applications that want to perform operations on the XML instead of using the item's properties.
Most properties of an item cannot be changed by a client application, with the exception of the common read/unread property.
The Enclosure object contains enclosure (or attached file) properties. One of the properties is the local path, which makes it possible for applications to get access to the enclosure via the file system.
Common RSS Feed List
The Common RSS Feed List is exposed primarily through the object model described above.
The feed list also provides an easily accessible copy of the user's feed list as a read-only OPML file under the user's "Documents" folder. The user can use this OPML file to save, e-mail, or import a copy of their feed list into another application.
Common RSS Data Store
Feed content is aggregated in the Common RSS Data store. The store allows for concurrent access by multiple applications. The store is designed to allow efficient updating and access of feeds and individual items.
Enclosure files are written into folders that a user can navigate to from within the Windows Explorer application.
The RSS Common Data Store provides a number of services including automatic purging of feed data and enclosures.
RSS Platform Sync Engine
The RSS sync engine handles on-demand as well as background updates and the merging of new feed content with existing content. Each time a feed is synchronized the feed:
- Is downloaded from the server via HTTP or HTTPS.
- Is parsed and normalized into a unified format.
- Identifies new and updated items.
- Merges into the store new and updated items.
- Is updated to reflect the last synchronization time.
New and updated items are identified by comparing each item in a feed with the items for that feed in the store. When an item is new it is then added to the store. When an item has been updated the new version of the item overwrites the one in the store.
New items are always marked as "unread." Updated items similarly get marked as "unread," even if they were "read" before.
Merging for feeds and lists
For feed (time-ordered) content, the content of the downloaded feed is merged with the local content in the following way:
- New items are appended to the local content.
- Updated items overwrite the corresponding items in the store.
The merge semantics for lists is somewhat different from the semantics for feeds.
For lists, the downloaded feed is merged in the same way, except that all items in the store that are not in the most recent feed from the server are deleted in the store. Lists also maintain an inherent sorting that is reapplied each time the feed is downloaded. This ensures that the content is resorted even if that is the only change.
Enclosures will be automatically downloaded for feeds that have enclosure download turned on. (By default, all feeds have enclosure download turned off.)
Enclosures of new and updated items will be downloaded in the background during idle network time (except when a user explicitly requests an enclosure download).
Enclosure download is integrated with the Attachment Execute Services (AES), a Windows service that maintains an allow/block list for certain file types. As a result, executable files are blocked by default from being downloaded by the Longhorn platform. Antivirus and antispyware applications can also block enclosures from being downloaded.
Each feed is synchronized in the background on a user-defined schedule so that the user has fresh information available without having to manually check for new content.
Each feed publisher can specify a default update schedule, which the service will respect unless the user indicates otherwise.
The RSS object model described above makes it easy for applications to deliver the benefit of RSS to users. The purpose of this section is to provide enough detail to let an application architect imagine how his application can deliver new scenarios by tapping into RSS feeds. We provide two examples:
- How a photo screen-saver can provide personalized and updated photos by supporting RSS photo-blogs. Every time the screensaver runs, the user sees new photos from people/sites they subscribe to.
- How a calendaring application can show calendars of games, conferences, school events, and so on by tapping into feeds of calendar events. The users can see their personal calendar side-by-side with these events and coordinate their schedule.
These demos were shown at GnomeDex 5.0 and in the Channel 9 interview. You can view them by going to http://blogs.msdn.com/ie.
Subscribing to photo blogs
Photo blogs, moblogs, and video blogs are a testament to the human desire to create and share photos and videos. In fact, it is easier to create an RSS feed with enclosures than it is to create a good-looking HTML site.
In this example, we'll describe how a screen-saver can use the Longhorn platform support for RSS to add support for updatable feeds of photos to its usual functionality.
After reading this, we hope you see how adding RSS functionality makes any existing application that deals with photos and videos more compelling.
- The user navigates to a photo blog—this could be the user's space on MSN or a community focused on photos.
- The site developer should provide an RSS feed with enclosures for each photo.
- The site developer should ensure that the main Web page refers to the RSS feed via the link HTML element in the header of the Web page. This will allow the browser to inform the user that an RSS feed is available for that page and they have the option to subscribe to it.
- The user likes the photos and subscribes to the RSS feed. The RSS sync engine downloads the RSS feed and the associated enclosures—in this case, the photo files.
- The sync engine keeps the content and photos up-to-date.
- When the screensaver activates, it uses the RSS object model to enumerate the feeds that have photo enclosures.
- The screensaver cycles through each photo-feed. For each RSS item, it shows the photo in the background and creates a caption by combining the RSS item's title and description.
- Every day, new photos will be downloaded to the user's machine.
- The screensaver can use the RSS APIs to construct the list of photos in reverse-chronological order. This ensures that the new photos are shown first.
- Now, whenever the screen-saver turns on, users will be staying connected with the latest pictures and photos from people and communities they care about.
Subscribing to calendar feeds
Creating a calendar RSS feed is a relatively simple task for publishers. Consuming a calendar feed using the Longhorn support for RSS is also relatively simple.
This combination presents the opportunity for the application developer to build a great calendaring experience that lets users stay on top of events.
- The user navigates to a site with a schedule of events on it. This could be a sports team's game schedule, a school's PTA schedule, or a department's recurring meetings. For this example, we will use the GnomeDex 5.0 conference schedule.
- The site developer should create an RSS feed corresponding to the Web page. The RSS feed should refer to iCal files for each event as enclosures in the RSS feed.
- The user is interested in staying up-to-date with the GnomeDex calendar and subscribes to the RSS feed.
- The RSS sync engine downloads the RSS feed and the associated enclosures—in this case, the iCal files.
- The sync engine regularly goes back to the site and keeps the feed content and enclosures up-to-date on the user's PC.
- The calendaring application calls the RSS APIs to determine if there are feeds that have calendar files as attachments.
- The calendaring application should perform this operation at application launch and register with the platform for notifications (to catch updates while the application is running without needing to poll the platform).
- By using the RSS APIs, the calendaring application will see that the user has subscribed to a feed for "GnomeDex schedule".
- The calendaring application can create a new calendar with the name of the feed. Within this calendar, the app should create an event for each iCal file.
- The next time the user uses the calendaring app, she will notice that a new calendar has been created and is already populated. She can use the familiar actions of the calendaring app to see this new calendar side-by-side with her personal calendar and drag and drop items into her personal calendar.
- If there are new events added to the calendar, the RSS feed will get updated and the RSS sync engine will pick up the updates. Using the RSS APIs, the calendaring application will add the new events to the calendar.
We would love to hear from you. You can contact us or others in the community in several ways:
- Longhorn RSS Wiki. Please visit http://channel9.msdn.com/wiki/default.aspx.
- Feedback on the RSS Team Blog. Please visit http://blogs.msdn.com/rssteam/ and post your comments.
- To send feedback or questions directly to the team, send e-mail to firstname.lastname@example.org.
- Microsoft Simple List Extensions
- RSS 1.0 specification:
- RSS 2.0 specification:
- Atom development site: