Developing Priorities: Fun First
Microsoft Developer Network
August 20, 2003
Summary: Duncan Mackenzie shows how the coolness of a feature can increase its chance of being finished early in a development project, as he creates an application to retrieve and display a RSS feed his way. (13 printed pages)
Microsoft® Visual Basic® .NET
People Keep Making Me Work
If you've been following my columns, you probably think that I spend all my time writing up fun and cool code samples, without having to work on anything boring. Well, that was certainly the plan, but it doesn't always work out that way. Lately, I've been so busy working on a relatively boring system that I don't really have time to write anything fun.
Recently, though, I was handed a list of new requirements for an internal system arranged by the order in which I was supposed to work on each feature. I quickly noticed that the really cool stuff was hidden down at the end of the list, marked as "Nice to Have," which is PM-speak for "We will never build this… but we are putting it on the list to humor you." Using the flimsy pretense that I was confused about the list's sorting order, I decided to tackle the cool features first and save those boring "Priority 1 / Must Have" features for some day after I've tidied my desktop and defragged my hard drive.
Planning Pages at MSDN
Before I can get into the "cool" feature that is the subject of this article, I should really give you some context about the system I'm working on. A few months ago, I built an internal system called Page Planner, which is used to design and build the pages that make up the MSDN Developer Centers (including the Visual Basic site that I plan content for). This system (as shown in Figure 1) allows us to update all of the individual pages on the site, including all of the technology-specific article pages like http://msdn.microsoft.com/vbasic/using/building/windows.
Figure 1. Page Planner at work
The main day-to-day work of maintaining these sites involves updating the appropriate technical information pages when new articles become available. This is normally done by manually entering URLs, or by dragging links in from the browser window. The recent release of the MSDN RSS feeds (which contain the up-to-date list of new content) and similar feeds for sites like 4GuysFromRolla.com produced the "nice to have" request for "Drag-and-Drop Links from an RSS feed."
Designing our New Feature
As a fairly low-priority item, the request wasn't described in any more detail than that one line, but I had a pretty good idea about what folks around MSDN were looking for, so I started with a rough design.
The RSS Viewer window will support the viewing of an RSS feed and the dragging of links from the feed into the rest of the system. The user will be able to select from a master list of feeds that are exposed to all of the system users or from their own personal list, and they will have the option to enter a feed URL directly (adding it to their personal list if desired).
I couldn't think of a really simple way to display a feed (with variable size content, XHTML support, and so on) inside a DataGrid control—not without a bit of custom code. So I decided to display the feed using the Web Browser control (hosting Microsoft® Internet Explorer on my Microsoft Windows® Form) instead. Using an XSL transform, I can convert the RSS information into HTML and then pass that HTML into the browser control for display.
Note A nice side-effect of this implementation decision is that I get the drag-and-drop functionality for free; the Web Browser control supports dragging links out of it and into other applications without any additional code.
The master list of links will be stored into some form of central repository, but the user's personal set of links will be stored locally and updated as needed. With some local preference editing still required, the final list of functionality is as follows:
- Retrieve master list of feeds from a central store.
- Retrieve personal list of feeds from a local store.
- Retrieve and transform an RSS feed into HTML.
- Display HTML in a Web browser.
- Allow the user to enter in the URL for an RSS feed.
- Allow the user to add/edit/delete from their personal store of feeds.
- Allow the user to add/edit/delete from the master list of feeds (note that this is certainly not suitable in all situations).
In my particular case, I am going to store the master feed list into the same shared database being used by the main Page Planner system and then use the SqlClient classes to retrieve/edit it, but you might need a different implementation (Web-service based, file-based central storage, and so on). To make it easy to swap out my particular method of storing the master and personal feed lists, I've designed those aspects of the program to be slightly abstracted through an IFeedList interface. You can implement this to create your own method of storing and retrieving the list of feeds.
Displaying the Feed
Of course, storing and updating feed lists is truly secondary to the purpose of this little project; the main functionality is to retrieve and display an RSS feed. So it is best if I start by showing you that piece of the system.
To load the feed itself, I use the Load method of an XMLDocument. I then load up the XSL from another URL (or file location) into an XSLTransform instance. Finally, I use the Transform method of the XSLTransform class to take the XMLDocument and transform it using the XSL. The output from the transform is written into a stream, so I created a String-based stream (an instance of IO.StringWriter) to accept the results.
Dim myDoc As New XmlDocument myDoc.Load(rssURL.Text) Dim result As New System.Text.StringBuilder Dim resultStream = New IO.StringWriter(result) Dim xslt As New XslTransform xslt.Load(xsltURL.Text) xslt.Transform(myDoc, _ New Xsl.XsltArgumentList, _ resultStream)
So far, this is really straightforward code, as the real work is being done in the XSL file itself (included in the download for this article). The XSL itself isn't capable of handling absolutely any RSS feed, as consistency isn't one of the strong-points of RSS implementations. Still, it has worked on the feeds from weblogs.asp.net, MSDN, and GotDotNet, so it should be sufficient for now. Once I have the results of the transform, I display it by putting it into the body of a Web page displayed in an embedded browser control.
Dim myDOMDoc As mshtml.HTMLDocument myDOMDoc = DirectCast( _ Me.embeddedBrowser.Document, _ mshtml.HTMLDocument) myDOMDoc.body.innerHTML = result.ToString
I wrote all of the code you've seen so far, and tested the various bits of XSL code, using a small sample application (see Figure 2). I ended up throwing that sample away as I continued on, but for now it is a good way to test my code against various RSS feeds.
Figure 2. Testing the code with a simple form
Making Sure Everything Is in Order
The first problem I ran into was that some feeds were not displaying in descending date order (most recent item first), which was fairly confusing. It turns out that while most RSS feeds are already sorted in descending order by date, the MSDN feeds are a notable exception. For consistency I decided to add sorting code to the XSL through the <xsl:sort> element. My failure to get sorting to work directly against the pubDate value resulted in one of the more interesting aspects of that transform, as I needed to include a user-defined function (written using Visual Basic .NET), to convert the string-based date value into a new format that can be sorted.
<msxsl:script language="vb" implements-prefix="utility"> function GetDate(pubDate As String) Try Dim myDate as Date = CDate(pubDate) Return myDate.ToString("yyyyMMddHHmmss") Catch End Try end function </msxsl:script>
The function itself is pretty straightforward, I just try to convert from string to date and then I output back to a string in a format that will allow for easy sorting. Using the <xsl:sort> function, I use the results of the GetDate function to perform the actual sort.
<xsl:template match="/rss/channel"> <xsl:for-each select="./item"> <xsl:sort order="descending" select="utility:GetDate(./pubDate)" /> <xsl:apply-templates select="." /> </xsl:for-each> </xsl:template>
The Try…Catch block around the GetDate function doesn't handle the error, but it allows the XSL to still function, even if the code is unable to parse the date string.
Variety Is the Spice of Life, Unless It Is in Your RSS
The specification for RSS is flexible in many ways, including the proper handling of HTML content and how dates should be specified, which is to say that it allows for multiple methods. This flexibility is great when you are writing a system to output RSS, but it makes life a little difficult when you are trying to read it in. This flexibility caused me a problem with the date first; most feeds I was testing used a pubDate element, but a few used a dc:date element instead. I dealt with the issue by adding an <xsl:choose> function to use and display whichever date attribute was available.
<xsl:choose> <xsl:when test='pubDate'> <p>Posted on: <xsl:value-of select="pubDate" /></p> </xsl:when> <xsl:when test='dc:date'> <p>Posted on: <xsl:value-of select="dc:date" /></p> </xsl:when> <xsl:otherwise> <p>Couldn't Retrieve Date</p> </xsl:otherwise> </xsl:choose>
I had to use a similar method for handling any HTML content inside of the downloaded RSS feed, using an <xsl:choose> to choose between the three main methods of handling HTML content inside an RSS feed. These three common methods are:
- Flag the HTML block with a content:encoding tag and HTML encode all of the tags.
- Place the HTML into an xhtml:body element, and leave the HTML tags intact.
- Just leave it unmarked and inline, just like plain text.
To make sure I could handle each of these three cases, I used another <xsl:choose> function (in the XSLT) to pick the right implementation for each specific format, and to just assume un-encoded content if I cannot determine the exact method being employed.
<xsl:choose> <xsl:when test='xhtml:body'> <xsl:copy-of select='xhtml:body'/> </xsl:when> <xsl:when test='content:encoded'> <xsl:value-of disable-output-escaping='yes' select='content:encoded'/> </xsl:when> <xsl:otherwise> <xsl:value-of disable-output-escaping='yes' select='description'/> </xsl:otherwise> </xsl:choose>
The end result should work for any feed that is using one of these three methods for exposing their content as a RSS feed (xhtml:body, description, or content:encoding), producing a final display similar to what is shown in Figure 3.
Figure 3. Displaying HTML content from a feed
Now, it is very important to note that whenever you are going to display HTML content that someone else has provided (such as the content inside of an RSS feed), you need to be aware of the possible risks, especially when you are using my method of replacing the contents of the "about:blank" page. When HTML is displayed in the embedded browser, it is running within the local zone, which will likely have much lower security restrictions than the Internet zone. Although there are ways you can clean up HTML before displaying it, it takes quite a bit of work to guarantee that it is completely safe. Check out this useful blog post describing some of the issues caused by HTML in RSS, and how to avoid them.
Storing and Retrieving the Feed Lists
Once I had the RSS feeds displaying, and I had tested the system with enough sample data (translation: it worked against my blog's feed) to ensure it was working correctly, it was time to move onto creating code to support retrieving and editing the personal and master feed lists. For now, I only implemented two classes that used the IFeedList interface: one for accessing SQL, and one that works with an xml settings file unique to the current user. See the code download for the source to the IFeedList interface and to the two implementations.
Public Interface IFeedList Function GetList() As Feeds Function AddFeed( _ ByVal newFeed As Feed) As Boolean Function DeleteFeed( _ ByVal feedToToast As Feed) As Boolean Function CanAdd() As Boolean Function CanDelete() As Boolean End Interface
For the personal-file–based version, I assume that you can add and remove items freely, but for the Microsoft® SQL Server™ version (which is supposed to be used for accessing a master list shared across multiple users), I needed a bit more security. I use integrated authentication, so you could potentially handle all of your security issues by restricting user permissions in SQL Server, but I decided to use a server role and to check the user's rights by looking at their role membership. Of course, any underlying table or database object security restrictions will also be in effect, providing a second layer of security. The implementation of CanAdd is shown below, including the call to a StoredProc that checks for role membership.
Public Function CanAdd() As Boolean _ Implements IFeedList.CanAdd 'does the currently logged on user 'have rights to add to a table? 'check if is in the '"FeedAdministrator" role in SQL Server Return IsInRole("FeedAdministrator") End Function Private Function IsInRole( _ ByVal Role As String) As Boolean Try Dim conn As New _ SqlClient.SqlConnection( _ Me.m_connectionString) conn.Open() Dim cmdIsInRole As New _ SqlClient.SqlCommand( _ "IsInRole", conn) cmdIsInRole.Parameters.Add( _ "@Role", SqlDbType.NVarChar, _ 128).Value = Role cmdIsInRole.Parameters.Add( _ "@RC", SqlDbType.Int) cmdIsInRole.Parameters( _ "@RC").Direction = _ ParameterDirection.ReturnValue cmdIsInRole.Parameters.Add( _ "@Result", SqlDbType.Bit) cmdIsInRole.Parameters( _ "@Result").Direction = _ ParameterDirection.InputOutput cmdIsInRole.Parameters( _ "@Result").Value = 0 cmdIsInRole.ExecuteNonQuery() Return CBool( _ cmdIsInRole.Parameters( _ "@Result").Value()) Catch ex As Exception Return False End Try End Function
I also updated the UI a bit, to support picking a feed from a list of available ones, and to allow you to add any loaded feed into your personal (local) list. Figure 4 shows the final interface, complete with the new Save button and a combo box that you can use either to pick from one of the saved feeds or to directly enter the URL of a RSS feed.
Figure 4. The final form includes a Save button
As I developed the system, I decided to break it up for easier reuse in the future. So the embedded browser is now combined with the XSL and RSS code into a custom control, which has been placed onto the form shown in Figure 4. To use this code in my actual application, I will likely make a few additional changes that will allow me to pass a SQL connection in and place the entire form and all of its associated code into a library project. In the end, I will have something that I can very easily launch from a button on my existing Windows Forms application. Yet I have built this sample as a standalone application so that you can run it all on its own.
As always, I needed to use some resources from various places on the Web to build my finished application. I didn't use any GotDotNet user samples in this particular sample, but I did use:
- Eric J. Smith's excellent CodeSmith utility to generate my strongly-typed Feeds collection.
- Some starter XSL stolen from the template folder of RSS Bandit (check out the workspace too).
- Various bits of XSL and "help-desk support" from Kent Sharkey.
I will also point you to some good sources of RSS data, great material to display using the code from this article, as well as some great reading.
- Weblogs @ ASP.NET, the main Microsoft® .NET blogging site, complete with an all-up RSS feed, is located at http://weblogs.asp.net.
- GotDotNet has feeds for all sorts of resources, including newly posted user samples, workspaces, and more. Check them all out at http://www.gotdotnet.com/community/resources/rsshome.aspx.
- MSDN also has RSS, providing listings of the most recently published articles for the whole site, or for individual topic areas such as Visual Basic. Read about them all at http://msdn.microsoft.com/aboutmsdn/rss.asp.
- I've compiled a selected list of .NET focused bloggers at http://msdn.microsoft.com/vbasic/support/community/blogs, and you can always test your code against my feed at http://weblogs.asp.net/duncanma/rss.aspx.
Of course, there are a great many other RSS feeds out there, but the feeds from these sites should be enough to keep you going for quite awhile.
At the end of some of my Coding4Fun columns, I will have a little coding challenge—something for you to work on if you are interested. For this article, the challenge is to create anything that outputs or consumes RSS. Managed code is preferred (Visual Basic .NET, C#, J#, or managed C++ please), but an unmanaged component that exposes a COM interface would also be good. Just post whatever you produce to GotDotNet and send me an e-mail message (at firstname.lastname@example.org) with an explanation of what you have done and why you feel it is interesting. You can send me your ideas whenever you like, but please just send me links to code samples, not the samples themselves (my inbox thanks you in advance).
Have your own ideas for hobbyist content? Let me know at email@example.com, and happy coding!