Click to Rate and Give Feedback
Related Articles
Here the author introduces SQL Server Data Services, which exposes its functionality over standard Web service interfaces.

By David Robinson (July 2008)
Here the author answers questions regarding the Entity Framework and provides an understanding of how and why it was developed.

By Elisa Flasko (July 2008)
Here we present techniques for programmatic and declarative data binding and display with Windows Presentation Foundation.

By Josh Smith (July 2008)
Systems that handle failure without losing data are elusive. Learn how to achieve systems that are both scalable and robust.

By Udi Dahan (July 2008)
More ...
Articles by this Author
In this installment, the author provides an enhanced implementation of the BST pattern and compares it to HTM solutions.

By Dino Esposito (July 2008)
AJAX is meant to go beyond mere partial page rendering. Find out where Dino Esposito thinks dynamic pages are headed in the future with ASP.NET AJAX.

By Dino Esposito (June 2008)
This month we begin a look at the Single Page Interface (SPI) model and some design patterns for designing AJAX applications.

By Dino Esposito (May 2008)
This month, use nested ListView controls to create hierarchical views of data and extend the eventing model of the ListView by deriving a custom ListView class.

By Dino Esposito (April 2008)
This month Dino Esposito shows you how to get Windows-style modal dialog boxes for your Web applications thanks to the Ajax Control Toolkit and some clever coding.

By Dino Esposito (Launch 2008)
This month Dino looks at AJAX control extenders again, adding more advanced features including masked editing and autocompletion.

By Dino Esposito (February 2008)
AJAX Extenders extend the behavior and features of ordinary Web controls so you can reduce postbacks and control input even better than with AJAX alone.

By Dino Esposito (January 2008)
Dino Esposito introduces the Microsoft AJAX Library and the JavaScript library for ASP.NET AJAX 1.0.

By Dino Esposito (December 2007)
More ...
Popular Articles
Here we introduce you to some of the concepts behind the new F# language, which combines elements of functional and object-oriented .NET languages. We then help you get started writing some simple programs.

By Ted Neward (Launch 2008)
Here the author answers questions regarding the Entity Framework and provides an understanding of how and why it was developed.

By Elisa Flasko (July 2008)
Jay Flowers demonstrates how to set up and use a Continuous Integration server using both discrete tools and the more comprehensive CI Factory solution.

By Jay Flowers (March 2008)
See how to build a document-level Visual Studio Tools for Office customization and integrate it with a content type in SharePoint.

By Steve Fox (May 2008)
More ...
Read the Blog
There are many things called threat modeling. Rather than argue about which is "the one true way," a good practice is to consider your needs and what your skills, abilities, and schedules are, and then work with a method that's best for you. In the July 2008 issue of MSDN Magazine, ...
Read more!
Want to develop games for Xbox Live? Want to get paid for it, too? Click on over to the XNA Team Blog to learn more about their initial rollout of the XNA Creators Club for XNA Game Studio. ...
Read more!
The Microsoft Entity Data Model (EDM), based on Dr. Peter Chen's Entity Relationship (ER) model, is the driving force behind the ADO.NET Entity Framework. The EDM is also the feature that most significantly differentiates the Entity Framework from other ORM-style technologies in the marketplace. In the July 2008 issue of MSDN ...
Read more!
System.IO.File is a handy helper class for reading and writing data, but its methods support only synchronous operation. Is there an easy way to provide File’s functionality for asynchronous file I/O? In the July 2008 issue of MSDN Magazine, Stephen Toub walks through several ...
Read more!
Remember .NET Terrarium, the interactive game meant to introduce .NET development techniques? Well, the Windows SDK team has released the source code for .NET Terrarium 2.0 on CodePlex. You can read more about this release on the Windows SDK blog and at Microsoft ...
Read more!
The Enumerable class plays an important role in every LINQ query you create. Because the Enumerable class's extension methods can process many other classes—including Array and List—you can use methods of the Enumerable class not only to create LINQ queries, but also to manipulate the behavior of arrays and other data structures. In the July 2008 issue of MSDN ...
Read more!
More ...
Real-World XML
Manipulate XML Data Easily with the XPath and XSLT APIs in the .NET Framework
Dino Esposito
Code download available at: XPathandXSLT.exe (166 KB)
Browse the Code Online

This article assumes you're familiar with Visual Basic .NET
Level of Difficulty 1 2 3
SUMMARY
XPath is emerging as a universal query language. With XPath, you can identify and process a group of related nodes in XML-based data sources. XPath provides an infrastructure that is integral to XML support in the .NET Framework. The XPath navigation model is even used under the hood of the XSLT processor. In this article, the author reviews the implementation details of the XPath navigator and the XSLT processor and includes practical examples such as asynchronous transformations, sorted node-sets, and ASP.NET server-side transformations.
One of the key advantages of XML is that it lets you mark portions of text with tags and attributes. You tag data inside text because you plan to retrieve it later. So how do you do this? You use XPath.
Though it doesn't have an XML-based syntax, XPath is the language defined to address parts of an XML document in a compact, relatively straightforward way. More importantly, XPath defines a common syntax so you can retrieve nodes from within classes that implement the XML document object model (DOM) as well as from XSLT.
In the Microsoft® .NET Framework, the XPath query language is fully supported through the classes defined in the System.Xml.XPath namespace. The .NET Framework implementation of XPath is based on a language parser and an evaluation engine. The overall architecture of an XPath query is similar to that of a database query. Just as with SQL commands, you prepare XPath expressions and submit them to a runtime engine for evaluation. The query is parsed and executed against an XML data source. Next, you get back some information representing the resultset of the query. An XPath expression can return a node-set (that is, an ordered collection of nodes), a Boolean value, a number, or a string.
In this article, I'll show how XPath integrates with the XmlDocument class and XSLT in the .NET Framework. I'll also dig out the intricacies of the XPathNavigator class, used by the .NET Framework to walk through XML documents.

XPath Expressions
XPath is a query language specifically designed for addressing both the elements and the text of an XML document. The XPath notation is essentially declarative. Any valid expression declares a node pattern using a notation that emphasizes the hierarchical relationship between the nodes. Similar to file system paths, XPath expressions proceed from a root (an axis in the XPath terminology) to a particular set of nodes or a value in the source document. The similarity to the file system does not end here though. XPath expressions are always evaluated in the context of a node. The context node is designated by the application and represents the starting point of the query. It is not much different from the idea of the current directory.
The context of an XPath query includes, but is not limited to, a context node and a context node-set. The context node-set is the overall set of nodes processed by the query. Typically, it is a superset of the set of nodes that is actually returned to the application. An XPath context also contains position and namespace information, variable bindings, and a standard library of functions that applications can extend. Any implementation of the XPath parser provides a function library that is used to evaluate expressions. Extension functions are defined within vendor-specific XPath implementations but can also be provided by specialized and XPath-based programming APIs such as XSL Transformation and XPointer. XPath expressions generally return node-sets, but Booleans, strings, numbers, and other types are supported.
The most frequently used type of XPath expression is the location path. A location path is an expression that looks a lot like a file system path in that it can be either absolute or relative to the context node. Absolute location paths begin with the forward slash. As shown in Figure 1, a fully qualified location path is made of three pieces: an axis, a node test, and one or more predicates. The axis information defines the initial context node-set for the expression, whereas the node test is a sequence of node names that identifies a path in the node-set. A predicate is a logical expression that defines the criteria for filtering the current node-set.
Figure 1 Location Path 
An XPath expression can contain any number of predicates. If no predicate is specified then all the children of the context node are returned in the query. Otherwise, the conditions set with the various predicates are logically concatenated using a short-circuited AND operator. Notice that predicates are processed in the order in which they appear so that the next predicate works on the node-set generated by the previous one.
While being processed, an XPath expression is tokenized in subexpressions called location steps and each is evaluated individually. The XPath processor is iteratively passed the subexpression and the context node-set generated at the previous step. It returns a possibly narrowed node-set that will be used as the input argument for the next subexpression. During this process, the context node, position, and size may vary while variable and function references as well as namespace declarations remain intact. Each location step is actually a location path and, as such, can be expressed in an abbreviated or fully qualified form, as appropriate. Location steps are separated by forward slashes.
In the .NET Framework, you can use XPath expressions through methods exposed by the XmlNode class on the XmlDocument class or by means of the XPathNavigator class.

The XPath Navigator
In the .NET Framework, the XmlDocument class represents the standard XML DOM (DOM Level 2 standard) as ratified by the W3C. On each child node you reach from XmlDocument, a couple of search methods are implemented. The XmlNode class provides the SelectNodes and SelectSingleNode methods that use XPath expressions to search for nodes in the document. The methods are nearly identical to similarly named methods in the COM-based MSXML library. SelectNodes returns a list of objects whereas SelectSingleNode returns only the first object that matches the search criteria. Here's how you use SelectNodes:
Dim doc As XmlDocument = New XmlDocument()
doc.Load(fileName)
Dim nodes As XmlNodeList
nodes = doc.SelectNodes(queryString)
The SelectSingleNode method differs from SelectNodes in that it returns an individual XmlNode object. I'll have more to say about these methods in a moment.
The XmlDocument support for XPath expressions has two goals. It smoothes the transition from MSXML COM code to the .NET Framework, while giving you a built-in mechanism to search for nodes on a memory-mapped XML document. The XmlDocument query API, though, is a simple high-level wrapper. The core .NET Framework API for processing XPath expressions is built around the XPathNavigator class. The navigator is an XPath processor that works on top of any XML data store that exposes the IXPathNavigable interface. Rendered through the interface defined in the XPathNavigator class, the navigator parses and executes expressions using the Select method. Unlike the XmlDocument methods, the navigator accepts expressions both as plain text and through precompiled objects. You can programmatically access the XPathNavigator object either from the XmlDocument class or the XPathDocument class. Figure 2 illustrates the two ways to access XPath functions in the .NET Framework.
Figure 2 Accessing XPath Functions 
The following code snippet shows how to create an XPathNavigator from an XML document and execute an XPath query:
Dim doc As XPathDocument
Dim nav As XPathNavigator
Dim iterator As XPathNodeIterator
doc = New XPathDocument(fileName)
nav = doc.CreateNavigator()
iterator = nav.Select(queryString)
While iterator.MoveNext()
' nav points to the node subtree
End While
The navigator returns an XPath iterator object. Iterators are nothing more than specialized enumerator objects which are used to move through the returned node-set. I'll discuss iterators shortly.

Documents, Navigators, and Readers
Prior to the advent of the .NET Framework, processing an XML file was a matter of working with documents rendered according to the SAX or the XmlDocument specification.
The XPath processor, which is the engine that parses and executes XPath queries, is built under the covers of the XPathNavigator class. As shown in Figure 2, XPath calculations are always delegated to a navigator, regardless of the high-level caller API. A navigator works on top of a particular data store, which is typically an instance of the XPathDocument class. Other data stores can be used as well as long as the data store class implements the IXPathNavigable interface, like this:
public interface IXPathNavigable {
    XPathNavigator CreateNavigator();
}
Besides the XPathDocument class, examples of data stores include XmlDocument and XmlDataDocument.
A data store is responsible for providing a navigator to explore in-memory XML content. An XPathNavigator implementation is always store specific and is built by inheriting from the XPathNavigator abstract class. Although in practice you always program navigators through the common reference type of XPathNavigator, each data store class has its own navigator object. They are internal, undocumented classes, programmatically inaccessible, and are often implemented in fairly different ways. Figure 3 lists the real navigator classes used by the three XPath data stores defined in the .NET Framework. The document-specific navigator exploits the internal layout of the document class in order to provide the navigation API.
The XPathDocument class provides a highly optimized and read-only in-memory store for XML documents. Specifically designed to implement the XPath data model, this class does not provide any identity for nodes. It simply creates an underlying tree of node references to let the navigator operate quickly and effectively. The internal architecture of the XPathDocument class looks like a linked list of node references. Nodes are managed through an internal class (XPathNode) that represents a small subset of the XmlNode class (see Figure 2).
Compared to XPathDocument, the XmlDocument class provides read-write access to the nodes of the underlying XML document. In addition, each node can be individually accessed.
The XmlDocument class also provides the ability to create a navigator object, as shown here:
Dim doc As XmlDocument = New XmlDocument()
doc.Load(fileName)
Dim nav As XPathNavigator = doc.CreateNavigator()
The XmlDocument's navigator class implements the IHasXmlNode interface. This interface defines one method, GetNode:
public interface IHasXmlNode {
XmlNode GetNode();
}
Using this method, callers can access and interrogate the currently selected node on the XmlDocument based on the XPathNavigator's position. This feature is simply impossible to implement for navigators based on XPathDocument because it does not expose the internal structure via the XmlNode class like the XmlDocument class does. This is by design. The XPathDocument minimizes the memory footprint and does not provide for node identity.
Because the GetNode method is implemented on the XPathNavigator class on the XmlDocument, callers can take advantage of it using a type cast, as shown in the following code snippet:
Dim doc As XmlDocument = New XmlDocument()
doc.Load(fileName)
Dim nav As XPathNavigator = doc.CreateNavigator()
Dim i As XPathNodeIterator = nav.Select(query)
Dim node As XmlNode = CType(i.Current, IHasXmlNode).GetNode()
At this point, the caller program has gained full access to the node and can read and update it at will.
Finally, the XmlDataDocument class is an extension of XmlDocument which is aimed at allowing the manipulation of a relational DataSet through XML. This is a neat example of the fact that the .NET Framework navigation API can be applied to XML-based data as well as XML-like (data that's structured like XML, but isn't XML) data.
If you look at the MSDN® documentation for XPath navigators, you'll see that a navigator reads data from XML-based data stores in a cursor-like fashion (forward and backward) and provides read-only access to the underlying data. In addition, it holds information about the current node and advances the internal pointer using a variety of move methods. When the navigator is positioned on a given node, all of its properties reflect the value of that node. This, of course, resembles XML readers, which I looked at in an article in the May 2003 issue of MSDN Magazine. So what's the difference between navigators and readers?
Navigators and readers are different animals. Readers are analogous to fire-hose cursors, providing basic read-only, forward-only movement. Navigators are read-only as well but they provide a significantly richer set of move methods including both forward and backward options. Navigators also provide several select methods to fine-tune the search. Readers are lower-level tools you can use to read XML-based or XML-like data, as well as build in-memory data structures. XML readers can be used to build the in-memory data structures that navigators rely upon.

Querying for Nodes
Suppose that you need to implement an XPath query in a .NET Framework-based application. Should you use XPath navigators or are you better off sticking to the XmlDocument's nodes interface? The XmlNode's SelectNodes method internally employs a navigator object to retrieve the list of matching nodes. The return value of the navigator's Select method is then used in order to initialize an internal node list object of type XPathNodeList, defined in the System.Xml.XPath namespace. As you may have already guessed, this class inherits from XmlNodeList which is a documented class. In addition, unlike SelectNodes, navigators can fully exploit compiled expressions.
The SelectNodes method always accepts the XPath expression as plain text. The string is then passed verbatim to the navigator. In only one case does the underlying navigator receive a compiled expression. If you use the overload of the SelectNodes method that handles namespace information, the XPath expression is first compiled and then passed to the processor. The overload is prototyped as follows:
Function SelectNodes( _
    xpathExpr As String, _
    nsm As XmlNamespaceManager) _
As XmlNodeList
The advantage of using compiled expressions becomes palpable when you frequently reuse the expression in the session, and they can be namespace aware. The XmlNamespaceManager is the class that allows the user to specify the prefix of the namespace to bind.
The SelectSingleNode method works as a special case of SelectNodes in that it returns only the first element of the returned node-set. Unfortunately, up until now, the implementation of SelectSingleNode has not been particularly efficient. If you need to locate only the first matching node, then calling SelectSingleNode or SelectNodes is nearly identical. Moreover, if you need to squeeze out every little bit of performance, you're probably better off using SelectNodes. The following pseudocode illustrates the current implementation of SelectSingleNode:
Function SelectSingleNode(xpathExpr As String) As XmlNode 
   Dim nodes As XmlNodeList = SelectNodes(xpathExpr)
   Return nodes(0)
End Function
The method internally calls SelectNodes and returns the first node that is a match. Note, however, that XmlNodeList is built dynamically—the next node is searched for only when requested.
A better way to query for a single node would be to pass SelectNodes with a more precise XPath expression that returns a single node. The idea is to avoid generic wildcard expressions like this:
NorthwindEmployees/Employee
You should place a stronger filter on the XPath expression so it returns a properly sized subset of nodes. To get only the first node, add an extra predicate that stops the query after the first match:
NorthwindEmployees/Employee[position() = 1]
This is an XPath best practice in general and is not due to the .NET Framework implementation in particular. The approach could be refined and generalized to size the node-set as is appropriate. This query string shows how to get the first n nodes that match:
NorthwindEmployees/Employee[position() < n+1]
I would suggest using XmlNode's SelectNodes instead of an instance of XPathNavigator to execute XPath queries when you need to perform operations on the selected nodes that assume some form of node identity. If you need to further manage the node as an instance of the XmlNode class, then using SelectNodes would simplify your code.

Programming with XPathNavigator
Let's learn a bit more about the programming interface of XPathNavigator. In general, regardless of the application-level API, the sequence of steps necessary to execute XPath queries on an XML data source is roughly the same:
  1. Get a reference to an XPath-enabled document class (for example, an instance of an XPathDocument or XmlDocument class).
  2. Create a navigator object for the specified data store.
  3. Optionally, precompile the XPath expression if you plan to reuse it later.
  4. Call the navigator's Select method to action.
The programming interface of the navigator object is defined in the XPathNavigator abstract class. Although you will mostly use the navigator object to execute XPath queries, it is worth noting that the XPathNavigator class represents a more general-purpose component. The navigator is a generic interface that acts as a cursor-like explorer of any data store that exposes its content as XML. Although functionally similar to an XML reader, the navigator is not as fast and effective for simple reads since it is a tree navigator and specializes in retrieval. If you just need to read an XML document, use an XML reader; if you need to perform queries, use a navigator. Remember, right now the navigator works on a fully memory-mapped data source.
Functionally speaking, the XPathNavigator class is not much different from a pseudo-class that just groups together all the methods needed to navigate the document contents. The big difference is in the fact that XPathNavigator is a distinct component that's completely decoupled from the document class. In other words, XPathNavigator represents an XML view of some store of data that's been mapped to an XML data model.
Figure 4 enumerates the properties available on the XPathNavigator class. Like XML readers and the XMLDocument class, XPathNavigator employs name tables to store repeated strings more efficiently. The set of properties looks like the subset of properties that characterize the current node in the XmlTextReader class.
It is worth repeating that the XPathNavigator's Select method returns an iterator object whose current element is referenced back as a navigator (XPathNavigator class). To access and process node information, you can only use the properties of the navigator. The properties in Figure 4 are read-only and, more importantly, don't map to an instance of the XmlNode class. If you need to manipulate the node as an XmlNode object (for example, to apply changes) make sure that you use XmlDocument as the data store class and then cast the current element of the iterator to IHasXmlNode. From the reference type, IHasXmlNode calls the GetNode method, which returns the XmlNode instance of the underlying node. In all other cases, access to nodes is read-only.
The navigator object provides a rich set of methods that I classify in three main groups based on their functionality: select, move, and miscellaneous. The following code snippet selects the descendants of a node. The code to get the ancestors is nearly identical:
Dim doc As XPathDocument = New XPathDocument(fileName)
Dim nav As XPathNavigator = doc.CreateNavigator()
nav.SelectDescendants(nodeName, nsUri, selfIncluded)
SelectDescendants takes the local name of the node and selects the descendants. The nsUri variable indicates the namespace URI of the descendant nodes, if any. The selfIncluded Boolean variable is a flag that indicates whether or not the context node should be included in the node set.
Figure 5 contains the list of XPathNavigator's move methods. You can jump in either direction—forward or backward, from sibling to sibling, and according to namespace restrictions. As you may have noticed, there are three different groups of move methods: for element, attributes, and namespace nodes. Only the MoveTo and MoveToRoot methods can be called on any node, irrespective of the type. In addition, attributes and namespaces also have methods to return their value: GetAttribute and GetNamespace. When selected, the navigator's Name property returns the namespace prefix. The Value property returns the URI.
Figure 6 groups all the other methods defined on the XPathNavigator class. A few of these methods have to do with XPath expressions. An XPath expression is a string that represents a location path, though it is a bit more than a plain command string. It has a surrounding context that the XPathExpression class encapsulates. The context of an expression includes the return type and the namespace information. The XPathExpression class is not publicly creatable. To get a new instance of it, you must take an XPath string expression and compile it into an XPathExpression object. The following code snippet shows how to compile an expression and display its expected return type:
Dim expr As XPathExpression = nav.Compile(xpathExpr)
Console.WriteLine(expr.ReturnType.ToString())
nav.Select(expr)
A compiled XPath expression can be consumed by the Select, Evaluate, and Matches methods. The term "compile" here does not mean that the XPath expression becomes an executable. More simply, the action of compiling must be seen as the process that produces an object by collecting various pieces of information. An expression can return values of types other than a node-set. In this case, you use the Evaluate method to evaluate the expression and then cast the generic object returned to the specific type. Select is a more specific method in that it assumes that the return type is a node-set and inserts the nodes in an iterator.

Sorting the Node-Set
An interesting extension built into the XPathExpression class is the ability to sort the node-set before it gets passed back to the caller. To add a sorting algorithm, you call the AddSort method on the XPathExpression object. AddSort is available in two flavors:
Sub AddSort(expr As Object, comparer As IComparer)

Sub AddSort(expr As Object, order As XmlSortOrder, _
    caseOrder As XmlCaseOrder, lang As String, _
    dataType As XmlDataType) 
The expr argument denotes the sort key. It can be a string representing a node name or another XPathExpression object that evaluates to a node name. In the first overload, the comparer argument refers to an instance of a class that implements the IComparer interface. The interface supplies a Compare method, which is actually used for comparing a pair of values. Use this overload if you need to specify a custom algorithm to sort nodes.
The second overload always performs a numeric or text comparison according to the value of the dataType argument. In addition, you can specify a sorting order (ascending or descending) and even the sort order for uppercase and lowercase letters (or you can ignore capitalization completely by using the value XmlCaseOrder.None). Finally, the lang argument specifies which language to use for comparison. The language name should also specify the locale. For example, to indicate U.S. English, it is preferable to use "us-en," rather than simply "en."
Let's delve a bit deeper into the IComparer interface. To sort arrays of objects, the .NET Framework provides a few predefined comparer classes including Comparer and CaseInsensitiveComparer. The comparer class compares objects (mostly strings) with respect to case. CaseInsensitiveComparer does the same but ignores case. To use both classes in your code, be sure to import the System.Collections namespace. The Comparer class has no public constructor but provides a singleton instance through the Default static property. For example:
expr.AddSort("lastname", Comparer.Default);
If needed, you can also create your own comparer class. This code shows a rather trivial Visual Basic® .NET implementation:
Class MyOwnStringComparer 
Implements IComparer
   Public Function Compare(x As Object, y As Object) _
          As Integer Implements IComparer.Compare
       Dim strX As String = CType(x, String)
       Dim strY As String = CType(y, String) 
       Return String.Compare(strX, strY)
   End Sub
End Class
The Compare method should return 0 if the two strings are equal; it returns a value greater than 0 if x precedes y, and a negative value if y precedes x.
The class can also be defined within the body of your application and does not require a separate assembly. This code associates a custom comparer with an XPath expression:
Dim comp As MyOwnStringComparer = New MyOwnStringComparer()
expr.AddSort("lastname", comp)
Figure 7 shows an application of this technique. After creating a navigator for an XML doc, the sample console app compiles the expression to use. The assumed data source has this schema:
<MyDataSet>
<NorthwindEmployees>
<Employee>
   <employeeid>...</employeeid> 
   <lastname>...</lastname> 
   <firstname>...</firstname> 
   <title>...</title> 
</Employee>
•••
</NorthwindEmployees>
</MyDataSet>
Sorting by a single node is easy—you just pass the node name to the AddSort method. It's more complicated to sort by multiple fields. The idea is that you indicate a comma-separated list of node names. However, the argument string must be an XPath expression that evaluates to the comma-separated list of node names. If you simply specify the sorting key as, say, "title, lastname" you'll get a runtime error because the XPath processor mistakes it for the actual node name. What's really needed is an expression that the XPath processor would translate at run time into the desired title and lastname, like this:
Dim sortKey As String = "concat(concat(title, ','), lastname)"
The concat keyword identifies one of the predefined helper functions that XPath implementations provide.
In