The XML Files: XPath Selections and Custom Functions, and More

Article
10/22/2019

XML Files

XPath Selections and Custom Functions, and More

Aaron Skonnard

Code download available at:XMLFiles0302.exe(147 KB)

QUsing the Microsoft® .NET Framework, is it possible for me to evaluate XPath expressions without having to load the entire document into memory?

AIn the .NET Framework, XPath evaluation is exposed through the XPathNavigator abstract class. The .NET Framework ships only two implementations of XPathNavigator: one that operates on the standard W3C DOM implementation (XmlNode) and another that operates on an in-memory tree representation called XPathDocument. Both implementations load the entire tree into memory before you can begin evaluating XPath expressions.

For instance, the following example uses the .NET Framework DOM implementation to sum the invoice Price elements:

const string EXPRESSION = "/Invoices/Invoice/LineItems/LineItem/Price/text()"; static void UseDOM() { XmlDocument doc = new XmlDocument(); doc.Load("invoices.xml"); XmlNodeList selection = doc.SelectNodes(EXPRESSION); double total = 0; foreach(XmlNode n in selection) total += XmlConvert.ToDouble(n.InnerText); Console.WriteLine("sum: {0}", total); }

The call to Load parses through the stream of XML (using an XmlTextReader) and builds an in-memory DOM tree whose nodes can be traversed.

The following example, on the other hand, uses XPathDocument and XPathNavigator to accomplish the same task:

static void UseNavigator() { XPathDocument doc = new XPathDocument("invoices.xml"); XPathNavigator nav = doc.CreateNavigator(); XPathNodeIterator it = nav.Select(EXPRESSION); double total = 0; while (it.MoveNext()) total += XmlConvert.ToDouble(it.Current.Value); Console.WriteLine("sum: {0}", total); }

Here, when you instantiate the XPathDocument class, you must provide an XML stream that is parsed and loaded into an in-memory tree optimized for XPath and XSLT evaluation. Then you call the Select method of XPathNavigator to evaluate the expression and XPathNodeIterator to iterate over the selection.

This cursor-based programming model and the DOM node-oriented approach are quite different. However, if you check out the implementation of XmlNode's SelectNodes method, you'll see that it's also using XPathNavigator, but exposing the results as a collection of nodes over an XPathNavigator cursor. Hence, both models use the same XPath evaluation interface; the difference is in the implementation of the trees and traversal logic (I'll provide more details in the next question).

As a quick side note, one advantage of XPathNavigator is that it also comes with a method (also named Evaluate) for evaluating XPath expressions that return strings, numbers, or Booleans—something other than a node-set (XPathNodeIterator). This allows you to greatly simplify the summing logic through XPath's sum function as shown here:

const string SUMEXPRESSION = "sum(/Invoices/Invoice/LineItems/LineItem/Price/text())"; static void UseNavigatorEvaluate() { XPathDocument doc = new XPathDocument("invoices.xml"); XPathNavigator nav = doc.CreateNavigator(); Console.WriteLine("sum: {0}", nav.Evaluate(SUMEXPRESSION)); ); }

These are the only two mechanisms for evaluating XPath expressions in the .NET Framework out of the box today. So if you're looking for a solution supported by Microsoft, you won't find it. You must load the tree into memory in order to evaluate XPath expressions. Typically, you're better off using XPathDocument since it has been optimized for XPath/XSLT evaluation. However, many developers still prefer the DOM programming model because they're already familiar with it. (And if you're targeting the .NET Compact Framework, it's your only choice.)

There is another alternative if you're willing to write a little code. Thanks to the extensible design of System.Xml, it's possible to write an extension class that provides XPath support in a streaming fashion. What you really want is the ability to evaluate XPath expressions over an underlying XmlTextReader object. To do this, you need to write a new class that derives from XPathNavigator and that uses an XmlTextReader object internally as the data source (see the code in Figure 1).

Figure 1 Extension Class

public class XPathReader: XPathNavigator { public XmlTextReader Node; public XPathReader( String url ) { Node = new XmlTextReader(url); } public XPathReader( Stream stream ) { Node = new XmlTextReader(stream); } ... // implementations of the various // XPathNavigator overrides }

Then you need to override all of XPathNavigator's abstract members and translate the calls to the underlying XmlTextReader object. I've provided a sample implementation with this column (thanks to Mark Fussell at Microsoft) that you can use to experiment. Figure 2 shows the equivalent functions for evaluating the invoice total using XPathReader.

Figure 2 Evaluating Invoice Total

static void UseReader() { XPathReader r = new XPathReader("invoices.xml"); XPathNodeIterator it = r.Select(EXPRESSION); double total = 0; while (it.MoveNext()) total += XmlConvert.ToDouble(it.Current.Value); Console.WriteLine("sum: {0}", total); } static void UseReaderEvaluate() { XPathReader r = new XPathReader("invoices.xml"); Console.WriteLine("sum: {0}", r.Evaluate(SUMEXPRESSION)); }

You have to be careful with your XPath expressions when using XPathReader because it's limited to the capabilities of the underlying XmlTextReader; for instance, XmlTextReader can only move forward through the stream. So if you use an XPath expression that needs to move backwards (such as "parent::node()"), the implementation will throw an exception. Regardless of the limitations, however, it does make it possible to evaluate XPath expressions without incurring the cost of loading the entire document into memory. The complete sample code for this column is available from the link at the top of this article.

QIs it more efficient to use XmlNode's SelectSingleNode than SelectNodes? Or should I use XPathNavigator's Select?

AFirst of all, let's think about which of the following two lines of code is more efficient:

XmlNode n1 = doc.SelectSingleNode("//Price"); XmlNode n2 = doc.SelectNodes("//Price")[0];

You'd think that it would be more efficient to use XmlNode's SelectSingleNode than SelectNodes since the latter returns an XmlNodeList while the former returns only the first XmlNode that matches. Due to constraints in the DOM specification, the XmlNodeList implementation should come with more overhead than just accessing the first matching node. However, thanks to the Microsoft implementation, there isn't a significant difference between the two techniques.

AFirst of all, let's think about which of the following two lines of code is more efficient:

XmlNode n1 = doc.SelectSingleNode("//Price"); XmlNode n2 = doc.SelectNodes("//Price")[0];

For instance, consider the internal code for the call to SelectSingleNode, which (using ILDasm or anakrino) looks something like this:

public XmlNode SelectSingleNode(string xpath) { XmlNodeList local0; XmlNode local1; try { local0 = this.SelectNodes(xpath); local1 = local0.get_ItemOf(0); } catch (ArgumentOutOfRangeException) { local1 = null; } return local1; }

Notice that the code simply delegates to SelectNodes and returns the first item in the local XmlNodeList object. You may be wondering why they did something so seemingly foolish, but before judging too quickly let's take a close look at the actual implementation of SelectNodes:

public XmlNodeList SelectNodes(string xpath) { XPathNavigator local0; local0 = this.CreateNavigator(); return new XPathNodeList(local0.Select(xpath)); }

Note that the code creates an XPathNavigator for the current DOM tree, calls Select, and returns an XPathNodeList object (which happens to derive from XmlNodeList). The implementation of XPathNodeList uses XPathNavigator's traversal methods to move through the selection as they're requested, and along the way it caches the encountered XmlNode references just in case they're asked for again. Hence, ultimately both of those lines of code are equivalent.

Looking at these implementation details also answers the other question about using XPathNavigator's Select. Since both methods use XPathNavigator's Select internally, it's not any more efficient to use XPathNavigator directly. If you're really concerned with performance, however, you may want to move away from the DOM implementation altogether and use XPathDocument instead. XPathDocument has been optimized for evaluating XPath expressions, but you're forced to use XPathNavigator's Select method since XPathDocument only supportsXPathNavigator traversal. These conclusions are completely based on current (version 1.0) implementation details, which may change over time.

QIs it possible to extend the .NET Framework XPath implementation by creating custom functions?

AYes, it's possible to hook custom functions into the .NET Framework XPath processor's context at run time, although you'll have to write a few helper classes along the way. Specifically, you have to write a class that derives from IXsltContextFunction and another that derives from XsltContext. The class you derive from IXsltContextFunction acts as the dispatcher to your custom functions. You can build your custom functions right into that class or dispatch to another class if you like. The XsltContext-derived class is called by the XPath engine to determine when your IXsltContextFunction-derived class is needed.

For example, let's say that you want to extend XPath with a custom C# function that calculates the distance between two coordinates as shown in the following code:

double Distance(double x1, double y1, double x2, double y2) { return Math.Sqrt(Math.Pow(x1-x2, 2) + Math.Pow(y1-y2, 2)); }

First you need to write a class that derives from IXsltContextFunction—let's call it DistanceFunction—and override the required members (see Figure 3). The Minargs and Maxargs properties get the minimum and maximum number of arguments supported by the function. In this case I expect exactly four arguments, but you could use these properties to distinguish between overloaded functions. The ReturnType property specifies the type of the return value, in this case XPathResultType.Number. And finally, the ArgTypes property gets the types of the supplied arguments.

Figure 3 Custom XPath Function Classes

class DistanceFunction : IXsltContextFunction { private XPathResultType[] _argTypes; public DistanceFunction(XPathResultType[] argTypes) { _argTypes = argTypes; foreach(XPathResultType t in argTypes) if (t != XPathResultType.Number) throw new Exception( "incorrect argument type: number expected"); } public int Minargs { get { return 4; } } public int Maxargs { get { return 4; } } public XPathResultType ReturnType { get { return XPathResultType.Number; } } public XPathResultType[] ArgTypes { get { return _argTypes; } } public object Invoke(XsltContext xsltContext, object[] args, XPathNavigator docContext) { return Distance((double)args[0], (double)args[1], (double)args[2], (double)args[3]); } private double Distance(double x1, double y1, double x2, double y2) { return Math.Sqrt(Math.Pow(x1-x2, 2) + Math.Pow(y1-y2, 2)); } } class DistanceFunctionContext : XsltContext { public override IXsltContextFunction ResolveFunction( string prefix, string name, XPathResultType[] ArgTypes) { if (name == "distance") return new DistanceFunction(ArgTypes); else return null; } ... // remainder of class omitted }

When the class is constructed, I expect the argument types to be provided. Within the constructor implementation, I validate that all of the arguments are number values before I cache the array in a member variable for future use; otherwise, I throw an exception. If you want to support automatic coercions when the wrong type is supplied, you can do so within Invoke.

Invoke will be called by the XPath engine at run time if it encounters a function that's handled by this class. Notice that Invoke receives an array of arguments that you can use to dispatch to the function, in this case named Distance. You need to handle any necessary coercion at this point before dispatching. Then you need to return an object of the expected type from Invoke.

The other class you have to write is one that derives from XsltContext—let's call it DistanceFunctionContext in this case (see Figure 3). This class has a few required members that you have to provide, but the most important and interesting one is ResolveFunction. This is called by the XPath engine at run time when it encounters an unrecognized function. In my implementation if the function name matches the word "distance," I return a new DistanceFunction object which will in turn be used in order to invoke the function.

To piece it all together you need to associate the custom context, DistanceFunctionContext, with the compiled XPath expression object before evaluating the expression (via SetContext), as shown in the code in Figure 4.

Figure 4 Using SetContext

XmlDocument doc = new XmlDocument(); doc.Load("line.xml"); XPathNavigator nav = doc.CreateNavigator(); DistanceFunctionContext ctx = new DistanceFunctionContext(new NameTable()); XPathExpression expr = nav.Compile("distance(number(/line/point[1]/x), number(/line/point[1]/y), number(/line/point[2]/x), number(/line/point[2]/y))"); expr.SetContext(ctx); Console.WriteLine("distance: {0}", nav.Evaluate(expr));

Now you can evaluate XPath expressions that contain my custom distance function. For example, assume that line.xml contains the following data:

<line> <point> <x>0</x> <y>0</y> </point> <point> <x>3</x> <y>4</y> </point> </line>

Using this file, the code in Figure 4 will produce this output:

distance: 5

This is yet another example of System.Xml's powerful extensibility model that enables you to customize your XML processing framework. The complete sample is available for download from the MSDN Magazine Web site.

QIs it possible to remove the default XML Schema namespaces that are inserted by XmlSerializer's Serialize method?

AYes, you can control the serialized namespaces through an XmlSerializerNamespaces object. Assume you have written the following employee class

public class Employee { public Employee() {} public Employee(string n, string id, string p) { this.name = n; this.id = id; this.position = p; } public string name; public string id; public string position; }

and also assume that you use XmlSerializer to serialize an instance, as shown here:

Employee e = new Employee("Michelle", "342-33-8390", "Manager"); XmlSerializer ser = new XmlSerializer(typeof(Employee)); ser.Serialize(Console.Out, e);

The previous code produces the following XML instance:

<Employee xmlns:xsd="https://www.w3.org/2001/XMLSchema" xmlns:xsi= "https://www.w3.org/2001/XMLSchema-instance"> <name>Michelle</name> <id>342-33-8390</id> <position>Manager</position> </Employee>

AYes, you can control the serialized namespaces through an XmlSerializerNamespaces object. Assume you have written the following employee class

public class Employee { public Employee() {} public Employee(string n, string id, string p) { this.name = n; this.id = id; this.position = p; } public string name; public string id; public string position; }

and also assume that you use XmlSerializer to serialize an instance, as shown here:

Employee e = new Employee("Michelle", "342-33-8390", "Manager"); XmlSerializer ser = new XmlSerializer(typeof(Employee)); ser.Serialize(Console.Out, e);

The previous code produces the following XML instance:

<Employee xmlns:xsd="https://www.w3.org/2001/XMLSchema" xmlns:xsi= "https://www.w3.org/2001/XMLSchema-instance"> <name>Michelle</name> <id>342-33-8390</id> <position>Manager</position> </Employee>

To get rid of the XML Schema namespaces you need to construct an XmlSerializerNamespaces object, add at least one namespace (you can use Add("", "")), and then pass it during the call to Serialize:

••• XmlSerializerNamespaces ns = new XmlSerializerNamespaces(); ns.Add("", ""); ser.Serialize(Console.Out, e, ns);

In this case Serialize produces the following XML document, which I believe is what you're after:

<Employee> <name>Michelle</name> <id>342-33-8390</id> <position>Manager</position> </Employee>

QI'm using values of type QName in some of my elements while using XmlSerializer to generate XML from an object. Here's the problem: I can't get XmlSerializer to inject the namespace declarations required by the QName values. Is there any way that I can make this happen?

AIf I understand the situation correctly, you need to generate an XML document (using XmlSerializer) that looks something like the following document:

<Employee xmlns:pos="https://example.org/positions"> <name>Michelle</name> <id>342-33-8390</id> <position>pos:Manager</position> </Employee>

As you probably know, you can use XmlRoot and XmlType to specify the namespace for the root element and XmlSchema complexType, respectively:

[XmlRoot(Namespace="https://example.org/employee")] [XmlType(Namespace="https://example.org/employee")] public class Employee { ... }

If you do this, and assuming you're using an empty XmlSerializerNamespaces object (see the previous question), you'll get the following document:

<q1:Employee xmlns:q1="https://example.org/employee"> <q1:name>Michelle</q1:name> <q1:id>342-33-8390</q1:id> <q1:position>pos:Manager</q1:position> </q1:Employee>

AIf I understand the situation correctly, you need to generate an XML document (using XmlSerializer) that looks something like the following document:

<Employee xmlns:pos="https://example.org/positions"> <name>Michelle</name> <id>342-33-8390</id> <position>pos:Manager</position> </Employee>

As you probably know, you can use XmlRoot and XmlType to specify the namespace for the root element and XmlSchema complexType, respectively:

[XmlRoot(Namespace="https://example.org/employee")] [XmlType(Namespace="https://example.org/employee")] public class Employee { ... }

If you do this, and assuming you're using an empty XmlSerializerNamespaces object (see the previous question), you'll get the following document:

<q1:Employee xmlns:q1="https://example.org/employee"> <q1:name>Michelle</q1:name> <q1:id>342-33-8390</q1:id> <q1:position>pos:Manager</q1:position> </q1:Employee>

But that doesn't help with the "pos" prefix used in the position element. If you have control over the XmlSerializer directly, you can always use the technique described in the previous section and simply add the desired namespace binding to the XmlSerializerNamespaces collection before calling Serialize:

••• XmlSerializerNamespaces ns = new XmlSerializerNamespaces(); ns.Add("pos", "https://example.org/positions"); ser.Serialize(Console.Out, e, ns);

This call to Serialize produces the following XML document with a namespace declaration for "pos":

<Employee xmlns:pos="https://example.org/positions" xmlns="https://example.org/employee"> <name>Michelle</name> <id>342-33-8390</id> <position>pos:Manager</position> </Employee>

However, you may not always have direct access to the XmlSerializer used to serialize your objects, as is the case with WebMethods. In that type of situation, you need to use the XmlNamespaceDeclarations attribute on a class member of type XmlSerializerNamespaces. Then you just need to add namespaces to the member prior to the object serialization, and XmlSerializer will inject them into the XML document for you. For example, here is a new version of Employee that uses XmlNamespaceDeclarations:

[XmlRoot(Namespace="https://example.org/employee")] [XmlType(Namespace="https://example.org/employee")] public class Employee { [XmlNamespaceDeclarations] public XmlSerializerNamespaces namespaces = new XmlSerializerNamespaces(); public string name; public string id; public string position; ••• }

Now you can write a WebMethod that injects the right namespace binding into the XmlSerializerNamespaces object at run time, before the infrastructure serializes the returned Employee object as shown here:

[WebMethod] public Employee GetEmployee() { Employee e = new Employee("Monica", "555-23-3421", "p:President"); e.namespaces.Add("p", "https://example.org/positions"); return e; }

In this case the WebMethod produces the following XML document, which contains a namespace declaration for the value used in position:

<Employee xmlns:xsd="https://www.w3.org/2001/XMLSchema" xmlns:xsi="https://www.w3.org/2001/XMLSchema-instance" xmlns:p="https://example.org/positions" xmlns="https://example.org/employee"> <name>Monica</name> <id>555-23-3421</id> <position>p:President</position> </Employee>

The sample code illustrated in this question, as well as the previous question, can also be found in the code download.

QIs it possible to disable the stack trace from appearing within the SOAP Fault that gets produced by throwing a SoapException?

AYou can configure this in your web.config file using the customErrors element (/configuration/system.web/customErrors):

<configuration> <system.web> <customErrors mode="On"/> </system.web> </configuration>

AYou can configure this in your web.config file using the customErrors element (/configuration/system.web/customErrors):

<configuration> <system.web> <customErrors mode="On"/> </system.web> </configuration>

The customErrors element has a mode attribute that takes three values: On, Off, and RemoteOnly (default). If you set it to On, the stack trace will no longer appear. If you set it to Off, the stack trace will always appear. And if you set it to RemoteOnly, the stack trace will appear if the endpoint was invoked from the same machine (for example, when the developer tests it), but it will not appear if it was invoked from a remote client—this is typically the behavior you want. Consider the following WebMethod and assume I have mode set to On or RemoteOnly:

[WebMethod] public double Divide(double x, double y) { if (y == 0) throw new SoapException("Cannot divide by 0", SoapException.ClientFaultCode); return x/y; }

If someone attempts to invoke this WebMethod using 0 for the value of y, they'll receive the following SOAP Fault:

<soap:Envelope xmlns:soap="https://schemas.xmlsoap.org/soap/envelope/"> <soap:Body> <soap:Fault> <faultcode>soap:Client</faultcode> <faultstring>Cannot divide by 0</faultstring> <detail /> </soap:Fault> </soap:Body> </soap:Envelope>

If mode had been set to Off, the faultstring element would have also contained the stack trace.

Send your questions and comments for Aaron to xmlfiles@microsoft.com.

Aaron Skonnardis an instructor/researcher at DevelopMentor, where he develops the XML and Web Service-related curriculum. Aaron coauthored Essential XML Quick Reference (Addison-Wesley, 2001) and Essential XML (Addison-Wesley, 2000).

Additional resources