Walkthrough: Creating an IQueryable LINQ Provider

This advanced topic provides step-by-step instructions for creating a custom LINQ provider. When you are finished, you will be able to use the provider you create to write LINQ queries against the TerraServer-USA Web service.

The TerraServer-USA Web service provides an interface to a database of aerial images of the United States. It also exposes a method that returns information about locations in the United States, given part or all of a location name. This method, which is named GetPlaceList, is the method that your LINQ provider will call. The provider will use Windows Communication Foundation (WCF) to communicate with the Web service. For more information about the TerraServer-USA Web service, see Overview of the TerraServer-USA Web Services.

This provider is a relatively simple IQueryable provider. It expects specific information in the queries that it handles and it has a closed type system, exposing a single type to represent the result data. This provider examines only one type of method call expression in the expression tree that represents the query, that is the innermost call to Where. It extracts the data that it must have in order to query the Web service from this expression. It then calls the Web service and inserts the returned data into the expression tree in the place of the initial IQueryable data source. The rest of the query execution is handled by the Enumerable implementations of the standard query operators.

The code examples in this topic are provided in C# and Visual Basic.

This walkthrough illustrates the following tasks:

  • Creating the project in Visual Studio.

  • Implementing the interfaces that are required for an IQueryable LINQ provider: IQueryable(Of T)IOrderedQueryable(Of T), and IQueryProvider.

  • Adding a custom .NET type to represent the data from the Web service.

  • Creating a query context class and a class that obtains data from the Web service.

  • Creating an expression tree visitor subclass that finds the expression that represents the innermost call to the Queryable.Where method.

  • Creating an expression tree visitor subclass that extracts information from the LINQ query to use in the Web service request.

  • Creating an expression tree visitor subclass that modifies the expression tree that represents the complete LINQ query.

  • Using an evaluator class to partially evaluate an expression tree. This step is necessary because it translates all local variable references in the LINQ query into values.

  • Creating an expression tree helper class and a new exception class.

  • Testing the LINQ provider from a client application that contains a LINQ query.

  • Adding more complex query capabilities to the LINQ provider.

    Note Note

    The LINQ provider that this walkthrough creates is available as a sample. For more information, LINQ Samples.

This walkthrough requires features that are introduced in Visual Studio 2008.

NoteNote

Your computer might show different names or locations for some of the Visual Studio user interface elements in the following instructions. The Visual Studio edition that you have and the settings that you use determine these elements. For more information, see Customizing Development Settings in Visual Studio.

To create the project in Visual Studio

  1. In Visual Studio, create a new Class Library application. Name the project LinqToTerraServerProvider.

  2. In Solution Explorer, select the Class1.cs (or Class1.vb) file and rename it to QueryableTerraServerData.cs (or QueryableTerraServerData.vb). In the dialog box that pops up, click Yes to rename all references to the code element.

    You create the provider as a Class Library project in Visual Studio because executable client applications will add the provider assembly as a reference to their project.

To add a service reference to the Web service

  1. In Solution Explorer, right-click the LinqToTerraServerProvider project and click Add Service Reference.

    The Add Service Reference dialog box opens.

  2. In the Address box, type http://terraserver.microsoft.com/TerraService2.asmx.

  3. In the Namespace box, type TerraServerReference and then click OK.

    The TerraServer-USA Web service is added as a service reference so that the application can communicate with the Web service by way of Windows Communication Foundation (WCF). By adding a service reference to the project, Visual Studio generates an app.config file that contains a proxy and an endpoint for the Web service. For more information, see Windows Communication Foundation Services and WCF Data Services in Visual Studio.

You now have a project that has a file that is named app.config, a file that is named QueryableTerraServerData.cs (or QueryableTerraServerData.vb), and a service reference named TerraServerReference.

To create a LINQ provider, at a minimum you must implement the IQueryable(Of T) and IQueryProvider interfaces. IQueryable(Of T) and IQueryProvider are derived from the other required interfaces; therefore, by implementing these two interfaces, you are also implementing the other interfaces that are required for a LINQ provider.

If you want to support sorting query operators such as OrderBy and ThenBy, you must also implement the IOrderedQueryable(Of T) interface. Because IOrderedQueryable(Of T) derives from IQueryable(Of T), you can implement both of these interfaces in one type, which is what this provider does.

To implement System.Linq.IQueryable`1 and System.Linq.IOrderedQueryable`1

  • In the file QueryableTerraServerData.cs (or QueryableTerraServerData.vb), add the following code.

    Imports System.Linq.Expressions
    
    Public Class QueryableTerraServerData(Of TData)
        Implements IOrderedQueryable(Of TData)
    
    #Region "Private members" 
    
        Private _provider As TerraServerQueryProvider
        Private _expression As Expression
    
    #End Region
    
    #Region "Constructors" 
    
        ''' <summary> 
        ''' This constructor is called by the client to create the data source. 
        ''' </summary> 
        Public Sub New()
            Me._provider = New TerraServerQueryProvider()
            Me._expression = Expression.Constant(Me)
        End Sub 
    
        ''' <summary> 
        ''' This constructor is called by Provider.CreateQuery(). 
        ''' </summary> 
        ''' <param name="_expression"></param> 
        Public Sub New(ByVal _provider As TerraServerQueryProvider, ByVal _expression As Expression)
    
            If _provider Is Nothing Then 
                Throw New ArgumentNullException("provider")
            End If 
    
            If _expression Is Nothing Then 
                Throw New ArgumentNullException("expression")
            End If 
    
            If Not GetType(IQueryable(Of TData)).IsAssignableFrom(_expression.Type) Then 
                Throw New ArgumentOutOfRangeException("expression")
            End If 
    
            Me._provider = _provider
            Me._expression = _expression
        End Sub
    
    #End Region
    
    #Region "Properties" 
    
        Public ReadOnly Property ElementType(
            ) As Type Implements IQueryable(Of TData).ElementType
    
            Get 
                Return GetType(TData)
            End Get 
        End Property 
    
        Public ReadOnly Property Expression(
            ) As Expression Implements IQueryable(Of TData).Expression
    
            Get 
                Return _expression
            End Get 
        End Property 
    
        Public ReadOnly Property Provider(
            ) As IQueryProvider Implements IQueryable(Of TData).Provider
    
            Get 
                Return _provider
            End Get 
        End Property
    
    #End Region
    
    #Region "Enumerators" 
    
        Public Function GetGenericEnumerator(
            ) As IEnumerator(Of TData) Implements IEnumerable(Of TData).GetEnumerator
    
            Return (Me.Provider.
                    Execute(Of IEnumerable(Of TData))(Me._expression)).GetEnumerator()
        End Function 
    
        Public Function GetEnumerator(
            ) As IEnumerator Implements IEnumerable.GetEnumerator
    
            Return (Me.Provider.
                    Execute(Of IEnumerable)(Me._expression)).GetEnumerator()
        End Function
    
    #End Region
    
    End Class
    

    The IOrderedQueryable(Of T) implementation by the QueryableTerraServerData class implements three properties declared in IQueryable and two enumeration methods declared in IEnumerable and IEnumerable(Of T).

    This class has two constructors. The first constructor is called from the client application to create the object to write the LINQ query against. The second constructor is called internal to the provider library by the code in the IQueryProvider implementation.

    When the GetEnumerator method is called on an object of type QueryableTerraServerData, the query that it represents is executed and the results of the query are enumerated.

    This code, except for the name of the class, is not specific to this TerraServer-USA Web service provider. Therefore, it can be reused for any LINQ provider.

To implement System.Linq.IQueryProvider

  • Add the TerraServerQueryProvider class to your project.

    Imports System.Linq.Expressions
    Imports System.Reflection
    
    Public Class TerraServerQueryProvider
        Implements IQueryProvider
    
        Public Function CreateQuery(
            ByVal expression As Expression
            ) As IQueryable Implements IQueryProvider.CreateQuery
    
            Dim elementType As Type = TypeSystem.GetElementType(expression.Type)
    
            Try 
                Dim qType = GetType(QueryableTerraServerData(Of )).MakeGenericType(elementType)
                Dim args = New Object() {Me, expression}
                Dim instance = Activator.CreateInstance(qType, args)
    
                Return CType(instance, IQueryable)
            Catch tie As TargetInvocationException
                Throw tie.InnerException
            End Try 
        End Function 
    
        ' Queryable's collection-returning standard query operators call this method. 
        Public Function CreateQuery(Of TResult)(
            ByVal expression As Expression
            ) As IQueryable(Of TResult) Implements IQueryProvider.CreateQuery
    
            Return New QueryableTerraServerData(Of TResult)(Me, expression)
        End Function 
    
        Public Function Execute(
            ByVal expression As Expression
            ) As Object Implements IQueryProvider.Execute
    
            Return TerraServerQueryContext.Execute(expression, False)
        End Function 
    
        ' Queryable's "single value" standard query operators call this method. 
        ' It is also called from QueryableTerraServerData.GetEnumerator(). 
        Public Function Execute(Of TResult)(
            ByVal expression As Expression
            ) As TResult Implements IQueryProvider.Execute
    
            Dim IsEnumerable As Boolean = (GetType(TResult).Name = "IEnumerable`1")
    
            Dim result = TerraServerQueryContext.Execute(expression, IsEnumerable)
            Return CType(result, TResult)
        End Function 
    End Class
    

    The query provider code in this class implements the four methods that are required to implement the IQueryProvider interface. The two CreateQuery methods create queries that are associated with the data source. The two Execute methods send such queries off to be executed.

    The non-generic CreateQuery method uses reflection to obtain the element type of the sequence that the query it creates would return if it was executed. It then uses the Activator class to construct a new QueryableTerraServerData instance that is constructed with the element type as its generic type argument. The result of calling the non-generic CreateQuery method is the same as if the generic CreateQuery method had been called with the correct type argument.

    Most of the query execution logic is handled in a different class that you will add later. This functionality is handled elsewhere because it is specific to the data source being queried, whereas the code in this class is generic to any LINQ provider. To use this code for a different provider, you might have to change the name of the class and the name of the query context type that is referenced in two of the methods.

You will need a .NET type to represent the data that is obtained from the Web service. This type will be used in the client LINQ query to define the results it wants. The following procedure creates such a type. This type, named Place, contains information about a single geographical location such as a city, a park, or a lake.

This code also contains an enumeration type, named PlaceType, that defines the various types of geographical location and is used in the Place class.

To create a custom result type

  • Add the Place class and the PlaceType enumeration to your project.

    Public Class Place
        ' Properties. 
        Public Property Name As String 
        Public Property State As String 
        Public Property PlaceType As PlaceType
    
        ' Constructor. 
        Friend Sub New(ByVal name As String, 
                       ByVal state As String, 
                       ByVal placeType As TerraServerReference.PlaceType)
    
            Me.Name = name
            Me.State = state
            Me.PlaceType = CType(placeType, PlaceType)
        End Sub 
    End Class 
    
    Public Enum PlaceType
        Unknown
        AirRailStation
        BayGulf
        CapePeninsula
        CityTown
        HillMountain
        Island
        Lake
        OtherLandFeature
        OtherWaterFeature
        ParkBeach
        PointOfInterest
        River
    End Enum
    

    The constructor for the Place type simplifies creating a result object from the type that is returned by the Web service. While the provider can return the result type defined by the Web service API directly, it would require client applications to add a reference to the Web service. By creating a new type as part of the provider library, the client does not have to know about the types and methods that the Web service exposes.

This provider implementation assumes that the innermost call to Queryable.Where contains the location information to use to query the Web service. The innermost Queryable.Where call is the where clause (Where clause in Visual Basic) or Queryable.Where method call that occurs first in a LINQ query, or the one nearest to the "bottom" of the expression tree that represents the query.

To create a query context class

  • Add the TerraServerQueryContext class to your project.

    Imports System.Linq.Expressions
    
    Public Class TerraServerQueryContext
    
        ' Executes the expression tree that is passed to it. 
        Friend Shared Function Execute(ByVal expr As Expression, 
                                       ByVal IsEnumerable As Boolean) As Object 
    
            ' The expression must represent a query over the data source. 
            If Not IsQueryOverDataSource(expr) Then 
                Throw New InvalidProgramException("No query over the data source was specified.")
            End If 
    
            ' Find the call to Where() and get the lambda expression predicate. 
            Dim whereFinder As New InnermostWhereFinder()
            Dim whereExpression As MethodCallExpression = 
                whereFinder.GetInnermostWhere(expr)
            Dim lambdaExpr As LambdaExpression
            lambdaExpr = CType(CType(whereExpression.Arguments(1), UnaryExpression).Operand, LambdaExpression)
    
            ' Send the lambda expression through the partial evaluator.
            lambdaExpr = CType(Evaluator.PartialEval(lambdaExpr), LambdaExpression)
    
            ' Get the place name(s) to query the Web service with. 
            Dim lf As New LocationFinder(lambdaExpr.Body)
            Dim locations As List(Of String) = lf.Locations
            If locations.Count = 0 Then 
                Dim s = "You must specify at least one place name in your query." 
                Throw New InvalidQueryException(s)
            End If 
    
            ' Call the Web service and get the results. 
            Dim places() = WebServiceHelper.GetPlacesFromTerraServer(locations)
    
            ' Copy the IEnumerable places to an IQueryable. 
            Dim queryablePlaces = places.AsQueryable()
    
            ' Copy the expression tree that was passed in, changing only the first 
            ' argument of the innermost MethodCallExpression. 
            Dim treeCopier As New ExpressionTreeModifier(queryablePlaces)
            Dim newExpressionTree = treeCopier.Visit(expr)
    
            ' This step creates an IQueryable that executes by replacing  
            ' Queryable methods with Enumerable methods. 
            If (IsEnumerable) Then 
                Return queryablePlaces.Provider.CreateQuery(newExpressionTree)
            Else 
                Return queryablePlaces.Provider.Execute(newExpressionTree)
            End If 
        End Function 
    
        Private Shared Function IsQueryOverDataSource(ByVal expression As Expression) As Boolean 
            ' If expression represents an unqueried IQueryable data source instance, 
            ' expression is of type ConstantExpression, not MethodCallExpression. 
            Return (TypeOf expression Is MethodCallExpression)
        End Function 
    End Class
    

    This class organizes the work of executing a query. After finding the expression that represents the innermost Queryable.Where call, this code retrieves the lambda expression that represents the predicate that was passed to Queryable.Where. It then passes the predicate expression to a method to be partially evaluated, so that all references to local variables are translated into values. Then it calls a method to extract the requested locations from the predicate, and calls another method to obtain the result data from the Web service.

    In the next step, this code copies the expression tree that represents the LINQ query and makes one modification to the expression tree. The code uses an expression tree visitor subclass to replace the data source that the innermost query operator call is applied to with the concrete list of Place objects that were obtained from the Web service.

    Before the list of Place objects is inserted into the expression tree, its type is changed from IEnumerable to IQueryable by calling AsQueryable. This type change is necessary because when the expression tree is rewritten, the node that represents the method call to the innermost query operator method is reconstructed. The node is reconstructed because one of the arguments has changed (that is, the data source that it is applied to). The Call(Expression, MethodInfo, IEnumerable(Of Expression)) method, which is used to reconstruct the node, will throw an exception if any argument is not assignable to the corresponding parameter of the method that it will be passed to. In this case, the IEnumerable list of Place objects would not be assignable to the IQueryable parameter of Queryable.Where. Therefore, its type is changed to IQueryable.

    By changing its type to IQueryable, the collection also obtains an IQueryProvider member, accessed by the Provider property, that can create or execute queries. The dynamic type of the IQueryable°Place collection is EnumerableQuery, which is a type that is internal to the System.Linq API. The query provider that is associated with this type executes queries by replacing Queryable standard query operator calls with the equivalent Enumerable operators, so that effectively the query becomes a LINQ to Objects query.

    The final code in the TerraServerQueryContext class calls one of two methods on the IQueryable list of Place objects. It calls CreateQuery if the client query returns enumerable results, or Execute if the client query returns a non-enumerable result.

    The code in this class is very specific to this TerraServer-USA provider. Therefore, it is encapsulated in the TerraServerQueryContext class instead of being inserted directly into the more generic IQueryProvider implementation.

The provider you are creating requires only the information in the Queryable.Where predicate to query the Web service. Therefore, it uses LINQ to Objects to do the work of executing the LINQ query by using the internal EnumerableQuery type. An alternative way to use LINQ to Objects to execute the query is to have the client wrap the part of the query to be executed by LINQ to Objects in a LINQ to Objects query. This is accomplished by calling AsEnumerable(Of TSource) on the rest of the query, which is the part of the query that the provider requires for its specific purposes. The advantage of this kind of implementation is that the division of work between the custom provider and LINQ to Objects is more transparent.

Note Note

The provider presented in this topic is a simple provider that has minimal query support of its own. Therefore, it relies heavily on LINQ to Objects to execute queries. A complex LINQ provider such as LINQ to SQL may support the whole query without handing any work off to LINQ to Objects.

To create a class to obtain data from the Web service

  • Add the WebServiceHelper class (or module in Visual Basic) to your project.

    Imports System.Collections.Generic
    Imports LinqToTerraServerProvider.TerraServerReference
    
    Friend Module WebServiceHelper
        Private numResults As Integer = 200
        Private mustHaveImage As Boolean = False 
    
        Friend Function GetPlacesFromTerraServer(ByVal locations As List(Of String)) As Place()
            ' Limit the total number of Web service calls. 
            If locations.Count > 5 Then 
                Dim s = "This query requires more than five separate calls to the Web service. Please decrease the number of places." 
                Throw New InvalidQueryException(s)
            End If 
    
            Dim allPlaces As New List(Of Place)
    
            ' For each location, call the Web service method to get data. 
            For Each location In locations
                Dim places = CallGetPlaceListMethod(location)
                allPlaces.AddRange(places)
            Next 
    
            Return allPlaces.ToArray()
        End Function 
    
        Private Function CallGetPlaceListMethod(ByVal location As String) As Place()
    
            Dim client As New TerraServiceSoapClient()
            Dim placeFacts() As PlaceFacts
    
            Try 
                ' Call the Web service method "GetPlaceList".
                placeFacts = client.GetPlaceList(location, numResults, mustHaveImage)
    
                ' If we get exactly 'numResults' results, they are probably truncated. 
                If (placeFacts.Length = numResults) Then 
                    Dim s = "The results have been truncated by the Web service and would not be complete. Please try a different query." 
                    Throw New Exception(s)
                End If 
    
                ' Create Place objects from the PlaceFacts objects returned by the Web service. 
                Dim places(placeFacts.Length - 1) As Place
                For i = 0 To placeFacts.Length - 1
                    places(i) = New Place(placeFacts(i).Place.City, 
                                          placeFacts(i).Place.State, 
                                          placeFacts(i).PlaceTypeId)
                Next 
    
                ' Close the WCF client.
                client.Close()
    
                Return places
            Catch timeoutException As TimeoutException
                client.Abort()
                Throw 
            Catch communicationException As System.ServiceModel.CommunicationException
                client.Abort()
                Throw 
            End Try 
        End Function 
    End Module
    

    This class contains the functionality that obtains data from the Web service. This code uses a type named TerraServiceSoapClient, which is auto-generated for the project by Windows Communication Foundation (WCF), to call the Web service method GetPlaceList. Then, each result is translated from the return type of the Web service method to the .NET type that the provider defines for the data.

    This code contains two checks that enhance the usability of the provider library. The first check limits the maximum time that a client application will wait for a response by limiting the total number of calls that are made to the Web service, per query, to five. For each location that is specified in the client query, one Web service request is generated. Therefore, the provider throws an exception if the query contains more than five locations.

    The second check determines whether the number of results returned by the Web service is equal to the maximum number of results that it can return. If the number of results is the maximum number, it is likely that the results from the Web service are truncated. Instead of returning an incomplete list to the client, the provider throws an exception.

To create the visitor that finds the innermost Where method call expression

  1. Add the InnermostWhereFinder class, which inherits the ExpressionVisitor class, to your project.

    Imports System.Linq.Expressions
    
    Class InnermostWhereFinder
        Inherits ExpressionVisitor
    
        Private innermostWhereExpression As MethodCallExpression
    
        Public Function GetInnermostWhere(ByVal expr As Expression) As MethodCallExpression
            Me.Visit(expr)
            Return innermostWhereExpression
        End Function 
    
        Protected Overrides Function VisitMethodCall(ByVal expr As MethodCallExpression) As Expression
            If expr.Method.Name = "Where" Then
                innermostWhereExpression = expr
            End If 
    
            Me.Visit(expr.Arguments(0))
    
            Return expr
        End Function 
    End Class
    

    This class inherits the base expression tree visitor class to perform the functionality of finding a specific expression. The base expression tree visitor class is designed to be inherited and specialized for a specific task that involves traversing an expression tree. The derived class overrides the VisitMethodCall method to seek out the expression that represents the innermost call to Where in the expression tree that represents the client query. This innermost expression is the expression that the provider extracts the search locations from.

  2. Add using directives (Imports statements in Visual Basic) to the file for the following namespaces: System.Collections.Generic, System.Collections.ObjectModel and System.Linq.Expressions.

To create the visitor that extracts data to query the Web service

  • Add the LocationFinder class to your project.

    Imports System.Linq.Expressions
    Imports ETH = LinqToTerraServerProvider.ExpressionTreeHelpers
    
    Friend Class LocationFinder
        Inherits ExpressionVisitor
    
        Private _expression As Expression
        Private _locations As List(Of String)
    
        Public Sub New(ByVal exp As Expression)
            Me._expression = exp
        End Sub 
    
        Public ReadOnly Property Locations() As List(Of String)
            Get 
                If _locations Is Nothing Then
                    _locations = New List(Of String)()
                    Me.Visit(Me._expression)
                End If 
                Return Me._locations
            End Get 
        End Property 
    
        Protected Overrides Function VisitBinary(ByVal be As BinaryExpression) As Expression
            ' Handles Visual Basic String semantics.
            be = ETH.ConvertVBStringCompare(be)
    
            If be.NodeType = ExpressionType.Equal Then 
                If (ETH.IsMemberEqualsValueExpression(be, GetType(Place), "Name")) Then
                    _locations.Add(ETH.GetValueFromEqualsExpression(be, GetType(Place), "Name"))
                    Return be
                ElseIf (ETH.IsMemberEqualsValueExpression(be, GetType(Place), "State")) Then
                    _locations.Add(ETH.GetValueFromEqualsExpression(be, GetType(Place), "State"))
                    Return be
                Else 
                    Return MyBase.VisitBinary(be)
                End If 
            Else 
                Return MyBase.VisitBinary(be)
            End If 
        End Function 
    End Class
    

    This class is used to extract location information from the predicate that the client passes to Queryable.Where. It derives from the ExpressionVisitor class and overrides only the VisitBinary method.

    The ExpressionVisitor class sends binary expressions, such as equality expressions like place.Name == "Seattle" (place.Name = "Seattle" in Visual Basic), to the VisitBinary method. In this overriding VisitBinary method, if the expression matches the equality expression pattern that can provide location information, that information is extracted and stored in a list of locations.

    This class uses an expression tree visitor to find the location information in the expression tree because a visitor is designed for traversing and examining expression trees. The resulting code is neater and less error-prone than if it had been implemented without using the visitor.

    At this stage of the walkthrough, your provider supports only limited ways of supplying location information in the query. Later in the topic, you will add functionality to enable more ways of supplying location information.

To create the visitor that modifies the expression tree

  • Add the ExpressionTreeModifier class to your project.

    Imports System.Linq.Expressions
    
    Friend Class ExpressionTreeModifier
        Inherits ExpressionVisitor
    
        Private queryablePlaces As IQueryable(Of Place)
    
        Friend Sub New(ByVal places As IQueryable(Of Place))
            Me.queryablePlaces = places
        End Sub 
    
        Protected Overrides Function VisitConstant(ByVal c As ConstantExpression) As Expression
            ' Replace the constant QueryableTerraServerData arg with the queryable Place collection. 
            If c.Type Is GetType(QueryableTerraServerData(Of Place)) Then 
                Return Expression.Constant(Me.queryablePlaces)
            Else 
                Return c
            End If 
        End Function 
    End Class
    

    This class derives from the ExpressionVisitor class and overrides the VisitConstant method. In this method, it replaces the object that the innermost standard query operator call is applied to with a concrete list of Place objects.

    This expression tree modifier class uses the expression tree visitor because the visitor is designed to traverse, examine, and copy expression trees. By deriving from the base expression tree visitor class, this class requires minimal code to perform its function.

The predicate that is passed to the Queryable.Where method in the client query may contain sub-expressions that do not depend on the parameter of the lambda expression. These isolated sub-expressions can and should be evaluated immediately. They could be references to local variables or member variables that must be translated into values.

The next class exposes a method, PartialEval(Expression), that determines which, if any, of the sub-trees in the expression can be evaluated immediately. It then evaluates those expressions by creating a lambda expression, compiling it, and invoking the returned delegate. Finally, it replaces the sub-tree with a new node that represents a constant value. This is known as partial evaluation.

To add a class to perform partial evaluation of an expression tree

  • Add the Evaluator class to your project.

    Imports System.Linq.Expressions
    
    Public Module Evaluator
        ''' <summary>Performs evaluation and replacement of independent sub-trees</summary> 
        ''' <param name="expr">The root of the expression tree.</param> 
        ''' <param name="fnCanBeEvaluated">A function that decides whether a given expression node can be part of the local function.</param> 
        ''' <returns>A new tree with sub-trees evaluated and replaced.</returns> 
        Public Function PartialEval(
            ByVal expr As Expression, 
            ByVal fnCanBeEvaluated As Func(Of Expression, Boolean)
            )  As Expression
    
            Return New SubtreeEvaluator(New Nominator(fnCanBeEvaluated).Nominate(expr)).Eval(expr)
        End Function 
    
        ''' <summary> 
        ''' Performs evaluation and replacement of independent sub-trees 
        ''' </summary> 
        ''' <param name="expression">The root of the expression tree.</param> 
        ''' <returns>A new tree with sub-trees evaluated and replaced.</returns> 
        Public Function PartialEval(ByVal expression As Expression) As Expression
            Return PartialEval(expression, AddressOf Evaluator.CanBeEvaluatedLocally)
        End Function 
    
        Private Function CanBeEvaluatedLocally(ByVal expression As Expression) As Boolean 
            Return expression.NodeType <> ExpressionType.Parameter
        End Function 
    
        ''' <summary> 
        ''' Evaluates and replaces sub-trees when first candidate is reached (top-down) 
        ''' </summary> 
        Class SubtreeEvaluator
            Inherits ExpressionVisitor
    
            Private candidates As HashSet(Of Expression)
    
            Friend Sub New(ByVal candidates As HashSet(Of Expression))
                Me.candidates = candidates
            End Sub 
    
            Friend Function Eval(ByVal exp As Expression) As Expression
                Return Me.Visit(exp)
            End Function 
    
            Public Overrides Function Visit(ByVal exp As Expression) As Expression
                If exp Is Nothing Then 
                    Return Nothing 
                ElseIf Me.candidates.Contains(exp) Then 
                    Return Me.Evaluate(exp)
                End If 
    
                Return MyBase.Visit(exp)
            End Function 
    
            Private Function Evaluate(ByVal e As Expression) As Expression
                If e.NodeType = ExpressionType.Constant Then 
                    Return e
                End If 
    
                Dim lambda = Expression.Lambda(e)
                Dim fn As [Delegate] = lambda.Compile()
    
                Return Expression.Constant(fn.DynamicInvoke(Nothing), e.Type)
            End Function 
        End Class 
    
    
        ''' <summary> 
        ''' Performs bottom-up analysis to determine which nodes can possibly 
        ''' be part of an evaluated sub-tree. 
        ''' </summary> 
        Class Nominator
            Inherits ExpressionVisitor
    
            Private fnCanBeEvaluated As Func(Of Expression, Boolean)
            Private candidates As HashSet(Of Expression)
            Private cannotBeEvaluated As Boolean 
    
            Friend Sub New(ByVal fnCanBeEvaluated As Func(Of Expression, Boolean))
                Me.fnCanBeEvaluated = fnCanBeEvaluated
            End Sub 
    
            Friend Function Nominate(ByVal expr As Expression) As HashSet(Of Expression)
                Me.candidates = New HashSet(Of Expression)()
                Me.Visit(expr)
    
                Return Me.candidates
            End Function 
    
            Public Overrides Function Visit(ByVal expr As Expression) As Expression
                If expr IsNot Nothing Then 
    
                    Dim saveCannotBeEvaluated = Me.cannotBeEvaluated
                    Me.cannotBeEvaluated = False 
    
                    MyBase.Visit(expr)
    
                    If Not Me.cannotBeEvaluated Then 
                        If Me.fnCanBeEvaluated(expr) Then 
                            Me.candidates.Add(expr)
                        Else 
                            Me.cannotBeEvaluated = True 
                        End If 
                    End If 
    
                    Me.cannotBeEvaluated = Me.cannotBeEvaluated Or 
                                           saveCannotBeEvaluated
                End If 
    
                Return expr
            End Function 
        End Class 
    End Module
    

This section contains code for three helper classes for your provider.

To add the helper class that is used by the System.Linq.IQueryProvider implementation

  • Add the TypeSystem class (or module in Visual Basic) to your project.

    Imports System.Collections.Generic
    
    Friend Module TypeSystem
    
        Friend Function GetElementType(ByVal seqType As Type) As Type
            Dim ienum As Type = FindIEnumerable(seqType)
    
            If ienum Is Nothing Then 
                Return seqType
            End If 
    
            Return ienum.GetGenericArguments()(0)
        End Function 
    
        Private Function FindIEnumerable(ByVal seqType As Type) As Type
    
            If seqType Is Nothing Or seqType Is GetType(String) Then 
                Return Nothing 
            End If 
    
            If (seqType.IsArray) Then 
                Return GetType(IEnumerable(Of )).MakeGenericType(seqType.GetElementType())
            End If 
    
            If (seqType.IsGenericType) Then 
                For Each arg As Type In seqType.GetGenericArguments()
                    Dim ienum As Type = GetType(IEnumerable(Of )).MakeGenericType(arg)
    
                    If (ienum.IsAssignableFrom(seqType)) Then 
                        Return ienum
                    End If 
                Next 
            End If 
    
            Dim ifaces As Type() = seqType.GetInterfaces()
    
            If ifaces IsNot Nothing And ifaces.Length > 0 Then 
                For Each iface As Type In ifaces
                    Dim ienum As Type = FindIEnumerable(iface)
    
                    If (ienum IsNot Nothing) Then 
                        Return ienum
                    End If 
                Next 
            End If 
    
            If seqType.BaseType IsNot Nothing AndAlso
               seqType.BaseType IsNot GetType(Object) Then 
    
                Return FindIEnumerable(seqType.BaseType)
            End If 
    
            Return Nothing 
        End Function 
    End Module
    

    The IQueryProvider implementation that you added previously uses this helper class.

    TypeSystem.GetElementType uses reflection to obtain the generic type argument of an IEnumerable<T> (IEnumerable(Of T) in Visual Basic) collection. This method is called from the non-generic CreateQuery method in the query provider implementation to supply the element type of the query result collection.

    This helper class is not specific to this TerraServer-USA Web service provider. Therefore, it can be reused for any LINQ provider.

To create an expression tree helper class

  • Add the ExpressionTreeHelpers class to your project.

    Imports System.Linq.Expressions
    
    Friend Class ExpressionTreeHelpers
        ' Visual Basic encodes string comparisons as a method call to 
        ' Microsoft.VisualBasic.CompilerServices.Operators.CompareString. 
        ' This method will convert the method call into a binary operation instead. 
        ' Note that this makes the string comparison case sensitive. 
        Friend Shared Function ConvertVBStringCompare(ByVal exp As BinaryExpression) As BinaryExpression
    
            If exp.Left.NodeType = ExpressionType.Call Then 
                Dim compareStringCall = CType(exp.Left, MethodCallExpression)
    
                If compareStringCall.Method.DeclaringType.FullName = 
                    "Microsoft.VisualBasic.CompilerServices.Operators" AndAlso 
                    compareStringCall.Method.Name = "CompareString" Then 
    
                    Dim arg1 = compareStringCall.Arguments(0)
                    Dim arg2 = compareStringCall.Arguments(1)
    
                    Select Case exp.NodeType
                        Case ExpressionType.LessThan
                            Return Expression.LessThan(arg1, arg2)
                        Case ExpressionType.LessThanOrEqual
                            Return Expression.GreaterThan(arg1, arg2)
                        Case ExpressionType.GreaterThan
                            Return Expression.GreaterThan(arg1, arg2)
                        Case ExpressionType.GreaterThanOrEqual
                            Return Expression.GreaterThanOrEqual(arg1, arg2)
                        Case Else 
                            Return Expression.Equal(arg1, arg2)
                    End Select 
                End If 
            End If 
            Return exp
        End Function 
    
        Friend Shared Function IsMemberEqualsValueExpression(
            ByVal exp As Expression, 
            ByVal declaringType As Type, 
            ByVal memberName As String) As Boolean 
    
            If exp.NodeType <> ExpressionType.Equal Then 
                Return False 
            End If 
    
            Dim be = CType(exp, BinaryExpression)
    
            ' Assert. 
            If IsSpecificMemberExpression(be.Left, declaringType, memberName) AndAlso 
               IsSpecificMemberExpression(be.Right, declaringType, memberName) Then 
    
                Throw New Exception("Cannot have 'member' = 'member' in an expression!")
            End If 
    
            Return IsSpecificMemberExpression(be.Left, declaringType, memberName) OrElse 
                   IsSpecificMemberExpression(be.Right, declaringType, memberName)
        End Function 
    
    
        Friend Shared Function IsSpecificMemberExpression(
            ByVal exp As Expression, 
            ByVal declaringType As Type, 
            ByVal memberName As String) As Boolean 
    
            Return (TypeOf exp Is MemberExpression) AndAlso 
                   (CType(exp, MemberExpression).Member.DeclaringType Is declaringType) AndAlso 
                   (CType(exp, MemberExpression).Member.Name = memberName)
        End Function 
    
    
        Friend Shared Function GetValueFromEqualsExpression(
            ByVal be As BinaryExpression, 
            ByVal memberDeclaringType As Type, 
            ByVal memberName As String) As String 
    
            If be.NodeType <> ExpressionType.Equal Then 
                Throw New Exception("There is a bug in this program.")
            End If 
    
            If be.Left.NodeType = ExpressionType.MemberAccess Then 
                Dim mEx = CType(be.Left, MemberExpression)
    
                If mEx.Member.DeclaringType Is memberDeclaringType AndAlso 
                   mEx.Member.Name = memberName Then 
                    Return GetValueFromExpression(be.Right)
                End If 
            ElseIf be.Right.NodeType = ExpressionType.MemberAccess Then 
                Dim mEx = CType(be.Right, MemberExpression)
    
                If mEx.Member.DeclaringType Is memberDeclaringType AndAlso 
                   mEx.Member.Name = memberName Then 
                    Return GetValueFromExpression(be.Left)
                End If 
            End If 
    
            ' We should have returned by now. 
            Throw New Exception("There is a bug in this program.")
        End Function 
    
        Friend Shared Function GetValueFromExpression(ByVal expr As expression) As String 
            If expr.NodeType = ExpressionType.Constant Then 
                Return CStr(CType(expr, ConstantExpression).Value)
            Else 
                Dim s = "The expression type {0} is not supported to obtain a value." 
                Throw New InvalidQueryException(String.Format(s, expr.NodeType))
            End If 
        End Function 
    End Class
    

    This class contains methods that can be used to determine information about and extract data from specific types of expression trees. In this provider, these methods are used by the LocationFinder class to extract location information from the expression tree that represents the query.

To add an exception type for invalid queries

  • Add the InvalidQueryException class to your project.

    Public Class InvalidQueryException
        Inherits Exception
    
        Private _message As String 
    
        Public Sub New(ByVal message As String)
            Me._message = message & " " 
        End Sub 
    
        Public Overrides ReadOnly Property Message() As String 
            Get 
                Return "The client query is invalid: " & _message
            End Get 
        End Property 
    End Class
    

    This class defines an Exception type that your provider can throw when it does not understand the LINQ query from the client. By defining this invalid query exception type, the provider can throw a more specific exception than just Exception from various places in the code.

You have now added all the pieces that are required to compile your provider. Build the LinqToTerraServerProvider project and verify that there are no compile errors.

You can test your LINQ provider by creating a client application that contains a LINQ query against your data source.

To create a client application to test your provider

  1. Add a new Console Application project to your solution and name it ClientApp.

  2. In the new project, add a reference to the provider assembly.

  3. Drag the app.config file from your provider project to the client project. (This file is necessary for communicating with the Web service.)

    Note Note

    In Visual Basic, you may have to click the Show All Files button to see the app.config file in Solution Explorer.

  4. Add the following using statements (Imports statement in Visual Basic) to the Program.cs (or Module1.vb in Visual Basic) file:

    using System;
    using System.Linq;
    using LinqToTerraServerProvider;
    

    Imports LinqToTerraServerProvider
    
  5. In the Main method in the file Program.cs (or Module1.vb in Visual Basic), insert the following code:

    QueryableTerraServerData<Place> terraPlaces = new QueryableTerraServerData<Place>();
    
    var query = from place in terraPlaces
                where place.Name == "Johannesburg"
                select place.PlaceType;
    
    foreach (PlaceType placeType in query)
        Console.WriteLine(placeType);
    

    Dim terraPlaces As New QueryableTerraServerData(Of Place)
    
    Dim query = From place In terraPlaces 
                Where place.Name = "Johannesburg" 
                Select place.PlaceType
    
    For Each placeType In query
        Console.WriteLine(placeType.ToString())
    Next
    

    This code creates a new instance of the IQueryable(Of T) type that you defined in your provider, and then queries that object by using LINQ. The query specifies a location to obtain data on by using an equality expression. Because the data source implements IQueryable, the compiler translates the query expression syntax into calls to the standard query operators defined in Queryable. Internally, these standard query operator methods build an expression tree and call the Execute or CreateQuery methods that you implemented as part of your IQueryProvider implementation.

  6. Build ClientApp.

  7. Set this client application as the "StartUp" project for your solution. In Solution Explorer, right-click the ClientApp project and select Set as StartUp Project.

  8. Run the program and view the results. There should be approximately three results.

The provider that you have to this point provides a very limited way for clients to specify location information in the LINQ query. Specifically, the provider is only able to obtain location information from equality expressions such as Place.Name == "Seattle" or Place.State == "Alaska" (Place.Name = "Seattle" or Place.State = "Alaska" in Visual Basic).

The next procedure shows you how to add support for an additional way of specifying location information. When you have added this code, your provider will be able to extract location information from method call expressions such as place.Name.StartsWith("Seat").

To add support for predicates that contain String.StartsWith

  1. In the LinqToTerraServerProvider project, add the VisitMethodCall method to the LocationFinder class definition.

    Protected Overrides Function VisitMethodCall(ByVal m As MethodCallExpression) As Expression
        If m.Method.DeclaringType Is GetType(String) And m.Method.Name = "StartsWith" Then 
            If ETH.IsSpecificMemberExpression(m.Object, GetType(Place), "Name") OrElse
               ETH.IsSpecificMemberExpression(m.Object, GetType(Place), "State") Then
                _locations.Add(ETH.GetValueFromExpression(m.Arguments(0)))
                Return m
            End If 
        End If 
    
        Return MyBase.VisitMethodCall(m)
    End Function
    
  2. Recompile the LinqToTerraServerProvider project.

  3. To test the new capability of your provider, open the file Program.cs (or Module1.vb in Visual Basic) in the ClientApp project. Replace the code in the Main method with the following code:

    QueryableTerraServerData<Place> terraPlaces = new QueryableTerraServerData<Place>();
    
    var query = from place in terraPlaces
                where place.Name.StartsWith("Lond")
                select new { place.Name, place.State };
    
    foreach (var obj in query)
        Console.WriteLine(obj);
    

    Dim terraPlaces As New QueryableTerraServerData(Of Place)
    
    Dim query = From place In terraPlaces 
                Where place.Name.StartsWith("Lond") 
                Select place.Name, place.State
    
    For Each obj In query
        Console.WriteLine(obj)
    Next
    
  4. Run the program and view the results. There should be approximately 29 results.

The next procedure shows you how to add functionality to your provider to enable the client query to specify location information by using two additional methods, specifically Enumerable.Contains and List(Of T).Contains. When you have added this code, your provider will be able to extract location information from method call expressions in the client query such as placeList.Contains(place.Name), where the placeList collection is a concrete list supplied by the client. The advantage of letting clients use the Contains method is that they can specify any number of locations just by adding them to placeList. Varying the number of locations does not change the syntax of the query.

To add support for queries that have the Contains method in their 'where' clause

  1. In the LinqToTerraServerProvider project, in the LocationFinder class definition, replace the VisitMethodCall method with the following code:

    Protected Overrides Function VisitMethodCall(ByVal m As MethodCallExpression) As Expression
        If m.Method.DeclaringType Is GetType(String) And m.Method.Name = "StartsWith" Then 
            If ETH.IsSpecificMemberExpression(m.Object, GetType(Place), "Name") OrElse
               ETH.IsSpecificMemberExpression(m.Object, GetType(Place), "State") Then
                _locations.Add(ETH.GetValueFromExpression(m.Arguments(0)))
                Return m
            End If 
        ElseIf m.Method.Name = "Contains" Then 
            Dim valuesExpression As Expression = Nothing 
    
            If m.Method.DeclaringType Is GetType(Enumerable) Then 
                If ETH.IsSpecificMemberExpression(m.Arguments(1), GetType(Place), "Name") OrElse
                   ETH.IsSpecificMemberExpression(m.Arguments(1), GetType(Place), "State") Then
                    valuesExpression = m.Arguments(0)
                End If 
    
            ElseIf m.Method.DeclaringType Is GetType(List(Of String)) Then 
                If ETH.IsSpecificMemberExpression(m.Arguments(0), GetType(Place), "Name") OrElse
                   ETH.IsSpecificMemberExpression(m.Arguments(0), GetType(Place), "State") Then
                    valuesExpression = m.Object 
                End If 
            End If 
    
            If valuesExpression Is Nothing OrElse valuesExpression.NodeType <> ExpressionType.Constant Then 
                Throw New Exception("Could not find the location values.")
            End If 
    
            Dim ce = CType(valuesExpression, ConstantExpression)
    
            Dim placeStrings = CType(ce.Value, IEnumerable(Of String))
            ' Add each string in the collection to the list of locations to obtain data about. 
            For Each place In placeStrings
                _locations.Add(place)
            Next 
    
            Return m
        End If 
    
        Return MyBase.VisitMethodCall(m)
    End Function
    

    This method adds each string in the collection that Contains is applied to, to the list of locations to query the Web service with. A method named Contains is defined in both Enumerable and List(Of T). Therefore, the VisitMethodCall method must check for both of these declaring types. Enumerable.Contains is defined as an extension method; therefore the collection it is applied to is actually the first argument to the method. List.Contains is defined as an instance method; therefore the collection it is applied to is the receiving object of the method.

  2. Recompile the LinqToTerraServerProvider project.

  3. To test the new capability of your provider, open the file Program.cs (or Module1.vb in Visual Basic) in the ClientApp project. Replace the code in the Main method with the following code:

    QueryableTerraServerData<Place> terraPlaces = new QueryableTerraServerData<Place>();
    
    string[] places = { "Johannesburg", "Yachats", "Seattle" };
    
    var query = from place in terraPlaces
                where places.Contains(place.Name)
                orderby place.State
                select new { place.Name, place.State };
    
    foreach (var obj in query)
        Console.WriteLine(obj);
    

    Dim terraPlaces As New QueryableTerraServerData(Of Place)
    
    Dim places = New String() {"Johannesburg", "Yachats", "Seattle"}
    
    Dim query = From place In terraPlaces 
                Where places.Contains(place.Name) 
                Order By place.State 
                Select place.Name, place.State
    
    For Each obj In query
        Console.WriteLine(obj)
    Next
    
  4. Run the program and view the results. There should be approximately 5 results.

This walkthrough topic showed you how to create a LINQ provider for one method of a Web service. If you want to pursue additional development of a LINQ provider, consider these possibilities:

  • Enable the LINQ provider to handle other ways of specifying a location in the client query.

  • Investigate the other methods that the TerraServer-USA Web service exposes, and create a LINQ provider that interfaces with one of those methods.

  • Find a different Web service that you are interested in, and create a LINQ provider for it.

  • Create a LINQ provider for a data source other than a Web service.

For more information about how to create your own LINQ provider, see LINQ: Building an IQueryable Provider on MSDN Blogs.

Show: