Service Tutorial 12 (C#) - Distributed LINQ Queries

Article
04/19/2010

DSS supports sending the LINQ Filters to the subscription manager, even if it is on a remote node. This means the computation (filtering) happens where the data is produced, and because its completely defined by the user, not the author of the service that publishes the data, it can do arbitrary filtering with no modifications required by the service author.

To summarize LINQ as subscription filters provides:

Custom compiler verified filtering: Arbitrary filtering specified by the consumer of the service, with no prior knowledge required by the author of the service publishing data. Services can now just produce Replace notifications and not have custom Update notifications for every useful sub-part of their state.
Simplified publisher model: Computation moves automatically to the data source (any DSS service) with the potential to drastically reduce notifications and bandwidth on the wire. Developers no longer need to issue Get or Query operations to a service if they wish to perform arbitrary filtering.
Dynamic code transfer: DSS infrastructure will automatically transfer the filter implementation assembly to the remote node the publisher executes, enabling true distributed, concurrent queries
Isolated, restricted code execution: Filter expressions are a restricted set code that can issue arbitrary requests to the system and only has access to variables defined in the filter class. The state or operations they operate on are cloned versions of the service state.

This tutorial illustrates how to subscribe a LINQ Filter to the Directory Service and get filtered notifications from it. The way subscriptions are created is explained in more detail in the tutorial 5 Service Tutorial 5 (C#) - Subscribing

This tutorial is provided in the C# language. You can find the project files for this tutorial at the following location under the Microsoft Robotics Developer Studio installation folder:

 Samples\ServiceTutorials\Tutorial12\CSharp

This tutorial teaches you how to:

Step 1: Subscribe to the Directory Service
Step 2: Handle unfiltered notifications from the Directory Service.
Step 3: Create the LINQ Filter.
Step 4: Add the LINQ Filter to the Subscription.
Step 5: Handle filtered notifications from the Directory Service.

Prerequisites

This tutorial uses the service written in Service Tutorial 1 (C#) - Creating a Service. The service you will create is referred to as ServiceTutorial12. Feel free to start a new project or keep working in the previous project, keeping in mind that some names may differ.

Hardware

This tutorial requires no special hardware.

Software

This tutorial is designed for use with Microsoft Visual C#. You can use:

Microsoft Visual C# Express Edition
Microsoft Visual Studio Standard, Professional, or Team Edition.

You will also need Microsoft Internet Explorer or another conventional web browser.

Getting Started

Under the Microsoft.Dss.ServiceModel.Dssp namespace, the ServiceInfoType class is defined to represent DSS service instances. It contains the following information:

Service: The service name
Contract: The contract name
Partner List: A list of the partner services associated to this service.
Alias List and HttpServiceAlias: A list of the Aliases associated to this service.

For simplicity, this tutorial is going to use ServiceInfoType class to define services. The state of the ServiceTutorial12 node is going to be a list of ServiceInfoType instances.

 /// <summary>
/// The ServiceTutorial12 State
/// </summary>
[DataContract()]
public class ServiceTutorial12State
{
    private List<ServiceInfoType> _directoryServices = new List<ServiceInfoType>();

    /// <summary>
    /// The list of filtered directory entries
    /// </summary>

    [DataMember]
    public List<ServiceInfoType> DirectoryServices
    {
        get
        {
            return this._directoryServices;
        }
        set
        {
            this._directoryServices = value;
        }
    }
}

This tutorial describes how to subscribe to the Directory Service running in the same DSS node. In order to communicate with it, it is necessary to include the following directives in the ServiceTutorial12.cs file.

 using ds = Microsoft.Dss.Services.Directory;
using dssp = Microsoft.Dss.ServiceModel.Dssp;

In order to receive notifications from the Directory Service, it is necessary to create an instance of the Microsoft.Dss.Services.Directory.DirectoryPort. Any notification produced as a result of the subscription is going to be posted to this port.

 /// <summary>
/// Directory notification port
/// </summary>
private ds.DirectoryPort _directoryNotify = new ds.DirectoryPort();

The next step is to create and send the subscription message. The Microsoft.Dss.Services.Directory directive is used to create an instance of the subscription message. Then, the _directoryNotify is set as a notification port. At this point no LINQ filter is specified, therefore the complete set of services will be returned. How to add a LINQ filter to the subscription is explained in Step 4.

 // Create a new Subscribe message and specify the port to
// send notifications to
var subscribe = new ds.Subscribe();
subscribe.NotificationPort = _directoryNotify;

(Notice the use of var in the code snippet above. This feature of C# allows the compiler to select the correct data type based on the context and simplifies writing code.)

Since the subcription is addressed to the Directory Service running in the same DSS node, the Directory Service is accessible by base.DirectoryPort. Once that the subscription message has been posted, the result is expected to be received at the ResponsePort of the subscription message. Both successful and failed cases are logged. Every time a new service is registered at the Directory Service, this will send an Microsoft.Dss.Services.Directory.Insert notification to the _directoryNotify port. Notifications will be handled by NotifyInsertHandler method.

 // Subscribe to the Directory Service
// (The DirectoryPort is defined in DsspServiceBase)
base.DirectoryPort.Post(subscribe);

Activate<ITask>
(
    Arbiter.Choice(subscribe.ResponsePort,
    delegate(dssp.SubscribeResponseType success)
    {
        LogInfo("Subscription successful");
    },
    delegate(W3C.Soap.Fault failure)
    {
        LogInfo("Subscription Failed");

    })
);

Activate<ITask>(Arbiter.Receive<ds.Insert>(true, _directoryNotify, NotifyInsertHandler));

Step 2: Handle unfiltered notifications from the Directory Service

In this Step, the contents of the NotifyInsertHandler method are described:

 private void NotifyInsertHandler(ds.Insert insert)

The NotifyInsertHandler method is in charge of processing the information conveyed in the Microsoft.Dss.Services.Directory.Insert message posted by the Directory Service. The body of the message is of type Microsoft.Dss.ServiceModel.Dssp.ServiceInfoType .

 ServiceInfoType serviceInfo= new ServiceInfoType();

 serviceInfo = insert.Body.Record;

The instance of the Microsoft.Dss.ServiceModel.Dssp.ServiceInfoType class is posted to the internal mainport of the service by sending an InsertDirectoryRequest message and invoking the InsertDirectory operation.

 _mainPort.Post(new InsertDirectory(serviceInfo));

InsertDirectory operation is derived from the Microsoft.Dss.ServiceModel.Dssp.Update operation. Both, InsertDirectory operation and the InsertDirectoryRequest message are defined as follows:

 /// <summary>
/// ServiceTutorial12 InsertDirectory Operation
/// </summary>
public class InsertDirectory : Update<InsertDirectoryRequest, PortSet<DefaultUpdateResponseType, Fault>>
{
    /// <summary>
    /// ServiceTutorial12 InsertDirectory Operation
    /// </summary>
    public InsertDirectory()
    {
    }

    /// <summary>
    /// ServiceTutorial12 InsertDirectory Operation
    /// </summary>
    public InsertDirectory(ServiceInfoType record)
        : base(new InsertDirectoryRequest(record))
    {
    }
}


/// <summary>
/// ServiceTutorial12 InsertDirectoryRequest
/// </summary>
[DataContract]
[DataMemberConstructor]
public class InsertDirectoryRequest
{
    /// <summary>
    /// ServiceTutorial12 InsertDirectoryRequest
    /// </summary>
    public InsertDirectoryRequest()
    {
    }

    /// <summary>
    /// ServiceTutorial12 InsertDirectoryRequest
    /// </summary>
    public InsertDirectoryRequest(ServiceInfoType record)
    {
        _record = record;
    }

    private ServiceInfoType _record;


    /// <summary>
    /// ServiceTutorial12 InsertDirectoryRequest
    /// </summary>
    [DataMember]
    public ServiceInfoType Record
    {
        get { return this._record; }
        set { this._record = value; }
    }
}

The invokation of the InsertDirectory operation is handled by the InsertDirectoryHandler method, which adds the new Microsoft.Dss.ServiceModel.Dssp.ServiceInfoType instance to the public list DirectoryServices of the ServiceTutorial12State.

 /// <summary>
/// Insert Directory Handler
/// </summary>
[ServiceHandler(ServiceHandlerBehavior.Exclusive)]
public IEnumerator<ITask> InsertDirectoryHandler(InsertDirectory insertDirectory)
{
    _state.DirectoryServices.Add(insertDirectory.Body.Record);
    insertDirectory.ResponsePort.Post(DefaultUpdateResponseType.Instance);
    yield break;
}

Build and run the service in the usual way.

Now if you navigate to https://localhost:50000/servicetutorial12, you should see the service state looking something like:

Figure 1 - Viewing the unfiltered state of ServiceTutorial12 in browser.

Note that the services listed are the same services by https://localhost:50000/directory. Since no LINQ filter was specified, all the running services are present.

Step 3: Create the LINQ Filter

Sending a LINQ filter to a remote node means that the computation will happen where the data is generated. Filtering can be arbitrary and can be completely defined by the subscriber, not the publisher of the data. Consequently, for security reasons the filter must follow the following rules:

DataContractFilter: The class implementing the filter is a subscriber defined class that must derive from DataContractFilter, defined in Microsoft.Dss.Base.dll.
Assembly: The filter or set of filters derived from the DataContractFilter class must be part of a standalone C# assembly.
Types: The filter will operate on the proxy types of the publisher assembly.

In this tutorial, it is described how to create a LINQ filter that follows the rules above. In order to create a new standalone C# assembly:

Right-click on the root of the Solution Explorer and select Add/New Project.
Choose the class library template.

In order to create a class that derives from DataContractFilter and use the proxy types, it is necessary to include the following directives:

 using ds = Microsoft.Dss.Services.Directory.Proxy;
using dssp = Microsoft.Dss.ServiceModel.Dssp;

The LINQ filter is a System.Linq.Expressions.Expression, therefore, this directive is also necessary.

 using System.Linq.Expressions;

The DirectoryFilter class aims to filter the notifications sent by the Directory Service. Only services partnered with the Subscription Manager will be returned.

 public class DirectoryFilter:dssp.DataContractFilter
{
    public string myPartner;

    public DirectoryFilter()
    {
        Expression<Func<ds.InsertRequest, IEnumerable<dssp.ServiceInfoType>>> exp = (ds.InsertRequest record) =>
          from r in new dssp.ServiceInfoType[] { record.Record }
          from p in r.PartnerList
          where(p.Name.Name.Equals(myPartner))
          select r;

        base.QueryHandler = exp.Compile();
    }
}

As part of the DirectoryFilter class, a public string variable myPartner is included to filter the names of the partners associated to each service.

The Expression is formed by a delegate System.Func that takes the proxy type Microsoft.Dss.Services.Directory.Proxy.InsertRequest as an input argument. The query operates on it, and returns a collection of Microsoft.Dss.ServiceModel.Dssp.ServiceInfoType instances that pass the filter.

LINQ expressions can be applied on collection types. In this particular example, however, the body of the individual notification is a single instance instead of a collection. Notice that in order to apply the LINQ from keyword on an individual message, an array dynamically with a single element is created.

Step 4: Add the LINQ Filter to the Subscription

In Step 1, a Subscription to the Directory Service was created. However, this subscription did not include a LINQ filter, therefore unfiltered results were received in the notification messages.

Now the filter created in Step 3 is going to the added to the subscription created in step 1. First, it is necessary to add a reference to the standalone C# assembly created in Step 3. Since both projects are in the same solution, you can go to Add Reference/Projects and select the assembly. Then the directive needs to be added.

 using DirectoryFilters;

Just after the assignation of the _directoryNotify port, you can create an instance of the DirectoryFilter class that was defined in the standalone C# assembly. Then, you can specify the partner you would like to filter with. In this case, SubscriptionManager is introduced. Finally, the LINQ filter instance needs to be added to the header of the subscription message.

 // Add a filter to the Subscribe message before sending it
var directoryFilter = new DirectoryFilter();
directoryFilter.myPartner = "SubscriptionManager";
subscribe.AddHeader(directoryFilter);

Step 5: Handle filtered notifications from the Directory Service

In Step 2 the NotifyInsertHandler method for the unfiltered case was explained. Now, it is necessary to add some additional logic that provides the filtered results of the LINQ Filter.

When notifications are issued, the LINQ Filter results must be retrieved from the Headers field collection in the notification operation. The result of a LINQ expression over a service state or service notification can be arbitrary types of data that might not be directly serializable. The infrastructure gets around this by wrapping the results in the DataContractFilterResults class and places an instance in the headers of every notification.

 var filteredResults = insert.GetHeader<DataContractFilterResults<dssp.ServiceInfoType>>();
if (filteredResults != null)
{
    serviceInfo = filteredResults.QueryResults[0];
}

Note that in this particular case, it is known that the filter is applied to a single instance instead of a collection. Consequentely, the collection of results within DataContractFilterResults class will only contain a single instance in this case.

Finally, the complete NotifyInsertHandler method that handles both unfiltered and filtered cases is shown:

 private void NotifyInsertHandler(ds.Insert insert)
{

    ServiceInfoType serviceInfo= new ServiceInfoType();


    if (!insert.HasHeaders)
    {
        LogInfo("Unfiltered Directory Notification Received");

        serviceInfo = insert.Body.Record;

    }
    else
    {
        LogInfo("Filtered Directory Notification Received");


        var filteredResults = insert.GetHeader<DataContractFilterResults<dssp.ServiceInfoType>>();
        if (filteredResults != null)
        {
            serviceInfo = filteredResults.QueryResults[0];
        }

    }


    _mainPort.Post(new InsertDirectory(serviceInfo));


}

Build and run the service in the usual way.

Now if you navigate to https://localhost:50000/servicetutorial12, you should see the service state looking something like:

Figure 2 - Viewing the filtered state of ServiceTutorial12 in browser.

Note that the services listed is a reduced list compared to the list shown by https://localhost:50000/directory.

Summary

In this tutorial, you learned how to:

Step 1: Subscribe to the Directory Service
Step 2: Handle unfiltered notifications from the Directory Service.
Step 3: Create the LINQ Filter.
Step 4: Add the LINQ Filter to the Subscription.
Step 5: Handle filtered notifications from the Directory Service.

Service Tutorial 12 (C#) - Distributed LINQ Queries

This tutorial teaches you how to:

See Also:

Prerequisites

Hardware

Software

Getting Started

Step 2: Handle unfiltered notifications from the Directory Service

Step 3: Create the LINQ Filter

Step 4: Add the LINQ Filter to the Subscription

Step 5: Handle filtered notifications from the Directory Service

Summary

Additional resources

Service Tutorial 12 (C#) - Distributed LINQ Queries

This tutorial teaches you how to:

See Also:

Prerequisites

Hardware

Software

Getting Started

Step 1: Subscribe to the Directory Service

Step 2: Handle unfiltered notifications from the Directory Service

Step 3: Create the LINQ Filter

Step 4: Add the LINQ Filter to the Subscription

Step 5: Handle filtered notifications from the Directory Service

Summary

Additional resources