CLR Inside Out
.NET Application Extensibility, Part 2
Jack Gudenkauf and Jesse Kaplan
Code download available at: Add Ins 2007_03.exe(177 KB)
Contents
Pipeline Architecture
Pipeline Components
Pipeline Implementation
Changing the Host and Its Views
Host and Add-In Compatibility
Summing Up
In our first column, where we introduced the new System.AddIn namespace in the Base Class Libraries (BCLs), we walked through the typical process customers undertake in order to provide extensibility through their host applications. We discussed the functionality needed to offer a comprehensive add-in solution. We also looked at some sample host code, as well as the new System.AddIn APIs used by the host. And, finally, we showed some sample add-in code that clearly demonstrated how the add-in developer can remain blissfully ignorant of the underlying mechanics of consuming the object model provided by the host.
We postponed the discussion of the System.AddIn pipeline and we promised to show the utilization of the model to enable compatibility as your host and add-in version independently. This time we'll first go into detail about the architecture of the model. Then we'll look at version 2.0 of the application we reviewed last month, using it to get an in-depth view of how our pipeline works and to demonstrate how you can use our system to keep add-ins working even as the host changes.
Before you read on, it may be helpful to look at the "Terminology" sidebar. Note also that the System.AddIn namespace and associated classes will be in the version of Visual Studio® expected to follow Visual Studio 2005, code-named "Orcas." You may download the new libraries prior to the full release of Visual Studio "Orcas" via the Community Technology Preview. You can find additional information at the CLR Add-in Team's blog (blogs.msdn.com/clraddins).
Terminology
Here are some terms that are used throughout this column.
Add-ins (also known as add ins, add ons, extensions, plug-ins, plug ins, snap-ins, and snap ins) are custom code, usually written by a third party that is dynamically loaded and activated by a host application, based upon some context (such as host startup or document loading).
Backward compatibility is the ability of the current version of an application or component to consume or use components targeted at an earlier version of the application or component. For example, the common language runtime (CLR) 2.0 is backward compatible with version 1.1 because it can run 1.1-targeted applications or components.
From a host and add-in perspective, a host application is said to be backward-compatible with its previous versions if add-ins written against those previous versions run without modification against the new version.
Forward compatibility represents the ability of the current version of an application or component to consume or use components targeted at a future version of the application or component. Forward compatibility is therefore mostly theoretical since it relies on the future. Full forward compatibility is only possible if nothing ever changes in the future version. For example, an application that was written against the CLR 2.0 features would be forward compatible if it also ran against the CLR 1.1, but this is not supported by the CLR.
From a host and add-in perspective, a host application is forward compatible if add-ins written against later versions of it can run unmodified on the previous version.
Pipelines are the communication machinery that enables the host and add-in to communicate over a version-resilient protocol.
Pipeline Architecture
Figure 1 shows pipeline components that enable a host to communicate with an add-in. The pipeline components are used in activating and communicating with the add-in.
Each component in the diagram is actually an assembly with many different types: for every type of complex object exchanged between the host and the add-in there are corresponding view, contract, and adapter types on both sides of the boundary. It is important to note that the diagram is completely symmetrical and is mirrored across the isolation boundary, with each side following the model shown in Figure 2. The host and add-in are entities that provide the real implementation and direct value to the user. One of the high-level goals of this model, and a primary driver of these design patterns, is that they can communicate and interact with each other even as one or both sides change.
Figure 1** Communication Pipeline between Host and Add-in **(Click the image for a larger view)
Figure 1 illustrates a number of important facts. First, the host and add-in are segregated from each other via an isolation boundary. Each one only references its view and thus gains a layer of abstraction over the contracts used for communication. Furthermore, there is no direct reference to the host/add-in itself, thus allowing relative isolation from all other components communicating with it.
Figure 2** Host/Add-In Model **(Click the image for a larger view)
Pipeline Components
Contracts The contract assembly contains the non-versioning types (contracts) that are loaded in both sides of the isolation boundary as the communication mechanism over that boundary. By restricting to contracts all communication over the boundary, we prevent the host's and add-in's implementation types from leaking across the boundary and causing versioning issues. In our system, all contracts inherit from the base IContract which serves as the "object" type. Note that the contract assembly is not allowed to change, but that hosts and add-ins can use different contracts for communication without either entity being aware of it. If you are familiar with COM, you'll notice in the following code that IContract has similar functionality to IUnknown (QueryInterface, AddRef, and Release):
public interface IContract
{
int AcquireLifetimeToken();
int GetRemoteHashCode();
IContract QueryContract(string contractIdentifier);
bool RemoteEquals(IContract contract);
string RemoteToString();
void RevokeLifetimeToken(int token);
}
Contracts must inherit from IContract and must be interfaces; thus they do not allow a specific implementation. They provide a separation of integration and implementation types. Contract signatures are constrained to serializable types defined in mscorlib (such as Int32, String, and DateTime), other contracts, and enumerations defined in the contract assembly. Contract interfaces also reconcile with the contract model of Windows® Communication Foundation, to ensure that there is no implementation in these types and to allow multiple inheritance.
Views Views are assemblies containing abstract classes that represent the host's and add-in's view of each other and the types that flow between them. The purpose of views is to give the host and add-ins a stable view of the world that is not dependent on the contracts used to communicate between them. Views also separate the add-in and host from each other's implementation on either side of the boundary. The diagram shows that the host/add-in only depend on their view and that the view itself has no dependencies; this allows the adapters and contracts to be swapped out without impacting the host or add-in.
Adapters Adapters assemblies contain adapter classes that are used to convert to and from views and the contracts. We use the term "side" loosely in our diagram in order to discuss the side of the boundary an adapter resides. The reality is that depending on the direction of a call, the adapter is either converting from a view to a contract or a contract to a view. Therefore, if you have calls going in both directions, that is, the host calling into the add-in and the add-in calling into the host, you will have two adapters on each "side" of the boundary.
A class in the adapter assembly that converts a view to a contract is a view-to-contract adapter. This class implements the contract by calling into the view passed into its constructor and is marshaled across the boundary as a contract. The contract-to-view-adapter is a class in the adapter assembly that converts a contract to a view. This class inherits from the view and implements its abstract methods by calling into the contract passed into its constructor.
The diagram shows that the adapter assemblies reference both the view and the contract and thus provides the connection between the two. The adapters provide the glue that lets you keep hosts and add-ins connected and working even as one or both sides change dramatically. We'll have more details on this later.
Pipeline Implementation
Now let's take a look at the sample code and see how the implementation plays out in practice. We'll use version 2.0 of the sample application we discussed last time and explore how the host changed and how the pipeline hooks the add-in and host together. Then we'll see how to use our model to make sure the add-ins we built last time against the first version of the host can be made to work unchanged against the new version.
The system is involved only in activating the add-in and passing it back to the host. Once that occurs, it gets out of the way and lets the pipeline handle the flow of types. Since the activation path is the first step in host/add-in interaction, let's look at it in detail.
Activation of an add-in occurs after the host has discovered the available add-ins and has chosen which one to activate (for details see Program.cs in the download or last month's column). In our sample, activation occurs with this simple code:
//Activate the selected AddInToken in a new AppDomain
Calculator calculator =
calcToken.Activate<Calculator>(AddInSecurityLevel.Internet);
From the host's point of view, it simply calls activate and receives the add-in back to it typed as its view "Calculator". Behind the scenes, however, the system is starting up the add-in and hooking up the pipeline.
To do so, it first creates the AppDomain that the add-in is going to run in and then loads the add-in, add-in view, add-in side adapter, and contract assembly into the newly created domain. The add-in view is shown in Figure 3 as add-in base, which is simply an attribute the pipeline uses to designate an add-in view class that is used in activation. Next, the system instantiates the add-in with its default constructor and the add-in adapter and passes the add-in, typed as the add-in base, to the adapter's constructor. Then it passes the add-in adapter across the AppDomain boundary typed as its contract (in this case ICalculatorContract2), loads the host-side-adapter assembly, and instantiates an instance of the host adapter, passing the add-in adapter, typed as the contract, to its constructor. Finally, it returns the host-side-adapter to the host, typed as the host add-in view (in this case, Calculator).
Figure 3** Activation Path **(Click the image for a larger view)
This seems like a lot of overhead, but in reality the performance cost of using this model to communicate across the boundary, versus communicating across the boundary directly over an interface, is minimal.
If you look at the add-in activation path you'll notice that it is really just an example of an object being passed from the add-in to the host. The only thing different about it is that the system itself instantiates the add-in and connects the components; in order for the system to identify these components and instantiate them correctly, we require that each component involved in activation have a custom-attribute applied in order to identify it. Typically, one type in each pipeline component is involved in activation and will have one of these attributes applied to it with the only exception being the host-view. The host-view is provided directly to the system via our APIs and thus doesn't need to be attributed.
In our sample, the add-in is providing itself as a service to the host, and the only data types passed between them are doubles and strings. Because of this, the sample only includes types involved in the activation pathway. Please see our team blog for examples of hosts and add-ins passing complex data in both directions.
Changing the Host and Its Views
The host application we discussed previously needed the add-ins to provide it a service: specifically it needed them to supply basic calculator functionality in the form of the Add, Subtract, Multiply, and Divide methods.
The original host's view of its add-ins looked like this:
public abstract class Calculator {
public abstract double Add(double a, double b);
public abstract double Subtract(double a, double b);
public abstract double Multiply(double a, double b);
public abstract double Divide(double a, double b);
}
In our new version, however, we've decided that the host needs more functionality from the add-ins and more flexibility in what calculator services they should provide. Thus, it's defined this new view of its add-ins:
public abstract class Calculator {
public abstract string Operations { get; }
public abstract double Operate(
string operation, double a, double b);
}
Rather than relying on add-ins for four specific operations, the host now wants to know what operations the calculator supports and then have the ability to run those operations. This type of change is typical of hosts that rely on add-ins for a service: either they discover they need additional functionality from them over time, or they decide to make the extension point more flexible. With our model, the host will only program against this updated view and will rely on the pipeline to connect the legacy add-ins to this new view.
You can find the definition of the new view in CalculatorView2.cs in the HostCalculatorView project in the sample. For details on how the host utilizes this new view, see Program.cs in SampleExtensibleCalculator.csproj.
Our custom contract (ICalculatorContract2), is the type used to communicate across an isolation boundary. The original contract looked like this:
[AddInContract]
public interface ICalculatorContract : IContract
{
double Add(double a, double b);
double Subtract(double a, double b);
double Multiply(double a, double b);
double Divide(double a, double b);
}
Now that the host has changed its view of add-ins, it is moving to a different contract as well and now uses the following:
[AddInContract]
public interface ICalculatorContract2 : IContract
{
string GetAvailableOperations();
double Operate(String operation, double a, double b);
}
You can see that the host view and the changes made to it are typically mirrored in its contracts.
The add-in view typically mirrors the host view and it changes in the same way the contracts do. In this sample, the only thing differentiating the add-in view from the host view is the assembly it is contained in and the attribute marking the add-in Base (the type used in activation):
[AddInBase]
public abstract class Calculator2
{
public abstract string Operations{ get; }
public abstract double Operate(
string operation, double a, double b);
}
Add-ins must inherit from the add-in base classes and apply an attribute to themselves identifying them for activation. You can see the sample code for the full implementation in the download, but the add-in's definition looks like this:
[AddIn("Basic V2 Add-in", Version="2.0.0.0")]
public class SampleV2AddIn : Calculator2
{
public override string Operations
{
get { ... }
}
public override double Operate(
string operation, double a, double b)
{
...
}
}
In our sample code, the add-in is providing a service to the host application and therefore the host needs to call into the add-in for functionality that extends the host. In this scenario, the add-in side adapter is providing the role of View To Contract Adapter.
The sample code shown in Figure 4 is also in CalculatorView2ToContract2AddInAdapter.cs and may be found in the AddInSideAdaptersV2.csproj project in the download. The AddInAdapterAttribute is an attribute used by System.AddIn.Hosting APIs to find and activate the add-in adapter class.
Figure 4 CalculatorViewToContractAddInAdapter
[AddInAdapter]
public class CalculatorViewToContractAddInAdapter : ContractBase,
ICalculatorContract2
{
private Calculator_view;
public CalculatorViewToContractAddInAdapter(
Calculator2 calculator) {
view = calculator;
}
public string GetAvailableOperations(){
return _view.Operations;
}
public double Operate(string operation, double a, double b){
return _view.Operate(operation, a, b);
}
}
The add-in adapter derives from ContractBase, which is a base class provided by System.AddIn.Pipeline. This class derives from MarshalByRefObject, IContract, and ISponsor. We will discuss ISponsor and lifetime management in the host adapter section below. The add-in adapter also derives from ICalculatorContract which is the custom contract. This contract is used to proffer up the new add-in calculator's GetAvailableOperations and Operate methods. Both of these methods are implemented in version 2.0 of our add-in (SampleV2AddIn.cs). The add-in adapter constructor takes in the add-in (SampleV2AddIn) as the add-in view (CalculatorContractsBase.Calculator). It implements the required functionality of ICalculatorContract2 by calling into the add-in (typed as the add-in Base Calculator 2).
In our sample code, the host is consuming a service provided by the add-in and therefore the host needs to call into the add-in for functionality that extends the host. In this scenario, the host side adapter is providing the role of contract-to-view adapter. The host will call through the functionality provided by the add-in adapter. The code in Figure 5 (from CalculatorContract2ToView2HostAdapter.cs in the HostSideAdapters.csproj project) illustrates.
Figure 5 CalculatorContractToViewHostAdapter
[HostAdapter]
public class CalculatorContractToViewHostAdapter : Calculator
{
private ICalculatorContract2 _contract;
private System.AddIn.Pipeline.LifetimeTokenHandle _handle;
public CalculatorContractToViewHostAdapter(
ICalculatorContract2 contract)
{
_contract = contract;
_handle = new LifetimeTokenHandle(contract);
}
public override string Operations {
get { return _contract.GetAvailableOperations(); }
}
public override double Operate(string operation, double a, double b) {
return _contract.Operate(operation, a, b);
}
}
In this example, HostAdapterAttribute is used by the System.AddIn.Hosting APIs to find and activate the host adapter class (CalculatorContractToViewHostAdapter). The host adapter derives from the host view. As the label implies, the host view is the host's view of the add-in. The host adapter constructor takes in the add-in adapter typed as the custom contract (in this case ICalculatorContract2).
You'll notice that the host adapter code looks very similar to the add-in adapter code. The main difference is that it is adapting from a contract to a view, rather than from a view to a contract. The host adapter takes the contract into its constructor and inherits from the view while the add-in adapter does the opposite. The only other interesting piece of code shown here is the LifetimeTokenHandle, which we'll address below.
The host is calling through to the add-in for functionality, therefore it needs to hold a reference to the remote object, which resides in the add-in's AppDomain. As a result, the lifetime of the object needs to be determined by the host, not by the add-in. We provide a simple mechanism, built on remoting, which ensures the add-in-provided object in the add-in's AppDomain remains alive as long as the host holds a reference to the object. When the host releases this reference and the Garbage Collector (GC) collects the host-referenced object, the remote object lease will also expire. This mechanism of lifetime management hides the complexities of the underlying mechanics from the host and add-in. Both the add-in developer and host developer may program as though the objects (views) they consume and provide are local resources with the memory for these resources managed by the GC.
This reference mechanism works for the types that are implemented by either the host or the add-in. The same mechanism is also used in determining the lifetime of the add-in. When the host releases the reference to the add-in (it is an object after all), the add-in is unloaded. If the add-in is the last object running in the AppDomain that the host owns, the pipeline will then unload the AppDomain.
The following sample code in CalculatorContract2ToView2HostAdapter.cs shows you the incredible ease of implementing lifetime management:
private System.AddIn.Pipeline.LifetimeTokenHandle _handle;
_handle = new System.AddIn.Pipeline.LifetimeTokenHandle(contract);
The LifetimeTokenHandle class works with the implementer of IContract, in most cases the system-provided ContractBase class, to manage the lifetime of objects passed across the boundary.
When the LifetimeTokenHandle class is instantiated in the ContractToViewAdapter (which is the side receiving the contract), it acquires a LifetimeToken from the contract. The GC finalizes the object once it has determined that no more references to that adapter exist, and the GC then revokes the LifetimeToken. On the other side, in the ViewToContractAdapter (which is the side sending the contract), the adapter keeps track of all of the outstanding lifetime tokens; when the count drops to zero, the adapter allows itself to be garbage collected. Additionally, if this is the add-in's AppDomain and the host has no other references into the domain, it unloads the AppDomain.
Host and Add-In Compatibility
If the host (host view) or its data types change from one version to the next, in order to get the old add-ins working on the new host view, the host creates a second add-in side adapter, which inherits from the new contract and understands the new data types and converts them to the original add-in view. The host or pipeline developer will need to provide default functionality for new methods (see Figure 6).
You can find the sample code for such an adapter in AddInSideV1toV2Adapter.csproj in the download. For full details on how to implement this, please see the sample, but the important part is the definition of the class and its constructor.
Figure 6** New Host, Old Add-Ins for Backward Compatibility **(Click the image for a larger view)
The key takeaway from Figure 7 is that to adapt the V1 add-ins to the V2 contract, you just need to build an add-in adapter that inherits from the V2 contracts (ICalculatorContracts2) and takes in the V1 add-in base in its constructor (Calculator).
Figure 7 CalculatorV1ViewToV2ContractAddInAdapter
[AddInAdapter]
public class CalculatorV1ViewToV2ContractAddInAdapter : ContractBase,
ICalculatorContract2
{
CalculatorContractsBase.Calculator _view;
public CalculatorV1ViewToV2ContractAddInAdapter(Calculator calc)
{
_view = calc;
}
public string GetAvailableOperations()
{
return "+, -, *, /" ;
}
public double Operate(string operation, double a, double b)
{
switch (operation)
{
case "+": return _view.Add(a, b);
case "-": return _view.Subtract(a, b);
case "*": return _view.Multiply(a, b);
case "/": return _view.Divide(a, b);
default:
throw new InvalidOperationException(
"This add-in does not support: " + operation);
}
}
}
The benefit of this approach is that old add-ins will continue to work even with the changes and that new add-ins can build and take advantage of the new object model. The application itself doesn't need to keep track of different versions of the add-ins and the host only deals with a single host view that the different add-in side adapters convert into the different add-in versions. This greatly simplifies the host's code base and lets a host drop support for legacy versions by simply not shipping an adapter for that version.
Summing Up
Our goal in designing this model was to enable dynamic composition of version-resilient, isolatable components. Our primary concern was to ensure the add-in developer experience would not be complicated by the mechanics involved in meeting this goal.
The System.AddIn model consists of the BCLs found in the System.AddIn namespace, as well as guidance on the design patterns you apply in communicating between components. This column discussed the pipeline architecture, an implementation of the pipeline, lifetime management, and utilization of the model to enable compatibility as your host and add-in version independently.
Send your questions and comments for James to <clrinout@microsoft.com.>.
Jack Gudenkauf is a Software Architect on the CLR team at Microsoft. He is currently focused on engaging with ISV/VAR/SI customers to ascertain .NET architectural adoption issues. Jack has held positions including Development Manager, MIS Director, software engineering, and consulting services. Before he joined Microsoft, he ran his own consulting company, The Logical Choice.
Jesse Kaplan is the Program Manager for Extensibility and add-ins for the CLR team at Microsoft. His past responsibilities include compatibility and managed/native interoperability.