RPC and C++: Build a Template Library for Distr...

We were unable to locate this content in de-de.

Here is the same content in en-us.

This article may contain URLs that were valid when originally published, but now link to sites or pages that no longer exist. To maintain the flow of the article, we've left these URLs in the text, but disabled the links.
MSDN Magazine

RPC and C++: Build a Template Library for Distributed Objects Containing Multiple Interfaces

Ajai Shankar
This article assumes you�re familiar with C++ and RPC
Level of Difficulty     1   2   3 
Download the code for this article: Shankar1100.exe (37KB)
Browse the code for this article at Code Center: RPC Demo
SUMMARY Building a C++ template library for developing distributed object-oriented applications using Remote Procedure Calls (RPC) allows the programmer to design RPC applications that are composed of components that implement multiple interfaces. This article demonstrates the use of techniques such as assembly-level thunks to build machine code on the fly, and C++ features such as templates, conversion operators, virtual destructors and virtual function tables. Along the way, familiar C++ interfaces and classes are transformed into a distributed application. The benefits of using distributed objects rather than raw remote procedures, which include writing fewer lines of code, is explained.
I n this article I will show how assembly-level thunks can be used to transform a remote procedure call (RPC) into a call to a virtual method on an object, and vice versa. I'll start with a brief overview on developing RPC applications using context handles and entry point vectors (EPV), then discuss the code for this article, including a template library for developing distributed objects using remote procedure calls. With apologies to C++ purists, I include a rather esoteric technique of building a virtual function table on the fly and a surprising use of C++ operator overloading.
      Specifically, I take familiar C++ interfaces and classes and transform them into a distributed application. Most importantly, I use only RPC to achieve my objectives, rather than relying on other component technologies. Figure 1 shows the C++ interfaces and classes used in the sample for this article.
      Two interfaces, IShape and IRotate, are declared. The Circle class implements the IShape interface and the Square class implements both interfaces. The IRotate interface is really not necessary since the rotate method could be a part of the IShape interface itself. But I've included it as an example of a class implementing multiple interfaces. Also the Square and the Circle classes give their own distinct implementation of the IShape interface. The virtual methods of the interfaces are declared using the __cdecl calling convention. This ensures that the this pointer of the object is placed on the stack rather than in a register when the method is invoked. I will put this to good use later.
      In the case of a nondistributed application, objects of these classes can be created and used like this:
Circle cl;

IShape *shape = &cl;
shape->move_to(100, 200);

Square sq;

shape = &sq;
float f = shape->area();

IRotate *r = &sq;

      In the distributed version, remote clients need to be able to create Circle and Square objects on the server as easily as this. Similarly, the clients should be able to get the interfaces as easily as they can in the nondistributed version. But first, the client has to be able to create and delete objects on the server.

RPC Interface for Creating and Deleting Objects

      Let's take a brief look at the steps in developing an RPC application. (You can find a more detailed reference and examples in the MSDN® Library and at http://msdn.microsoft.com/library/psdk/rpc/rpcstart_084L.htm.) First, declare the interfaces in an interface definition language (IDL) file. Then run the MIDL compiler on the IDL file to generate the client-side proxy, the server-side stub, and the header file that declares the interface specifications and the entry point vectors. Next, provide implementations for the functions on the server side and link them with the server-side stub generated by MIDL. When the server application runs, it registers the interfaces and their implementations with the RPC runtime. Finally, link the client with the proxy file generated by MIDL. The client can now call the functions in the interface once it obtains a handle to the server.
      What exactly is an RPC interface? An RPC interface is a set of related methods that are grouped together and have a globally unique identifier. The first RPC interface is the following code from the objxx.idl file:
interface IObject
    typedef [context_handle] void* POBJECT;
    void IObject_create(handle_t hBinding, [out] POBJECT *ppObj);
    void IObject_delete([in, out] POBJECT *ppObj);
      The IObject interface consists of just two methods, IObject_create and IObject_delete. The IObject_create method accepts a binding handle and returns a context handle. The IObject_delete method deletes the context handle returned by IObject_create.
      What are these binding and context handles? Before a client can invoke a remote procedure, it has to specify which server to connect to and which protocol sequence (such as TCP/IP or named pipes) to use. A binding handle contains this information. It also contains endpoint information and an objectid. The endpoint information identifies which port, or set of ports, the server is listening on for remote procedure calls. I will explain what an objectid is later, but endpoint information really isn't relevant to our discussion. In fact, the framework uses dynamic endpoints, which means the client won't need to specify the endpoint information in the binding handle. Note that I have tested this project over the ncacn_ip_tcp (TCP) and ncadg_ip_udp (UDP) protocol sequences, but it does not work over named pipes and the local RPC. This is because when a context handle is created from a binding handle, the objectid in the binding handle does not seem to get transferred to the context handle.
      To invoke the IObject_create method on the server rpc.example.com which is listening for remote procedure calls over TCP/IP, the client has to create a binding handle:
handle_t handle;
char str[128];
sprintf(str, "%s:%s", "ncacn_ip_tcp", "rpc.example.com");
RpcBindingFromStringBinding((unsigned char*)str, &m_handle);
Then it invokes the remote procedure IObject_create like this:
POBJECT context;
IObject_create(handle, &context);
      When the client invokes the remote procedure, what it is actually doing is making a call to a proxy method, which marshals the arguments that are passed and sends them over the network to the server. On the server, the call goes to a stub, which unmarshals these arguments and calls the actual method that the server implements. The return values and the [out] arguments are also marshaled and unmarshaled over the network.
      Running midl.exe on the IDL file creates the proxies and stubs. For my interface, the following files are created.
  • objxx_c.câ€"the client-side proxy
  • objxx_s.câ€"the server-side stub file
  • objxx.hâ€"the header file which declares the remote procedures, the interface specification, and so on.
      A context handle is nothing but a pointer that makes sense on the server. The client never tries to interpret what it means. In our example, the context handle would be a pointer to a Circle or Square object. But most importantly, a context handle retains all information that was present in the binding handle that was used to create it. This is why a client can call the remote procedure IObject_delete using a context handle in place of a binding handle like this:

Multiple Implementations on the Server

      As you can see, I have declared only one interface to create and delete objects of any type. This means that the server needs to somehow provide multiple implementations for the same IObject interface, one that creates Square objects and another that creates Circle objects. The client should also be able to specify the object it wants to create. The server provides multiple implementations for the same interface using EPVs.
      As you can see in Figure 2, an EPV is nothing but an array of pointers to functions. In this case there are two EPVs, one for the Circle class and another for the Square class (see Figure 3).
      The server registers each of these vectors with a different manager typeid. A manager typeid is yet another GUID, and is associated with an objectid. For purposes of this discussion, a manager typeid is the same as the objectid, so I will use the term classid to refer to the objectid or manager typeid.
      Thus, the Square class has its own classid and the Circle class has a different classid. The server registers the different EPVs for the IObject interface as follows:

typedef IObject_v1_0_s_ifspec IObject_if_spec;

// associate manager typeid with objectid
RpcObjectSetType(Circle::get_clsid(), Circle::get_clsid()); 
// register the IObject interface for Circle
RpcServerRegisterIfEx(IObject_if_spec, Circle::get_clsid(), 

RpcObjectSetType(Square::get_clsid(), Square::get_clsid());
RpcServerRegisterIfEx(IObject_if_spec, Square::get_clsid(), 
      Since these calls need to be made for any number of interfaces that a class implements, the file rpcxx.h contains the functions Register and Unregister. These functions take a classid and an array of interface specification/entry point vector pairs and registers and unregisters the interfaces, as you can see in Figure 4.
      If the client wants to create a Square object, it would specify the classid in the binding handle like this, and call IObject_create:
handle_t handle;
char str[128];
sprintf(str, "%s:%s", "ncacn_ip_tcp", "rpc.server.com");
RpcBindingFromStringBinding((unsigned char*)str, &m_handle);
RpcBindingSetObject(handle, Square::get_clsid());

POBJECT context;
IObject_create(handle, &context);
      The function RpcBindingSetObject associates an objectid with the binding handle. When the call arrives on the server, the RPC runtime checks the objectid in the binding handle or the context handle. Then it looks up the manager typeid for this objectid (in this case, the classid) and calls the method in the EPV registered with this manager typeid (in this example, the EPV for the Square class).
      The file rpcxx.h contains the class ObjectProxy that wraps the client-side calls, and the macro DECLARE_CLSID that declares the get_clsid method. Figure 5 shows the client-side versions of Circle and Square objects that use this class.
      The client can obtain context handles to remote objects with the following code, while the destructor takes care of releasing the context handles:
Circle cl("rpc.example.com"); // get context handle to a circle
Square sq("rpc.example.com");

A Generic IObject Implementation

      Now you have seen how the server provides different methods (Square_create, Circle_destroy, for instance) that create and destroy Square and Circle objects, and how it registers the entry point vectors. This would work if it weren't for a small but significant hitch. Let's say a client holds a context handle to an object on the server, but then the client crashes before it can release the context handle. If this happens, the object on the server will never be destroyed. The RPC runtime handles this situation as follows. If a named context handle exists (in this case POBJECT), the server is supposed to provide a context rundown routine which will delete the object. The runtime calls this routine whenever there is a communication breakdown with the client for any reason. In this case the following rundown routine must be provided:
void __RPC_USER POBJECT_rundown(POBJECT pobj) \
    { // delete the object; ???
    pobj = 0; }
      Since the rundown routine does not know what type of pointer pobj holds, it cannot delete it! In a C program it would just have to free the memory pointed to by pobj. But a delete in C++ implies that the destructor has to be called. This is important for nontrivial classes (unlike the Circle and Square classes).
      It can be a hassle to write functions like Square_create and Square_delete for every remotable class that you create. Let's see an elegant solution using the features provided by C++. Figure 6 shows the C++ class that wraps the IObject interface.
      The only significant thing about this class is that the destructor is declared as virtual. If I ensure that all remotable classes are derived from IObject, I can define a generic delete function like this.
inline void _delete(void **ppobj)
    {delete static_cast<IObject*>(*ppobj);
    *ppobj = 0; }
      This will work, as the destructor will be called through a virtual function table. The rundown routine would be:
void __RPC_USER POBJECT_rundown(POBJECT pobj) \
    {_delete(&pobj); }
I can declare a generic class that will provide the EPV for any class:
template<class _Ty>
class ObjectStub
    static IObject::epv_type* stub()
        {static IObject::epv_type _stub = {_create, _delete };
        return &_stub; }
    static void _create(handle_t handle, void **ppobj)
        {*ppobj = static_cast<IObject*>(new _Ty()); }
      The _create function creates a new object of the class _Ty on the heap, and returns the IObject pointer to that object. The static_cast ensures that the class _Ty does indeed derive from IObject. The fact that the code returns a pointer to IObject means that the _delete function will work as expected.
      The entry point vectors for the Circle and Square classes are:
IObject::epv_type *Circle_epv = ObjectStub<Circle>::stub();

IObject::epv_type *Square_epv = ObjectStub<Square>::stub();
      The only restrictions imposed on remotable classes are that they should be derived from IObject, and they should have a constructor that accepts no arguments.
      The file rpcxx.h contains the macros BEGIN_STUB_IMPL and END_STUB_IMPL that declare the beginning and the end of an array of interfaces that a class implements. Figure 7 shows the minimal changes that had to be made to write the remote version of the Circle and Square classes.
      The END_STUB_IMPL macro also declares the RegisterClass and UnregisterClass methods that register and unregister the interfaces and the entry point vectors for this class. With these macros, the server version of the Square class becomes:
class Square
    : public IShape, public IRotate, public IObject
    DECLARE_CLSID(0xeebbf0c0, 0x1544, 0x11d4, 0x83, 0xef, 0x00, 0xc0,
                  0x4f, 0x6e, 0xe8, 0x17)
    // IShape implementation
    // IRotate implementation

The implementation for the Circle class is similar.

Implementing Other Interfaces

      Though the client can easily create and destroy objects on the server, it can't do anything very useful with the objects it creates. This is because the RPC versions of the IShape and the IRotate interfaces have not been declared. Figure 8 shows how to declare these interfaces.
      The most important fact to note in Figure 8 is that the RPC versions of the interfaces accepts a context handle as the first argument. This is because the client has to specify the object on which it wants to invoke the method. For example, the client would have obtained context handles to two Square objects as shown in the following lines of code, and would have to specify which object to rotate, move, and so on. And just as a reminder, the context handle retains the objectid associated with the binding handle that was used to create it.
Square sq1("localhost");
Square sq2("rpc.example.com");

IShape_move(sq2->m_context, 100, 200);
IRotate_rotate(s2->m_context, 30);
      I ran midl.exe on demo.idl to obtain the proxy, stub, and header file that declare the interface specifications and the entry point vectors for these interfaces. As usual, the server has to implement the methods and register the entry point vectors. The IShape_draw method for the Circle and Square would have to be implemented as follows.
void Circle_IShape_draw(POBJECT obj)
    {IObject *o = static_cast<IObject*>(obj);
    Circle *c = static_cast<Circle*>(o);
    IShape *s = static_cast<IShape*>(c);
    s->draw(); }

void Square_IShape_draw(POBJECT obj)
    {IObject *o = static_cast<IObject*>(obj);
    Square *sq = static_cast<Square*>(o);
    IShape *s = static_cast<IShape*>(sq);
    s->draw(); }

      Boy, that's a lot of code. I will have to write forwarding functions for each method of each interface that a class implements. This seems like a lot of trouble just to prove that you can write object-oriented distributed applications using only RPC. But the question is, do you really need to write these forwarding functions? The answer is no, if you use thunks.

Using Thunks

      A thunk is a block of machine code that does weird, undocumented things like modifying the program stack, then jumping to another address. Before I examine this topic in detail, let's see what the memory layout of a Square looks like. Let's say the client holds a context handle to a Square object located at memory address 1000 in the server's address space. Figure 9 shows how the object will be organized in memory.
      When the _create function returns the context handle to this object, the IObject part of the Square object (static_cast<IObject*>((Square*) 1000) = 1008) is returned. This value can be arrived at using the classoffset macro:
#define classoffset(class, super) ((unsigned long)
static_cast<super*>((class*)8) - 8)
This macro finds the offset of a superclass from within a subclass. The class offsets of the various interfaces for the Square class are:
classoffset(Square, IShape) is 0
classoffset(Square, IRotate) is 4
classoffset(Square, IObject) is 8
Given an IObject pointer at 1008, the pointer to the Square object is as follows:
IObject *obj = (IObject*) 1008;
Square *sq = (Square*) ((unsigned long) obj - classoffset(Square, 
                        IObject)); // 1000 = 1008 - 8
Similarly, given a pointer to a Square object, the pointer to the various interfaces that Square implements are:
Square *sq = (Square*) 1000;

IShape *s = (IShape*) ((unsigned long) sq + classoffset(Square,  
            IShape));  // 1000 = 1000 + 0

IRotate *r = (IRotate*) ((unsigned long) sq + classoffset(Square, 
             IRotate)); // 1004 = 1000 + 4
Given an IObject pointer, the pointers to the other interfaces that Square implements are:
IObject *obj = (IObject*) 1008;

IShape *s = (IShape*)
            ((unsigned long) obj + classoffset(Square, IShape) - 
            classoffset(Square, IObject)); // 1000 = 1008 + 0 - 8

IRotate *r = (IRotate*) 
             ((unsigned long) obj + classoffset(Square, IRotate) - 
             classoffset(Square, IObject)); // 1004 = 1008 + 4 - 8
      As you know, virtual functions declared as using the __cdecl calling convention are called like this:
  1. Push the this pointer on the stack.
  2. Get the ((unsigned long) *this) value from the vtable.
  3. Call the function at index 0, 1, 2, and so on.
      Figure 10 shows how these steps are translated into assembly language.
      Instead of writing the forwarding function Square_IShape_draw, let's say there is a block of actual machine code. When the code is called with a POBJECT as an argument, it does the following:
  1. Changes the POBJECT passed as the first argument to point to the IShape pointer.
  2. Jumps to the function at index 0 (the draw method of IShape).
      Similarly, instead of the forwarding functions Square_IShape_area and Square_IShape_move_to, there are blocks of code that change the POBJECT to point to an IShape and jump to the functions at index 1 and 2.
      Let's assume these thunks are registered as entry point vectors for the Square class and that IShape interface, and a client holds a context handle to the IObject located at address 1008. If the client calls IShape with this code
IShape_move_to(context, 100, 200);
then the following would occur:
  1. The RPC runtime stub sees that a remote procedure call is being made.
  2. It checks the objectid in the context handle and finds that it belongs to class Square.
  3. It finds the entry point vector associated with the IShape interface for this class (the thunks that were registered).
  4. It calls the method move_to (function at index 2 in the EPV) with these arguments: move_to(1008, 100, 200).
  5. When the thunk gets control, the stack looks like this:
    200             [esp + 10]
    100             [esp + 8]
    1008            [esp + 4]
    return-address  [esp]
  6. The thunk knows the distance from the IObject interface to the IShape interface for class Square is -8, and it changes [esp + 4] to 1000.
  7. It gets the IShape vtable, which is the address inside 1000.
  8. It jumps to the function at index 2.
      The net effect is the same as it would be if the RPC runtime executed the following C++ code:
IObject *o = static_cast<IObject*>(obj); // o is 1008
Square *sq = static_cast<Square*>(o);    // sq is 1000
IShape *s = static_cast<IShape*>(sq);    // s is 1000
s->move_to(100, 200); // call function at index 2
                      // in the vtable at [1000]
Figure 11 shows what a thunk really looks like.
      The hexadecimal codes that you see in Figure 11 are the actual machine codes for the assembly language instructions. When a thunk is initialized, it is given the offset to the interface from IObject and the index of the function in the virtual function table. The entry point vector consisting of thunks for any class and any interface can be obtained using the generic class shown in Figure 12.
      The number of functions in the RPC interface is the size of:
(epv_type of that interface) / sizeof(a pointer)
The stub declares an array of thunks to virtual functions of the interface _If implemented by the class _Impl. In this case, m_epv is the actual entry point vector holding pointers to each of these thunks. The entry point vectors for each interface for each of the classes are the following:
Stub<Square, IShape>::stub();
Stub<Square, IRotate>::stub();
Stub<Circle, IShape>::stub();

      Figure 13 shows the macro DECLARE_IF_STUB in rpcxx.h and the resulting server-side code. These classes can be registered with the RPC runtime using RegisterClass and UnregisterClass. The only restrictions imposed on the C++ version of the interfaces are that all virtual methods must be declared as using the __cdecl calling convention, each interface should typedef its entry point vector type as epv_type, and each interface should have a method get_s_ifspec that returns the interface specification for the RPC version of this interface.

The Client-side Code

      You have seen that by deriving the client's version of the classes from ObjectProxy, you can easily obtain a context handle to an object of any class on the server and release said object. But the same cannot be said for invoking methods using this context handle. In fact, in the previous section you saw a non-object-oriented version of how a client invokes the move_to method on the IShape interface:
IShape_move_to(context, 100, 200);

      There is a lot of room for improvement here. The initial solution would be, as usual, to write forwarding functions for each method of each interface. Figure 14 shows what the client's version of the Circle class would look like if I chose this solution. This is perfectly fine, provided I have the patience to write all these forwarding functions. But there is an easier way.

Client-side Thunks using a vtable

      As you've probably guessed, I will use thunks again, but this time with a vengeance. As I mentioned in the introduction to this article, I am going to show how to actually build some virtual function tables on the fly. The entries in these tables are going to be pointers to thunks to the proxy functions generated by midl.exe. I will also use C++ operator overloading in a novel way to ensure that calls go through these vtables.
      First I need to add a static function, get_proxy_epv, to each of the C++ interfaces. This function returns a pointer to the entry point vector for the RPC version of the interface, consisting of pointers to the RPC client-side proxies for this interface, as you can see in Figure 15.
      Declare this template class as:
template<class _Impl, class _If>
class Proxy
    operator _If*()
        {return reinterpret_cast<_If*>(this); }
    void **m_vtable;
      This class takes a class and an interface as template arguments and overloads the _If* conversion operator to return a pointer to itself. The only member of the class is the double pointer m_vtable. This seems like a crazy thing to do right now, but let's go ahead and make some minor changes to the client version of the Square class:
class Square
    : public ObjectProxy
    , public Proxy<Square, IShape>
    , public Proxy<Square, IRotate>
    DECLARE_CLSID(0xeebbf0c0, 0x1544, 0x11d4, 0x83, 0xef, 0x00, 0xc0,
                  0x4f, 0x6e, 0xe8, 0x17)
    Square(char *address = "rpc.example.com")
        : ObjectProxy(get_clsid(), address) {}
      The Proxy class overloads the IShape* and the IRotate* allows the following conversions:
Square sq("rpc.example.com");

IShape *s = sq; // through Proxy<Square, IShape>
IRotate *r = sq; // through Proxy<Square, IRotate>
As Square derives from Proxy<Square, IShape> and also Proxy<Square, IRotate>, the resulting memory layout of an object of this class located at address 1000 in the client address space is shown in Figure 16.
      The previous code uses the conversion operators and returns the following memory addresses:
IShape *s = sq; // "this" pointer for Proxy<Square, IShape> = 1008

IRotate *r = sq; // "this" pointer for Proxy<Square, IRotate> = 1012
      If the client calls s->draw, the compiler generates code similar to the following:
IShape *s = (IShape*) 1008;

push s ; push "this" pointer
mov ecx, s;
move ecx, [ecx];
// get at the vtable, i.e. get at the m_vtable
// of the Proxy class ! (whatever is inside address 1008)
call [ecx + 0]; // call the draw function at index 0
      Thus, the m_vtable of the Proxy class has to be initialized to point to some valid code. Otherwise the result is a GPF error. But what should it be initialized to? No prizes for guessing this oneâ€"it should be initialized to pointers to thunks that call the client-side proxy functions. These thunks must replace the IShape pointer on the stack (at [esp + 4]) with m_context of the ObjectProxy and jump to the actual proxy method.
      The thunk at m_vtable[0] should jump to the function at IShape::get_proxy_epv()[0], the thunk at index 1 should jump to the function at IShape::get_proxy_epv()[1], and so on. The actual proxy functions will be called via the thunks with the context handle as the first argument.
      The offset of class Proxy<Square, IShape> to the m_context data member of class ObjectProxy can be obtained from the following expression:
#define offsetof(class, field)((unsigned long)&((class*)0)->field)

typedef Proxy<Square, IShape> _Pxy;

unsigned long _Ctx = 
classoffset(Square, ObjectProxy)+offsetof(ObjectProxy, m_context) -
classoffset(Square, _Pxy);
In other words, 0 + 4 - 8, or -4.
      The offsetof macro gives the offset of a field within a class. The expression I just cited can be verified for the Square object at address 1000. The value of m_context is the value at address 1004 (1008 - 4). Similarly, for class Proxy<Square, IRotate> the offset is -8. Figure 17 shows the actual client-side thunk for a function .
      ProxyThunk is initialized using the offset of the Proxy<class, interface> from the address of the m_context member of class ObjectProxy, and a pointer to an actual proxy function to call. Each class/interface pair (Proxy<class, interface>) would have its own array of thunks, and a vtable containing pointers to these thunks. The member m_vtable would point to this vtable.

      Figure 18 shows how the Proxy class is initialized. The ProxyVtbl class initializes the virtual function table for each class/interface pair to point to thunks that jump to the correct RPC proxy function when called. The constructor for the Proxy class initializes the m_vtable pointer to the vtable built this way. With this framework in place, the client-side code is quite small (see Figure 19).


      That completes the tour of distributed object-oriented programming using RPC. The sample code with this article builds a demo server and client in fewer than 140 lines of code, one of the real advantages of using distributed objects in place of raw remote procedures. This article showed how templates, virtual destructors, thunks, and the rest could be put to good use, reminding the developer that C++ is a powerful language that is limited only by your own creativity.
For related articles see:

For background information see:
The Remote Procedure Call reference

Ajai Shankar is a programmer/analyst for IRIS Software Inc. in New Jersey. He is presently working as a consultant for Wal-Mart in Bentonville, Arkansas.

From the November 2000 issue of MSDN Magazine.

Page view tracker