Performance Comparison: .NET Remoting vs. ASP.NET Web Services
Microsoft Developer Network
Summary: Compares relative performance of Microsoft ASP.NET Web services, which provide the highest levels of interoperability with full support for WSDL and SOAP over HTTP; and Microsoft .NET Remoting, which is designed for common language runtime type-system fidelity, and supports additional data format and communication channels. (14 printed pages)
ASP.NET Web services and .NET Remoting provide a full suite of design options for cross-process communication in distributed applications. In general, ASP.NET Web services provide the highest levels of interoperability with full support for WSDL and SOAP over HTTP, while .NET Remoting is designed for common language runtime type-system fidelity and supports additional data format and communication channels. For more information, see ASP.NET Web Services or .NET Remoting: How to Choose.
This article focuses on comparing relative performance of these techniques.
ASP.NET Web Services
ASP.NET provides a Microsoft® IIS-hosted infrastructure that supports industry standards such as SOAP, XML, and WSDL. Although.NET Remoting supports IIS hosting and SOAP over HTTP, ASP.NET is designed to provide the highest level of SOAP interoperability including support for SOAP Section 5 and document/literal.
ASP.NET can leverage the features available with IIS, such as security and logging. IIS hosting is also robust in that it will re-spawn Aspnet_wp.exe if it terminates. Also, ASP.NET Web services are much easier to create and consume than services exposed using .NET Remoting because configuration is simplified both at the server and client.
For more details, see Building XML Web Services Using ASP.NET in the .NET Framework Developer's Guide.
.NET Remoting is more versatile and extensible in terms of enabling communication between objects using different transport protocols and serialization formats. TCP, HTTP, and custom protocols are supported as are Binary, SOAP, and custom formats. Multiple-object creation and lifetime modes are supported including Singleton, SingleCall, and Client-Activated. Remoting requires a host process, which can be IIS, or a Microsoft® Windows service or executable written in .NET.
A .NET Remoting object using the SOAP formatter can be exposed as an XML Web service when hosted in IIS with ASP.NET. Since the payload is encoded in SOAP over HTTP, any client that supports the SOAP Encoding format can access the objects over the Internet. Another advantage of using the HTTP protocol is that it generally passes unobstructed across most firewalls. The TCP channel and the Binary formatter can be employed in an intranet environment where both the server and client are running on the .NET Framework. This approach is optimized for performance since raw sockets are used to transmit data across the network using a custom protocol that is more efficient than HTTP. Though this approach offers excellent performance in a closed environment, it cannot be employed in cross-platform scenarios where the client is not running the .NET Framework.
IIS hosting provides secure communication for wire-level protection using Secure Sockets Layer (SSL), and you can also leverage IIS and ASP.NET authentication and authorization. The TCP channel as well as the HTTP channel with non-IIS hosting do not support secure socket transmission, so data is transmitted in clear text. You are required to implement your own security if you are using the TCP channel or the HTTP channel hosted in processes other than IIS.
For more details, please refer to the .NET Remoting Overview in the .NET Framework Developer's Guide.
Performance of cross-process communication in distributed applications depends on a number of factors:
Channels including TCP and HTTP used to transport messages between applications across remoting boundaries impose various amounts of overheads. TCP sockets are more efficient than HTTP.
Serialization is another factor. The serialized stream can be encoded using XML, SOAP, or a compact binary representation. ASP.NET Web services use the XMLSerializer class to serialize an object into an XML stream, which is extremely fast with the exception that there is some overhead associated with XML parsing. Remoting uses BinaryFormatter and SOAPFormatter to serialize an object into binary and SOAP format, respectively. Since these formatters use reflection, they are fast for referenced objects, but can be slow for value types which have to be boxed and unboxed to pass through the reflection API. The SOAPFormatter has some additional overheads of generating encoded SOAP messages.
The tests used in this comparison are based on common business scenarios accessing customer and order data. To make the test as realistic as possible, the database contains over 100,000 rows of Customer accounts. The data is in a Microsoft® SQL Server™ 2000 database and the SQL Server .NET data provider is used to connect to SQL Server.
The following methods are used in the performance comparison:
The GetOrderStatus method accepts an OrderId and returns an integer representing the status of the order.
The GetCustomers method accepts a CustomerId and a parameter to specify the number of customer rows you want to read. The top n rows with CustomerId greater than the CustomerId passed to the Web Service method are read.
We performed tests with paging through a small set of customer rows with different page sizes: 20, and 50.
In our tests an ASPX Web page calls the remote object containing the test code. Although we would get better absolute performance if we were to test the techniques directly, rather than behind a Web server, testing in a stateless environment is realistic for common application scenarios. Moreover, there are a number of test tools available that can appropriately stress Web pages providing multithreaded testing and reliable performance statistics.
For our tests, we used Microsoft Application Center Test (ACT), which is designed to stress test Web servers and analyze performance and scalability problems with Web applications, including ASPX pages and the components they use. Refer to ACT documentation for details on how to create and run tests. Application Center Test can simulate a large group of users by opening multiple connections to the server and rapidly sending HTTP requests. It also allows us to build realistic test scenarios where we can call the same method with a randomized set of parameter values. This is an important feature whereby users are not expected to call the same method with the same parameter values over and over again. The other useful feature is that Application Center Test records test results that provide the most important information about the performance of the Web application.
As I mentioned earlier, there are two types of activation for remoting objects—server activation and client activation. The server directly controls the lifetime of a server-activated object. There are two modes of activation for server-activated objects—Singleton and SingleCall. Singleton types never have more than one instance running at one time. All client requests are serviced by a single instance, and hence, using this activation mode you can maintain state between clients. On the other hand, each client request is serviced by a separate instance in the case of SingleCall types. Client-activated objects are created on the server, but the calling application domain controls the lifetime of those objects. Using this mode allows you to maintain state between requests from the same client. ASP.NET only supports SingleCall, i.e., each request is serviced by a new instance, and if the service is to be stateful, then it must be managed using cookies and SessionState or custom SOAP headers. In all of the tests that have been performed to compare the various remoting options, SingleCall mode is used for a fair comparison.
Although ASP.NET Web services support SOAP, HTTP-GET, and HTTP-POST options we used only the SOAP option in our tests for consistency.
HTTP/1.1 recommends that any one client can only have up to two connections to a single server. Therefore, when using the HTTP protocol for communication (as in remoting with the HTTPChannel and ASP.NET), it only opens 2 connections by default at any given time to a given server, while the TCP channel opens as many connections as there are threads making requests to the server. For simulating multiple clients sending simultaneous requests to the remote object, we changed the default of 2 to 100 connections to the server per client using the client's configuration file:
When doing remoting with the HTTP channel—use the clientConnectionLimit attribute in the client's .config file:
<system.runtime.remoting> <application> … <channels> <channel ref="http" clientConnectionLimit="100"> <clientProviders> <formatter ref="soap" /> </clientProviders> </channel> </channels> </application> </system.runtime.remoting>
For ASP.NET Web Services, use the maxConnection attribute in <system.net> in the client's .config file:
<system.net> <connectionManagement> <add address="*" maxconnection="100" /> </connectionManagement> </system.net>
Since unlimited HTTP connections are allowed to the "localhost", i.e., when both the client and the server are on the same machine, you don't need to change the configuration file.
The following tables provide a brief summary of the test bed configuration used to perform the tests.
Table 1. Client Machine Configuration
|Number of clients||Machine/CPU||# Of CPUs||Memory||Disk||Software|
|1||Compaq Proliant 1130 MHz||2||1GB||16.9 GB||
Table 2. Web Server Configuration
|Number of servers||Machine/CPU||# Of CPUs||Memory||Disk||Software|
|3||Compaq Proliant 1000 MHz||2||1 GB||16.9 GB||
Table 3. Application Server Configuration
|Number of servers||Machine/CPU||# Of CPUs||Memory||Disk||Software|
|1||Compaq Proliant 1126 MHz||2||1 GB||16.9 GB||
Table 4. Database Server Configuration
|Number of servers||Machine/CPU||# Of CPUs||Memory||Disk||Software|
|1||Compaq Proliant 700 MHz||4||4 GB||18 GB||
Throughput and latency are the key performance indicators. For a given amount of data being returned, throughput is the number of client requests processed within a certain unit of time, typically within a second. Because peak throughput may occur at a response time that is unacceptable from a usability standpoint, we tracked latency—measured as response time using the report generated by Application Center Test for each of the tests run.
In the cross-machine scenario, the remoting client and the remoting object are on different computers. The ACT clients send requests to an ASPX Web page that in turn calls the method on the remote object.
Figure 1. Cross-machine scenario
As Figure 1 shows, the tests were performed behind a Web server, and there were database access and network hops to external machines, which added on extra overhead. Therefore, the performance numbers generated for throughput and latency act merely as a basis to compare these technologies. They do not represent absolute performance. Exact throughput and latency for distributed systems built using ASP.NET Web services and .NET Remoting will depend on the architecture used.
Here we compare the behavior of various techniques when getting a single value from the database.
Figure 2. GetOrderStatus: Throughput and latency
- ASMX: ASP.NET Web service
- All the other options represent various .NET Remoting configurations.
- In all of the other options, the name hints the host, transport protocol and format that are being used, i.e., Host_TransaportProtocol_Format.
- 'WS' is used as an abbreviation for Windows Service, which hosts the remoting type.
- IIS_HTTP_Binary/SOAP are hosted in IIS with ASP.NET.
WS_TCP_Binary, as shown in Figure 2, in which the object is configured to use the TCP channel and the Binary formatter with host being a Windows service, outperforms other distribution techniques. The reason for this is that this approach transmits binary data over raw TCP sockets, which are more efficient than HTTP. and since the data does not have to be encoded/decoded, there is less processing overhead. We see around 60% performance spread between WS_TCP_Binary and the slowest approach.
Though IIS_HTTP_Binary produces the same binary payload as WS_HTTP_Binary does, it is slower, as there is an extra process hop from IIS (Inetinfo.exe) to Aspnet_wp.exe. The same reason explains the performance difference between IIS_HTTP_SOAP and WS_HTTP_SOAP.
WS_HTTP_Binary and WS_TCP_SOAP offer very similar performance. Although the former has the additional overhead of parsing HTTP, and the latter has the additional overhead of parsing SOAP, it seems the overhead of HTTP parsing and SOAP parsing come out about the same in this case.
The ASP.NET Web service outperforms IIS_HTTP_SOAP and WS_HTTP_SOAP, since ASP.NET XML serialization is more efficient than .NET Remoting SOAP serialization. And as can be seen in Figure 2, the ASP.NET Web service is very similar to IIS_HTTP_Binary in performance.
GetCustomers Using a Custom Class
In this section, the remote method retrieves the customer records from the database into a DataReader, populates a Customers object using the data in the DataReader and returns the Customers object. We performed tests with a result set of 20 and 50 rows to see how the amount of data returned impacts performance.
Figure 3. GetCustomers(n=20): Throughput and latency
The relative performance of other options gains substantially over WS_TCP_Binary, as the cost of marshalling the Customers object becomes a significant factor here. In this test we are serializing a Customers object containing 20 Customers as compared to serializing an integer in the previous test.
IIS_HTTP_Binary is slightly slower than WS_HTTP_Binary because of the extra process hop involved in the former case as mentioned earlier. Note that the ASP.NET Web service offers very similar performance as IIS_HTTP_Binary.
The performance of WS_TCP_SOAP is substantially reduced with more data and it is now comparable to WS_HTTP_SOAP and IIS_HTTP_SOAP. The reason being that most of the time is spent serializing data in this case, which becomes the significant factor affecting performance.
Figure 4. GetCustomers(n=50): Throughput and latency
- ASMX_SOAPExtn: Uses a SOAPExtension to work around a buffering issue with ASP.NET.
As you see in Figure 4, with increase in size of data, the performance difference between WS_TCP_Binary and other options is reduced all the more.
Note that the ASP.NET Web service has fallen behind IIS_HTTP_Binary. This is because of a known issue with buffering of large messages in ASP.NET that has been fixed in the upcoming version. With the buffering issue fixed, the ASMX option will be in line with IIS_HTTP_Binary, as we saw in the previous tests.
We worked around the buffering issue by implementing a SOAPExtension that provides some buffering between the XmlSerializer and the network. As you see in the graph, the ASMX_SOAPExtn option (which uses the SOAPExtension that we implemented) improves substantially over the ASMX option and is slightly behind IIS_HTTP_Binary. The SOAPExtension is included in the Building Distributed Applications with .NET downloadable sample code.
WS_TCP_SOAP, WS_HTTP_SOAP and IIS_HTTP_SOAP are similar in performance with WS_TCP_SOAP being slightly faster.
GetCustomers Using a DataSet
In the next set of tests, the remote method retrieves the customer records from the database in a DataSet, which is returned to the client.
Figure 5. GetCustomers(n=20): Throughput and latency
The relative performance between WS_TCP_Binary and other approaches has decreased as the overhead associated with dataset marshalling becomes a significant factor here.
IIS_HTTP_Binary maintains a slight lead over WS_HTTP_Binary. The cost of extra process hop in the latter has become negligible as compared to the data marshalling cost.
Note that the performance of the ASP.NET Web service has dropped so much so that it performs similar to WS_TCP_SOAP, WS_HTTP_SOAP and IIS_HTTP_SOAP. This drop in performance is due to two known issues in ASP.NET, which will be fixed in upcoming versions. One is the buffering issue as discussed earlier; other is related to DataSet serialization in ASP.NET. With these issues fixed, the ASP.NET Web service will be nearly comparable to IIS_HTTP_Binary. By working around the buffering issue with large messages, ASMX_SOAPExtn has gained largely over ASMX, as you see in Figure 5, however, the performance is still degraded because of the DataSet serialization issue.
Figure 6. GetCustomers(n=50): Throughput and latency
The relative performance between WS_TCP_Binary and other approaches has substantially dropped, as the size of the dataset is larger than in the previous test. We see very similar performance offered by WS_HTTP_SOAP, and IIS_HTTP_SOAP.
The ASP.NET Web service approach has fallen behind all the more because of the known issues we discussed earlier. Note that just by working around the buffering issue, ASMX_SOAPExtn gains significantly over ASMX.
As these tests demonstrate, the various design choices available with ASP.NET Web services and .NET Remoting carry varying amounts of overhead and therefore have vastly different performance characteristics. The size of data being passed is also significant and multiplies the differences between the other design choices. This comparison covers only stateless, synchronous remote procedure calls and does not cover the performance implications of authentication, authorization, and data privacy, which are other significant factors affecting the performance of distributed applications.
Though both the .NET Remoting infrastructure and ASP.NET Web services can implement inter-process communication, each is designed with a particular level of expertise and flexibility in mind to benefit a different target audience. If your application needs interoperability with other platforms or operating systems, you would be better off using ASP.NET Web services, as they are more flexible in that they support SOAP section 5 and Document/Literal. On the other hand, use .NET Remoting when you need the richer object-oriented programming model. See ASP.NET Web Services or .NET Remoting: How to Choose for details. In scenarios where performance is the main requirement with security and process lifecycle management is not a major concern, .NET Remoting TCP/Binary is a viable option; however, keep in mind that you can increase the performance of IIS-hosted implementations by adding a few more machines into the system, which may not be possible when using a .NET Remoting TCP/Binary implementation.