Export (0) Print
Expand All

Test Scenario 1: Single VM Using a Physical Server (Baseline)

The purpose of test scenario one was to determine how a single physical BizTalk Server performed compared to a virtualized BizTalk Server running on the same hardware. To ensure a fair comparison, this scenario restricted the amount of available memory and logical processors available to the machine to be equivalent to that available to the virtualized BizTalk Server. This was done using the /maxmem and /numproc boot.ini switches. For more information, see “Boot INI Options Reference” at http://go.microsoft.com/fwlink/?LinkId=122139.

Hyper-V supports one or two virtual processors for each virtual machine that runs Windows Server 2003 R2 64-bit or 32-bit editions. The test software stack was based on the 64-bit platform and tested the performance of both one and two virtual processors running against the same specification. The specification of the physical machines used to run the physical BizTalk Server and the Hyper-V servers were identical aside from the networking configuration (see table below for details).

Physical BizTalk Server Windows Server 2008 Hyper-V Server

Model

HP DL380 G5

HP DL380 G5

Processor

Dual Processor Intel Xeon 2.33Ghz Quad core

Dual Processor Intel Xeon 2.33Ghz Quad core

Memory

4 GB

4 GB

Networking

HP NC3T3i Multifunction Gigabit Server adapter

Broadcom BCM5708C NetXtreme II GigE Intel ® Pro/1000PT Desktop Adapter

Both the physical and virtual machines used in this scenario were part of the same BizTalk group hosted by the same physical SQL Server machine. The specification of the physical machine (Physical_SQL01) used for scenario one tests is detailed in the table below. The SAN was configured as per the specification detailed earlier in this document.

Physical SQL Server (Physical_SQL_01)

Model

HP DL380 G5

Processor

Dual Processor Intel Xeon 2.33Ghz Quad core

Memory

4 GB

Networking

HP NC3T3i Multifunction Gigabit Server adapter

For scenario one, the infrastructure consisted of five load generator machines one of which ran the testing process using Visual Studio 2005 Team Test Edition. A monitoring machine was used to perform live analysis of data using perfmon. The SQL Server used was a physical server identical in configuration to the physical BizTalk Server machine. SAN attached storage was used for the SQL Server and configured as per the table in the “Physical Infrastructure Specifics” section in Overview of Test Scenarios.

Identical test runs were performed against both the physical BizTalk Server 1 and the virtual BizTalk Server 1 (see diagram below). Virtual BizTalk Server 1 was hosted on Hyper-V_01 as a virtual machine on Hyper-V.

Physical BizTalk Server Scenario

Cc768517.12967217-5101-452a-b8ac-4af89c610046(en-US,BTS.10).gif

Virtual BizTalk Server Scenario

Cc768517.d4ebe0bc-0979-4f3a-8d78-b2255e2ae7f1(en-US,BTS.10).gif

Test Configuration

The test configuration for all tests in this scenario was as follows:

  • 10,000 messages sent in total by one load generator

  • Loadgen configuration

    • Sleep Interval – 1

    • Threads Per Section – 35

    • LotSize – 4

Optimizations Applied

In order to tune the system to attain these results, the following optimizations (in addition to the standard platform optimizations such as database file separation) were performed:

  • The MaxReceiveInterval for all service classes was set to 100 ms.

  • To ensure that outbound HTTP connections was not a bottleneck, the max connections for any endpoint was set to 300 for all machines. For more information, see the “Setting SOAP and HTTP Adapter Concurrent Connections” section of the BizTalk Server Operations Guide at http://go.microsoft.com/fwlink/?LinkId=122140.

For more information on tuning BizTalk Server, see the “Optimizing BizTalk Server Performance” section of the Microsoft BizTalk Server Performance Optimization Guide at http://go.microsoft.com/fwlink/?LinkId=122141.

This scenario tested the effect of virtualizing BizTalk Server with one and two virtual processors assigned to the virtual machines. In a one virtual processor configuration, the virtualized BizTalk Server machine was able to process documents at 73% of the rate of the same physical server configuration. Latency was 30% lower on the virtual BizTalk Server compared to the same physical configuration.

With two virtual processers assigned to the BizTalk Server machine, the throughput improved by 34% compared to a single-processor virtual machine. Latency was within 3% of a one virtual processer BizTalk Server machine. In comparison to a physical BizTalk Server with two logical processors, throughput was approximately 76% of a physical two-processor server; latency was approximately 17% lower on the virtual BizTalk Server than the physical server.

Cc768517.note(en-US,BTS.10).gifNote
It is important to note that there is generally a trade-off in BizTalk systems between throughput and latency. Higher throughput typically equates to increased latency. With all other things being equal, a BizTalk Server solution running on physical hardware will provide higher throughput than a BizTalk Server solution running on virtual machines. This is due to the overhead that the virtual processor has when executing hypervisor code. In this scenario, it is possible that the BizTalk Server solution running on virtual machines will actually provide improved latency as compared to the BizTalk Server solution running on physical hardware.

To verify that networking is not a bottleneck, measure the network counters detailed in Measuring Performance on Hyper-V.

1-Core Physical BizTalk Server1-Processor Virtual BizTalk Server% Difference2-Core Physical BizTalk Server2-Processor Virtual BizTalk Server% Difference

Documents/Sec

36.6

26.8

-26.78%

47.4

36

-24.05%

Request-Response Duration Average (ms)

619

438

-29.24%

540

450

-16.67%

Cc768517.de38adcd-53ec-4229-ba04-e839b93fcc1a(en-US,BTS.10).gif

Cc768517.99d93802-ecc1-4c76-83e3-7ead968468f9(en-US,BTS.10).gif

In both the physical and virtual BizTalk Server environments, CPU utilization of the BizTalk Server was the bottleneck. The results indicate that in a single server configuration, BizTalk Server is able to attain 73% to 76% of the performance of the same physical configuration when CPU in the BizTalk Server tier is the main resource under contention.

Machine Configuration

For this comparison, both the physical and virtualized machines were part of the same BizTalk Server group with its databases hosted on the same physical SQL Server (Physical_SQL01). The physical and virtual machines were run with the following settings:

  • One processor \ available to the operating system

  • 1.5 GB of memory allocated to both machines

Results Analysis

The table and bar charts below present a summary of the results of a one-core physical against a single processor virtualized BizTalk Server. The main point of interest was that latency was consistently lower on the virtual BizTalk Server (438 ms) compared to the physical BizTalk Server (619 ms), but the number of processed documents per second was significantly higher on the physical server (36.6) compared to the Hyper-V server (26.8). The resource bottleneck during the tests for both the physical and Hyper-V environments was the CPU on the BizTalk Server. The PAL tool was used to perform a full analysis of each of the tests run and other resources including network, disk I/O and memory were not a resource bottleneck.

Name Run Number Request-Response Duration Average (ms) Documents/sec BizTalk CPU Utilization Hyper V % Guest Run Time (BizTalk VM instance) SQL\Processor(_Total)\% Processor Time

1-Core Physical BizTalk Server

63

619

36.6

91.7

N\A

18.6

1-Processor (Core) Virtual BizTalk Server

59

438

26.8

89.9

82

15.1

For this first comparison, the graphs of each of the key metrics are explained below. For the remaining scenarios, a summary of the data is provided.

Cc768517.567ad83b-2d22-4fd0-8e06-311ede90492f(en-US,BTS.10).gif

Cc768517.5252c181-5bc8-49c9-be24-3656a01cc88a(en-US,BTS.10).gif

Documents/Sec

This was 36.8 on a physical machine against 26.8 on the virtual machine. On this metric, Hyper-V was able to process at approximately 73% of the rate of a physical server. The physical BizTalk Server consistently processed at a higher rate of documents per second than the Hyper-V machine during the test.

Latency

The latency observed on a physical machine was approximately 40% higher (619 milliseconds) compared to the virtual server (438 milliseconds). The physical BizTalk Server consistently displayed latency between 0.5 and 0.7 seconds, whereas the virtual BizTalk Server was between 0.4 and 0.5 seconds. There is generally a trade-off in BizTalk systems between throughput and latency. The higher latency on the physical BizTalk Server is attributable to the fact that it was processing documents at a higher rate per second; higher throughput typically means higher latency.

CPU Load

Processor utilization as represented by the %Processor Time performance monitor counter was comparable for both runs (91.7% in a physical environment vs. 89.9% in a virtual environment). The trend of the CPU utilization is fairly similar throughout the test runs as reported by the performance monitor counter inside the physical and Hyper-V machine respectively.

As detailed previously in Measuring Performance on Hyper-V, the processor utilization counter within a virtual machine is skewed, with the effect that it reports a higher utilization than the virtual processor for the machine was actually under. This can be verified by looking at the Hyper-V HyperVisor Virtual Processor counter for the virtual machine, which is published on the root partition. The actual processor load on the virtual processor during the test was consistently below 90%. The average reported during the duration of the test was 82%, which is 7% lower than reported within the Hyper-V machine performance monitor counter.

Cc768517.note(en-US,BTS.10).gif
This difference occurs because the Virtual Processor processes two types of load; load directly related to the guest operating system and load related to running hypervisor code. The %Guest Run Time counter shown in these results reflects the percentage of time that the virtual processor is executing instructions in the guest operating system.

SQL Load

The physical server placed a marginally higher load on the SQL Server, which measured 18.6% CPU utilization vs. 15.1%. The processor utilization on SQL Server was significantly higher when the physical BizTalk Server was using it, compared to SQL Server processor utilization by the virtual BizTalk Server. With BizTalk Server, the load on SQL Server increases with the number of documents per second processed by BizTalk Server, therefore, this behavior is to be expected.

Machine Configuration

Hyper-V provides functionality that enables multiple processors to be assigned to a virtual machine. On a Windows Server 2003 guest machine, Hyper-V supports assigning up to two virtual processors. As with the previous test, both the physical and virtualized machines were part of the same BizTalk Server group with its databases hosted on the same physical SQL Server Physical_SQL01. The physical and virtual machines were conducted using the following settings:

  • Two processors available to the operating system

  • 1.5 GB of memory allocated to both machines

Results Analysis

The documents-per-second increased to 47.4 on the physical machine from 36.6 when one core was used, which represents an improvement of 30%. An increase from 26.8 to 36 documents per second was observed on the Hyper-V virtual machine, which represents a difference of approximately 34%. Latency was lower again on the Hyper-V virtual machine compared to the physical computer (450 ms vs. 540 ms).

Name Run Number Request-Response Duration Average (ms) Documents/sec BizTalk CPU Utilization Hyper V % Guest Run Time (BizTalk VM instance) SQL\Processor(_Total)\% Processor Time

2-Core Physical BizTalk Server

64

540

47.4

93.5

N\A

28.2

2-Processor (Core) Virtual BizTalk Server

78

450

36

91.4

81.6

21.7

Cc768517.1af08762-b7ca-4e98-8d72-777bf6e2c345(en-US,BTS.10).gif

Cc768517.7df15471-43ca-4640-82f2-f9b80a19cf33(en-US,BTS.10).gif
Show:
© 2015 Microsoft