Best Practices for Testing Engine Performance

The information in this topic refers to the tests explained in Test Scenarios for Measuring MST of the Engine.

The following guidelines should be followed when testing BizTalk Messaging Engine performance:

Know your load behavior profile

  • As the three load tests have illustrated, it is critical to understand the profile of your load in terms of messages processed over time. The better this is understood, the more accurately you can test and adjust your system capacity. If all you know is your peak throughput requirement, then the most conservative approach is to size your system so that your maximum sustainable throughput is the same or higher than your peak load. However, if you know that you have predictable peaks and valleys in your load, you can better optimize your system to recover between peaks, resulting in a smaller, less expensive overall deployment.

Test performance early

  • Run performance tests on your system that simulates the expected load profile as early as possible in your development cycle. After investing significant effort to design and test the functionality of your solution, avoid waiting until the last minute to test performance on production hardware. If you have to change anything in your design or architecture to achieve your performance goals, knowing the performance results for your system early will give you time to adjust and test again.

Emulate your expected load profile when testing performance

  • There are two primary components to this:

    1. Emulate the load profile over time.

    2. Run the test long enough to evaluate if it is sustainable.

  • If your cycles are daily, you should plan to run performance tests for at least one day to validate sustainability. The longer that you run the tests, the better.

Emulate the production configuration

  • For example, the number and type of ports, the host and host instance configuration, database configuration, and adapter setup. Don’t assume that configuration changes will not significantly impact performance.

Use real messages

  • Message sizes and message complexity affect performance, so be sure and test with the same message schemas and instances that you plan to see in production.

Emulate your normal operations during performance tests

  • Though the load tests did not include them, standard operations activities such as periodic database queries, backups, and purging will affect your sustainable throughput, so make sure that you perform these tasks during your performance and capacity test runs. This includes both DTA and BAM tracking, if you plan to use them in production.

The speed of the I/O subsystem for the MessageBox database is a key success factor

  • The load tests performed made use of a fast Storage Area Network for the MessageBox database files dedicated to the build-out. Even in this case, the cleanup jobs were able to drive the idle time to near zero for the SQL data file. Because the I/O subsystem is frequently a bottleneck in production systems, if possible, optimize SQL I/O by placing the database data file(s) and log file(s) on separate physical drives.

Make sure the SQL Agent is running on all MessageBox database servers

  • Clearly, the cleanup jobs will never run if the SQL Agent is not running, so make sure these are running.

Spool depth is a key indicator

  • Regardless of other indicators, this measure gives you a quick and easy way to assess if your system is overdriven or not.