Jacek Czerwonka Tester Spotlight

Before joining Microsoft, Jacek Czerwonka worked as a developer for a software company whose expertise was in design and development of distributed backoffice applications. Formal testing processes there were not well established at that time, so he started looking at the testing theory and best practices and saw that this field is still very young and with a lot of room to grow. In 2001, Jacek joined Microsoft's Windows Sustained Engineering team and testing became his primary job. We asked Jacek about his current work, his thoughts on the biggest challenges in testing, and also asked him to tell us how he got involved in PICT and combinatorial testing.

Tester Center: What are you doing now?
Jacek: After few years of working as a tester and later a test lead on the servicing team, I got involved in development of code and test analysis tools. Effective re-testing is of primary concern in software maintenance and the tools we're developing expose information on risks of fixes and possible mitigations with a goal of assisting developers and testers in maintaining Windows. Currently, I'm spending a few months in Bangalore working with Microsoft Research India on the second iteration of the tools.

Tester Center: What have been your biggest challenges in testing?
Jacek: Even though the rate of changes after the release of any software is much lower than when software is under active development, it brings its own set of challenges. Time allowed and resources available for re-testing are much smaller than in pre-release phase. Risks, on the other hand, are very high as customers have low tolerance for mistakes. With such constraints, the primary challenges for us are around targeting testing in a way that allows us to re-test with very high bug finding ratio and without executing all of the tests we have available (millions of test cases). Only this way we can maintain quick turnaround time and high quality releases.

Another challenge is scaling up test execution without exploding costs. In Windows, we have to be prepared for 10+ years of maintenance after the release. At any point in time, we are servicing multiple releases of the server and client versions of Windows with all their different flavors. Creating and maintaining highly reliable (no false positives) servicing test suites which require almost no manual intervention is of utmost importance for our ability to service the code.

Tester Center: How did you get involved in combinatorial testing and PICT?
Jacek: I read a couple of papers by Bell Labs on pairwise testing and, inspired by their findings, I implemented my first tool for creating n-way test suites. In the process, I learned about the work of David Erb, at the time an architect at Microsoft -- who developed the first version of PICT. Each of our tools had advantages: David's tool had a very fast generation core while mine had features that made it much easier to use, like support for constraints. (Interestingly, they were almost exactly the same in the number of tests they would generate for the same test domain.) In 2002, David and I decided to merge the code bases and I spent a couple of weekends re-writing the code for what would become PICT 3.0. There've been incremental revisions since then, which added a number of interesting new features.