Proof-of-Concept Design

Odysseas Pentakalos, Ph.D.

January 2008

Summary: This article shows how the development of a proof-of-concept can bridge the gap very effectively between how the software product is envisioned during requirements definition and how it is ultimately delivered to the customer. (6 printed pages)

Contents

Introduction
Building Expectations
Controlling Expectations
Conceptual Versus Deliverable
Conclusion
Critical-Thinking Questions
Sources
Glossary
 

Introduction

A few years ago, I worked on a multiyear project in which we were tasked with the staged development of a custom-made enterprise application for a large organization. Each stage of the project lasted 8 to 12 months and involved the development of a distinct component of the overall solution. Development of the deliverables of each stage was treated as a separate software-development project in which we went through the entire software-development life cycle (SDLC) in producing the deliverables. Over the years, the customer had settled on its preferred software-development process internally, which resembled the waterfall model [Wikipedia] and which the client wanted us to use. Despite our initial resistance to it, we were ultimately forced to follow it.

Building Expectations

The goal of the first stage of development was fairly limited in scope, compared with the goals of later stages of the project. The single deliverable consisted of the development of a simple prototype that would help the customer team illustrate to their end users the goals of the project. Due in part to everyone’s excitement at being involved in a new project and in part to the limited scope of the deliverable, we were able to complete the fully functional deliverable early. The customer was very pleased with the results and developed considerable respect in our abilities, which contributed to our team developing a high-level of confidence.

The next stage involved considerably more functionality. The requirements were not as clear, and the time that was allotted was not much longer than what we had available during the first stage of development (despite the much greater scope of the deliverables in this stage). Of course, with our recently acquired confidence, none of those issues was much cause for concern at the time. After a lot of work, late nights, and stress, we managed to complete development.

Next, we prepared for the meeting with the customer. Given our experience with the first stage of the project, we expected that the customer would be awed with our results and that, after offering considerable amounts of praise, they would return to their office completely satisfied with yet another successful delivery. We expected also that they, in turn, would expect us to deliver a fully functional system that did exactly what we all had originally envisioned. From the beginning of the meeting, however, it quickly became clear that what they had envisioned and what we actually had delivered were considerably different concepts. The situation became increasingly tense, as we came to realize that feature after feature that we had provided totally failed (in their opinion) their expectations.

Controlling Expectations

After that meeting, having barely survived a cancellation of the contract, and before moving on to begin the requirements-definition cycle for the next stage of the project, we jointly decided that for the next stage of development we would provide the customer with a proof-of-concept (POC) system at two checkpoints before the overall project would be due. This would allow both our customer and us to confirm that the solution that we were developing was in-line with their expectations; and, if not, to allow us to get back on track before it was too late.

The decision to incorporate the development of POC systems for the rest of the stages of that project was a considerable factor towards successful completion of the overall project. Through that experience, we learned a number of lessons with regard to the value of a POC system.

In [DeGrace, et al. 1990], the authors list four reasons for the failure of the waterfall approach for software development [Sutherland 2004]:

· Requirements are not fully understood before the project begins.

· Users know what they want only after they see an initial version of the software.

· Requirements change often during the software-construction process.

· New tools and technologies make implementation strategies unpredictable.

In our experience with the project that was described earlier, the development of a POC system provided a cure for three of these issues. First of all, by developing a POC for the customer, we were forced to understand fully the requirements early on. The understanding was much deeper than what one normally receives at the early stages of a new project by simply reading through the requirements, or while incorporating them into the architecture of the system.

The failure that we experienced on that project after the second stage of development was in part due to the second issue that was listed previously. The users were disappointed with our delivery, not only because we misinterpreted the requirements, but also because they did not really know what they wanted until they had seen the deliverable—which, unfortunately, was not what they had in mind. After the customer got a chance to review the POC—which we provided them in the later stages of the project—they got a better idea of what they wanted the final deliverable to look and behave like. Having the POC system also gave us the opportunity to communicate to the user the look and feel of the final product much more vividly than through the use of design documents and design reviews. Seeing the POC allowed them, on the one hand, to adjust their requirements to match exactly what they wanted and, on the other hand, to better define their expectations for the final deliverable. As a result of these adjustments, the customer was much happier with the end result.

At times during the development, we chose to incorporate new technologies; at other times, we were forced to do so. Many of the details that were needed during the design and development of software become known only during the implementation stages [Parnas 1986]. It has been shown consistently that design mistakes that are found early in the software-development life cycle are cheaper to fix than when they are detected later down the road [Eeles 2006]. The POC system provided us with lots of feedback and information of which we were not aware, and allowed us to adjust our design decisions well before the cost of backtracking became too high. The development of a POC system gave us the opportunity to understand and evaluate how to best incorporate those technologies into our design, without having to worry about the complexities of developing the full scope of the system.

As soon as we had a good handle on the capabilities and idiosyncrasies of each new technology (by observing its behavior and operation during the POC development), we were in much better shape to incorporate it into the final product. The risks that we undertook when using the new technologies had been reduced simply due to the fact that, after testing the technology within the scope of the POC system, they were no longer new to our team.

Conceptual Versus Deliverable

It is important to keep in mind that a POC is just a prototype and does not represent the deliverable. POC systems are usually developed quickly and without a lot of testing, so that they do not make good candidates for early versions of the final deliverable. In cases in which the deliverable includes a user interface, the POC is a façade that illustrates the look and feel of the interface; but there is no functionality behind the façade, much like the houses that are used in Hollywood movie studios. In cases in which the product is an application programming interface (API), the POC illustrates the methods and functionality that the API will provide; but the implementations of the methods are simply stubs that will not perform real work.

When the POC has served its purpose, it is best to throw it away and start with the development of the deliverable. Developers must resist the urge to start development of the final deliverable by enhancing the existing POC. At the same time, the development team must communicate to the customer that the POC is a prototype that looks like the desired system, but, in reality, is just smoke and mirrors. Otherwise, one runs the risk of raising expectations to the point at which the customer will expect the delivery of the rest of the system at the same pace that the prototype was developed.

The features that are implemented as part of the POC should have the key features of the project—especially, the parts of the system that have many unknowns or represent increased risk. At the same time, components of the system that are repetitive implementations of a given concept should be excluded. Implementation of a single instance of a given concept in the POC will provide all of the information that is needed to match a customer’s expectations successfully.

The decision with regard to whether to incorporate one or more POC systems into the schedule also depends on the software-development process that is being used. Despite much criticism—even including some by its own founder [Parnas 1986]—the waterfall model is still quite popular. Because the waterfall model and some of its close relatives do not incorporate iterations that allow for the revision of the requirements and design decisions as development progresses, it is especially important to include the development of a POC system. Other development processes, such as the Spiral [Boehm 1985] or Scrum [Sutherland 2004], prescribe the development of a prototype or early versions of the deliverable that are to be refined over time. The processes can preclude the need for—or the value derived from—the explicit development of a POC system.

Conclusion

As it became very clear to us, when engaging in a new project, it is imperative that the development of one or more POC systems be considered before one settles on the architecture and design of the final deliverable. Development of a POC system provided us with many benefits including, among others, the:

· Very clear understanding of requirements.

· Understanding of the capabilities and limitations of new technologies.

· Ability to assess design decisions early in the process.

· Ability for the customer to visualize early on the look-and-feel of the solution.

· Reduction in the overall risk of project failure.

We had to be careful of the features that we incorporated into our POC system. We had to be very clear to the customer, as well as to the development team, that the result was a POC system and not an early version of the final deliverable. Although the use of a POC provided benefits, we had to be careful about how we incorporated it into the existing software-development process.

Critical-Thinking Questions

· What software-development process are you using? How does the possibility of a POC system fit in with it?

· What is the nature of the POC system for the project at hand? Are you developing an application with a user interface, an API that will be used by third-party developers, or a product that is defined by your marketing team?

· What key features must go into the POC? What features can safely be left out? What aspects of the design must be evaluated and tested, to ensure that the correct decisions have been made?

· What new technologies are being incorporated into your project? How can they be used in the POC system?

Sources

· [Boehm, 1985] Boehm, Barry W. "A Spiral Model of Software Development and Enhancement." Proceedings of an International Workshop on Software Process and Software Environments, Coto de Caza, Trabuco Canyon, CA. March 27-29, 1985.

· [DeGrace, et al. 1990] DeGrace, Peter, and Leslie Hulet Stahl. Wicked Problems, Righteous Solutions: A Catalogue of Modern Software Engineering Paradigms. Englewood Cliffs, NJ: Yourdon Press, 1990.

· [Eeles 2006] Eeles, Peter. "The Process of Software Architecting." developerWorks. April 15, 2006.

· [Parnas 1986] Parnas, David L., and Paul C. Clements. "A Rational Design Process: How and Why to Fake It."IEEE Transactions on Software Engineering. February 1986.

· [Sutherland 2004] Sutherland, Jeff. "Agile Development: Lessons Learned from the First Scrum." Cutter Agile Project Management Advisory Service: Executive Update, 2004, 5(20): pp. 1-4.

· [Wikipedia] Various. "Waterfall model." Wikipedia, the Free Encyclopedia. December 26, 2007.

Glossary

Proof-of-concept—A short and/or incomplete realization of a certain method or idea to demonstrate its feasibility, or a demonstration in principle whose purpose is to verify that some concept or theory is probably capable of exploitation in a useful manner [Wikipedia].

Software-development process—A structure that is imposed on the development of a software product [Wikipedia].

About the author

Odysseas Pentakalos is Chief Technology Officer of SYSNET International, Inc., where he focuses on providing his clients consulting services with the architecture, design, and development of large distributed systems that use Java, XML, and J2EE technologies. He holds a Ph.D. in Computer Science from the University of Maryland. In addition, Odysseas has published dozens of papers in conference proceedings and journals, is a frequent speaker at industry conferences, and coauthored Windows 2000 Performance Guide (Sebastopol, CA: O'Reilly, 2002).

This article was published in Skyscrapr, an online resource provided by Microsoft. To learn more about architecture and the architectural perspective, please visit skyscrapr.net.