Modeling for Software Development and Operations
Software development modeling
Summary: We use many models to design, develop, deploy, and manage technology solutions. A partial list of these models might include business cases, use case diagrams, entity relationship models, object models, code, test suites, deployment plans, logical data center models, and exception management plans. Software lifecycle methodologies have traditionally struggled to automate the production of "downstream" models from "upstream" models, or even to keep the different system models synchronized. This brief paper describes the modeling space and outlines current efforts to use models more effectively throughout the technology lifecycle. (4 printed pages)
Models are typically "scaled down" representations of real-world objects and systems. The reduction in scale may include size, detail, and aspect (a model that focuses on cost or durability while ignoring unrelated qualities of the system, for example). In the development of information technology solutions, models are used to manage complexity and to communicate system requirements between business stakeholders, solution and system architects, developers, and operations personnel.
Think about the many models used in engineering projects. Use-case models may be used to express the high-level functional requirements of the system and the roles that need to be supported. Risk models may define the prioritization of work by focusing on risk reduction as early as possible in the project timeline. Entity relationship models capture the basic information to be managed by the solution, and suggest appropriate factoring across tables, objects, and services. Logical system models form the basis of communication between development and operations. Prototypes permit early validation of requirements and solution strategies. The list goes on.
All of these models permit project participants to contribute their unique expertise to the development and management of the resulting system. All of these models help reduce both project cost and project risk by heading off misconceptions and oversights as early in the process as possible. Modeling tools and frameworks encourage the alignment of business and information technology groups by providing higher levels of traceability, visibility, and accountability across all levels of the solution design.
Microsoft is focused on making models more useful across the entire software lifecycle—from conception through development to operations and management. Historically, the only model that has maintained currency across the lifecycle has been the cryptic, detailed, and sadly incomplete model described by compilable code: the "source" of the system. Disharmony between system models causes frequent errors of perception and communication both within and across organizational disciplines, resulting in project and system failures.
Microsoft's goal is to permit all stakeholders—technical and non-technical, specialist and generalist—to have access to current, accurate, and appropriate representations of the organization's systems, expressed in languages with which each is familiar. Microsoft's intent is to empower all stakeholders—from the business analyst to the data architect to the security specialist to the network engineer—to contribute their expertise to the evolution of a solution, with a minimum of information loss.
Some models attempt to represent a complete system at varying levels of abstraction. Other models focus on specific aspects of a system, such as how security is maintained or performance tracked from one end of the system to the other. Some models relate to the creation and evolution of the solution, while others predict and analyze its behavior in operation.
Use case diagrams are probably the simplest "complete" model in software engineering: the collection of use cases for a system establishes the functional expectations, identifies roles for the users of the system, and suggests operational requirements. Use cases may be evolved into business process models, yielding a detailed requirements model, a data model, an object model, and ultimately a compilable model. This cascade of models suggests the familiar "waterfall" conception of software engineering.
A better conception of system modeling would be a series of locks. Systematic process supported by tooling—not the whims of nature—should manage the handoff from one model to the next. That handoff should be calm and predictable. Most importantly, that handoff should be bi-directional. Use cases should be transformable into business process models, and business process models should be transformable into use cases (although you cannot perform a full-fidelity roundtrip from detailed to abstract to detailed because of information loss). Ideally, a stepwise flow of mappings should permit synchronization between all of the models describing a system.
Support for such a collection of mapped models will lead to model-driven development (MDD). In MDD, models become first-class artifacts in the development process. When a systems architect models allowable and forbidden message paths across the security domains in the enterprise data center, that model instructs and constrains the service model, likely affecting service factoring. When a data architect models data access methods, tools generate proxy code and language bindings. MDD is not a revolutionary idea; examples of successful implementations of model synchronization include multi-view data modeling tools—where iconographic models, entity-based models, and relational schemas can all be kept synchronized—and the designers in integrated development environments which synchronize visual and code models of systems under development. We can leverage the lessons of those successes to greatly broaden the scope of MDD.
One of the challenges is to develop modeling tools that can be used successfully by the many stakeholders who contribute to the system lifecycle. The obstacle here is often language: a security analyst needs to be able to express requirements using the vocabulary and syntax of security. Ideally, an iconographic representation of that language will support "drawing" the specifications with intuitive symbols and connectors. Such tools already support disciplines such as database design, user-interface design, and object modeling. Such domain specific languages (DSLs) can be readily developed to empower other stakeholders, from the corporate compliance officer to the financial analyst.
In addition to DSLs and modeling tools, we must rigorously define the relationships between the models. What is the complete set of models we support? What is the model of record for each aspect of the system? How is each model mapped into each related model? How are manual changes to a model synchronized across the rest of the system models? Development environments must incorporate this "meta-model" and enforce the relationships between the models, especially constraining modifications to the model of record for each aspect.
Effective model-driven development can greatly increase the level of automation in the software lifecycle. Unambiguous models produced by domain experts will greatly reduce information loss in the requirements gathering process. Industry-specific models will reduce the risk of inconsistent or improper implementations, improving interoperability across diverse platforms. Synchronization of development and operational models will head off problems at deployment time. Adherence to a modeling framework will protect against "overlooked" requirements. Automated transformations between models will reduce coding errors and development time. Maintaining the currency of all models will permit timely informed feedback from all stakeholders, heading off the disastrous costs of projects gone awry.
Microsoft is aggressively pursuing this goal both in its Dynamic Systems Initiative (DSI) and in Visual Studio Team System through an effort called the Software Factories Initiative.
DSI uses the System Definition Model (SDM)—a model for describing distributed systems and the platforms on which they operate—to effect synchronization between development and operations. The SDM can be used to model software components in terms of the resources they require to function (CPU cycles for example), and their dependency on other components and services. Likewise, the SDM can be used to model logical machine types and security zones in the data center in terms of the resources they can provide to the software components. With the SDM, parallel models of solutions and hosting environments can thus be built, permitting effective simulation, validation of design, planning, and dynamic allocation of resources once the solution is deployed and operating. The SDM is supported in Visual Studio .NET 2005 modeling tools, as well as future versions of Microsoft Operations Manager (MOM) and Microsoft Systems Management Server (SMS).
Software Factories promote the use of DSLs and domain-specific frameworks to achieve increased automation of the development process. The goal is to permit the Microsoft integrated development environment, Visual Studio .NET, to be configured into a "factory" for creating specific types of software components and solutions. For example, a factory for securities-trading applications would incorporate models, patterns, and constraints appropriate to the structure of financial service organizations, accounting practices, and regulatory oversight of trading practices. For an introduction to the concepts of Software Factories, refer to The Case for Software Factories by Jack Greenfield. For a more complete exposition, read Software Factories by Jack Greenfield and Keith Short, et al, from Wiley Publishing.
Today, most system models have very short useful lives, quickly becoming "pretty pictures." Without unambiguous mapping and round-tripping between models, non-code representations of systems immediately become unreliable representations of what the system once might have been. With the right end-to-end modeling tools and frameworks, we can make software development, deployment, and management more consistent, more accountable, and better aligned with business goals.