.gif)
by Joseph Hofstader
Contents
Introduction
Who Are Those Guys?
Problem Space
Solution Space
Don't Build "The Homer"
To Make a Short Story Long...
Resources
Introduction
The role of an architect in software development has come
under attack of late. On software projects, the title Architect is often
ambiguously defined and the value provided by architects is not easily quantifiable.
The perception that architects live in an "ivory tower" disassociated from the
realities of delivering a software solution contributes to some of the
animosity toward those of us with the title.
This article presents a defense of the practice of
architecture in software development. It defines the qualities of an effective
architect and describes the skills required to succeed in this profession. The
article examines widely held perceptions of architects and some of the mistakes
that architects make which contribute to negative perceptions. Ultimately, the
intent is to show the value good architects bring to a software development
effort.
Who Are Those Guys?
In the field of information technology, no title conjures up
more raw emotion than Architect. I have witnessed and been involved in many
debates over the definition of the term. When I introduce myself at meetings,
reactions range from "we're not worthy" to "we do not need an architect"—the
former, although friendly, reflecting the lofty image of architects, and the
latter implying that an architect's knowledge and skills are irrelevant. Both
responses demonstrate a lack of understanding of what architects really do.
At the OOPSLA conference in 2005, I attended a "Birds of a
Feather" (BOF) hosted by Grady Booch. The topic of
the BOF was his then upcoming "Handbook of Software Architecture." One of the
attendees related some negative experiences he had had with architects, both in
IT and in construction. One story was about the architect who drew plans for
his house expansion. The attendee said that he viewed the drawings with
engineering software and the plans were off by a few feet, and that the actual
construction could not and did not follow the architect's specification. He was
making the point, which I have heard echoed by many qualified individuals, that
architects are detached from the reality of delivering tangible results and
that their responsibilities should be relegated to the engineers and builders
who are fully engaged in product development.
That meeting, and many subsequent conversations with others,
led me to wonder what exactly the role of an architect is on a software product
and what the characteristics of good architects are. The most concise
definition I have come up with is: The role of the IT architect is to solve
a problem by defining a system that can be implemented using technology. Good
architects define systems by applying abstract knowledge and proven methods to
a set of technologies with the goal of creating an extendible and maintainable
solution.
From this concise definition, we can extrapolate that good architects draw upon a foundation of knowledge to be
successful in their role. To "solve a problem," the architect must have a good
understanding of the problem domain. To "define a system using technology,"
implies that the architect has technical acumen. "Abstract knowledge" requires
the architect to be able to conceptualize the technical solution. "Proven
methods" assumes an understanding of the patterns used to solve similar
problems. Figure 1 depicts the key skills of an architect.
.jpg)
Figure 1. The key skills
of an architect
The key benefit an architect brings to a project is the
ability to apply those skills in the definition and development of a robust
software solution. An effective architect is part of the continuum of all
phases of a software development project, with skills critical to defining the
problem space, such as domain knowledge and the ability to conceptualize a
software solution, as well as the ability to define the solution space using
appropriate technologies and patterns. The risk of not having an architect
actively involved in product development increases the likelihood that the
product will take too long to develop or be developed incorrectly. Figure 2
illustrates the phases of a development project where the skills of an
architect are applied.
.jpg)
Figure 2. Architect
skills in phases of software development
Describing the architectural skills required for a
successful project is not as straightforward as it may seem. Many areas,
especially technical acumen and patterns, are often debated regarding the level
of expertise necessary for an architect. The following sections, divided by
problem space and solution space, offer an explanation of each of these skill
sets and a rationalization of how these skills make an architect invaluable to
a software development effort.
Problem Space
Defining the problem space and ultimately setting the scope
of a software solution requires an understanding of what will be built, as well
as domain knowledge and a conceptualization of how information will flow
through the solution. As the following sections detail, these skills are
essential to understanding the purpose of a software solution and communicating
the proposed solution to all stakeholders.
Domain Knowledge
The problem domain for a software solution can be horizontal
or vertical. A horizontal domain is applicable across industries, like workflow
automation. Vertical domains are specific to a particular industry, like
telecommunications. Problem domains can be further decomposed into subdomains, or aggregated into larger domains. An example
of a subdomain within a vertical domain is network
activation in telecommunications. An example of the aggregation of a subdomain into a larger horizontal domain is workflow in an
enterprise application integration product.
There are many standards organizations and vertical industry
groups that specify standards and protocols that need to be considered when
defining a software solution. These organizations can be specific to a vertical
industry domain or a horizontal industry domain. The TMForum
is an example of a vertical organization that specifies management protocols
for the telecommunications industry. The W3C specifies standards for the
horizontal World Wide Web domain including technologies like Web services.
The value of domain knowledge is sometimes underestimated by
IT managers. I once worked for a telecommunications company whose IT leadership
wanted to change the organization from being structured around "centers of
excellence" focused on a business domain to being structured with "pools" of
resources based on technical skills without regard to domain knowledge. For
some of the resources assigned to horizontal domains, like Web development,
this model worked well. Many products require Web interfaces and the skills
were applicable across verticals. Where the "resource pool" structure failed
was in industry specific subverticals, like network
activation. Understanding how to provision and activate services requires
detailed knowledge of the provisioning process as well as the interfaces,
protocols and standards of all network devices that comprise the
telecommunications services.
Deep domain knowledge often involves a steep learning curve.
If the staff on every project is required to learn the intricacies of the
domain for every release of the project, productivity is significantly reduced.
Assuming features are sufficiently decomposed so that the amount of time to
deliver the feature is constant, then productivity suffers proportional to the
amount of time spent acquiring domain knowledge. Conversely, maintaining a
"center of excellence" around each vertical domain supported by a business can
also be an expensive proposition. Development work is seldom evenly balanced
throughout a given timeframe and keeping a fixed number of resources assigned
to a project that is not actively engaged in delivering solution features can
drain productivity on other development efforts.
A balance that is common in technology companies is having an
architect be a domain expert and a pool of resources available to different
projects. This strategy increases productivity by minimizing the amount of time
obtaining domain knowledge within a vertical. It also allows some preliminary
research to be done prior to engaging development staff, helping to ensure that
the development is consistently productive. This approach provides the company
the added benefit of a flexible staffing model, in that development staff can
be augmented with contractors without having valuable domain knowledge walk out
the door at the end of the contract.
Conceptual Thinking
One of the main responsibilities of an architect is to
communicate the solution to technical and nontechnical stakeholders. The
ability to conceptualize a software solution is essential to communicating a
software solution to stakeholders who care about delivery of functional
requirements, and may not know or care about technical details. Defining a
conceptual architecture prior to commencing the development of a software
solution helps facilitate the feedback loop needed to define the scope of a
product and can help determine an initial level of effort and cost estimate for
a product.
A conceptual model is the artifact most associated with
software architecture. The conceptual model typically shows the components of a
software system that will fulfill the functional requirements and where they
apply in a software solution (user interface, domain layer, and so forth). The
conceptual model is often accompanied by a number of diagrams showing the flow
of information through the proposed solution. In the case where the software
system consists of components from other products or solutions, the conceptual
architecture often contains the protocols that will be used to access different
parts of the solution.
Applying the correct level of granularity is the main
challenge in defining the conceptual model. The conceptual architecture should
not contain any references to a particular platform or technology, other than
protocols used to access subsystems. I once took over as architect of a project
that was stalled in "analysis paralysis" for over a year. As I was reviewing
documents to get up to speed, I noticed that the conceptual architecture
consisted of a number of boxes containing the names of technologies with no
reference to system functionality. After reviewing that document I understood
why the system could not be developed: There was no mention of the required
features, making it hard to understand what needed to be developed.
Solution Space
It is in the area of defining the solution space that
opposition to architecture is most obvious. Developers will usually accept the
architect working in the problem space, but may be resistant to having the
architect define the solution space. In many instances, developers have valid
arguments about architects meddling in the solution space, especially if the
architects have not kept their technical knowledge up-to-date.
A colleague of mine illustrates the attitudes developers have
toward architects when he says "architects talk in bubbles and developers talk
in code" (see Figure 3). The idea that code is the only artifact that matters in a
software development project is so prevalent that it is one of the values
listed in the Agile Manifesto: "We have come to value […] working software over
comprehensive documentation." A good architect understands that code is
undeniably the most critical part of a software solution, and many of the
modern development environments now produce code from "bubbles," including
tools that support Model Driven Architecture (MDA) and Domain Specific
Languages (DSL).
.jpg)
Figure 3. An architect
thinking in bubbles
That being said, a good architect also understands that a
software solution consists of more than the functional requirements implemented
in custom code (see Figure 4)—for example, development platforms, frameworks, and
infrastructure technologies. Consideration also needs to be given to the
nonfunctional requirements of a software solution, like: deployment, security,
scalability, and performance. Neglecting the nonfunctional requirements
increases the likelihood that the system will fail.
.jpg)
Figure 4. A developer
seeing part of the picture
Another critical piece of solution space knowledge is the
patterns used to implement a software solution. Patterns allow a software
solution to be developed with thought toward extendibility and reuse, which are
critical to reducing the total cost of ownership of a software
solution—especially in later phases of development or product maintenance.
Technical Acumen
Technology has been rapidly evolving for as long as anybody
can remember. I've implemented countless technologies over the last dozen
years, including technologies for process distribution, user experience
programming languages, enterprise application integration, object-relational
mapping, virtualization, and data persistence.
Understanding how a technology works is not enough to
develop a robust software solution—understanding where the technology is
applicable within a solution is essential to the development of a quality
product. On several occasions, I have reviewed architecture documentation
consisting of little more than a number of boxes on a Visio diagram each
representing a technology, with no mention of how the technology was intended
to be used or any reference to the functional requirements to be fulfilled by
the technology. Such practices give architects a bad name.
It is impossible for anybody to understand every detail of
every technology. But an architect should understand, at a minimum, the intent
and applicability of any technology prior to requiring its usage in the context
of a software solution. The architect should also map the technology to the
functional requirements, or to the applicable part of the conceptual
architecture. Too often, I encounter the bad practice in which an architect
makes a technical edict without explaining the intended application of the
technology. To be honest, I made the same mistake on occasion earlier in my own
career.
Architects sometimes allow their passion for technology to
overshadow the problems that they need to solve. The art of choosing technology
for a software solution is finding the minimum amount of technology required to
meet the system requirements, both functional and nonfunctional. Delivering a
software product that meets all quality standards, such as performance and
scalability, will require a good amount of technical knowledge, and it is the
job of the architect to define the platform for development and deployment.
With all of the advances in technology, keeping abreast of
the latest trends can be daunting, but it is critical for an architect. One
company that I worked with had a problem with the performance of a
client/server reporting application. The application had been extended for
years without a technology update. The architect responsible for the solution
was adamant that building an object-layer over the relational model would solve
the problems with his application. The proposed solution would have been status
quo a decade ago, but database technologies have advanced greatly over the last
decade and now contain optimized reporting services as part of the platform.
The solution proposed by the architect would have increased the total cost of
ownership of the solution (development, maintenance, licensing) and most likely
would have adversely affected performance. Luckily, the architect was open to
learning about the newer technologies and the product upgrade took advantage of
the capabilities in the newer database platform.
Patterns
One critical skill possessed by all great architects is an
understanding of how to leverage patterns in the definition of a software
solution. Patterns have been a mainstay of software development for over two
decades. Through the seminal Design Patterns by Gamma et al., the Pattern
Oriented Software Architecture (POSA) series of books, various publications
from industry luminaries, and the work of organizations like the Hillside
Group, patterns have become a key measure of how software is judged. When
reading about a new technology or a framework, I often try to list the
different patterns that were used in the solution in order to assess the quality
of the design.
The mainstream usage of patterns in software development has
made it a critical skill for architects. Learning how to leverage patterns
takes time and effort. Some architects, like me, have had the good fortune of
working with experts in patterns who provide mentoring in the discipline of
architecture and design; others are left to acquire these skills on their own.
With or without mentors, developing a proficiency in patterns does not happen
by accident and requires dedication. There is a large learning curve in
acquiring the patterns vocabulary and an even larger learning curve in
understanding where patterns can be applied in a software solution. The effort
put into learning patterns is paid back tenfold, giving an architect the necessary
skills to intelligently discuss design with their peers and to create
extendable software systems.
One of the main motivations to leveraging patterns in a software
solution is the ability to design frameworks that allow the solution to be
easily extended. A software solution uses frameworks to define the solution's
architecture in the context of the problem domain. Good software frameworks are
usually defined using general-purpose patterns that are reusable over a number
of domains. One of the main drivers behind domain-specific languages is to
increase developer productivity by providing graphical tools to customize
general frameworks. As mentioned previously, patterns are an essential component of
defining frameworks.
I can provide many examples in which understanding patterns
has increased productivity, but the best example was in the project I mentioned
earlier which I inherited after it was stalled for over a year. Having designed
similar solutions in the past, I understood the patterns that were necessary to
build an extendable domain model. While equipment was being ordered for the lab
and development staff was being interviewed, I was able to define the domain
model and frameworks with patterns that I had used on similar efforts in the
past, accelerating the development phase. The initial frameworks I developed
were a key factor in being able to deliver the product in a short time frame.
Don't Build "The Homer"
With all of the skills possessed by good architects, it is
often challenging to narrow the numerous options to be used in a software
solution. Between the standards defined for specific domains, the alternatives
for conceptualizing a software system, the plethora of technological options,
and the numerous patterns to promote extendibility and reuse, it is easy to
over-architect a solution that bears a greater resemblance to alphabet soup than
to a robust software system (see Figure 5).
.jpg)
Figure 5. An architect
making alphabet soup
The first step in simplifying a software system is to
understand that no matter how fun it is to try out new techniques and
technologies, they must be applied in the context of a system requirement. It
was no coincidence that the first system I designed after reading Design
Patterns contained almost all of the patterns defined in that book.
Many years later, after a few painful experiences, I have
learned that a minimalist approach, using the least amount of technology to
fulfill the requirements, always yields the best results.
There is an episode of The Simpsons I often think about when
defining a software solution. In the episode, Homer, the dim-witted father, was
asked to design a car for an auto manufacturer. The design for Homer's car was
full of nonessentials like multiple horns, cup holders, and shag
carpeting—without thought to the overall cost or customer appeal of the car.
The prototype, appropriately called "The Homer," was so costly and unappealing
that the product bankrupted the company. Whenever I design a solution, or
review the design of a solution, I frequently stop and ask myself if the
resulting product is beginning to resemble "The Homer." If the answer is yes, I
sharply refocus on the essential functionality and technology required to
deliver the software system.
To Make a Short Story Long...
A couple of months ago, while waiting to make a presentation
at a customer event, I was introduced to a developer attending the event as an
architect from Microsoft. Expecting one of the usual greetings like "pleased to
meet you," I was surprised that the first thing he said was "I think that architects are obsolete." Not wanting to be
roped into an argument before my presentation, I responded with a grin and
said, "I hope not."
Upon reflection, I have wondered why he felt he needed to
make that statement from the outset: Was he having a bad day? Did he have a bad
experience with an architect? Perhaps he is unaware of the contributions of
architects in software development. Maybe his solution was so well architected
that the development staff only needed to consider coding and testing
functional requirements without needing to understand the considerations
necessary to create a robust software solution.
A poorly architected project languishes without results, or
is delivered with poor results. On the other hand, a successful software
solution is well architected, by someone with domain knowledge, the ability to
think conceptually with technical acumen, and the ability to leverage patterns
to solve problems. Industry titles for this role have come to include
architect, technical lead, and a number of others, but the importance of the
role is clear. We should stop wasting time arguing the virtues of the role and
continue solving problems with technology.
Resources
The Agile Manifesto
http://agilemanifesto.org/
Design Patterns: Elements of Reusable Object-Oriented
Software, Erich Gamma et al. (Addison-Wesley Professional, November 10,
1994, ISBN: 978-0201633610)
Handbook of Software Architecture
http://www.booch.com/architecture/index.jsp.
The Hillside Group
http://hillside.net/
The Homer Wikipedia Entry
http://en.wikipedia.org/wiki/Oh_Brother,_Where_Art_Thou
"Using Patterns to Define a
Software Solution," Joseph Hofstader, November, 2006.
http://msdn2.microsoft.com/en-us/library/bb190165.aspx
"Using Patterns for Rapid Development," Joseph Hofstader, February, 2007.
http://www.iasahome.org/c/portal/layout?p_l_id=PUB.1.366
About the author
Joseph Hofstader is an
Architect for Microsoft's Developer and Platform Evangelism organization. He is
also an adjunct professor in the Department of Information Technology and Electronic
Commerce at the University of Denver. Prior to joining Microsoft, he was a
Distinguished Member of Technical Staff at Avaya. Joe has spent his career
architecting, designing, and developing solutions in the telecommunications
industry. He is happy to receive feedback at joe.architect@live.com.
This article
was published in the Architecture Journal, a print and online publication
produced by Microsoft. For more articles from this publication, please visit
the Architecture Journal Web site.