Architecture in Turbulent Times

Mike Walker
Microsoft Corp

June, 2009

Summary: This article aims to show architects a way to understand key forces on the business of IT, what they can do to add value, and the key areas of focus and technologies that will help them deliver value back to the business.

Contents

Introduction
Shifting Demands for Architects
How Architects Can Add Value
Methods for Maximizing Architecture Decisions
Four Key Architectural Imperatives
Conclusion

Introduction

The current economic condition in which we reside (as of this writing) has given rise to particularly challenging times for information- technology (IT) professionals.Events in the financial sectors and other corresponding industry sectors have had a substantial impact on technology products and services. Although there is much debate by economists on the length, depth, and impact of the economic crisis, we can safely assume that it will not go away in the foreseeable future (see Forrester; October 15, 2008). Its duration is enough to affect business and IT priorities significantly for the next fiscal year or two.

Despite the economic slump, businesses that continue to invest and innovate will have significant competitive advantage both during and after it. Furthermore, organizations that look past the immediate challenges and toward the opportunities that the economic condition presents will have longevity and sustainability well after it. A great example of this is growth by acquisitions. Making key technology assessments and acquisitions can change the position of a company in the market, reduce risk, and increase stockholder value.

IT will be at the center of this innovation; it isa key enabler for companies. IT decision makers such as architects will find themselves in high demand, and their expertise is one that will beused to its full extent. An architect poses the technical know-how to make complex and holistic decisions that affect millions of dollars for an organization.

This article will focus on how the economy affects both architecture in the enterprise and the architect who is a technical decision maker in many companies. By exploring the economic condition along with forces that are imposed on companies, key technology focus areas will emerge. Those technology areas will lead tospecific technologies that will satisfy business demands.

Shifting Demands for Architects

With this new economic condition, companies will start to realign their IT decision makers in accordance with their priorities as a business. Creating alignment with the business and architects is imperative. The architect is at the center of most major IT decisions, by either making the decision or being an advisor to the decisions in question. This realignment will naturally shift the priorities for architects. The exciting, new, strategic technology projects will fade away, and the more tactical projects will prevail.

This can be seen with the current trends and activities in the market, such as:

• Doing more with less. Large corporations have stated that their IT budgets will be cut by as much as 50 percent or more, but that they will continue to have the same service-level agreements (SLA) as they did in the past.
• Trimming existing project costs. In the same vein as "doing more with less," trimming project costs are immediate and tactical activities that will determine the course of specific IT and business directions. Specifically, big service-oriented architecture (SOA) projects will become more pragmatic and actionable, instead of ambitious and multiyear initiatives.
• Mergers and acquisitions (M&A). As market conditions become more climactic, industries will consolidate. IT systems have proliferated through every aspect of the business. Typically, companies have various implementations of similar process and technologies; nevertheless, they are different. More than ever, architects are needed who can understand and provide insight into technologies in the M&A decision-making process.
• Revitalizing the skills base. People are an asset; they represent the whole of a company. IT decision makers have an opportunity to revitalize their leadership and technical acumen in these tough times to make better decisions, grow the business, and take advantage of the economic crisis as a time of innovation.
• New approaches to outsourcing. While leveraging tried-and-true methods, there are new enabling technologies that will provide value to companies. These include Platform as a Service (PaaS) and Software as a Service (SaaS) vendors. As an example, Microsoft provides a wide variety of services -- from its Azure cloud-services infrastructure, with hosted data storage, workflow, and application development, to SaaS solutions such as Dynamics CRM Live.

For architects, business as usual will change; they will have to leverage their diverse skill set to address the demands of the industry and their company. The skills that architects will leverage are the following:

• Motivation and inspiration -- Sometimes, complex IT decisions that have elusive return on value require additional persuasion for the enterprise to buy in. Architects will leverage this skill to rally the enterprise for the right causes.
• Negotiation -- There will be times at the decision-making table when an architect must negotiate to get things accomplished. Most architects are individual contributors and do not have organizational power, so an ability to negotiate is key.
• Critical thinking -- Being able to think quickly and on one's toes often is required for architects, especially in times when making the right decision is critical for one's business.
• Problem solving -- Architects will need to find new and innovative approaches to solving traditional problems. With various forces on their business, the problems of the past have changed and will require quantitative and qualitative ways to evaluate and solve problems.
• Big thinking -- Even more so now, IT personnel will need to look more holistically, given pressures on cost control and return-on-investment (ROI) needs. Avoiding tunnel vision and being able to look at a problem from multiple angles to test one's own rationale represent a skill set that architects possess and will be able to demonstrate to the rest of the IT organization.
• Business savvy -- Having a deep understanding of the business is key to making the right IT decisions; to do so, architects will use their business savvy to communicate and learn from their business leaders and subject-matter experts (SMEs). Understanding the industry in which one works is essential; it helps architects understand how the technology decisions that are made affect the business objectives and how it does within the industry.
• Process orientation -- Thinking in terms of process is essential for an architect; this is the native language for the business. Thinking in terms of a business leader is essential, as is building repeatable and reusable processes both as artifacts from the work that they do and how they work themselves.

The skills that were mentioned -- such as process orientation, business savvy/technical acumen, and critical thinking -- will aid architects in better understanding how their specific company operates. Each company is different in how it interrupts the industry and correspondingly reacts to that understanding.
To understand the priorities of an organization, we must take a step back to understand what influences the direction that our companies take. There are direct and indirect influences -- or forces -- that determine the course of a company. These forces can change as time progresses, so that predicting them can be very difficult. Business SMEs often understand these forces very well, as they keep their eyes and ears on them to qualify their decisions. Architects must do the same.
IT has become less of a function of strategic value to companies -- no longer a necessary evil or cost center, but a real differentiating factor in the business of a company. Architects who have an understanding of forces will provide inherent alignment with the goals and objectives of a company.
Forces can be grouped into three high-level groups. These groups include the following:

• External -- Forces that are outside the organization forces and cannot be controlled.
• Business -- Purely business-related forces that can be derived from the inside or outside.
• Internal -- Forces that originate from the specific culture and operating model of a company.

CIOs, COOs, and IT architects will take into consideration these forces, as they drive technology decisions, initiatives, projects, and purchasing. Understanding these forces will enable architects to snap to the business priorities and imperatives with more ease and less uncertainty.
Figure 1 shows more detail on the industry forces and their effect on companies.

Figure 1. Industry forces on enterprises (Click on the picture for a larger image)

External forcesare events or circumstances in the broader market that affect an organization. Unlike business forces, these often are uncontrollable external events that eventually will need to be addressed by the business.

The following are usually the drivers for business forces:

• Faltering economy — There is no doubt that the faltering economy has affected businesses dramatically. Gartner predicts that worldwide IT growth will drop significantly—from what was first forecast as 8.9 to 7.3 percent (2008 prediction) to the current forecast of 5.8 to2.3 percent (see Gartner; December 31, 2008).
• Regulatory compliance — The need to fulfill compliance requirements has always been a requirement of companies. However, there is now increased pressure, fueled by the economic crisis. Governments will mandate a combination of green IT, security and privacy, auditing, and industry-specific regulations. Interestingly, 43 percent of technology leaders think that the 2008 election will affect IT decisions going further (see CDW IT Monitor;September 10, 2008).
• Natural disasters — Companies are increasingly dependent on IT-systems availability and information integrity. This dependency, combined with the sprawl of natural disasters, forces architects toevaluate how systems where built. There is and will continue tobe a need for business-continuity planning, data-replication strategies, and disaster-recovery mechanisms.

Business forcesare forces that relate directly to the business; they affect how the business operates and makes decisions. The following list shows the major business forces that will affect us in these times of uncertainty:

• Increased competition — Businesses are increasingly more competitive, as the market tightens up. Increased competition forces architects to focus on mission-critical solutions for the business that they support. Furthermore, Gartner furthers this by stating that enterprises do not want to disclose that they are aggressively cutting their IT budgets for competitive reasons (see Gartner; December 31, 2008).
• Increased M&A — In an economy such as the current one (as ofthis writing), it iscommon practice that there be significant acquisitions. We have seen this in the financial sector, but it will have a ripple effect across industries. M&A activities will spur a wealth of IT activities. Architects will play a major part in technology-portfolio evaluations, trade-off analysis of systems, architecture-gap analysis, security analysis and evaluation, and integration analysis.
• Business-optimization activities — With increased pressure from competition and modest projections, the theme of most companies will beto do more with less. Architects will need to look atenabling technologies to measure, track, and analyze business activities.

Internal forcesare the result of both external and business forces. These are the forces on which many architects act directly; they are muchcloser to the business of IT:

• Process improvement — Given some of the business-optimization needs of companies, process improvement and streamlining will occur. Architects should look at both:
• • Efficiencies in the systems that measure, track, and analyze processes.
  • Playing a part in optimizing the process itself to adapt to changing conditions. Forrester’s Gene Leganza states, “Enterprise architects need to work across development, support, and operations teams to drive decisions” (see Forrester; October 28, 2008).
• Infrastructure optimization—Both infrastructure and application optimization are direct results of “doing more with less.” As budgets are squeezed and server budgets dwindle, architects will need to evaluate infrastructure efficiencies. More often than not, there are opportunities to consolidate, virtualize, centralize, and repurpose infrastructure investments.
• Application portfolio management—Not only will the infrastructure need to be evaluated, but the applications will, too. Applications often can be elusive and not completely visible to architects or senior, technical decision makers. With pressure from both cost and regulatory perspectives, it is critical that applications be well understood and managed.

How Architects Can Add Value

As discussed earlier, the role of an architect is a unique one. Architects are some of the more unbiased technology decision makers, and they have a holistic view of solutions in an enterprise. In times of uncertainty, this can be a great asset to organizations. This skill set will be used now more than ever in a faltering economy.

We find that the cutting of budgets for IT does not correlate directly to IT not being important. It does show the need to streamline and improve IT. So, in essence, it means more IT activities. For architects, it means a realignment of architectural priorities.

Shifting of Priorities

Architectural priorities have changed, and the activities that an architect once made a high priority have now changed their course. What was once strategic to the organization is now labeled as risky or nice to have. Prudent low-risk initiatives are now in order. In the coming years, architects will engage in mission-critical, high–return- on-investment (ROI), low–total-cost-of- ownership (TCO) activities.

Table 1 shows examples of the IT priorities that are expected to change in the coming years.

Table 1. Priorities of IT

From Table 1, it is clear that priorities have shifted from large-scale, sometimes risky IT investments to tactical activities and projects that have a near-term ROI.

Dynamic, dexterous architects will have few or no problems adapting to this change.

Methods for Maximizing Architecture Decisions

In the previous sections, we discussed that we must change our mindsets and make our solutions much pragmatic. To do so, we must use pragmatic methods.

Pragmatic methods should both quantify and qualify our architecture decisions. The challenge is that there are not many methods to do this, as such. Traditional challenges were that architects quantified decisions in their own minds. The challenge here is that this is subject to bias and interpretation of the challenges; it does not provide a repeatable method that produces consistent results.

There is a method that can be used to produce this level of analysis in a repeatable fashion. The method is called the Architecture Qualification Method (AQM).

The Architecture Qualification Method (AQM)

The AQM is a pragmatic way of aligning with business architecture and, ultimately, fully qualifying our architecture decisions. This is performed by architects and can plug in to existing architecture frameworks (for example, TOGAF, FEAF, EACOE, and IASA), architecture standards, and proven practices. The AQM provides a set of tools that aid architects to achieve the following:

• Use of a method for capturing quality attributes for the enterprise
• Mechanisms for capturing definitions of quality attributes thatcan be tailored to your organization
• Intelligence that allows for the classification and variability of quality attributes
• Provision of a framework to rationalize attributes to the solution
• A way to govern how architectures are classified and rationalized throughout the organization

The AQM directly solves issues by using complexity and the understanding of architectures. Figure 2 shows an example of only a small subset of inputs to the architecture decision-making process.

Figure 2. Inputs to architecture decision making (Click on the picture for a larger image)

As you can see, there is input from all sides.

The AQM solves the following challenges:

• Provide a way to capture architecture analysis in a repeatable way.
• Qualify the quality attributes of the architecture.
• Reduce complexity and confusion in the build-out of solution architectures.
• Create a classification framework that allows for the orchestration of architecture standards and design patterns.
• Ease governance through the qualification of usage of standards and design patterns.
• Make architecture reuse proactively via reduction of complexity.

Architectures have multiple cross-cutting concerns that can be difficult to understand and quantify.

How Does the AQM Do This, and How Is It Different?

The AQM takes a structured approach to solving architectural problems. This is different from other solutions, because it provides a way for enterprises to define their own measurements, based on the business that they are in. See Figure 3, for example.

Figure 3 shows how the priorities of these two industries are different. This is furthered by the fact that the companies in these industries have slightly different ways of running their businesses, which can have a dramatic impact on how we create architectures.

Figure 3. Differences in industry priorities (Click on the picture for a larger image)

This is illustrated in Jeanne Ross’s book Enterprise Architecture asStrategy: Creating a Foundation for Business Execution (Boston, MA: Harvard Business School Press, 2006), in which she compares the differences between two companies in the same industry (such as FedEx and UPS) and shows vast differences in the IT systems that support their businesses.

What we get when we look at all of this is what is shown in Figure 4.

Figure 4. The AQM and business priorities (Click on the picture for a larger image)

Providing a way to have full traceability is a core tenet of the AQM. It is absolutely critical to be able not only to justify our decisions, but also to quantify and qualify them. The only way to do that is to align our decisions with business imperatives.

The AQM is used with the following components, as illustrated in Figure 4.

• Process—Specific to the AQM, this component provides the integration into existing processes and its own unique process for executing the AQM.
• Classification system—The classification system covers two aspects:
  • Definition of quality attributes into the terminology and the specific taxonomies that are well understood by your enterprise. (The unfortunate reality is that we all have various definitions of architecture terms.)
  • Creation of classifications, properties, and thresholds on quality attributes. This is one of the more important aspects of the AQM. This mechanism provides a way to fully qualify quality attributes through proper definition of what they mean to your enterprise. Through these classifications, properties, and thresholds, the AQM is able to both quantify and qualify architectures.
• Asset and pattern mapping matrix—As soon as a classification system has been created through the AQM, you can start to correlate it with real things—”real things” meaning your design patterns, standards, policies, and existing IT assets (system or application). By doing, so you can have a systematic way of prescribing what patterns to use for solutions—subsequently automating decision making, making it traceable, and easing governance.
  An example of this is a common scenario that I run into. Say that you organization has standards on a set of technologies for communications. Say also that those standards are FTP, SFTP, Web Services, CIFS/SMB, Connect Direct, and Microsoft SQL Server Integration Services (SSIS). Because all of these are in the standards list, developers or architects might not choose the optimal one for a variety of reasons that might include cost factors, complexity, or lack of understanding of the technology or personal biases.
  In the scenario, then, the architect chooses FTP because it is cheap and faster to implement. The problem is that the drivers for the solution call for the transactions to be highly secure and have a high level of resiliency. FTP is not the choice here. So, with the AQM, we can put qualifiers on these standards to show in what circumstances you should use each standard. This is applied to all of the architecture quality attributes (architecture “–ilities,” such as scalability, security, reliability, interoperability, and so on).
• Qualification tool—The qualification tool comes into play when the architect wants to build out a solution. As mentioned in the preceding scenario, the tool provides a way to put all of the pieces together in an easy way.

All of the complexity is removed from the architect, so that the architect can focus on creating a solution, instead of thumbing though thousands of standards pages to find (ultimately) something that might or might not be useful.

So, How Do I Use the AQM?

It is one thing to define a methodology, but it is an entirely different thing to make it actionable. The following list shows how the AQM is rationalized and immediately actionable and pragmatic:

• Office Excel–based templates that can be used by nearly every architect from Day 1
• Integration into an architecture meta-data repository
• Standard process definitions in Office SharePoint

Figure 5 shows screen captures of various aspects of the AQM. Data has not been populated in these, for very specific purposes. However, in the sample, data is populated in the classification matrix to show sample values. This is very simplistic and shows just what kind of data would be represented. In a real-world implementation, the repository would handle a great deal of this by joining multiple relationships with information and classifiers of a quality attribute. The spreadsheet is a viewing mechanism.

Figure 5. Aspects of the AQM (Click on the picture for a larger image)

We took a very high-level look at the AQM and showed the purpose, methods, and tools that are associated with it. Through this method, you can expect a greater level of accuracy and quality of architectural decision making.

Four Key Architectural Imperatives

Figure 6 shows the imperatives that architects should keep in mind in these times of uncertainty.

Figure 6. Key architectural imperatives (Click on the picture for a larger image)

These imperatives comprise four overarching imperatives:

• Align—Find direct links to business imperatives.
• Optimize—Do more with what you have.
• Externalize—Move IT assets outside of the IT operating environment, if they do not add value.
• Consolidate—Reduce unnecessary redundancies.

Align

Pressure from many forces on businesses will force IT alignment with business objectives. Now more than ever, companies are striving for this alignment. Architects will need to invest in better qualification of architecture decisions to ensure that value is added to the business. This qualification often is elusive and difficult to ascertain.

Understanding what areas to look at is critical. In the following list, you will find the major areas in which you can start to quantify architecture decisions. This is not a comprehensive list, but a starting point for architects to gather these key information points:

• Key metrics—All architects struggle with ways of quantifying their efforts, especially higher-level architects (such as enterprise architects). Instead of defining IT-specific metrics, architecture organizations will need to operate more like a business. By doing so, they will need not only to demonstrate their effectiveness, but also to quantify it.
• Assessments—Repeatable and consistent ways of evaluating solutions for their business fit is a must. These assessments should drive how decisions are made, and they illustrate how architectural trade-offs occurred. Assessments include architecture-viability assessments, architecture trade-off analysis, architecture decisions documents, and standards RFP assessments.
• Requirements management—Why should architects care about requirements? Simply put, it drives not only the functionality, but also the architecture. Capturing functional and nonfunctional requirements in reusable ways will help align architectures to the business. This is the fastest, easiest way to get such alignment.
• Architecture management—The next area of concern is to look at what has already been built and how it fits into the existing and new imperatives of the business. Architecture management links into standard processes, such as application portfolio management and PMO processes.

Optimize

As IT budgets shrink and the “big bang” projects dissolve, eyes will turn to optimization of existing solutions. Driving to get more value (from what the enterprise already has) will be a key imperative. Architects will be need to evaluate current solutions by determining how they are used, whether they are running optimally, whether they are redundant, and whether they fit the use that the business intended.

These questions are very difficult to answer. To answer them,

a great deal of information is required that might or might not be captured. There is a strong dependency on process. If the company has lax processes around software development and architectural processes, the quality or the information itself might be absent.

Key activities that will aid architects in obtaining this information are:

• Portfolio management. Reviewing the IT portfolio of applications will allow architects to either inventory existing systems or review the systems that have been cataloged. When choosing to optimize the business, portfolio management is very important; it surfaces all of the key aspects of a solution and tells us information such as:
  • How the solution links back to the business and (sometimes) a business process or capability.
  • Scorecarding of the solution across all other solutions.
  • Overall cost of the solution.
  • Links to how the solution is supported.
• Application life-cycle management (ALM).Whereas portfolio management looked holistically across the enterprise, ALM is much more focused. ALM improvements can include process optimizations to streamline efforts, new tools to automate and accelerate application development, and key information- gathering points to support architecture efforts and quantify business value.
• Revisiting architecture and development tooling. Optimizing the tooling that is used in the actual development and architectural planning will be essential for companies. In times of slowed project work, it is optimal to retool.
• Optimizing solutions. Not only will there be process improvements, but those solutions that we evaluated and classified in portfolio management will need to be optimized in some way. This will open up opportunities for architects to be creative with solutions. With a wave of new innovations in social computing, context-aware architectures, cloud-based architectures, and SaaS (to name a few), there are ways to introduce lower risk, cost, and support for your company.

Externalize

As companies classify solutions through methods such as portfolio management and ALM, they will be able to determine which applications add strategic value. This will drive business decisions for those solutions. Several options are available to these companies; it will be up to architects to help IT and business leaders determine the best course of action.

A trend since the very beginning of IT is the notion of externalizing IT assets. We saw this with traditional outsourcing, application service providers (ASPs), and managed services providers (MSPs).

Figure 7. Evolution of externalized services (Click on the picture for a larger image)

As Figure 7 shows, the concept of externalizing services, applications, or entire business processes is not entirely new but, instead, evolved.

For each solution, it is clear that there are distinct benefits and drawbacks. Some, such as ASP models, are just rendered obsolete, while outsourcing still strives in key problem areas such as the mainframe. What is most evident is that these technologies and methods are evolving.

There is no better time to start to think about externalizing IT services. Independent solution vendors (ISVs) and platform providers such as Microsoft (with its Live Services and Azure Cloud Services) solve traditional technology problems.

As with all technology solutions, if they are done correctly, SaaS and cloud-based solutions will provide companies with significant benefits:

• Accelerated technology adoption—The barrier to entry on access to new technologies is as easy as a subscription—a relatively small investment in prototype solutions on innovative technologies that in the past would have software licensing, procurement, deployment issues, and support staff training (among others) associated with them.
• Complexity—SaaS specifically enables companies to reduce the complexity in the solutions in which they would traditionally build or buy. Combining the power of strong Internet, security, and XML standards with SaaS-vendor solution expertise reduces many of the technical complexities that once loomed.
• Lower total cost of ownership (TCO)—By reducing the number of staff, acquisition of physical servers, software licenses, and overall operational costs, the TCO of solutions or entire IT services can be lowered.
• Agility—Allowing companies to speed up the delivery of solutions on modern technologies and reducing the complexities of line-of-business (LOB) application integration will make companies much more agile than ever before.

Consolidate

Pressure from all sides will force architects to think about doing more with less. This will include reducing the complexity and redundancy

in the enterprise. As architects optimize their enterprise through portfolio management, ALM, and tool rejuvenation, they will look at ways to optimize and consolidate applications.

Key areas in which consolidation will occur are:

• IT infrastructure. The hardware backbone of an enterprise often is the first to be consolidated—sometimes, the easiest of all consolidation efforts with mainstream and ever-evolving virtualization solutions. While it is easy to virtualize, there is careful planning needed, as it could lead to the same problem that you have with server sprawl.
• IT services. Collaboration, VoIP, e-mail, business intelligence, portals, system monitoring, and project-management systems (to name a few) are all IT services that can be streamlined by creation of standards, consolidation of multiple vendors, or moving them outside of the firewall to a service provider.
• Solution architectures. Companies often find redundancies in solutions across LOBs or functional areas. Consolidating solutions will be key to lowering costs and complexity within the organization.
• Process. Process management often is overlooked, but it can be a valuable exercise to consolidate disconnected and redundant processes. This streamlines your architectural efforts by providing repeatable and predictable measurements.

Through these four architectural imperatives, weathering the turbulent economic storm will be tolerable. Benefits for architects are huge. Not only will they be tightly aligned to the business, but they also will be seen as true business partners in IT. Architects will be able to add significant value in reducing TCO.

Conclusion

In the fragile economic state in which we find ourselves currently (as of this writing), enterprises can differentiate themselves with technology innovation and optimization. Those that capitalize on market conditions as an opportunity to streamline, consolidate, and acquire IT assets and/or entire businesses will have longevity well past the economic crisis.

This article has aimed to show architects a way to understand key forces on the business of IT, what they can do to add value, and the key areas of focus and technologies that will help them deliver value back to the business.

About the Author

Mike Walker (Mike@MikeTheArchitect.com) is a principal architect for Microsoft who is responsible for building the strategy for managing, delivering, and communicating the Microsoft position on enterprise architecture. He is responsible for driving Microsoft’s worldwide Enterprise 2.0 and Enterprise Architecture strategies in key industry segments.

Mike joined Microsoft in early 2006. His background is as a financial-services enterprise architect and strategist, specializing in business transformation around technology, strategic infrastructure planning, portfolio management of technology projects, and solution architecture. As a thought leader, Mike combines this experience with a strong focus on strategic execution.

Visit Mike Walker’s blog.

Follow up on This Topic

• Application Platform Optimization
• Application Lifecycle Management
• Externalizing to the Cloud