Adopting and Benefiting from Agile Processes in Offshore Software Development
by Andrew Filev
Summary: In modern software development, there are two trends that allow people to get more for less: agile development and offshore outsourcing. Let's look at how and when to combine both successfully, to raise the competitiveness of your business.
During the post-bubble era, IT budgets were cut more than were demands for their services, which prompted managers to search for more cost-effective solutions and empowered the trend for outsourcing software development to emerging market countries (offshore development). The economic driving force is not the only force for this trend. The recent rapid growth triggered by an improved communications infrastructure also plays a major role.
With respect to working in distributed teams in general and to offshore outsourcing in particular, usable voice over Internet protocol (VoIP) software, instant messengers, e-mail clients, and wikis have made online communications easier. Moreover, now it is often preferable to use online tools such as wikis over personal communications because these tools not only help to communicate the information, but also help to structure and store it. These tools are also effective when distributing information to many recipients.
These globally available, fast Internet connections power other tools, adding to this trend. Modeling tools help to make documentation more self-explanatory in distributed teams. Bug trackers, source control servers, Web portals, and online collaboration tools all help coordinate the distributed projects. Terminal services and virtual machines facilitate remote testing and administration.
The Internet also brought emerging market countries onto the track of high technologies. Because the Internet bypasses political borders, thousands of young people in developing countries like Russia and China use it to learn technologies that are cutting edge and improve their English skills. This new wave of Internet-educated software engineers came just in time to reinforce the offshoring trend.
The recent rise of offshore outsourcing made it big enough to become the target for political debates. For this discussion assume offshore development is an existing reality, and we'll focus on maximizing return from such outsourcing engagements. We'll skip the politics, but consult the list of resources for McKinsey Global Institute's research that quantifies the benefits of offshoring for the U.S. economy and debunks several myths about it.
Let's return back onshore. Here, the hearts and minds of many managers and engineers are conquered by another modern trend, agile software development. Slow heavyweight methods didn't prove themselves in today's dynamic business environment. Slender budgets demand more results, while bureaucracy was never the best choice in terms of return on investment (ROI). The power of agile methods lies in collaboration, flexibility, and dedication to the business value of software as reflected in core principles of the Agile Manifesto: individuals and interactions over processes and tools, working software over comprehensive documentation, customer collaboration over contract negotiation, and responding to change over following a plan (see Resources).
Agile methods greatly suit the new wave of Internet-based start-ups (often called Web 2.0). Agile software development lets some of these start-ups achieve more for less and release significant projects with small teams and tiny budgets. The short iterations and working software principles are reflected in a practice called constant beta after several Google products with the word "beta" incorporated in their logo.
However, agile methods are not one size fits all. They work well for small co-located teams facing rapidly changing conditions.
Figure 1. Point-to-point and service bus integration (Click on the picture for a larger image)
While there are cases in which the applicability of agile software development is subject to question—such as in distributed development with offshore outsourcing—my successful five years of experience applying principles of agile development in distributed teams demonstrates that it is possible and that it gives great returns when used properly.
Figure 2. Three integration layers (Click on the picture for a larger image)
There are other scenarios where the use of agile software-development processes remains questionable. Examples are large development teams (more than 20 people working on one independent project), systems where predictability is paramount (life-critical applications), and bureaucratic environments. We won't look at such scenarios here, and furthermore we assume a company has a corporate culture favoring agile development and intends to apply the ideas presented here to software teams of fewer than 20 people (that is, 20 people on a particular team or project, not on the entire development team). We'll instead look at the application of agile methods to distributed development in general and offshore outsourcing in particular.
Offshore software-development deals represent a whole spectrum of different engagements, from hiring one offshore developer from rentacoder.com on one side, to billion-dollar deals with U.S. companies who own overseas subsidiaries on the other. Some of these deals are arranged in a way that prevents the companies from using an agile software-development process, even if one of the sides wants to.
To implement an agile process, the selected outsourcing model should encourage communications and collaboration, assume flexibility, and justify releasing often. Though dozens of criteria may be applied to outsourcing deals, not many of them are as important for our further discussions as the pricing model. See Figure 1 for the mapping of the most common pricing schemas.
Predictable results imply predictable processes. In Figure 2 you can see groups of the development processes aligned on the predictive-adaptive scale. If we use a predictive-adaptive criterion, predictable schemas, which are closer to the left end of the scale (see Figure 1), require more predictability from the software-development process; while agile development processes lie on the opposite side of the software-development processes continuum.
The predictability concern is especially related to fixed scope-fixed price engagements. When this type of contract is applied to an outsourcing deal, the client and provider are naturally tied to predictive software-development processes and their consequences. So this contract is not a good fit for agile software development. When designing an outsourcing engagement, remember that responsiveness to changes encourages using pricing models like time and material, which gives flexibility both to client and provider and is a better fit for agile development.
When structuring the deal, consider how the selected schema will affect communications and collaboration. An agile development process requires an open environment, tight in-team integration, shared common goals, understood business value, and frequent communication. The more barriers we have among engineers, customers, users, managers, and other stakeholders, the harder it is to provide a base for agile software development, which means reducing middlemen to allow maximum transparency and integration between teams.
Big players work this out by establishing their own branches in other countries, which makes economical sense if a company wants to have more than 100 engineers in its overseas development center. The exact number depends heavily on the other country and such factors as how easily the company may recruit talented people there.
When considering this alternative, do not repeat the mistakes of some small and midsize businesses, who underestimated the hidden expenses like top management time, travel budgets, local attorney's fees, and other costs. Also, the rapid growth of economies in developing countries causes shortages of talent, which doesn't ease the life of small foreign newcomers. While local players are better networked in their local community, branches of big companies may set this off by well-known brands, bigger salaries, and better social packages.
Such luxuries are often out of reach of small companies, which may want to have only 15 developers overseas. Successful small and midsize players overcome the costs of handling remote office setup by using dedicated development teams, offshore development centers (ODCs), build-operate-transfer (BOT) models, virtual offices, virtual teams, and so forth. Regardless of the name and details, one thing is common in these models: the offshore provider is more focused on providing physical, legal, IT, and HR infrastructure to the client than on participating in the development process. The responsibility for project delivery in such deals is shared between client and provider.
To reap the full benefits of such deals, engineers must be assigned to one client, and team retention rates must be good in both companies. The provider must provide transparent communications to the client on every level, including developer-to-developer communications. People must speak one language without a translator.
In successful engagements we sometimes see that although people in remote offices get paid through the provider, they associate themselves more with the client and share the values and corporate culture of both companies.
There are well-known practices used by successful software-development teams: common coding standards; a source-control server; one-click, build-and-deploy scripts; continuous integration; unit testing; bug tracking; design patterns; and application blocks. These practices must be applied to distributed teams more strictly than to local teams.
For example, consider continuous integration. It might be extremely frustrating to come to work and get a broken build from the source-control server, while the person responsible for it is several thousand miles away and might be seeing dreams at the moment. This issue might not be big if done by the guy in the next office, but it might become a major problem in a distributed scenario hurting productivity and communications. You can minimize such risks by sticking to continuous integration practices team-wide and installing the corresponding server (such as Microsoft Team Foundation Server, CruiseControl.NET, and CruiseControl).
Teams working on the Microsoft .NET platform are in a great position with the features provided by Microsoft Visual Studio Team System right out of the box. You get prescriptive Microsoft Solutions Framework for Agile Development and supporting tools. This product is extremely helpful for teams who need more guidance with agile development in distributed environments. For experienced teams, it's an integrated solution that provides great ROI.
Another Microsoft product, which provides great value for distributed teams, is Windows SharePoint Services (WSS). Wikis naturally fit and help agile development in distributed teams, and the next version of WSS is planned to have wiki among its enhancements. WSS is also tightly integrated with Visual Studio Team System, which makes it the best choice for the team's Web portal.
From an IT infrastructure point of view, I recommend using a virtual private network (VPN), giving the teams equal access to shared resources. The VPN environment, being less strict than a public network, allows using such features as Windows Live Messenger's application sharing, video and voice calls, remote assistance, and whiteboard.
Working remotely, small misunderstandings quickly grow into bigger problems. In distributed development teams managers must pay attention to communication practices, which they sometimes omit without negative consequences in local development. This attention includes regular (daily/weekly) reports and status update meetings, which allow the team members to synchronize, discuss achievements, and reveal problems. Managers should also try to build personal human relations in teams through introductory meetings, on-site visits, team-building activities, and other methods.
In offshore outsourcing deals, development managers should be aware of language, cultural, and time-zone barriers, and must find ways to surmount these obstacles. Globalization slowly but constantly erases the cultural distinctions in the professional environment, but there are still cases when cultural differences bring confusion. There are many country-specific issues in this topic, and they are out of the scope of this discussion. Language issues are much easier to detect, though it doesn't mean that they're easier to overcome. Where companies face a language barrier, it is common and highly desirable to have company-sponsored language training for employees. In most of the offshore development countries professionals are motivated to learn English, so it is usually people in these locations who get language training.
Variations in time zones specifically make the process more difficult. But it turns out that in countries with developed outsourcing industries, software engineers are usually ready to adapt their working schedule to work with overseas counterparts. There are two strategies to handle time-zone differences. The first is to separate teams by activity; for example, have quality assurance and product managers on site and developers overseas. This arrangement allows implementing a cycle, where developers implement fixes and new requirements while their counterparts are sleeping and vice versa. Of course, there should be intersections in working schedules (in the beginning/end of a working day). The second approach is to divide projects into blocks, and try to assign each block to one location, delegating as many functions as possible to this location. The second approach forces better communication and thus better serves agile development, but both work, and sometimes there is no choice.
Choosing the right model is also very important, but it doesn't guarantee success. It's highly recommended that at least one party has experience in agile development, preferably in a distributed environment. The lack of face-to-face communication, along with time, cultural, and language differences, requires attention and investing additional efforts to get desired results. The benefits of having a good offshore partner—cost-savings, on-demand staff augmentation, and outsourcing infrastructure-related tasks—(which might be summarized as "getting more for less") far outweigh investment in building the productive relations. This positive balance would be impossible without modern tools empowered by the great communications infrastructure now available globally.
Andrew Filev (MCA, MVP) is vice president responsible for offshore operations at Murano Software. He establishes offshore development centers, and leads and motivates teams. An excellent communicator, Andrew fills the gap between different cultures and builds lasting partnerships with clients.
"Exploding the Myths of Offshoring," Martin N. Baily and Diana Farrell, The McKinsey Quarterly (McKinsey & Company, 2004) (Note: registration required)
This article was published in the Architecture Journal, a print and online publication produced by Microsoft. For more articles from this publication, please visit the Architecture Journal website.