May I Use Your Desktop to Save the World, Please?

Nita Menon
October 2008

Applies to:
   Distributed Computing

Summary: Among the questions that this article asks—and answers—are: Given that it seems like a lot of work to deploy and manage a distributed system, why would anyone do so, and what are the advantages of doing so? (5 printed pages)

Contents

Introduction
What Is Distributed Computing?
Anatomy of a Distributed System
Advantages
Conclusion

Introduction

When Paul Allen and Bill Gates started Microsoft Corporation with the dream of "a computer in every home and on every desk," the thought was so remote that many would not have believed it to be possible. However, over 30 years later, we live in a world in which the desktop is in every home and on every desk, and the lives and functions of individuals and corporations revolve around it.

The evolution from the first personal computer in the 1900s to the connected, mobile personal-computing era of today has been fascinating. What, however, do we do with all of the processing power and capabilities on desktops today? Regardless of all of the demonstrations of capabilities of new desktops and of all of the projects that you intend to do with your new computer—gaming, e-mail, word processing, spreadsheets, and so on, combined with the busy lives of people—we use less than a fraction of the processing capabilities of today's desktops.

Would it not be great, then, to tap in to the capabilities and idle time on these desktops? If we joined the processing power of all of these desktops, we would have one of the most complex and fantastic systems in the world. How would anyone manage such a system? What could you use it to do? Would this even be possible? Let us see some examples.

One sterling example of such a distributed computing system is the World Community Grid: It aims to be the world's largest public computing grid, the largest distributed system, built from nothing but the millions of idle desktops across the world. What is more, the projects that can be executed on the World Community Grid are meant for the benefit of humanity!

What Is Distributed Computing?

Here is a question that is heard commonly on software-development projects: If one person can do a job in ten days, can ten people do the job in one day? The answer to this question in a software-development project is not always "Yes." However, in the distributed computing world, the answer would always be "Yes." If, for example, we had to unlock a 64-bit encryption code, we would need to try more than 72 quadrillion keys. It would take a single computer several hours to do so; but, if we could split this across multiple computers, the job would be done far more quickly. Simply defined, this is what distributed computing is all about.

How exactly does this work, though? Can multiple computers be synchronized in such harmony that they can all work toward a common goal?

Anatomy of a Distributed System

If you were to decide tomorrow that you would need 100 people to work for you on an important project, what is the first thing that you would do? Coordinating tasks along with planning and managing a 100-person team would not be a trivial task. Would you not need someone or perhaps a team of people dedicated to doing just that? That is the role of a manager.

All distributed systems rely on a manager or a set of management servers to manage and deploy units of work across the myriad computers that participate in the distributed system.

The management server does one of many things. For example, it:

  • Divides larger tasks into smaller ones.
  • Allocates tasks to idle computers.
  • Distributes the workload efficiently across computers.
  • Monitors work that is done across computers.

Depending on the scale of the distributed system, additional complexities might be involved in managing a distributed system. For example, deciding on the workload that is to be processed by a computer would depend on the processing capability of the computer. In large-scale distributed systems, this can vary between lower-end e-mail and word-processing desktop machines and higher-end quad core servers. Managing the capabilities of the resources, therefore, would be an important aspect of such a distributed system.

But, as in all such systems, communicating effectively is the key. Coming back to the initial example, what would the manager of your project do if that person had no means to communicate with the 100-person team? No telephone, e-mail, or face-to-face? How would that person recognize capabilities, distribute tasks, monitor progress, and do everything that must be done for the project to run efficiently?

Here, we come to the second most important part of a distributed-computing environment: the network. Without the means to connect all of the computers that participate in the distributed system, there is no system. However, the network does not have to be complex or special; it can be something as simple as a LAN or as common as the Internet. The network enables these computers to be reachable, so that information can be exchanged and progress can be ascertained easily across all of the computers.

The network would help enable all of the computers to:

  • Communicate their availability.
  • Communicate status of assigned tasks.
  • Communicate results of assigned tasks.
  • Communicate processing capability.
  • Accept new jobs.

Are we done with just a manager, a network, and computers? If we were to take just two computers and put them on a LAN, and deploy a server on the LAN that could communicate with the two computers, would we be able to use all of the processing power on the two computers? The answer is, No. There must be something else.

Let us revisit the example of the 100-person team. We hired our managers and got our team in place. Also, we laid out e-mail addresses and telephone lines across our team and managers, so that they could communicate. However, what if we were to discover that 80 of the 100 people in our team did not speak English fluently, and that English was the only language that our managers spoke? How would they communicate? We would need someone to translate requirements to them, and to communicate their task status and availability to us. That is the role of an agent.

Most large-scale distributed systems today rely on software agents. Software agents are small, lightweight pieces of software that are installed on the myriad servers that form part of the distributed-computing network.

The software agent is responsible for:

  • Communicating to the management server.
  • Publishing the availability of the computer on which it is deployed.
  • Making known the capability of the computer on which it is deployed.
  • Accepting assigned tasks.
  • Performing the assigned tasks by using local computer resources.
  • Communicating the results of the task.

These three core components largely handle all of the aspects of a distributed system. However, there are many other aspects to consider.

Security, for example, is one aspect. If the distributed system were tackling the task of breaking an encryption key that was deployed by a hacker, we would not want what it was doing to be common knowledge to all of the computers in the distributed system. Management must be in place to define which computers get access to what kind of information.

Resource management is another aspect. For example, if a task that is assigned to Computer A is required to be completed before several other tasks can be distributed to other computers, and if Computer A seems to be non-reachable, what would the management server do?

There are several modes of operation that must be considered. For example, the management server could:

  • Broadcast the task across the network, and let computers respond to their availability and willingness to take up the task.
  • Reassign the task to another available computer.
  • Keep re-polling Computer A till it became available.

It does seem like a lot of work to deploy and manage a distributed system. Why would anyone do so? What would be the advantages of doing so?

Advantages

Cost

Many jobs that are performed today on distributed systems, such as the World Community Grid, could be achieved by using a single powerful mainframe or a supercomputer. However, not all organizations can afford, maintain, or even rent a supercomputer—hence, the evolution of distributed computing. Making use of every spare processing capability of a traditional desktop-PC–like computer or a slightly more powered server, these systems are far more reachable and within the means of most organizations.

SETI (Search for Extra-Terrestrial Intelligence) is yet another example. A nonprofit and educational research organization, SETI uses a public computing grid that is similar to the World Community Grid and is used for exactly what its name says: the search for extraterrestrial intelligence. SETI states that the most powerful supercomputer, ASCI White, cost over $110 million, whereas SETI, with similar processing capabilities, so far has cost just about $500,000.

Reliability

Compared to traditional supercomputers, distributed-computing systems offer a far more reliable advantage, too. In using a traditional supercomputer, compromising the availability of the supercomputer would put to a halt all the jobs that depended on it. In a distributed-computing environment, however, the lack of availability of a single computer could be managed with far lesser repercussions.

Scalability

Distributed-computing systems are also easy to scale. Although supercomputers are built to handle large workloads, they typically are not easy to scale. In a distributed environment, however, one could add into the environment as many computers as one wanted.

There are also several intangible benefits. Research says that over 90 percent of CPU cycles on a typical desktop are unused. As the world forges toward better utilization of power, space, and other environmental resources, being able to use these computers effectively would mean also a cleaner, better environment for our future generations.

Conclusion

Needless to say, distributed computing is here to stay. We just have to find more uses for it. So, the next time that you step away from your computer for coffee, a chat by the water cooler, or lunch—or if, perhaps, you are lucky enough to go back to the comforts of your home and family—you could be donating your desktop to a good cause!

There! Does that not make you feel good, now?

About the author

Nita Menon is a Senior Architect at a financial-services firm in New York City. Nita has strong experience in building and leading teams of professionals to deliver high-quality software applications. Also an excellent strategist, Nita has experience with building strategies for large corporations to:

  • Develop the long-term vision of the company.
  • Promote the role that technology has to play in the long-term vision.
  • Communicate and help adopt the long-term vision.
  • Build successful in-house development teams.
  • Enable offshore development successfully.

 

Show: