System Topology
The following figure illustrates a simple LINQ to HPC topology. This topology can be implemented on an on-premises Windows HPC Server 2008 R2 cluster or in a Windows Azure Scheduler deployment in Windows Azure.
.png "An HPC cluster running DryadLINQ")
Figure 1. An HPC cluster that runs LINQ to HPC
The following sections discuss each of the elements in the illustration.
Client
The client is a computer that runs applications that perform LINQ to HPC queries. To enable communication with the head node, the client computer is either connected to the same network as the head node, or communicates with the head node through a web-based protocol. To support LINQ to HPC applications, the LINQ to HPC provider is installed on the client computer.
LINQ to HPC provider
The LINQ to HPC provider initiates the HPC jobs that execute LINQ to HPC queries. The provider examines the LINQ to HPC queries in an application and constructs an execution plan. It starts new LINQ to HPC jobs by contacting the job scheduler.
The LINQ to HPC provider is analogous to other LINQ providers, such as the LINQ to Objects provider, and the LINQ to SQL provider.
DSC client service
The DSC client service manages the distributed data that is used by LINQ to HPC queries. The DSC client communicates with a DSC service that runs on the head node.
Within an application, the DSC client service is represented by an instance of the DscService class.
Head node
The head node performs management tasks for the entire Windows HPC cluster or Windows Azure Scheduler deployment. To support LINQ to HPC workloads, the head node runs the HPC job scheduler and the DSC Service.
DSC Service
The DSC is comprised of the DSC database and the DSC service. The DSC manages all the data that is used by a LINQ to HPC query. It can be configured so that it controls some or all of the compute nodes of an HPC cluster. The compute nodes that are controlled by the DSC are called DSC nodes.
The DSC database is an instance of the Microsoft® SQL Server® data management software. It holds the catalog of DSC files and DSC file sets, where a file set is the data used by the application, and a file is a subset of that data. It also manages the metadata for the cluster. Cluster metadata includes mappings of files to file sets, and files to DSC nodes, as well as the attributes of the file sets, DSC files, and DSC nodes. File set properties include lease time (the time quota for a temporary file), the size of the file set, the creation time, and the time of last use. Attributes for DSC files include the path names of their associated NTFS files and the file size. Node attributes include the DSC node state, such as read-only, read-write, or unavailable, as well as the host name, the amount of free disk space on the node, and the amount of disk space used by DSC files. For more information about DSC file sets and files, see DSC File Sets and Files.
Note
A command-line utility named Dsc.exe gives you some control over the DSC. You can use Dsc.exe to create and delete file sets, add DSC files to file sets, and specify which compute nodes of the cluster will be used to store DSC files and execute queries. You can also configure settings. For example, you can set the replication factor value, which sets the number of copies the cluster maintains for each new DSC file that it creates. This enables you to control aspects of the DSC, such as fault tolerance and query performance. For more information, see DSC Command-Line Reference in the LINQ to HPC Programmer's Guide.
HPC Job Scheduler
The HPC job scheduler is responsible for allocating compute resources and starting jobs. The scheduling policy, which can be configured by an administrator, determines the order in which to run jobs from the job queue and how to allocate cluster resources. For more information, see Job Scheduler Configuration in Windows HPC Server 2008 R2
For LINQ to HPC jobs, the job scheduler allocates compute nodes exclusively (no other jobs run on those nodes at the same time). When a LINQ to HPC job is ready to start, the job scheduler creates an instance of the LINQ to HPC graph manager on a compute node. The graph manager does not need to run on a DSC node. It can run on any compute node that has the LINQ to HPC compute node features installed.
LINQ to HPC graph manager
The execution of the individual tasks (vertices) that make up a particular LINQ to HPC job is managed by the LINQ to HPC graph manager. When the HPC job scheduler starts a LINQ to HPC job, it creates an instance of the graph manager to manage the execution of the LINQ to HPC query. A new instance of the graph manager is created for each job.
The graph manager uses the execution plan to distribute the computation across the DSC nodes. To preserve data locality, it must know where the DSC has distributed the files that make up the input file set. It receives this information from the DSC service that runs on the head node. After it has assigned vertices to DSC nodes, the graph manager starts executing the vertices that comprise the first stage of the computation. It starts and stops vertices as the job progresses through each stage.
The graph manager monitors all the vertices. If a vertex fails, the graph manager attempts to reschedule the vertex 6 times. It tries to preserve data locality, but will run on any available node. For more information about the graph manager, see How a LINQ to HPC Job Runs.
DSC nodes
DSC nodes are compute nodes that have the LINQ to HPC compute node features installed and that have been added to the DSC. An administrator can add compute nodes to the DSC by using the DSC command line utility. For more information, see DSC Command-Line Reference in the LINQ to HPC Programmer's Guide.
The DSC nodes store DSC files and perform the computations of each vertex. Not all vertices necessarily run the same processing logic. Each DSC node performs the file replication tasks that are assigned to the node by the DSC service on the head node. The DSC node also reclaims temporary storage used by vertices, performs file validation, and sends heartbeats to the DSC service on the head node.