[This documentation is for preview only, and is subject to change in later releases. Blank topics are included as placeholders.]
The Concurrency Runtime Task Scheduler schedules and coordinates tasks at run time. A task is a unit of work that performs a specific job. The work that is performed by task group items, parallel algorithms, and asynchronous agents are all examples of tasks.
The Task Scheduler manages the details that are related to efficiently scheduling tasks on computers that have multiple computing resources. The Task Scheduler also uses the newest features of the underlying operating system. Therefore, applications that use the Concurrency Runtime automatically scale and improve on hardware that has expanded capabilities.
Concurrency Runtime Versus Other Concurrency Models describes the differences between preemptive and cooperative scheduling mechanisms. The Task Scheduler uses cooperative scheduling and a work-stealing algorithm together with the preemptive scheduler of the operating system to achieve maximum usage of processing resources.
The Concurrency Runtime provides a default scheduler so that you do not have to manage infrastructure details. However, to meet the quality needs of your application, you can also provide your own scheduling policy or associate specific schedulers with specific tasks. This topic describes the important features that let you extend the Task Scheduler to fine-tune your applications.
The Scheduler and CurrentScheduler Classes
The Task Scheduler enables applications to use one or more scheduler instances to schedule work. The Scheduler class represents a scheduler instance and encapsulates the functionality that is related to scheduling tasks.
Scheduler instances let you associate specific scheduling policies with various kinds of workloads. For example, your application can require that some tasks run at the typical thread priority and other tasks run at a higher priority. Scheduler instances also let you divide the available processing resources and assign a fixed set of resources to each scheduler. For example, consider a parallel algorithm that does not scale beyond four processors. You can create a scheduler policy that limits its tasks to use no more than four processors concurrently.
A thread that is attached to a scheduler is known as an execution context, or just context. One scheduler can be active on the current context at any given time. The active scheduler is also known as the current scheduler. The Concurrency Runtime uses the CurrentScheduler class to provide access to the current scheduler. The current scheduler for one context can differ from the current scheduler for another context. The runtime does not provide a process-level representation of the current scheduler.
Typically, the CurrentScheduler class is used to access the current scheduler. The Scheduler class is useful when you have several schedulers and you want to provide your own scheduling policy or associate specific schedulers with specific tasks.
The following sections describe how to create and manage a scheduler instance. For a complete example that illustrates these tasks, see How-to: Manage a Scheduler Instance.
Creating a Scheduler Instance
There are these three ways to create a Scheduler object:
If no scheduler exists, the runtime creates a default scheduler for you when you use runtime functionality, for example, a parallel algorithm, to perform work. The default scheduler becomes the current scheduler for the context that initiates the parallel work.
The CurrentScheduler::Create method creates a Scheduler object that uses a specific policy and associates that scheduler with the current context.
The Scheduler::Create method creates a Scheduler object that uses a specific policy, but does not associate it with the current context.
Allowing the runtime to create a default scheduler enables all concurrent tasks to share the same scheduler. Typically, the functionality that is provided by the Parallel Patterns Library (PPL) or the Asynchronous Agents Library is used to perform parallel work. Therefore, you do not have to work directly with the scheduler to control its policy or lifetime. When you use the PPL or the Agents Library, the runtime creates the default scheduler if it does not exist and makes it the current scheduler for each context. If you create a scheduler and set it as the current scheduler, then the runtime uses that scheduler. Create additional scheduler instances only when you require a specific scheduling policy. For more information about the policies that are associated with a scheduler, see Scheduler Policies later in this topic.
Managing the Lifetime of a Scheduler
The runtime uses a reference-counting mechanism to control the lifetime of Scheduler objects.
When you use the CurrentScheduler::Create method or the Scheduler::Create method to create a Scheduler object, the runtime sets the initial reference count of that scheduler to one. The runtime increments the reference count when you call the Scheduler::Attach method. The Scheduler::Attach method associates the Scheduler object with the current context, making it the current scheduler. When you call the CurrentScheduler::Create method, the runtime both creates a Scheduler object and attaches it to the current context (and sets the reference count to one). You can also use the Scheduler::Reference method to increment the reference count of a Scheduler object.
The runtime decrements the reference count when you call the CurrentScheduler::Detach method to detach the current scheduler, or call the Scheduler::Release method. When the reference count reaches zero, the runtime destroys the Scheduler object after all scheduled tasks finish. A running task is allowed to increment the reference count of the current scheduler. Therefore, if the reference count reaches zero and a task increments the reference count, the runtime does not destroy the Scheduler object until the reference count again reaches zero and all tasks finish.
The runtime maintains an internal stack of Scheduler objects for each context. When you call the Scheduler::Attach or CurrentScheduler::Create method, the runtime pushes that Scheduler object onto the stack for the current context. This makes it the current scheduler. When you call CurrentScheduler::Detach, the runtime pops the current scheduler from the stack for current context and sets the previous one as the current scheduler.
The runtime provides several ways to manage the lifetime of a scheduler instance. The following table shows the appropriate method that releases or detaches the scheduler from the current context for each method that creates or attaches a scheduler to the current context.
Create or attach method | Release or detach method |
|---|
CurrentScheduler::Create | CurrentScheduler::Detach |
Scheduler::Create | Scheduler::Release |
Scheduler::Attach | CurrentScheduler::Detach |
Scheduler::Reference | Scheduler::Release |
Calling the inappropriate release or detach method produces unspecified behavior in the runtime.
When you use functionality, for example, the PPL, that causes the runtime to create the default scheduler for you, do not release or detach this scheduler. The runtime manages the lifetime of any scheduler that it creates.
Because the runtime does not destroy a Scheduler object before all tasks have finished, you can use the Scheduler::RegisterShutdownEvent method or the CurrentScheduler::RegisterShutdownEvent method to receive a notification when a Scheduler object is destroyed. This is useful when you must wait for every task that is scheduled by a Scheduler object to finish.
Methods and Features
This section summarizes the important methods of the CurrentScheduler and Scheduler classes.
Think of the CurrentScheduler class as a helper for creating a scheduler for use on the current context. The Scheduler class lets you control a scheduler that belongs to another context.
The following table shows the important methods that are defined by the CurrentScheduler class.
Method | Description |
|---|
Create | Creates a Scheduler object that uses the specified policy and associates it with the current context. |
Get | Retrieves a pointer to the Scheduler object that is associated with the current context. This method does not increment the reference count of the Scheduler object. |
Detach | Detaches the current scheduler from the current context and sets the previous one as the current scheduler. |
RegisterShutdownEvent | Registers an event that the runtime sets when the current scheduler is destroyed. |
CreateScheduleGroup | Creates a ScheduleGroup object in the current scheduler. |
ScheduleTask | Adds a lightweight task to the scheduling queue of the current scheduler. |
GetPolicy | Retrieves a copy of the policy that is associated with the current scheduler. |
The following table shows the important methods that are defined by the Scheduler class.
Method | Description |
|---|
Create | Creates a Scheduler object that uses the specified policy. |
Attach | Associates the Scheduler object with the current context. |
Reference | Increments the reference counter of the Scheduler object. |
Release | Decrements the reference counter of the Scheduler object. |
RegisterShutdownEvent | Registers an event that the runtime sets when the Scheduler object is destroyed. |
CreateScheduleGroup | Creates a ScheduleGroup object in the Scheduler object. |
ScheduleTask | Schedules a lightweight task from the Scheduler object. |
GetPolicy | Retrieves a copy of the policy that is associated with the Scheduler object. |
SetDefaultSchedulerPolicy | Sets the policy for the runtime to use when it creates the default scheduler. |
ResetDefaultSchedulerPolicy | Restores the default policy to the one that was active before the call to SetDefaultSchedulerPolicy. If the default scheduler is created after this call, the runtime uses default policy settings to create the scheduler. |
[go to top]
This section describes the role of scheduler policies. For examples that use specific scheduler policies to control the behavior of the scheduler, see How-to: Specify Specific Scheduler Policies and How-to: Create Agents that Use Specific Scheduler Policies.
You can specify specific policies when you create a Scheduler object. You cannot change the scheduler policy after you create the Scheduler object. Policy values let you control the strategy that the scheduler uses when it manages tasks. For example, your application can require some tasks to use user-mode schedulable (UMS) threads and other tasks to use the normal threading mechanism.
Each scheduler uses its own policy when it schedules tasks. Therefore, the policies that are associated with one scheduler do not affect the behavior of any other scheduler.
The CurrentScheduler::Create, Scheduler::Create, and Scheduler::SetDefaultSchedulerPolicy methods each take a SchedulerPolicy object that contains a collection of key-value pairs that specify the behavior of the scheduler. The SchedulerPolicy constructor takes a variable number of arguments. The first argument is the number of policy elements that you are about to specify. The remaining arguments are key-value pairs for each policy element. The following example creates a SchedulerPolicy object that specifies three policy elements. The runtime uses the default values for the policy keys that are not specified.
SchedulerPolicy policy(3,
MinConcurrency, 2,
MaxConcurrency, 4,
ContextPriority, THREAD_PRIORITY_HIGHEST
);
The PolicyElementKey enumeration defines the policy keys that are associated with the Task Scheduler. The following table describes the policy keys and the default value that the runtime uses for each of them.
Policy Key | Description | Default Value |
SchedulerKind | A SchedulerType value that specifies whether to use normal threads or UMS threads to schedule tasks. | UMSThreadDefault |
MaxConcurrency | An unsigned int value that specifies the maximum number of concurrency resources that the scheduler uses. | MaxExecutionResources |
MinConcurrency | An unsigned int value that specifies the minimum number of concurrency resources that the scheduler uses. | 1 |
TargetOversubscriptionFactor | An unsigned int value that specifies how many threads to allocate to each processing resource. | 1 |
LocalContextCacheSize | An unsigned int value that specifies the maximum number of contexts that can be cached in the local queue of each virtual processor. | 8 |
ContextStackSize | An unsigned int value that specifies the size of the stack, in kilobytes, to reserve for each context. | 0 (use the default stack size) |
ContextPriority | An int value that specifies the thread priority of each context. This can be any value that you can pass to SetThreadPriority or INHERIT_THREAD_PRIORITY. | THREAD_PRIORITY_NORMAL |
SchedulingProtocol | A SchedulingProtocolType value that specifies the scheduling algorithm to use. | EnhanceScheduleGroupLocality |
DynamicProgressFeedback | A DynamicProgressFeedbackType value that specifies whether to rebalance resources according to statistics-based progress information. | ProgressFeedbackEnabled |
When the runtime creates the default scheduler, it uses the policy values that are specified by the call to the Scheduler::SetDefaultSchedulerPolicy method. If you do not call the Scheduler::SetDefaultSchedulerPolicy method, the runtime uses the default policy values from the table.
Use the CurrentScheduler::GetPolicy and the Scheduler::GetPolicy methods to retrieve a copy of the scheduler policy. The policy values that you receive from these methods can differ from the policy values that you specify when you create the scheduler. For example, the UMSThreadDefault policy value specifies that the scheduler uses UMS threads if that feature is available on the operating system (for example, the 64-bit version of Windows 7). If UMS threads are not available, the scheduler sets this policy value to ThreadScheduler, which specifies that the scheduler is to use normal threads.
[go to top]
A schedule group groups related tasks together. A scheduler has one or more schedule groups. The SchedulingProtocol scheduler policy influences the order in which the scheduler executes the tasks in each schedule group. When SchedulingProtocol is set to EnhanceScheduleGroupLocality (which is the default), the Task Scheduler chooses the next task from the schedule group that it is working on when the current task finishes or cooperatively yields. The Task Scheduler searches the current schedule group for work before it moves to the next available group. Conversely, when SchedulingProtocol is set to EnhanceForwardProgress, the scheduler moves to the next schedule group after each task finishes or yields. For an example that compares these policies, see How-to: Use Schedule Groups to Influence Order of Execution.
The runtime uses the ScheduleGroup class to represent schedule groups. To create a ScheduleGroup object, call the CurrentScheduler::CreateScheduleGroup or Scheduler::CreateScheduleGroup method. The runtime uses a reference-counting mechanism to control the lifetime of ScheduleGroup objects, just as it does with Scheduler objects. When you create a ScheduleGroup object, the runtime sets the reference counter to one. The ScheduleGroup::Reference method increments the reference counter by one. The ScheduleGroup::Release method decrements the reference counter by one.
Every Scheduler object has a default schedule group for every scheduling node. A scheduling node maps to the underlying system topology. The runtime creates one scheduling node for every processor package or Non-Uniform Memory Architecture (NUMA) node, whichever number is larger. If you do not explicitly associate a task with a schedule group, the scheduler adds the task to a default group.
Many types in the Concurrency Runtime let you associate an object with a schedule group. For example, the agent class and message block classes such as unbounded_buffer, join, and timer, provide overloaded versions of the constructor that take a ScheduleGroup object. The runtime uses the Scheduler object that is associated with this ScheduleGroup object to schedule the task.
You can also use the ScheduleGroup::ScheduleTask method to schedule a lightweight task. For more information about lightweight tasks, see Lightweight Tasks later in this topic.
For an example that uses schedule groups to control the order of task execution, see How-to: Use Schedule Groups to Influence Order of Execution.
[go to top]
A lightweight task is a task that you schedule directly from a Scheduler or ScheduleGroup object. The Concurrency Runtime itself uses lightweight tasks to schedule asynchronous agents and send messages between asynchronous message blocks.
Lightweight tasks carry less overhead than asynchronous agents and task groups. For example, the runtime does not inform you when a lightweight task finishes. In addition, the runtime does not catch or handle exceptions that are thrown from a lightweight task. For more information about exception handling and lightweight tasks, see Exception Handling in the Concurrency Runtime.
The work function of a lightweight task resembles the work function that you provide to the Windows API CreateThread function. Therefore, lightweight tasks are useful when you adapt existing code to use the scheduling functionality of the Concurrency Runtime. For an example that demonstrates how to adapt existing code to use a lightweight task, see Walkthough: Adapting Existing Code to Use Lightweight Tasks.
For most tasks, we recommend that you use more robust functionality such as task groups and parallel algorithms because they let you more easily break complex tasks into more basic ones. For more information about task groups, see Task Parallelism. For more information about parallel algorithms, see Parallel Algorithms.
To create a lightweight task, call the ScheduleGroup::ScheduleTask, CurrentScheduler::ScheduleTask, or Scheduler::ScheduleTask method. To wait for a lightweight task to finish, wait for the parent scheduler to shut down or use a synchronization mechanism such as an event object.
[go to top]
The default scheduler creates the same number of threads as there are available hardware threads. You can use oversubscription to create additional threads for a given hardware thread.
For computationally intensive operations, oversubscription typically does not scale because it introduces additional overhead. However, for tasks that have a high amount of latency, for example, reading data from disk or from a network connection, oversubscription can improve the overall efficiency of some applications.
To enable oversubscription in the current context, call the Context::Oversubscribe method with the _BeginOversubscription parameter set to true. When you enable oversubscription on a thread that was created by the Concurrency Runtime, it causes the runtime to create one additional thread. After all tasks that require oversubscription finish, call Context::Oversubscribe with the _BeginOversubscription parameter set to false.
You can enable oversubscription multiple times from the current context, but you must disable it the same number of times that you enable it. Oversubscription can also be nested; that is, a task that is created by another task that uses oversubscription can also oversubscribe its context. However, if both a nested task and its parent belong to the same context, only the outermost call to Context::Oversubscribe causes the creation of an additional thread.
[go to top]
The wait function suspends the execution of the current context for a specified number of milliseconds. After the specified time has elapsed, the runtime reschedules the context for execution. Therefore, the wait function might suspend the current context longer than the value provided for the milliseconds parameter.
Passing 0 (zero) for the milliseconds parameter causes the runtime to suspend the current context until all other active contexts are given the opportunity to perform work. This lets you yield a task to all other active tasks.
The wait function cooperatively yields the current context. The runtime uses the yield time to perform other tasks. For an example that uses the wait function to yield the current context, and thus allow other contexts to run, see How-to: Use Schedule Groups to Influence Order of Execution.
[go to top]