Querying DSC File Sets
This section describes some of the most common ways to use LINQ to HPC to query data that resides in a DSC file set. See Creating DSC File Sets for information about how to create DSC file sets from existing data. If you want to learn about more complex queries, see Implementing Distributed Algorithms by Using LINQ to HPC.
|
Scenario |
To query the DSC file set data… |
|
You have a DSC file set that is made up of lines of text. For example, you want to analyze web server log files. |
Use the FromDsc method, and use LineRecord as the type parameter. For more information, see Using the LineRecord Class to Query a Text-Based File Set. |
|
You have a DSC file set that is in binary format. It contains serialized .NET objects. For example, you have adapted an existing data parallel .NET program to cluster computing. |
Use the FromDsc method, and use a type parameter that matches the type that was originally used to save the data. For more information, see Querying a Binary DSC File Set. |
|
You want to extract the results of a LINQ to HPC query into a locally running application. |
Enumerate the values of the query. For more information, see Enumerating the Results of a LINQ to HPC Query. |
|
You want to transform the data in a DSC file set, and then save the results as a new file set. |
Use the FromDsc method to read the input file set. Use the Select operator to process each item. Finally, use the ToDsc method to save the results as a new file set. See Saving the Results of a LINQ to HPC Query to a New DSC File Set. |
You can use almost any LINQ operator in a LINQ to HPC query. The following table lists the supported query operators.
|
Operator Type |
Operator |
|
Where |
|
|
Select, SelectMany |
|
|
Take, Skip, TakeWhile, SkipWhile |
|
|
Join, GroupJoin |
|
|
OrderBy, OrderByDescending, Reverse |
|
|
GroupBy |
|
|
Concat, Distinct, Union, Intersect, Except |
|
|
AsEnumerable, ToArray, ToList, ToDictionary, ToLookup, Cast |
|
|
First, FirstOrDefault, Last, LastOrDefault, Single, SingleOrDefault, DefaultIfEmpty |
|
|
Count, LongCount, Sum, Max, Min, Average, Aggregate, GroupBy |
|
|
All, Any, Contains |
Note |
|---|
| The following LINQ operators cannot be used in LINQ to HPC queries: ElementAt, ElementAtOrDefault, OfType, SequenceEqual, ThenBy, ThenByDescending. |
Unlike LINQ, LINQ to HPC allows delayed execution for scalar queries. LINQ to HPC supports the following operators that return queries instead of immediate results. See Creating Free-Form Compound Queries.
-
AggregateAsQuery
-
AllAsQuery
-
AnyAsQuery
-
AverageAsQuery
-
ContainsAsQuery
-
CountAsQuery
-
FirstAsQuery
-
FirstOrDefaultAsQuery
-
LastAsQuery
-
LastOrDefaultAsQuery
-
LongCountAsQuery
-
MaxAsQuery
-
MinAsQuery
-
SingleAsQuery
-
SingleOrDefaultAsQuery
-
SumAsQuery
Note