Merging Existing DSC Files
After a DSC file set has been sealed, it is not possible to add more files. If you want to create a file set containing the files from two or more existing file sets, you will need to create a completely new file set.
There are three approaches to this, depending on your scenario. The first option is to use the DSC APIs.
string fileSetA = ...;
string fileSetB = ...;
string fileSetAandB = ...;
DscService dsc = new DscService(SampleConfiguration.HeadNode);
DscFileSet result = dsc.CreateFileSet(fileSetAandB, DscCompressionScheme.None);
foreach (var file in dsc.GetFileSet(fileSetA).GetFiles())
result.AddExistingFile(file);
foreach (var file in dsc.GetFileSet(fileSetB).GetFiles())
result.AddExistingFile(file);
result.Seal();
The second option is to use a query containing a Concat operator to do this.
context.FromDsc<LineRecord>(fileSetA)
.Concat(context.FromDsc<LineRecord>(fileSetB))
.ToDsc(fileSetAandB)
.SubmitAndWait();
In most cases the Concat operator is the simplest query to use. However there is a third option, to do this within a query using the Union operator.
context.FromDsc<LineRecord>(fileSetA)
.Union(context.FromDsc<LineRecord>(fileSetB),
new FalseComparer()).ToDsc(fileSetAandB)
.SubmitAndWait();
Note |
|---|
| If you want to control the number of DSC files that the new file set will use to store the combined records, use the HashPartition operator before the ToDsc operator. |
In this case, FalseComparer always returns a false comparison so that the union operation does not remove identical records.
[Serializable]
public class FalseComparer : IEqualityComparer<LineRecord>
{
public bool Equals(LineRecord x, LineRecord y)
{
return false;
}
public int GetHashCode(LineRecord obj)
{
return obj.GetHashCode();
}
}
Note |
|---|
| The Union operator does not preserve the order of the records in the file set. Concat does preserve the ordering. |
Show:
Note