Merging Existing DSC Files

After a DSC file set has been sealed, it is not possible to add more files. If you want to create a file set containing the files from two or more existing file sets, you will need to create a completely new file set.

There are three approaches to this, depending on your scenario. The first option is to use the DSC APIs.

string fileSetA = ...;
string fileSetB = ...;
string fileSetAandB = ...;

DscService dsc = new DscService(SampleConfiguration.HeadNode);
DscFileSet result = dsc.CreateFileSet(fileSetAandB, DscCompressionScheme.None);
foreach (var file in dsc.GetFileSet(fileSetA).GetFiles())
    result.AddExistingFile(file);
foreach (var file in dsc.GetFileSet(fileSetB).GetFiles())
    result.AddExistingFile(file);
result.Seal();

The second option is to use a query containing a Concat operator to do this.

context.FromDsc<LineRecord>(fileSetA)
    .Concat(context.FromDsc<LineRecord>(fileSetB))
    .ToDsc(fileSetAandB)
    .SubmitAndWait();

In most cases the Concat operator is the simplest query to use. However there is a third option, to do this within a query using the Union operator.

context.FromDsc<LineRecord>(fileSetA)
    .Union(context.FromDsc<LineRecord>(fileSetB),
            new FalseComparer()).ToDsc(fileSetAandB)
    .SubmitAndWait();

Note

If you want to control the number of DSC files that the new file set will use to store the combined records, use the HashPartition operator before the ToDsc operator.

In this case, FalseComparer always returns a false comparison so that the union operation does not remove identical records.

[Serializable]
public class FalseComparer : IEqualityComparer<LineRecord>
{
    public bool Equals(LineRecord x, LineRecord y)
    {
        return false;
    }

    public int GetHashCode(LineRecord obj)
    {
        return obj.GetHashCode();
    }
}

Note

The Union operator does not preserve the order of the records in the file set. Concat does preserve the ordering.