How to: Deliver Changes in Batches (SQL Server)

Article
09/29/2009

This topic describes how to deliver changes in batches for database synchronization in Sync Framework that uses SqlSyncProvider, SqlCeSyncProvider, or DbSyncProvider. The code in this topic focuses on the following Sync Framework classes:

Understanding Batching

By default, Sync Framework delivers changes to each node in a single DataSet object. This object is held in memory as changes are applied to a node. The default behavior works well if there is sufficient memory on the computer where changes are applied and the connection to the computer is reliable. Some applications, however, can benefit from having changes divided into batches. Consider the following scenario for a synchronization application:

A large number of clients that use SqlCeSyncProvider synchronize periodically with a server that uses SqlSyncProvider.
Each client has a limited amount of memory and disk space.
The connections between the server and clients are low bandwidth and intermittent, often resulting in long synchronization times and dropped connections.
The size of the changes (in KB) for a typical synchronization session is large.

Batching changes is ideal for this type of scenario because it provides the following capabilities:

Enables the developer to control the amount of memory (the memory data cache size) that is used to store changes on the client. This can eliminate out-of-memory errors on the client.
Enables Sync Framework to restart a failed synchronization operation from the start of the current batch, rather than the start of the entire set of changes.
Can reduce or eliminate the need to re-download changes or re-enumerate changes on the server due to failed operations.

Batching is simple to configure for 2-tier and n-tier applications, and it can be used for the initial synchronization session and for subsequent sessions.

Configuring and Using Batching

Batching in Sync Framework works as follows:

The application specifies the memory data cache size for each provider that is participating in the synchronization session.

If both providers specify a cache size, Sync Framework uses the smaller value for both providers. The actual cache size will be no more than 110% of the smallest specified size. During a synchronization session, if a single row is greater than 110% of the size the session terminates with an exception.

A value of 0 (the default) disables batching. If one provider has batching enabled, and the other provider does not, batching is enabled for both upload and download.
The application specifies the location of the spooling files for each provider. By default, spooling files are written to the temp directory for the account under which the synchronization process runs.
The application calls Synchronize.
Sync Framework enumerates changes one row at a time. If the memory data cache size for the source provider is reached, changes are persisted to a local spooling file, and the in-memory data is flushed. This process continues until all changes are enumerated.
For n-tier scenarios, service and proxy code in the application streams the spooling files to the destination. For more information, see Code Specific to N-Tier in this topic. For two-tier scenarios, the local file is already at the destination because in this case all synchronization code runs at the destination.
Sync Framework de-serializes changes from the spooling files and applies those changes. This process continues until all changes are applied to the destination.

All batches are applied in one transaction. That transaction is not created until the last batch is received by the destination provider.
For two-tier scenarios, Sync Framework cleans up the spooling file. For n-tier scenarios, Sync Framework cleans up spooling files on the computer on which synchronization is initiated, but files on the middle tier should be cleaned up by the proxy (demonstrated in the sample Cleanup() method later in this topic). To handle cases in which a session is aborted, the middle tier should also use a process to clean up files that are older than a certain date.

Note

The data changes that will be applied to a node are available from the Context property of the DbChangesSelectedEventArgs object. When data is not batched, the ChangesSelected event fires only one time, and all changes are available from the Context property. When data is batched, ChangesSelected fires for each batch and only the changes from that batch are available at that time. If you require changes from all batches, respond to each ChangesSelected event and store the data that is returned.

The following table describes the types and members that are related to batching. The only property required for batching is MemoryDataCacheSize, but it is also recommended to set BatchingDirectory.

Type or Member	Description
BatchingDirectory	Gets or sets the directory in which batch files are spooled to disk. The path specified must be a directory that is local to the provider or proxy that is executing. UNC file paths and non-file URI paths are not supported. Important Spooling files contain raw database data. The directory to which files are written must be protected with the appropriate access controls.
CleanupBatchingDirectory	Gets or sets whether to clean up batching files after the changes in the files have been applied to the destination. The default is to clean up the files.
MemoryDataCacheSize	Gets or sets the maximum amount of memory, in KB, that Sync Framework uses to cache changes before spooling those changes to disk. Note This setting affects only the size of the data and metadata that are held in memory for changes that are sent to the destination. It does not limit the memory used by other Sync Framework components or user application components.
BatchApplied	The event that occurs after each batch of changes has been applied to the destination.
BatchSpooled	The event that occurs after each batch of changes has been written to disk.
DbBatchAppliedEventArgs	Provides data for the BatchApplied event, including the current batch number and the total number of batches to apply.
DbBatchSpooledEventArgs	Provides data for the BatchSpooled event, including the current batch number and batch size.
BatchFileName	Gets or sets the name of the file to which spooled changes are written.
IsDataBatched	Gets or sets whether data is sent in multiple batches or in a single DataSet object.
IsLastBatch	Gets or sets whether the current batch is the last batch of changes.
BatchedDeletesRetried	Gets or sets the number of delete operations that were retried during a synchronization session in which changes were batched. Deletes are retried for batches because of the ordering of primary key and foreign key deletes. If a foreign key delete does not exist in the current batch or an earlier batch, the corresponding primary key delete fails. Failed deletes are retried once after all batches are applied.
SelectIncrementalChangesCommand (relevant only for DbSyncProvider)	Gets or sets the query or stored procedure that is used to select incremental changes from the local database. Note It is recommended that the query that is specified include the clause `ORDER BY [sync_row_timestamp]`. Ordering rows by timestamp value ensures that if a synchronization session is restarted, the provider will begin to enumerate from the highest timestamp watermark (individual table watermarks are persisted with each batch) and not miss any changes.
DataTable	Gets or sets the DataTable object that contains the changes to be synchronized. If batching is enabled, accessing this property de-serializes the spooled file from disk. Any changes made to the tables are then persisted back to the spooled file.
DataSet	Gets or sets a DataSet object that contains the selected rows from the peer database. Returns null if IsDataBatched is true.

Common Code for Two-Tier and N-Tier

The code examples in this section demonstrate how to handle batching in 2-tier and n-tier scenarios. This code is taken from two of the samples that are included in the Sync Framework SDK: SharingAppDemo-CEProviderEndToEnd and WebSharingAppDemo-CEProviderEndToEnd. Each example is introduced with the location of the code, such as SharingAppDemo/CESharingForm. In terms of batching, the key difference between the two applications is the additional code required in the n-tier case to upload and download the spooled files and to create directories for each node that enumerates changes.

The following code example from the synchronizeBtn_Click event handler in SharingAppDemo/CESharingForm sets the memory data cache size and the directory to which spooling files should be written. The path specified for BatchingDirectory must be a directory that is local to the provider or proxy that is executing. UNC file paths and non-file URI paths are not supported. The path specified for BatchingDirectory is the root directory. For each synchronization session, Sync Framework creates a unique subdirectory in which to store spooling files for that session. This directory is unique for the current source-destination combination to isolate files for different sessions.

The following code example from the synchronizeBtn_Click event handler in WebSharingAppDemo/CESharingForm sets the same properties, but the batching directory for the destination is set for the proxy, rather than directly for the provider as it is in the 2-tier scenario:

//Set memory data cache size property. 0 represents non batched mode.
//No need to set memory cache size for Proxy, because the source is 
//enabled for batching: both upload and download will be batched.
srcProvider.MemoryDataCacheSize = this._batchSize;
 

//Set batch spool location. Default value if not set is %Temp% directory.
if (!string.IsNullOrEmpty(this.batchSpoolLocation.Text))
{
    srcProvider.BatchingDirectory = this.batchSpoolLocation.Text;
    destinationProxy.BatchingDirectory = this.batchSpoolLocation.Text;
}

The following code examples from the SynchronizationHelper file in both applications create methods to handle the BatchSpooled and BatchAppliedEvents that are raised by a provider during change enumeration and change application:

void provider_BatchSpooled(object sender, DbBatchSpooledEventArgs e)
{
    this.progressForm.listSyncProgress.Items.Add("BatchSpooled event fired: Details");
    this.progressForm.listSyncProgress.Items.Add("\tSource Database :" + ((RelationalSyncProvider)sender).Connection.Database);
    this.progressForm.listSyncProgress.Items.Add("\tBatch Name      :" + e.BatchFileName);
    this.progressForm.listSyncProgress.Items.Add("\tBatch Size      :" + e.DataCacheSize);
    this.progressForm.listSyncProgress.Items.Add("\tBatch Number    :" + e.CurrentBatchNumber);
    this.progressForm.listSyncProgress.Items.Add("\tTotal Batches   :" + e.TotalBatchesSpooled);
    this.progressForm.listSyncProgress.Items.Add("\tBatch Watermark :" + ReadTableWatermarks(e.CurrentBatchTableWatermarks));
}

void provider_BatchApplied(object sender, DbBatchAppliedEventArgs e)
{
    this.progressForm.listSyncProgress.Items.Add("BatchApplied event fired: Details");
    this.progressForm.listSyncProgress.Items.Add("\tDestination Database   :" + ((RelationalSyncProvider)sender).Connection.Database);
    this.progressForm.listSyncProgress.Items.Add("\tBatch Number           :" + e.CurrentBatchNumber);
    this.progressForm.listSyncProgress.Items.Add("\tTotal Batches To Apply :" + e.TotalBatchesToApply);
}

//Reads the watermarks for each table from the batch spooled event. //The watermark denotes the max tickcount for each table in each batch.
private string ReadTableWatermarks(Dictionary<string, ulong> dictionary)
{
    StringBuilder builder = new StringBuilder();
    Dictionary<string, ulong> dictionaryClone = new Dictionary<string, ulong>(dictionary);
    foreach (KeyValuePair<string, ulong> kvp in dictionaryClone)
    {
        builder.Append(kvp.Key).Append(":").Append(kvp.Value).Append(",");
    }
    return builder.ToString();
}

Code Specific to N-Tier

The remainder of the code examples apply only to the n-tier scenario in WebSharingAppDemo. The relevant n-tier code is contained in three files:

The service contract: IRelationalSyncContract
The Web service: RelationalWebSyncService
The proxy: RelationalProviderProxy

The two providers SqlSyncProvider and SqlCeSyncProvider both inherit from RelationalSyncProvider, so this code applies to both providers. Additional store-specific functionality is separated into proxy and service files for each type of provider.

To understand how batching works in an n-tier scenario, consider a synchronization session in which the server is the source and the client is the destination. After changes have been written to the local directory on the server, the following process occurs for downloaded changes:

The GetChangeBatch method is called on the client proxy. As demonstrated later in the sample code, this method should include specific code to handle batching.
The service gets a batch file from SqlSyncProvider. The service removes the complete path information and sends only the file name over the network. This prevents exposing the directory structure of the server to the clients.
The proxy call to GetChangeBatch returns.
1. The proxy detects that changes are batched so it calls DownloadBatchFile by passing the batch file name as an argument.
2. The proxy creates a unique directory (if one doesn’t exist for the session) under RelationalProviderProxy.BatchingDirectory to hold these batch files locally. The directory name is the replica ID of the peer that is enumerating changes. This ensures that the proxy and service have one unique directory for each enumerating peer.
The proxy downloads the file and stores it locally. The proxy replaces the filename in the context with the new full path to the batch file on the local disk.
The proxy returns the context back to the orchestrator.
Repeat steps 1 through 6 until the last batch is received by proxy.

The following process occurs for uploaded changes

The orchestrator calls ProcessChangeBatch on the proxy.
The proxy determines that it is a batch file, so it performs the following steps:
1. Removes the complete path information and sends only the file name over the network.
2. Calls HasUploadedBatchFile to determine if the file has already been uploaded. If it has, step C is not necessary.
3. If HasUploadedBatchFile returns false, calls UploadBatchFile on the service, and uploads the batch file contents.
  
  The service will receive the call to UploadBatchFile and store the batch locally. Directory creation is similar to step 4 above.
4. Calls ApplyChanges on the service.
The server receives the ApplyChanges call and determines that it is a batch file. It replaces the filename in the context with the new full path to the batch file on the local disk.
The server passes the DbSyncContext to local SqlSyncProvider.
Repeat steps 1 through 6 until the last batch is sent.

The following code example from IRelationalSyncContract specifies upload and download methods that are used to transfer spooled files to and from the middle tier:

[OperationContract(IsOneWay = true)]
void UploadBatchFile(string batchFileid, byte[] batchFile);

[OperationContract]
byte[] DownloadBatchFile(string batchFileId);

The following code examples from RelationalWebSyncService expose the UploadBatchFile and DownloadBatchFile methods defined in the contract, and include additional batching related logic in the following methods:

Cleanup: cleans up any spooled files from a specified directory or the temp directory if one is not specified.
GetChanges: checks if data is batched and if so it removes the directory path of the spooled file so that the path is not sent over the network. In n-tier scenarios, it is a security risk to send full directory paths over a network connection. The file name is a GUID.
HasUploadedBatchFile: returns whether a particular batch file has already been uploaded to the service.
ApplyChanges: checks if data is batched, and if so it checks if the expected batch file has already been uploaded. If the file has not been uploaded, an exception is thrown. The client should have uploaded the spooled file prior to calling ApplyChanges.

public abstract class RelationalWebSyncService: IRelationalSyncContract
{
    protected bool isProxyToCompactDatabase;
    protected RelationalSyncProvider peerProvider;
    protected DirectoryInfo sessionBatchingDirectory = null;
    protected Dictionary<string, string> batchIdToFileMapper;
    int batchCount = 0;

    public void Initialize(string scopeName, string hostName)
    {
        this.peerProvider = this.ConfigureProvider(scopeName, hostName);
        this.batchIdToFileMapper = new Dictionary<string, string>();
    }

    public void Cleanup()
    {
        this.peerProvider = null;
        //Delete all file in the temp session directory
        if (sessionBatchingDirectory != null && sessionBatchingDirectory.Exists)
        {
            try
            {
                sessionBatchingDirectory.Delete(true);
            }
            catch 
            { 
                //Ignore 
            }
        }
    }

    public void BeginSession(SyncProviderPosition position)
    {
        Log("*****************************************************************");
        Log("******************** New Sync Session ***************************");
        Log("*****************************************************************");
        Log("BeginSession: ScopeName: {0}, Position: {1}", this.peerProvider.ScopeName, position);
        //Clean the mapper for each session.
        this.batchIdToFileMapper = new Dictionary<string, string>();

        this.peerProvider.BeginSession(position, null/*SyncSessionContext*/);
        this.batchCount = 0;
    }

    public SyncBatchParameters GetKnowledge()
    {
        Log("GetSyncBatchParameters: {0}", this.peerProvider.Connection.ConnectionString);
        SyncBatchParameters destParameters = new SyncBatchParameters();
        this.peerProvider.GetSyncBatchParameters(out destParameters.BatchSize, out destParameters.DestinationKnowledge);
        return destParameters;
    }

    public GetChangesParameters GetChanges(uint batchSize, SyncKnowledge destinationKnowledge)
    {
        Log("GetChangeBatch: {0}", this.peerProvider.Connection.ConnectionString);
        GetChangesParameters changesWrapper = new GetChangesParameters();
        changesWrapper.ChangeBatch  = this.peerProvider.GetChangeBatch(batchSize, destinationKnowledge, out changesWrapper.DataRetriever);

        DbSyncContext context = changesWrapper.DataRetriever as DbSyncContext;
        //Check to see if data is batched
        if (context != null && context.IsDataBatched)
        {
            Log("GetChangeBatch: Data Batched. Current Batch #:{0}", ++this.batchCount);
            //Dont send the file location info. Just send the file name
            string fileName = new FileInfo(context.BatchFileName).Name;
            this.batchIdToFileMapper[fileName] = context.BatchFileName;
            context.BatchFileName = fileName;
        }
        return changesWrapper;
    }

    public SyncSessionStatistics ApplyChanges(ConflictResolutionPolicy resolutionPolicy, ChangeBatch sourceChanges, object changeData)
    {
        Log("ProcessChangeBatch: {0}", this.peerProvider.Connection.ConnectionString);

        DbSyncContext dataRetriever = changeData as DbSyncContext;

        if (dataRetriever != null && dataRetriever.IsDataBatched)
        {
            string remotePeerId = dataRetriever.MadeWithKnowledge.ReplicaId.ToString();
            //Data is batched. The client should have uploaded this file to us prior to calling ApplyChanges.
            //So look for it.
            //The Id would be the DbSyncContext.BatchFileName which is just the batch file name without the complete path
            string localBatchFileName = null;
            if (!this.batchIdToFileMapper.TryGetValue(dataRetriever.BatchFileName, out localBatchFileName))
            {
                //Service has not received this file. Throw exception
                throw new FaultException<WebSyncFaultException>(new WebSyncFaultException("No batch file uploaded for id " + dataRetriever.BatchFileName, null));
            }
            dataRetriever.BatchFileName = localBatchFileName;
        }

        SyncSessionStatistics sessionStatistics = new SyncSessionStatistics();
        this.peerProvider.ProcessChangeBatch(resolutionPolicy, sourceChanges, changeData, new SyncCallbacks(), sessionStatistics);
        return sessionStatistics;
    }

    public void EndSession()
    {
        Log("EndSession: {0}", this.peerProvider.Connection.ConnectionString);
        Log("*****************************************************************");
        Log("******************** End Sync Session ***************************");
        Log("*****************************************************************");
        this.peerProvider.EndSession(null);
        Log("");
    }

    /// <summary>
    /// Used by proxy to see if the batch file has already been uploaded. Optimizes by not resending batch files.
    /// NOTE: This method takes in a file name as an input parameter and hence is suseptible for name canonicalization
    /// attacks. This sample is meant to be a starting point in demonstrating how to transfer sync batch files and is
    /// not intended to be a secure way of doing the same. This SHOULD NOT be used as such in production environment
    /// without doing proper security analysis.
    /// 
    /// Please refer to the following two MSDN whitepapers for more information on guidelines for securing Web servies.
    /// 
    /// Design Guidelines for Secure Web Applications - https://msdn.microsoft.com/en-us/library/aa302420.aspx (Refer InputValidation section)
    /// Architecture and Design Review for Security - https://msdn.microsoft.com/en-us/library/aa302421.aspx (Refer InputValidation section)
    /// </summary>
    /// <param name="batchFileId"></param>
    /// <returns>bool</returns>
    public bool HasUploadedBatchFile(String batchFileId, string remotePeerId)
    {
        this.CheckAndCreateBatchingDirectory(remotePeerId);

        //The batchFileId is the fileName without the path information in it.
        FileInfo fileInfo = new FileInfo(Path.Combine(this.sessionBatchingDirectory.FullName, batchFileId));
        if (fileInfo.Exists && !this.batchIdToFileMapper.ContainsKey(batchFileId))
        {
            //If file exists but is not in the memory id to location mapper then add it to the mapping
            this.batchIdToFileMapper.Add(batchFileId, fileInfo.FullName);
        }
        //Check to see if the proxy has already uploaded this file to the service
        return fileInfo.Exists;
    }

    /// <summary>
    /// NOTE: This method takes in a file name as an input parameter and hence is suseptible for name canonicalization
    /// attacks. This sample is meant to be a starting point in demonstrating how to transfer sync batch files and is
    /// not intended to be a secure way of doing the same. This SHOULD NOT be used as such in production environment
    /// without doing proper security analysis.
    /// 
    /// Please refer to the following two MSDN whitepapers for more information on guidelines for securing Web servies.
    /// 
    /// Design Guidelines for Secure Web Applications - https://msdn.microsoft.com/en-us/library/aa302420.aspx (Refer InputValidation section)
    /// Architecture and Design Review for Security - https://msdn.microsoft.com/en-us/library/aa302421.aspx (Refer InputValidation section)
    /// </summary>
    /// <param name="batchFileId"></param>
    /// <param name="batchContents"></param>
    /// <param name="remotePeerId"></param>
    public void UploadBatchFile(string batchFileId, byte[] batchContents, string remotePeerId)
    {
        Log("UploadBatchFile: {0}", this.peerProvider.Connection.ConnectionString);
        try
        {
            if (HasUploadedBatchFile(batchFileId, remotePeerId))
            {
                //Service has already received this file. So dont save it again.
                return;
            }
            
            //Service hasnt seen the file yet so save it.
            String localFileLocation = Path.Combine(sessionBatchingDirectory.FullName, batchFileId);
            FileStream fs = new FileStream(localFileLocation, FileMode.Create, FileAccess.Write);
            using (fs)
            {
                    fs.Write(batchContents, 0, batchContents.Length);
            }
            //Save this Id to file location mapping in the mapper object
            this.batchIdToFileMapper[batchFileId] = localFileLocation;
        }
        catch (Exception e)
        {
            throw new FaultException<WebSyncFaultException>(new WebSyncFaultException("Unable to save batch file.", e));
        }
    }

    /// <summary>
    /// NOTE: This method takes in a file name as an input parameter and hence is suseptible for name canonicalization
    /// attacks. This sample is meant to be a starting point in demonstrating how to transfer sync batch files and is
    /// not intended to be a secure way of doing the same. This SHOULD NOT be used as such in production environment
    /// without doing proper security analysis.
    /// 
    /// Please refer to the following two MSDN whitepapers for more information on guidelines for securing Web servies.
    /// 
    /// Design Guidelines for Secure Web Applications - https://msdn.microsoft.com/en-us/library/aa302420.aspx (Refer InputValidation section)
    /// Architecture and Design Review for Security - https://msdn.microsoft.com/en-us/library/aa302421.aspx (Refer InputValidation section)
    /// </summary>
    /// <param name="batchFileId"></param>
    /// <returns></returns>
    public byte[] DownloadBatchFile(string batchFileId)
    {
        try
        {
            Log("DownloadBatchFile: {0}", this.peerProvider.Connection.ConnectionString);
            Stream localFileStream = null;

            string localBatchFileName = null;

            if (!this.batchIdToFileMapper.TryGetValue(batchFileId, out localBatchFileName))
            {
                throw new FaultException<WebSyncFaultException>(new WebSyncFaultException("Unable to retrieve batch file for id." + batchFileId, null));
            }

            localFileStream = new FileStream(localBatchFileName, FileMode.Open, FileAccess.Read);
            byte[] contents = new byte[localFileStream.Length];
            localFileStream.Read(contents, 0, contents.Length);
            return contents;
        }
        catch (Exception e)
        {
            throw new FaultException<WebSyncFaultException>(new WebSyncFaultException("Unable to read batch file for id " + batchFileId, e));
        }
    }

    protected void Log(string p, params object[] paramArgs)
    {
        Console.WriteLine(p, paramArgs);
    }

    //Utility functions that the sub classes need to implement.
    protected abstract RelationalSyncProvider ConfigureProvider(string scopeName, string hostName);


    private void CheckAndCreateBatchingDirectory(string remotePeerId)
    {
        //Check to see if we have temp directory for this session.
        if (sessionBatchingDirectory == null)
        {
            //Generate a unique Id for the directory
            //We use the peer id of the store enumerating the changes so that the local temp directory is same for a given source
            //across sync sessions. This enables us to restart a failed sync by not downloading already received files.
            string sessionDir = Path.Combine(this.peerProvider.BatchingDirectory, "WebSync_" + remotePeerId);
            sessionBatchingDirectory = new DirectoryInfo(sessionDir);
            //Create the directory if it doesnt exist.
            if (!sessionBatchingDirectory.Exists)
            {
                sessionBatchingDirectory.Create();
            }
        }
    }
}

The following code examples from RelationalProviderProxy set properties and call methods on the Web service:

BatchingDirectory: enables the application to set the batching directory for the middle tier.
EndSession: cleans up any spooled files from a specified directory.
GetChangeBatch: downloads change batches by calling the DownloadBatchFile method.
ProcessChangeBatch: uploads change batches by calling the UploadBatchFile method.

public abstract class RelationalProviderProxy : KnowledgeSyncProvider, IDisposable
{
    protected IRelationalSyncContract proxy;
    protected SyncIdFormatGroup idFormatGroup;
    protected string scopeName;
    protected DirectoryInfo localBatchingDirectory;

    //Represents either the SQL server host name or the CE database file name. Sql database name
    //is always peer1
    //For this sample scopeName is always Sales
    protected string hostName;

    private string batchingDirectory = Environment.ExpandEnvironmentVariables("%TEMP%");

    public string BatchingDirectory
    {
        get { return batchingDirectory; }
        set 
        {
            if (string.IsNullOrEmpty(value))
            {
                throw new ArgumentException("value cannot be null or empty");
            }
            try
            {
                Uri uri = new Uri(value);
                if (!uri.IsFile || uri.IsUnc)
                {
                    throw new ArgumentException("value must be a local directory");
                }
                batchingDirectory = value;
            }
            catch (Exception e)
            {
                throw new ArgumentException("Invalid batching directory.", e);
            }
        }
    }

    public RelationalProviderProxy(string scopeName, string hostName)
    {
        this.scopeName = scopeName;
        this.hostName = hostName;
        this.CreateProxy();            
        this.proxy.Initialize(scopeName, hostName);
    }

    public override void BeginSession(SyncProviderPosition position, SyncSessionContext syncSessionContext)
    {
        this.proxy.BeginSession(position);
    }

    public override void EndSession(SyncSessionContext syncSessionContext)
    {
        proxy.EndSession();
        if (this.localBatchingDirectory != null && this.localBatchingDirectory.Exists)
        {
            //Cleanup batching releated files from this session
            this.localBatchingDirectory.Delete(true);
        }
    }

    public override ChangeBatch GetChangeBatch(uint batchSize, SyncKnowledge destinationKnowledge, out object changeDataRetriever)
    {
        GetChangesParameters changesWrapper = proxy.GetChanges(batchSize, destinationKnowledge);
        //Retrieve the ChangeDataRetriever and the ChangeBatch
        changeDataRetriever = changesWrapper.DataRetriever;

        DbSyncContext context = changeDataRetriever as DbSyncContext;
        //Check to see if the data is batched.
        if (context != null && context.IsDataBatched)
        {
            if (this.localBatchingDirectory == null)
            {
                //Retrieve the remote peer id from the MadeWithKnowledge.ReplicaId. MadeWithKnowledge is the local knowledge of the peer 
                //that is enumerating the changes.
                string remotePeerId = context.MadeWithKnowledge.ReplicaId.ToString();

                //Generate a unique Id for the directory.
                //We use the peer id of the store enumerating the changes so that the local temp directory is same for a given source
                //across sync sessions. This enables us to restart a failed sync by not downloading already received files.
                string sessionDir = Path.Combine(this.batchingDirectory, "WebSync_" + remotePeerId);
                this.localBatchingDirectory = new DirectoryInfo(sessionDir);
                //Create the directory if it doesnt exist.
                if (!this.localBatchingDirectory.Exists)
                {
                    this.localBatchingDirectory.Create();
                }
            }

            string localFileName = Path.Combine(this.localBatchingDirectory.FullName, context.BatchFileName);
            FileInfo localFileInfo = new FileInfo(localFileName);
            
            //Download the file only if doesnt exist
            FileStream localFileStream = new FileStream(localFileName, FileMode.Create, FileAccess.Write);
            if (!localFileInfo.Exists)
            {
                byte[] remoteFileContents = this.proxy.DownloadBatchFile(context.BatchFileName);
                using (localFileStream)
                {
                    localFileStream.Write(remoteFileContents, 0, remoteFileContents.Length);
                }
            }
            //Set DbSyncContext.Batchfile name to the new local file name
            context.BatchFileName = localFileName;
        }

        return changesWrapper.ChangeBatch;
    }

    public override FullEnumerationChangeBatch GetFullEnumerationChangeBatch(uint batchSize, SyncId lowerEnumerationBound, SyncKnowledge knowledgeForDataRetrieval, out object changeDataRetriever)
    {
        throw new NotImplementedException();
    }

    public override void GetSyncBatchParameters(out uint batchSize, out SyncKnowledge knowledge)
    {
        SyncBatchParameters wrapper = proxy.GetKnowledge();
        batchSize = wrapper.BatchSize;
        knowledge = wrapper.DestinationKnowledge;
    }

    public override SyncIdFormatGroup IdFormats
    {
        get
        {
            if (idFormatGroup == null)
            {
                idFormatGroup = new SyncIdFormatGroup();

                //
                // 1 byte change unit id (Harmonica default before flexible ids)
                //
                idFormatGroup.ChangeUnitIdFormat.IsVariableLength = false;
                idFormatGroup.ChangeUnitIdFormat.Length = 1;

                //
                // Guid replica id
                //
                idFormatGroup.ReplicaIdFormat.IsVariableLength = false;
                idFormatGroup.ReplicaIdFormat.Length = 16;


                //
                // Sync global id for item ids
                //
                idFormatGroup.ItemIdFormat.IsVariableLength = true;
                idFormatGroup.ItemIdFormat.Length = 10 * 1024;
            }

            return idFormatGroup;
        }
    }

    public override void ProcessChangeBatch(ConflictResolutionPolicy resolutionPolicy, ChangeBatch sourceChanges, object changeDataRetriever, SyncCallbacks syncCallbacks, SyncSessionStatistics sessionStatistics)
    {
        DbSyncContext context = changeDataRetriever as DbSyncContext;
        if (context != null && context.IsDataBatched)
        {
            string fileName = new FileInfo(context.BatchFileName).Name;

            //Retrieve the remote peer id from the MadeWithKnowledge.ReplicaId. MadeWithKnowledge is the local knowledge of the peer 
            //that is enumerating the changes.
            string peerId = context.MadeWithKnowledge.ReplicaId.ToString();

            //Check to see if service already has this file
            if (!this.proxy.HasUploadedBatchFile(fileName, peerId))
            {
                //Upload this file to remote service
                FileStream stream = new FileStream(context.BatchFileName, FileMode.Open, FileAccess.Read);
                byte[] contents = new byte[stream.Length];
                using (stream)
                {
                    stream.Read(contents, 0, contents.Length);
                }
                this.proxy.UploadBatchFile(fileName, contents, peerId);
            }

            context.BatchFileName = fileName;
        }
        this.proxy.ApplyChanges(resolutionPolicy, sourceChanges, changeDataRetriever);
    }

    public override void ProcessFullEnumerationChangeBatch(ConflictResolutionPolicy resolutionPolicy, FullEnumerationChangeBatch sourceChanges, object changeDataRetriever, SyncCallbacks syncCallbacks, SyncSessionStatistics sessionStatistics)
    {
        throw new NotImplementedException();
    }

    protected abstract void CreateProxy();

    #region IDisposable Members

    public void Dispose()
    {
        this.proxy.Cleanup();
        this.proxy = null;
        GC.SuppressFinalize(this);
    }

    #endregion
}

How to: Deliver Changes in Batches (SQL Server)

Understanding Batching

Configuring and Using Batching

Common Code for Two-Tier and N-Tier

Code Specific to N-Tier

See Also

Concepts

Additional resources