How to: Filter a Replica

Article
09/29/2009

This topic describes how to use a managed language to implement a Sync Framework provider that represents a filtered replica. A filtered replica stores data only for items or change units that are in its filter.

This topic assumes a basic familiarity with C# and Microsoft .NET Framework concepts.

The examples in this topic focus on the following Sync Framework class and member:

Understanding Filtered Replicas

A filtered replica stores item and change unit data only for the items and change units that are in its filter, as well as ghosts, which are metadata for items and change units that were in the filter but have moved out. A filtered replica also tracks its filter, and may track other filters as well. A filtered replica can negotiate a filter with the source provider, in which case the source provider produces a filtered change batch. If the source provider cannot produce a filtered change batch, the filtered provider can filter the changes itself and apply only those that are in its filter.

A filtered provider implements IFilteredReplicaNotifyingChangeApplierTarget to communicate with the change applier about items that have moved in relation to the filter. A filtered provider also typically implements IRequestFilteredSync so that it can negotiate the filter that the source provider uses for enumerating changes.

Build Requirements

.NET Framework 2.0 or later.
Reference to Microsoft.Synchronization.
Reference to Microsoft.Synchronization.MetadataStorage.

Example

The example code in this topic shows how to implement a filtered destination provider. The filtered provider requests a filtered change batch that includes ghosts, and applies the filtered changes and ghosts to the destination replica. The replica in this example is a text file that stores contact information as a list of comma-separated values. The items to synchronize are the contacts that are contained in this file. A filter is a string that causes a contact to be included only if the filter string is found in the address field of the contact.

Negotiating the Filter

When Sync Framework calls the SpecifyFilter method of the destination replica, the destination replica requests the filter that the source provider uses to enumerate changes. This example specifies the first filter in the list of filters tracked by the destination replica and throws an exception if the source provider rejects the filter.

public void SpecifyFilter(FilterRequestCallback filterRequest)
{
    // Use the first tracked filter as the filter for sync.
    if (0 < _ContactStore.TrackedFilters.Count)
    {
        _filterForSync = _ContactStore.TrackedFilters[0];
    }

    // The source provider must agree to send a filtered change batch.
    if (!filterRequest(_filterForSync, FilteringType.CurrentItemsAndVersionsForMovedOutItems))
    {
        throw new SyncInvalidOperationException("Filter specified by SpecifyFilter was rejected.");
    }
}

Applying Filtered Changes

The destination provider uses a change applier to process the change batch. The metadata storage service does not support custom filtering, so the list of local versions must be updated to mark ghost changes before the list is sent to the change applier.

public override void ProcessChangeBatch(ConflictResolutionPolicy resolutionPolicy, ChangeBatch sourceChanges, object changeDataRetriever, SyncCallbacks syncCallbacks, SyncSessionStatistics sessionStatistics)
{
    // Use the metadata storage service to get the local versions of changes received from the source provider.
    IEnumerable<ItemChange> localVersions = _ContactStore.ContactReplicaMetadata.GetLocalVersions(sourceChanges);
    // Copy and fix up the local version list to include ghost information.
    List<ItemChange> fixedLocalVersions = new List<ItemChange>();
    ChangeKind fixedChangeKind;
    foreach (ItemChange localVersion in localVersions)
    {
        fixedChangeKind = localVersion.ChangeKind;
        if (localVersion.ChangeKind != ChangeKind.UnknownItem && _ContactStore.IsGhost(localVersion.ItemId))
        {
            fixedChangeKind = ChangeKind.Ghost;
        }
        fixedLocalVersions.Add(new ItemChange(IdFormats, localVersion.ReplicaId, localVersion.ItemId, fixedChangeKind, localVersion.CreationVersion,
            localVersion.ChangeVersion));
    }

    // Use a NotifyingChangeApplier object to process the changes. Note that the provider object is passed as the INotifyingChangeApplierTarget
    // object that will be called to apply changes to the item store.
    NotifyingChangeApplier changeApplier = new NotifyingChangeApplier(ContactStore.ContactIdFormatGroup);
    changeApplier.ApplyChanges(resolutionPolicy, sourceChanges, (IChangeDataRetriever)changeDataRetriever,
        fixedLocalVersions, _ContactStore.ContactReplicaMetadata.GetKnowledge(), 
        _ContactStore.ContactReplicaMetadata.GetForgottenKnowledge(), this, _sessionContext, syncCallbacks);
}

The change applier calls the SaveItemChange method to save changes. A filtered replica handles change actions that affect ghosts.

case SaveChangeAction.CreateGhost:
case SaveChangeAction.UpdateGhost:
{
    try
    {
        _ContactStore.UpdateGhostFromSync(change, _filterKeyMap);
    }
    catch (Exception ex)
    {
        RecoverableErrorData errData = new RecoverableErrorData(ex);
        context.RecordRecoverableErrorForItem(errData);
    }

    break;
}

case SaveChangeAction.MarkItemAsGhost:
{
    try
    {
        // Delete the item from the contact store and update the metadata to indicate it is a ghost.
        _ContactStore.MarkItemAsGhost(change, _filterKeyMap);
    }
    catch (Exception ex)
    {
        RecoverableErrorData errData = new RecoverableErrorData(ex);
        context.RecordRecoverableErrorForItem(errData);
    }

    break;
}

case SaveChangeAction.UnmarkItemAsGhost:
{
    try
    {
        // Create the item in the contact store and update the metadata to indicate the item is not a ghost.
        _ContactStore.UnmarkItemAsGhost(change, (string)context.ChangeData, _filterKeyMap);
    }
    catch (Exception ex)
    {
        RecoverableErrorData errData = new RecoverableErrorData(ex);
        context.RecordRecoverableErrorForItem(errData);
    }

    break;
}

case SaveChangeAction.DeleteGhostAndStoreTombstone:
{
    try
    {
        _ContactStore.DeleteGhostFromSync(change.ItemId, change.ChangeVersion);
    }
    catch (Exception ex)
    {
        RecoverableErrorData errData = new RecoverableErrorData(ex);
        context.RecordRecoverableErrorForItem(errData);
    }

    break;
}

public void UpdateGhostFromSync(ItemChange itemChange, FilterKeyMap providerFilterKeyMap)
{
    // Find the ghost metadata in our list or load it from the metadata store.
    ItemMetadata itemMeta = null;
    if (_ContactGhostMetaList.ContainsKey(itemChange.ItemId))
    {
        itemMeta = _ContactGhostMetaList[itemChange.ItemId];
    }
    else
    {
        itemMeta = _ContactReplicaMetadata.FindItemMetadataById(itemChange.ItemId);
    }
    
    // The ghost does not exist, so create it and add it to the metadata store.
    if (null == itemMeta)
    {
        itemMeta = _ContactReplicaMetadata.CreateItemMetadata(itemChange.ItemId,
            itemChange.CreationVersion);

        InitializeFilterTrackingFields(itemMeta);

        // Create values for all index fields in the metadata store.
        itemMeta.SetCustomField(FirstNameField, itemChange.ItemId.ToString());
        itemMeta.SetCustomField(LastNameField, "0");
        itemMeta.SetCustomField(PhoneNumberField, "0");
    
        _ContactGhostMetaList.Add(itemMeta.GlobalId, itemMeta);
    }

    // Set the version metadata for the change unit by using the metadata storage service.
    itemMeta.ChangeVersion = itemChange.ChangeVersion;

    // Update the filter tracking metadata for filter change metadata sent from the source provider.
    for (int iFilter = 0; iFilter < _trackedFilters.Count; iFilter++)
    {
        // Get filter change metadata from the source provider for this change, if it exists.
        FilterChange filterChange = GetFilterChange(itemChange, iFilter, providerFilterKeyMap);

        // If filter change metadata is present, use it to update the item metadata.
        if (null != filterChange)
        {
            SetIsInFilter(itemMeta, iFilter, filterChange.IsMoveIn);
            SetMoveVersion(itemMeta, iFilter, filterChange.MoveVersion);
        }
    }
}

public bool IsGhost(SyncId itemId)
{
    bool isGhost = false;

    ItemMetadata itemMeta = _ContactReplicaMetadata.FindItemMetadataById(itemId);
    if (null != itemMeta)
    {
        // The item is a ghost if it is not deleted and it does not exist in the contact store.
        isGhost = (!itemMeta.IsDeleted && !_ContactList.ContainsKey(itemId));
    }

    return isGhost;
}

public void MarkItemAsGhost(ItemChange itemChange, FilterKeyMap providerFilterKeyMap)
{
    // Delete the item from the contact store.
    _ContactList.Remove(itemChange.ItemId);

    // Move the item from the active metadata list to the ghost list.
    ItemMetadata ghostMeta = _ContactItemMetaList[itemChange.ItemId];
    _ContactGhostMetaList.Add(itemChange.ItemId, ghostMeta);
    _ContactItemMetaList.Remove(itemChange.ItemId);

    // Update the filter tracking metadata for filter change metadata sent from the source provider.
    for (int iFilter = 0; iFilter < _trackedFilters.Count; iFilter++)
    {
        // Get filter change metadata from the source provider for this change, if it exists.
        FilterChange filterChange = GetFilterChange(itemChange, iFilter, providerFilterKeyMap);

        // If filter change metadata is present, use it to update the item metadata.
        if (null != filterChange)
        {
            SetIsInFilter(ghostMeta, iFilter, filterChange.IsMoveIn);
            SetMoveVersion(ghostMeta, iFilter, filterChange.MoveVersion);
        }
    }
}

public void UnmarkItemAsGhost(ItemChange itemChange, string changeData, FilterKeyMap providerFilterKeyMap)
{
    // Get the metadata for the ghost.
    ItemMetadata itemMeta = null;
    if (_ContactGhostMetaList.ContainsKey(itemChange.ItemId))
    {
        itemMeta = _ContactGhostMetaList[itemChange.ItemId];
    }
    else
    {
        itemMeta = _ContactReplicaMetadata.FindItemMetadataById(itemChange.ItemId);
    }

    if (null == itemMeta)
    {
        throw new SyncInvalidOperationException("UnmarkItemAsGhost received an item but has not metadata for the item.");
    }

    // Create a new contact and add it to the contact store.
    Contact contact = new Contact();
    _ContactList.Add(itemMeta.GlobalId, contact);
    _ContactList[itemChange.ItemId].FromString(changeData);

    // Move the metadata from the ghost list to the active list.
    _ContactItemMetaList.Add(itemMeta.GlobalId, itemMeta);
    _ContactGhostMetaList.Remove(itemMeta.GlobalId);

    // Update the metadata for the item.
    UpdateContactMetadataInternal(itemChange.ItemId, itemChange.ChangeVersion, itemChange, providerFilterKeyMap);
}

// Mark a ghost as deleted in the metadata store.
public void DeleteGhostFromSync(SyncId itemId, SyncVersion changeVersion)
{
    // Find the item in the ghost metadata list or load it from the metadata store.
    ItemMetadata itemMeta = null;
    if (_ContactGhostMetaList.ContainsKey(itemId))
    {
        itemMeta = _ContactGhostMetaList[itemId];
    }
    else
    {
        itemMeta = _ContactReplicaMetadata.FindItemMetadataById(itemId);
    }

    if (null == itemMeta)
    {
        throw new SyncInvalidOperationException("DeleteGhostFromSync received item but has no metadata.");
    }

    //Mark item as deleted.
    itemMeta.MarkAsDeleted(changeVersion);

    // Clear the index name field so it doesn't collide with future items.
    itemMeta.SetCustomField(FirstNameField, itemMeta.GlobalId.ToString());

    // Move the item to the deleted list.
    _ContactDeletedItemMetaList.Add(itemMeta);
    _ContactGhostMetaList.Remove(itemMeta.GlobalId);
}

Enumerating Items Moved into the Filter

The filtered provider implements IFilteredReplicaNotifyingChangeApplierTarget to communicate with the change applier about items that have moved in relation to the filter. This example enumerates all the items in the metadata store and adds an item to the returned list when the item is in the filter for the replica and the move version of the item is not contained in the specified base knowledge.

public IEnumerator<SyncId>  GetNewMoveInItems(SyncKnowledge baseKnowledge)
{
     List<SyncId> newMoveInIdList = new List<SyncId>();
    IEnumerable<ItemMetadata> allItems = _ContactStore.ContactReplicaMetadata.GetAllItems(false);
    SyncKnowledge mappedBaseKnowledge = _ContactStore.ContactReplicaMetadata.GetKnowledge().MapRemoteKnowledgeToLocal(baseKnowledge);
    foreach (ItemMetadata itemMeta in allItems)
    {
        FilterChange filterChange = _ContactStore.GetTrackedFilterMetadata(itemMeta, _filterForSync);
        if (filterChange.IsMoveIn)
        {
            if (!mappedBaseKnowledge.Contains(_ContactStore.ContactReplicaMetadata.ReplicaId, itemMeta.GlobalId, filterChange.MoveVersion))
            {
                newMoveInIdList.Add(itemMeta.GlobalId);
            }
        }
    }
    return newMoveInIdList.GetEnumerator();
}

Next Steps

Next, you might want to add filter negotiation to your provider so that it can communicate with the destination provider to establish which filter to use for change enumeration. For more information on how to negotiate filters, see How to: Negotiate a Filter.

You may also want to enable your provider to track filters. Filter-tracking replicas keep the knowledge size small when sending changes to filtered replicas. For more information on how to implement a filter-tracking provider, see How to: Track Filters and Enumerate Filtered Changes.

Share via