Using Proxy Providers

Many standard and custom controls do not implement native Microsoft UI Automation providers. To be accessible to UI Automation client applications, these controls must be furnished with client-side providers, also known as proxies.

UI Automation supplies a set of proxies for most standard controls such as those used in Win32 and Windows Forms applications. Third-party proxies for unsupported controls can be written and added to the table of proxies that are available to client applications.

This topic contains the following sections:

• What is a Proxy?
• What is a Proxy Factory?
• Proxy Factory Mapping
• Managing Default Proxies
• Examples
• Related Topics

What is a Proxy?

A client-side provider, or proxy, is an object that implements the IRawElementProviderSimple interface on behalf of a control that does not have an IRawElementProviderSimple implementation of its own. Without a proxy, such a control is largely opaque to UI Automation, which can supply only basic information available from the window handle (HWND), such as the control location.

What is a Proxy Factory?

Each proxy requires a corresponding proxy factory, which is an object that exposes the IUIAutomationProxyFactory interface. UI Automation maintains an internal table of proxy factory entries, each of which contains a reference to the proxy factory for each proxy, and a set of conditions. When UI Automation encounters a control that does not have a native IRawElementProviderSimple implementation, it searches for a proxy factory entry whose conditions indicate that it supports the control. UI Automation searches the table from the beginning, and when it finds a matching entry, UI Automation calls the factory's IUIAutomationProxyFactory::CreateProvider method. If the matching proxy is successfully created, the UI Automation stops searching and uses the newly created proxy object; otherwise, UI Automation continues searching.

A client application creates an instance of a proxy factory entry by using the IUIAutomation::CreateProxyFactoryEntry method, which returns an IUIAutomationProxyFactoryEntry interface pointer. Clients use methods exposed by IUIAutomationProxyFactoryEntry to specify the set of conditions that the proxy factory uses for creating the proxy.

When it calls IUIAutomationProxyFactory::CreateProvider, UI Automation passes parameters that the proxy factory object can use to determine whether the proxy adequately supports the custom control. If so, the proxy factory creates an instance of the proxy and returns the IRawElementProviderSimple interface pointer; otherwise, it returns a NULL pointer.

Proxy Factory Mapping

By default, UI Automation searches through the proxy factory table in the following order.

Proxies 7 and 8 are duplicate entries for the SysListView32 control. Without modification, the proxy 7 is always used for the SysListView32 control, and proxy 8 is never used. Proxy 8 is used only for visible list items, and is typically used by client applications that work only with visible elements, or that have strict performance requirements. These clients can remove proxy 7.

Proxy 9, the Microsoft Active Accessibility to UI Automation proxy, should always be the last entry in the table. This enables Microsoft Active Accessibility fallback functionality for controls that implement Microsoft Active Accessibility, but not UI Automation.

When modifying entries in the proxy factory table, you should carefully evaluate the new position of the entries. We recommend that entries for custom proxies be placed after the non-control and container proxies, but before the Microsoft Active Accessibility to UI Automation proxy. Also, while it is possible to have code in the call to IUIAutomationProxyFactory::CreateProvider determine whether it should support a given window handle (HWND), it is more efficient to let UI Automation select the proxy based on the class name, and keep conditional code in the CreateProvider method to a minimum.

UI Automation maintains a separate proxy factory table for each client. When a client changes its proxy table, the changes affect only the client itself; other clients are not affected.

Managing Default Proxies

When a client application creates the CUIAutomation object, the proxy factory table initially contains entries only for the default proxy providers for standard controls. By using the IUIAutomationProxyFactoryMapping interface, clients can add new entries, remove unwanted entries, change the order of entries, and so on. A client can retrieve an IUIAutomationProxyFactoryMapping interface pointer by calling the IUIAutomation::get_ProxyFactoryMapping method.

The table of available proxies contains an IUIAutomationProxyFactoryEntry interface for each proxy. Each IUIAutomationProxyFactoryEntry specifies the IUIAutomationProxyFactory and the control class that the proxy serves, and defines how events are to be handled.

The table of proxies is represented by an IUIAutomationProxyFactoryMapping interface, which can be obtained from the IUIAutomation::ProxyFactoryMapping property. An application can use IUIAutomationProxyFactoryMapping methods to add and delete proxies. To create a new entry to add to this table, use IUIAutomation::CreateProxyFactoryEntry to obtain the interface, and then use the IUIAutomationProxyFactoryEntry methods to define the applicable control class and the behavior of the proxy.

Examples

The following example code enumerates the entries in the proxy factory table and displays the class name of the supported control. For proxies that are not supplied with the operating system, the image name is displayed.

HRESULT GetProxyTable()
{
    IUIAutomationProxyFactoryMapping* pMap;
    IUIAutomationProxyFactoryEntry* pEntry;
    UINT count;
    BSTR className;
    BSTR imageName;

    HRESULT hr = g_pAutomation->get_ProxyFactoryMapping(&pMap);
    if (SUCCEEDED(hr))
    {
        if (SUCCEEDED(pMap->get_Count(&count)))
        {
            for (UINT x = 0; x < count; x++)
            {
                if (SUCCEEDED(pMap->GetEntry(x, &pEntry)))
                {
                    pEntry->get_ClassName(&className);
                    if (className)
                    {
                        std::wcout << className << L"\t";
                        SysFreeString(className);
                    }
                    if (SUCCEEDED(pEntry->get_ImageName(&imageName)))
                    {
                        if (imageName)
                        {
                            std::wcout << imageName;
                            SysFreeString(imageName);
                        }
                        std::wcout << L"\n";
                    }
                }
                pEntry->Release();
            }
        }
        pMap->Release();
    }

    return hr;
}

The following example code implements a simple proxy for controls that have the "Button" class name, and adds an entry for the proxy to the proxy factory table. This example uses the Font dialog box of Notepad to demonstrate the proxy.

// Registers a proxy for controls with the class name "BUTTON" and
// demonstrates it on the Font dialog of Notepad (which should be
// open when running this code).#include "stdafx.h"
#include <stdio.h>
#include <tchar.h>

#include <windows.h>
#include <UIAutomation.h>

template <class T> void SafeRelease(T **ppT)
{
    if (*ppT)
    {
        (*ppT)->Release();
        *ppT = NULL;
    }
}

// A simple proxy for the proxy factory to create.
class ReallySimpleProxy : public IRawElementProviderSimple
{
public:
    ReallySimpleProxy(HWND hwnd):_refCount(1), _hWnd(hwnd) { }
    virtual ~ReallySimpleProxy() {}

    // IUnknown methods.
    IFACEMETHODIMP_(ULONG) AddRef() 
    { 
        return InterlockedIncrement(&_refCount);
    }
    IFACEMETHODIMP_(ULONG) Release()
    {
        long val = InterlockedDecrement(&_refCount);
        if(val == 0)
        {
            delete this;
        }
        return val;
    }

    IFACEMETHODIMP QueryInterface(REFIID riid, void** ppInterface)
    {
        if(riid == __uuidof(IUnknown))
            *ppInterface =(IUnknown*)((IRawElementProviderSimple*)this);
        else if(riid == __uuidof(IRawElementProviderSimple))
            *ppInterface =(IRawElementProviderSimple*)this;
        else
        {
            *ppInterface = NULL;
            return E_NOINTERFACE;
        }

        ((IUnknown*)(*ppInterface))->AddRef();
        return S_OK;
    }

    // IRawElementProviderSimple methods.
    IFACEMETHODIMP get_ProviderOptions(ProviderOptions * pRetVal)
    {
        *pRetVal = ProviderOptions_ClientSideProvider;
        return S_OK;
    }

    IFACEMETHODIMP GetPatternProvider(PATTERNID patternId, IUnknown ** pRetVal)
    {
        *pRetVal = NULL;
        return S_OK;
    }

    IFACEMETHODIMP GetPropertyValue(PROPERTYID propertyId, VARIANT * pRetVal)
    {
        pRetVal->vt = VT_EMPTY;

        // A String value to test
        if(propertyId == UIA_NamePropertyId)
        {
            pRetVal->vt = VT_BSTR;
            pRetVal->bstrVal = SysAllocString( L"ReallySimpleProxy Control" );
        }

        // Provider Id
        if(propertyId == UIA_ProviderDescriptionPropertyId)
        {
            pRetVal->vt = VT_BSTR;
            pRetVal->bstrVal = SysAllocString( L"Sample: ReallySimpleProxy" );
        }

        return S_OK;
    }

    IFACEMETHODIMP get_HostRawElementProvider(IRawElementProviderSimple ** pRetVal)
    {
        return UiaHostProviderFromHwnd(_hWnd, pRetVal); 
    }

private:
    ULONG _refCount;

    HWND _hWnd; // Window handle for this object.
};

// A simple proxy factory that creates a simple proxy.
class SimpleProxyFactory : public IUIAutomationProxyFactory
{
public:
    SimpleProxyFactory():_refCount(1) {}
    virtual ~SimpleProxyFactory() {}

    // IUnknown methods.
    IFACEMETHODIMP_(ULONG) AddRef() 
    { 
        return InterlockedIncrement(&_refCount);
    }
    IFACEMETHODIMP_(ULONG) Release()
    {
        long val = InterlockedDecrement(&_refCount);
        if(val == 0)
        {
            delete this;
        }
        return val;
    }

    IFACEMETHODIMP QueryInterface(REFIID riid, void** ppInterface)
    {
        if(riid == __uuidof(IUnknown))
            *ppInterface =(IUnknown*)((IUIAutomationProxyFactory*)this);
        else if(riid == __uuidof(IUIAutomationProxyFactory))
            *ppInterface =(IUIAutomationProxyFactory*)this;
        else
        {
            *ppInterface = NULL;
            return E_NOINTERFACE;
        }

        ((IUnknown*)(*ppInterface))->AddRef();
        return S_OK;
    }

    // IUIAutomationProxyFactory methods.
    IFACEMETHODIMP CreateProvider (UIA_HWND hwnd, LONG idObject, LONG idChild, IRawElementProviderSimple **ppRetVal )
    {
        *ppRetVal = new ReallySimpleProxy((HWND)hwnd);
        return S_OK;
    }

    IFACEMETHODIMP get_ProxyFactoryId ( BSTR *pRetVal )
    {
        *pRetVal = SysAllocString(L"Simple Proxy Factory");
        return S_OK;
    }

private:
    ULONG _refCount;
};


HRESULT CheckElement(IUIAutomation *uia, HWND hwnd)
{
    IUIAutomationElement * buttonEl = NULL;
    BSTR name = NULL;
    BSTR providerDescription = NULL;

    wprintf( L"Getting AutomationElement for the button...\n" );
    HRESULT hr = uia->ElementFromHandle(hwnd, &buttonEl);
    if (FAILED(hr))
        goto cleanup;
    if (buttonEl == NULL)
    {
        hr = E_FAIL;
        goto cleanup;
    }

    hr = buttonEl->get_CurrentName(&name);
    if (FAILED(hr))
        goto cleanup;

    hr = buttonEl->get_CurrentProviderDescription(&providerDescription);
    if (FAILED(hr))
        goto cleanup;

    wprintf( L"Got Element:\n  %s\n  %s\n\n", name, providerDescription);
cleanup:
    SysFreeString(providerDescription);
    SysFreeString(name);
    SafeRelease(&buttonEl);

    return hr;
}


int _tmain(int argc, _TCHAR* argv[])
{
    IUIAutomation * uia = NULL;
    IUIAutomationProxyFactoryMapping * proxyMapping = NULL;
    IUIAutomationProxyFactoryEntry * simpleEntry = NULL;
    IUIAutomationProxyFactory *simpleFactory = NULL;
    BSTR bstrClassName = NULL;

    wprintf( L"Initializing COM...\n" );
    HRESULT hr = CoInitializeEx( NULL, COINIT_APARTMENTTHREADED );
    if (FAILED(hr))
        goto cleanup;

    wprintf( L"CoCreating Automation Object...\n" );
    hr = CoCreateInstance(__uuidof(CUIAutomation), NULL, CLSCTX_INPROC_SERVER,
        __uuidof(IUIAutomation), (void **)&uia);
    if (FAILED(hr))
        goto cleanup;


    wprintf( L"Finding Notepad Font Window...\n" );
    HWND fontHwnd = FindWindow(NULL,L"Font");
    if (fontHwnd == NULL)
        goto cleanup;

    wprintf( L"Finding the OK button in Font Window...\n" );
    HWND buttonHwnd = FindWindowEx(fontHwnd, NULL, L"BUTTON", L"OK");
    if (buttonHwnd == NULL)
        goto cleanup;

    wprintf( L"Checking Name and Provider Description without our Proxy:\n" );
    CheckElement(uia, buttonHwnd);

    wprintf( L"Getting the Proxy Factory Mapping Object...\n" );
    hr = uia->get_ProxyFactoryMapping(&proxyMapping);
    if (FAILED(hr))
        goto cleanup;
    if (proxyMapping == NULL)
        goto cleanup;

    wprintf( L"Creating the Proxy Factory...\n" );
    simpleFactory = new SimpleProxyFactory();
    if (simpleFactory == NULL)
        goto cleanup;

    wprintf( L"Creating the Proxy Factory Entry...\n" );
    hr = uia->CreateProxyFactoryEntry(simpleFactory, &simpleEntry);
    if (FAILED(hr))
        goto cleanup;
   bstrClassName = SysAllocString(L"BUTTON");
    if (bstrClassName == NULL)
        goto cleanup;
    hr = simpleEntry->put_ClassName(bstrClassName);       // Match HWNDs with Button class name 
    if (FAILED(hr))
        goto cleanup;
    hr = simpleEntry->put_ImageName(NULL);            // Match any process ImageName
    if (FAILED(hr))
        goto cleanup;
    hr = simpleEntry->put_AllowSubstringMatch(FALSE); // Do not allow substring matching
    if (FAILED(hr))
        goto cleanup;

    // Enter it into the mapping.
    hr = proxyMapping->InsertEntry(0, simpleEntry);
    if (FAILED(hr))
        goto cleanup;

    wprintf( L"Checking Name and Provider Description with our Proxy:\n" );
    CheckElement(uia, buttonHwnd);

cleanup:

    SafeRelease(&simpleEntry);
    SafeRelease(&simpleFactory);
    SafeRelease(&proxyMapping);
    SafeRelease(&uia);
    SysFreeString(bstrClassName);
    CoUninitialize();

    return 0;
}

Related Topics

• Creating UI Automation Providers