This article may contain URLs that were valid when originally published, but now link to sites or pages that no longer exist. To maintain the flow of the article, we've left these URLs in the text, but disabled the links.

Software Driving Software:

Active Accessibility-Compliant Apps Give Programmers New Tools to Manipulate Software

Dmitri Klementiev

SUMMARY Active Accessibility was developed to allow people with disabilities to work on PCs-it's used in magnifiers, screen readers, and tactile mice. It can also be used to create applications that drive other software, and its ability to emulate user input is particularly well suited to the design of testing software.       Starting from the basics of Active Accessibility, this article leads you through the development of a software testing application. You'll see how this testing application interacts with common controls and other UI elements, then processes the resulting WinEvents.

icrosoft® Active Accessibility® is a relatively recent technology (version 1.0 was released in May, 1997) that provides programmatic access to user interface elements by providing a consistent way of extracting information from any UI element. With this functionality, programmers can get information about UI elements and interact with them. For example, you can programmatically push a button, choose an item in a list view, or drop down the list of a combobox. Originally created to allow people with disabilities (for instance those with impaired vision) to use software, this technology is used in many applications such as magnifiers, screen readers, and tactile mice.
      Active Accessibility is supported on Windows® 98 and Windows 2000. To get Active Accessibility support on Windows 95 (English only) and Windows NT® 4.0 (Service Pack 6 or greater), you can install the Active Accessibility RDK and SDK from https://www.msdn.microsoft.com/library/default.asp?url=/nhp/Default.asp?contentid=28000544. Additional information relating to Active Accessibility can also be found at this address.
      The nature of Active Accessibility makes it very useful in applications that drive other software products. One class of applications that can make good use of Active Accessibility is testing programs. Another class might be programs that perform a user-defined sequence of keyboard or mouse input along with UI element state verificationâ€"for example, a program that keeps the Windows Mediaâ„¢ player trying to connect to a busy server by catching the moment when the "Windows Media Player Error" message is displayed, dismissing this dialog, and clicking the necessary menu item to continue the connection process.
      There are two types of Active Accessibility-related programs: Active Accessibility-compliant applications, and tools to manipulate them. This article discusses the second aspect: creating tools to manipulate other software products. Specifically, I will focus on the development of testing applications. These are not only useful, but they also demonstrate almost all of the features of Active Accessibility.

Active Accessibility Basics

      The main idea behind Active Accessibility is to provide the functionality to access UI elements programmatically to get information about or manipulate these elements. UI elements that support this functionality are called accessible. In most cases this means that a UI element supports the IAccessible interface. You can also say that an accessible UI element is represented by the IAccessible interface in the world of Active Accessibility.
      Whenever you need to get information about an element, perform an action on it, or do anything else using Active Accessibility, you generally refer to the element by using a method or property of the IAccessible interface that represents this element. Later I'll show you how an IAccessible interface/child ID pair generally represents a UI element. For now I'll focus on the IAccessible interface.
      There are several ways to get a pointer to the IAccessible interface for an accessible UI element. The most common way is to use one of the functions provided by Active Accessibility such as AccessibleObjectFromPoint, AccessibleObjectFromWindow, and so on, and methods supported by IAccessible such as get_accChild and get_accParent. These functions and methods will be described in detail later in this article.
      The IAccessible interface supports properties that allow you to get information about the corresponding UI element. The most important properties for use in testing software are name, role, and state. These are most easily defined by example. Let's take a look at the Windows NT 4.0 Windows Explorer Find: All Files dialog box shown in Figure 1. It contains a combobox labeled "Look in:" The name of this combobox is Look in:, the role is combobox, and the state is visible. The state of a UI element reflects the element's current state.

Figure 1 The IAccessible Example

      The Active Accessibility SDK provides convenient tools that you can use to become more familiar with Active Accessibility and for developing related applications. One of these tools, the Object Inspector, displays the properties of a UI element pointed to by the mouse cursor. The Verification Tool shows how the world of Active Accessibility is populated by controls that support the IAccessible interface in a chosen window.
      Besides retrieving information about the element and manipulating it through the IAccessible interface, Active Accessibility offers two other important features that are useful for testing applications: monitoring events generated by UI elements and emulating keyboard and mouse input. Events fired by accessible elements are called WinEvents. They occur when an accessible element is created or changes its name, state, location, or keyboard focus. The list of supported events can be found in the WINABLE.H file. Each event's name begins with EVENT_OBJECT or EVENT_SYSTEM. These events are also discussed in the Active Accessibility SDK documentation.
      The event mechanism is similar to a standard Windows hooking mechanism. Both the monitoring events and emulation input features will be discussed with the sample code.
      To understand how IAccessible methods and properties are supported, let's take a look inside Active Accessibility.

Inside Active Accessibility

      The core functionality in Active Accessibility is provided by OLEACC.DLL. Each time you call a function that returns a pointer to an IAccessible interface corresponding to a particular UI element, OLEACC.DLL verifies whether this element natively supports IAccessible. Native support means that IAccessible is implemented programmatically for this element.
      When a UI element doesn't support IAccessible natively, OLEACC.DLL verifies the Windows class name for this element. If this class is a USER or COMCTL32-supported class, OLEACC.DLL creates a proxy that implements IAccessible on behalf of the UI element. Mostâ€"but not allâ€"COMCTL32 controls have IAccessible support provided by OLEACC.DLL.
      Examples of UI elements that natively support IAccessible are custom controls, owner-drawn, or windowless controls. Since developers who create software that contains these kinds of UI elements also implement the interfaces for these elements, they are also responsible for providing correct support for methods and properties. In practice that means some methods or properties might be implemented improperly or not at all. This also means that a developer who implements the interface defines its properties, such as the name and role.
      If a UI element doesn't support IAccessible natively and OLEACC.DLL doesn't recognize its class name as supported, OLEACC.DLL creates a default proxy that provides minimal HWND-based IAccessible support, such as location and whether the window is enabled and visible. The default proxy doesn't provide any control-specific information.
      An understanding of how runtime Active Accessibility works is important when developing applications that use standard controls. These apps become compatible with Active Accessibility automatically. This also means that you don't need to rewrite your apps. Active Accessibility names will be given based on the Win32® controls' names. Roles will be defined based on the controls' functionality.
      With an understanding of how UI elements are represented and supported in the world of Active Accessibility, let's consider how UI elements relate to each other.

Parent/Child Relations of UI Elements

      As I mentioned before, the world of Active Accessibility is populated by IAccessible interfaces. Each UI element has a corresponding pointer to an IAccessible interface and an additional identifier called a child ID.
      All accessible UI elements have parent/child relationships in terms of Active Accessibility. For instance, the Look in: combobox has the Name & Location property page as an ancestor, though not as an immediate parent. Figure 2 illustrates these relationships in Windows NT 4.0. In this example, the element name is shown first, followed by its role; Name & Location is the element name, and property page is its role. Note that not all child interfaces are shown here, just those that are essential for the explanation. This picture might be different for other operating systems. For instance, on Windows 98 some names are different and there are additional intermediate children.

Figure 2 Levels of IAccessible

       Figure 2 reflects the general rule that each HWND-based UI element has two levels of IAccessible in it: the parent representing the overall HWND, and children representing the client and non-client areas (such as window scrollbars, menus, and so on). A control is called labeled if it follows a label. In this case the Look in: combobox is a labeled control with the label Look in:. For labeled controls, the parent/child tree contains two entries with the label's name and role window. Interfaces corresponding to the static text and to the control are children of these entries. The property page's ancestor is the Find: All Files dialog. The topmost parent is the Desktop window.
      Generally, the parent/child hierarchy for HWND-based controls is created and supported by OLEACC.DLL based on the HWND hierarchy. Developers of accessible windowless controls must create and manage their internal hierarchy manually. Since there are usually many intermediate IAccessible interfaces in a parent/child tree, this tree is often complex.
      One more question should be answered before I discuss how to use Active Accessibility to create testing software. Why does an IAccessible interface/child ID pair, not just the IAccessible interface, represent a UI element? The next section takes a look at this issue.

The IAccessible Interface/Child ID Pair

      Let's consider a control that supports the IAccessible interface and has many children, such as a listbox with many items. There are two ways to make this control accessible. The first is to support the IAccessible interface for the listbox itself and a separate IAccessible interface for each item. Another is to support just one control's interface that would provide functionality to access its children based on some sort of identification for each child.
      In the first case, separate COM objects would be created for the control and each of its children. This will increase memory consumption compared to the second case, where each child doesn't support its own IAccessible and is instead accessed via its parent's interface. In the latter case, an additional parameterâ€"the child IDâ€"is used along with the parent's IAccessible interface to represent a child. The child ID is always a VARIANT of type VT_I4, and contains either a unique value defined programmatically or just an ordinal for the child. Ordinal means that the first child has an ID of 1, the second has an ID of 2, and so on incrementally.
      Thus, if a child doesn't support its own IAccessible interface, but its parent does support it, then this child is represented by the parent's IAccessible interface/child ID pair. Generally, the UI parent element that supports IAccessible is also represented by this pair, where child ID is defined as CHILDID_SELF, which has a value of 0.
      Remember that the child ID is always relative to the IAccessible interface. For instance, an accessible element can have a non-CHILDID_SELF child ID relative to its parent's IAccessible interface, and its child ID will be CHILDID_SELF relative to its own IAccessible interface, if supported.
      If you use Active Accessibility functions, you should always provide an IAccessible interface/child ID pair to interact with an accessible UI element. Almost all Active Accessibility functions and IAccessible methods require both of these parameters, so don't forget to supply the correct child ID. This ID will be CHILDID_SELF for UI elements that support IAccessible.
      If a child doesn't support IAccessible and the parent supports the IEnumVARIANT interface, the child ID can be retrieved by the standard IEnumVARIANT interface method called Next. If the parent does not support the IEnumVARIANT interface, the child ID is just its ordinal. It is important to mention that if a child supports IAccessible, Next can return either a child ID or an IDispatch corresponding to the child's IAccessible interface. The implementation of Next is up to the developer and may be different for different UI elements.
      The fact that a UI element is accessible doesn't necessarily imply that it has its own IAccessible interface. If a UI element supports its own IAccessible interface, the child's IAccessible interface/CHILDID_SELF pair is used to access it. If the element does not support its own IAccessible interface, the pair that represents the element will be the parent's IAccessible interface/child ID pair, where child ID is not equal to CHILDID_SELF and is defined programmatically. You should always call get_accChild to verify whether a UI element supports its own IAccessible interface.

Creating Testing Software

      To write a testing program, you must obtain information about UI elements, use WinEvent hooks to monitor events, access controls using pointers to the IAccessible interface, perform actions on controls, and emulate keyboard and mouse inputs. These and other challenges are addressed in the next sections using my sample program.
      The sample program simulates a user's interaction with the Find: All Files dialog. This program uses the standard Win32 FindWindow function to determine whether the Find: All Files dialog exists. If it doesn't exist, the application waits for it by monitoring WinEvents. As soon as the dialog appears, the application obtains a pointer to the IAccessible interface that corresponds to the dialog, navigates to the Containing text: edit box and to the Named: combobox, sets their values, and pushes the Find Now button. Finally, it closes the dialog by emulating an Alt-F4 keyboard input. Note that on Windows 2000, the corresponding window and its fields differ from those on Windows NT 4.0 and Windows 9x. To accommodate this, the sample program verifies the current operating system and, if necessary, initializes the bIsWin2K variable that allows it to treat Windows 2000 as a separate case.
      Automated testing software must perform sequential manipulations on software products via UI controls, and then verify the results of those manipulations. Thus, the first problem you need to solve when writing a testing program is how to efficiently manipulate UI controls. You must be able to find the IAccessible interface/child ID pairs corresponding to UI elements in a reasonable amount of time so you can obtain information about them and perform actions on them.
      To verify results, you can retrieve a UI element's relevant properties (such as value, state, or name), or use keyboard input emulation to select text, put it into the clipboard, and then compare it with the expected value.
      Performing an action on a UI element and verifying the results takes less time than searching for the accessible element. Thus, your attention should be focused on decreasing the search time.

Getting Element Information

      To obtain information about a UI element, you need the IAccessible interface/child ID pair that corresponds to the UI element. I'll discuss how to obtain this pair shortly, but for now let's assume that you already have it.
      The first four functions in Figure 3 show how to get the name, role, state, and Window class that correspond to a UI element. The first three, GetUIElementName, GetUIElementRole, and GetUIElementState are wrappers for the IAccessible methods get_accName, get_accRole, and get_accState. They take the IAccessible interface/child ID pair that corresponds to a UI element as input parameters, and return a string (and its length) that contains the requested information. GetUIElementRole and GetUIElementState return DWORD values for role and state, respectively.
      When using get_accRole, the additional function GetRoleText is needed to convert an integer role representation into its string representation:

hr = pacc->get_accRole(*pvarChild, &varRetVal);if (hr == S_OK && varRetVal.vt == VT_I4){ GetRoleText(varRetVal.lVal, lpszRole, cchRole);}else lstrcpyn(lpszRole, "unknown role", cchRole);

      The state of a UI element is also represented in the form of an integer. Since a state can have multiple values, such as selectable and focusable, the integer is a combination of bits that reflects those values. The function GetUIElementState converts the combination of state bits into the corresponding comma-separated state strings within the loop:

for(dwStateBit = 1; cCount; cCountâ€", dwStateBit <<= 1){���}

      Inside this loop you verify which state bits from the minimal state value to the maximum state value are contained in varRetVal.lVal, and return strings for these states using the function GetStateText. The case when the state is normal (varRetVal.vt == 0) is treated separately.
      The fourth function, GetWindowClassForUIElement, takes a pointer to IAccessible and uses the function WindowFromAccessibleObject to get the window handle that corresponds to this pointer. The fact that WindowFromAccessibleObject uses just IAccessible and not an IAccessible interface/child ID pair means that if an accessible element doesn't have its own support for IAccessible, then the parent's IAccessible represents the element and this element has the same class as its parent. The Win32 function GetClassName is called to retrieve the Window class name.

Monitoring WinEvents

      Monitoring WinEvents is very similar to Windows message monitoring using Windows hooks. The important difference is that when monitoring WinEvents fired by UI elements from another process, you don't need to create a separate DLL to load into the process address space.
      There are two options for hooking: you can hook out of context or in context by choosing the corresponding last argument in the SetWinEventHook function. When hooking out of context there is no need for an additional DLL, and the callback function's code runs outside of the target process. When hooking in context the callback function's code sits in the required additional DLL, which is loaded in the target's process address space. This second case is harder to write code for, but is potentially more efficient.
      The function SetWinEventHook is used to set a WinEvent hook with a WinEventProc callback function. Once this hook is set, each time a WinEvent is fired, the WinEventProc function will be called by the system.
      In my sample program, I verify whether the Find: All Files window exists. If it does not, I set the WinEvent hook to detect the moment when this window is created. If the window exists, I post the message WM_TARGET_WINDOW_FOUND to inform the main thread that the window is available. The following code from WinMain demonstrates this approach:

if(NULL == (hWndMainWindow = FindWindow(NULL, szMainTitle))){ hEventHook = SetWinEventHook( EVENT_MIN, // eventMin ID EVENT_MAX, // eventMax ID NULL, // always NULL for outprocess hook WinEventProc, // call back function 0, // idProcess 0, // idThread WINEVENT_SKIPOWNPROCESS | // always the same for WINEVENT_OUTOFCONTEXT); //outproc hook}else PostThreadMessage(GetCurrentThreadId(), WM_TARGET_WINDOW_FOUND, 0, 0);

      Since I passed zeros as the fifth and sixth parameters to SetEventHook, the callback function WinEventProc will be called for any process or thread in the system that fires a WinEvent.
      Not all UI elements behave the same way in terms of firing WinEvents. Some of them might not fire their own EVENT_ OBJECT_CREATE events when created. An event that tells you its parent is being created can be fired instead. Some windows might fire the EVENT_OBJECT_CREATE event just once when they are created for the first time, and then fire EVENT_OBJECT_HIDE when closed and EVENT_OBJECT_SHOW when reopened. It is always a good idea to use the Accessible Event Watcher tool from the Active Accessibility SDK to verify events while developing WinEvent support.
      In the sample app, I am interested in the EVENT_OBJECT_ CREATE and EVENT_OBJECT_SHOW events because they are fired when an accessible element (in this case, a window) is created or becomes visible. Since I want the callback function to only see windows coming up, it returns if the triggering event was not EVENT_OBJECT_CREATE or EVENT_OBJECT_SHOW, or if the window handle for the corresponding element was NULL.
      Another approach is to filter the WinEvents to be processed in the callback function by using the first and second parameters to SetEventHook. These parameters define an event interval containing the WinEvents that should be processed. This approach makes sense if you know for sure what this interval should be. But if you haven't finished the final design of your application, I recommend that you don't filter the events in the callback function.
      There are two exceptions to this rule of thumb. The first is if there are performance problems, but this normally is not an issue. The other exception occurs when the accessible element (or the COM server, to be exact) doesn't process Active Accessibility function calls executed from the callback function while processing certain events. This is usually not the case when working with controls supported by OLEACC.DLL. When UI elements support IAccessible natively, a developer of the application that contains these UI elements is completely responsible for the implementation of this support. Incorrect implementation can cause problems, so you can filter the events that cause the problem in the SetWinEventHook call.
      A callback function is called asynchronously when it is running out of context. In this case it is a good idea to be careful when calling IAccessible methods from the callback function since when an IAccessible method is called, the UI element corresponding to this IAccessible interface might not exist anymore. Be particularly careful with the EVENT_OBJECT_DELETE event that signifies that the UI element that fired this event is no longer available. The AccessibleObjectFromEvent function, discussed in the next section, will return an error in this case.
      Also, when you call IAccessible methods from a callback function and this IAccessible represents an HWND-based element that fired the event, you might want to call the Win32 IsWindowVisible function first because hidden UI elements might not be fully initialized.

Obtaining a Pointer to IAccessible

      There are four ways to obtain a pointer to the IAccessible interface. One way is to call the Active Accessibility function AccessibleObjectFromEvent from the WinEventProc function. All in parameters for the AccessibleObjectFromEvent functions are passed to WinEventProc. The out parameters compose the IAccessible interface/child ID pair corresponding to the UI element that fired the event.
      In the sample code, AccessibleObjectFromEvent is called when the app detects either an EVENT_OBJECT_CREATE or an EVENT_OBJECT_SHOW event triggered by a window. The name of the accessible element that corresponds to this pair is the title of the window that came up. If this title matches szMainTitleâ€"the title you are waiting forâ€"the message WM_TARGET_WINDOW_FOUND is posted to the current thread to notify it that the window is available.
      The next two methods to obtain the pointer to the IAccessible interface utilize the AccessibleObjectFromPoint and AccessibleObjectFromWindow functions. The function AccessibleObjectFromPoint returns an IAccessible interface/child ID pair for the UI element located at the position defined by the first POINT parameter.
      The function Acces-sibleObjectFromWindow returns a pointer to an IAccessible interface representing a window, a client area, or some non-client area control, depending on the second parameter, which is the object ID. The object ID value in this parameter can be one of the standard object identifier constants or a custom object ID. The child ID to be used with the pointer to IAccessible that is returned by AccessibleObjectFromWindow is always equal to CHILDID_SELF.
      Usage of AccessibleObjectFromWindow is shown in the sample program. It is the first call in WinMain after the window handle to the Find: All Files dialog is found. This function returns the pointer to IAccessible that represents the found window:

if(S_OK == (hr = AccessibleObjectFromWindow(hWndMainWindow, OBJID_WINDOW,IID_IAccessible,(void**)&paccMainWindow))){���}

      Since there are more IAccessible interfaces than windows, searching for a window using Win32 functions will take significantly less time than searching for an IAccessible interface corresponding to this window using the Active Accessibility tree. This means that to increase performance you might want to find the window closest to the desired UI element using Win32 functions such as FindWindow and EnumWindows, and then get the IAccessible interface/child ID pair for this element using Active Accessibility navigation.
      Of course, you must use common sense about when to use Win32 functions versus Active Accessibility functions. For instance, if you need to access several controls that belong to the same dialog, you might try to search for window handles for each control and then use AccessibleObjectFromWindow if those controls are HWND-based. However, a more general approach would be to find a window handle to the dialog. Once the parent window is found, use the AccessibleObjectFromWindow function to get a pointer to the IAccessible interface corresponding to this window, and use it as an accessible parent for navigating to the UI elements that belong to this dialog.
      The fourth method of retrieving an IAccessible interface/child ID pair is to navigate through the parent/child Active Accessibility tree. While developing custom, owner drawn, or windowless controls, it's good programming practice for the role and name combination to be unique for each UI element belonging to the same window. However, if for some reason two UI elements in a window have the same role and name, an additional parameterâ€"window classâ€"can be used to uniquely identify the element.
      The function FindChild shows the implementation of a search routine based on Active Accessibility parent/child navigation. This function takes six parameters. The first four contain information passed to the function, and the last two hold the IAccessible interface/child ID pair for the accessible element if one is found.
      The first parameter is a pointer to the parent's IAccessible interface. When FindChild is called the first time, this is the pointer to the IAccessible interface corresponding to the window to which the UI element belongs. FindChild is then called recursively for each accessible element that supports its own IAccessible interface. The next three parameters define the name, role, and class of the UI element for which you are searching. If one of those parameters is NULL, the search routine skips the verification based on this parameter.
      FindChild consists of the following steps. It finds the number of children for the parent. Then it retrieves a child ID for each child, and based on this ID, an IAccessible interface/child ID pair is found. With this pair, FindChild gets the name, role, and class name for each child and compares them with the requested values. If there is a match, FindChild returns. It is important to note that the correct approach is to compare roles and states as integers, not as strings. Then your application will work correctly when role and state strings change and when your application is run on localized platforms. The string values for these properties are convenient while you are debugging an application, for instance by using OutputDebugString as demonstrated in the sample code.
      If the desired accessible element is not found and the child supports its own IAccessible interface, FindChild is called recursively with the child's pointer to IAccessible as the first argument.
      The first step in enumerating children is to verify whether the accessible UI element (whose pointer to IAccessible is sent to FindChild as the parent) supports the IEnumVARIANT interface. If it does, query for it and reset it:

hr = paccParent -> QueryInterface (IID_IEnumVARIANT, (PVOID*) & pEnum);if(hr == S_OK && pEnum) pEnum -> Reset();

      The next step is to obtain the number of accessible children owned by the parent by calling the get_accChildCount function. Then, for each child, find the child ID and verify whether this child supports the IAccessible interface by getting a pointer to the IDispatch interface and querying it for an IAccessible interface.
      If a parent supports the IEnumVARIANT interface, the method Next is called. This method returns either a pointer to the IDispatch interface or a child ID. If a parent does not support the IEnumVARIANT interface, the child ID is just the ordinal or index for this child. In this case, or if the Next method returned a child ID, you call get_accChild. This function servers two purposes. First, it verifies whether a child supports the IAccessible interface. Second, it retrieves a pointer to the IDispatch corresponding to this IAccessible interface. To make your application work correctly, you should always verify whether a UI element supports its own IAccessible interface. If it does, use that interface, not the parent's. The code in Figure 4 demonstrates this approach.
      Once you have a pointer to IDispatch, either from Next or get_accChild, query this interface for an IAccessible interface that corresponds to the child. If a pointer to this interface is not NULL, it goes to the IAccessible interface/child ID pair as the first component, and the second component becomes CHILDID_SELF. If the pointer is NULL, the parent's IAccessible interface is used in the pair that defines the child. The child ID in this case is either the value returned by the Next method or the child's ordinal. The following code, extracted from FindChild, illustrates this:

if(paccChild){ VariantInit(&varChild); varChild.vt = VT_I4; varChild.lVal = CHILDID_SELF;}else{ paccChild = paccParent; paccChild->AddRef();}

      Coding becomes easier from this point on. You have the IAccessible interface/child ID pair that represents an accessible UI element. You have methods that retrieve a name, role, class, and state for this element. All you need to do is call those methods and make the proper verifications. In my sample program, I skip elements that have state unavailable.

Acting on Elements

      The IAccessible interface/child ID pair opens the doors for calling IAccessible methods such as get_accDescription to get a description of the UI element, get_accValue to get a value, and so on. One of the most important functions is accDoDefaultAction. A default action is defined for each accessible UI element role. For instance, the default action for a button is "push this button," and for a selected checkbox it is "uncheck." To find the default action for a particular element, refer to the documentation for Active Accessibility or call get_accDefaultAction.
      The sample application clicks the Find Now button by calling accDoDefaultAction, as shown here:

if(FindChild (paccMainWindow, lpstrName, "push button", "Button", &paccControl, &varControl)){ hr = paccControl->accDoDefaultAction(varControl); paccControl->Release(); VariantClear(&varControl);}

      You can combine Active Accessibility techniques and standard Windows-based programming techniques. The code for setting a value for the Containing text: edit box and Named: combobox illustrates this approach:

if(FindChild (paccMainWindow, "Containing text:", "editable text", "Edit", &paccControl, &varControl)){ WindowFromAccessibleObject(paccControl, &hWnd); SendMessage(hWnd, WM_SETTEXT, 0, (LPARAM)"My Text"); paccControl->Release(); VariantClear(&varControl);}

Emulating Keyboard and Mouse Input

      Let's suppose that you need to manipulate one of the new UI elements that doesn't fully support Windows messages or IAccessible interface methods. If it doesn't support the messages and methods that you need, the easiest solution is keyboard and mouse input emulation. For instance, you can use Tab emulation to navigate to the desired control.
      The function that enables you to do this is SendInput, which is a regular USER API. Though not a part of Active Accessibility, it is very natural to use SendInput in conjunction with Active Accessibility. SendInput takes three parameters: the number of keyboard or mouse actions to be performed, the array of INPUT structures, and the size of the INPUT structure. Each element of INPUT details an action to be performed. Note that pushing a button down and releasing a button are two different actions, so two separate INPUT elements must be created.
      In the sample code, the function SendInput is used to emulate the Alt-F4 keyboard sequence to close the Find: All Files dialog:

INPUT input[4]; memset(input, 0, sizeof(input));input[0].type = input[1].type = input[2].type = input[3].type = INPUT_KEYBOARD;input[0].ki.wVk = input[2].ki.wVk = VK_MENU;input[1].ki.wVk = input[3].ki.wVk = VK_F4;// Then release it. THIS IS IMPORTANTinput[2].ki.dwFlags = input[3].ki.dwFlags = KEYEVENTF_KEYUP;input[0].ki.time = input[1].ki.time = input[2].ki.time = input[3].ki.time = GetTickCount();SendInput(4, input, sizeof(INPUT));

In this code snippet, input[ ].type = INPUT_KEYBOARD specifies that keyboard input is emulated. input[ ].ki.dwFlags and input[ ].ki.wVk specify the action to be performed and the key on which to perform this action, respectively. Pay particular attention to this statement:

input[2].ki.dwFlags = input[3].ki.dwFlags = KEYEVENTF_KEYUP;

Without it, the keys will never be released.
      Each time you call SendInput emulating keyboard input, you might want to use GetAsyncKeyState to verify the state of a key, or BlockInput to block keyboard and mouse inputs that may interfere with SendInput.

Problems and Workarounds

      Since Active Accessibility is a relatively new technology, it has some flaws in practical use. Most problems arise because there are many controls with only partial supportâ€"or no support at allâ€"for Active Accessibility. This section applies mostly to the controls that natively support the IAccessible interface, and to those controls that are not supported by OLEACC.DLL. Since the IAccessible interface support depends on the developer of the software product, there is no guarantee that a particular product is fully compliant with Active Accessibility. To be fully compliant, an application must use controls that support Active Accessibility, and its UI elements must support three critical features:

• Methods for obtaining information about the UI element (IAccessible properties)
• Methods for performing actions on the UI element
• Firing WinEvents when appropriate, for instance when its state or value has changed

      Since Active Accessibility was created to allow people with disabilities to use software, obtaining information about the UI element is most commonly supported. This is because the main purpose of utilities such as Microsoft Narrator (included with Window 2000) is to give information about the specific UI element.
      Typically the third feature, firing WinEvents, is partially supported. In other words, only essential events are supported. For instance, EVENT_OBJECT_FOCUS and EVENT_OBJECT_SELECTION events are very important because you can get information about which control has keyboard focus and which one is selected based on them.
      Thus, there are two common situations where Active Accessibility cannot be fully exploited. First, some applications have custom, owner drawn, or windowless controls that do not natively support the IAccessible interface. Second, some programs support only the "read information" methods of IAccessible. In the first situation, all you can do is emulate keyboard and mouse input as described earlier. In well-written software you can generally use accelerators, or navigate to controls using hotkeys and the Tab key. When the desired control has the focus, you can emulate an Enter keypress.
      The disadvantage to this approach is that your application will stop working correctly if the software that it drives changes its hotkeys, accelerators, or Tab order. This can happen when a new version of the software is released, and it can be a serious problem if you are developing a testing application, since you are continually using new versions of the software.
      You can avoid this problem if the controls of the driven program support the read information methods of IAccessible. This lets you navigate to a UI element using Active Accessibility, as demonstrated in the sample program's FindChild function. Once you obtain an IAccessible interface/child ID pair for the UI element you're looking for, you can call the IAccessible method accLocation to get the coordinates of the UI element, position the mouse cursor over this element, and emulate a mouse action. Of course, if names, roles, and other properties of accessible elements change, a testing application would also fail to work correctly. But this is less likely than changing the order of controls or control items, thus making an approach that uses the read information methods of IAccessible more reliable than just navigation by keyboard.
      Usually the mouse action is a button click to select an item from a listbox, set the focus to an edit box, and so on. Once focus is set, you can use keyboard input to set the value.
      The code in Figure 5 illustrates how to click on a button. It assumes that the corresponding IAccessible interface/child ID pair has been found. This code warrants two comments. First, several controls do not behave correctly when SendInput(n, input, sizeof(INPUT)) is called and n is greater than 1. In other words, each keyboard or mouse action should be sent separately in this case, and some time should be allowed to process this action. The last three lines in the code snippet reflect this.
      Second, if you would like the code to work for both right-handed and left-handed mouse settings, you can add the following simple code:

if (GetSystemMetrics(SM_SWAPBUTTON)){ input[0].mi.dwFlags = MOUSEEVENTF_RIGHTDOWN; input[1].mi.dwFlags = MOUSEEVENTF_RIGHTUP;}

      The only restriction in using this approach is that the control on which the action is performed must be visible and unobscured by other windows. If an IAccessible method is used to perform the action, this restriction depends on the implementation of the method. Some implementations may use SendInput internally, so the restriction is applicable. Others may use different techniques that work for invisible and obscured UI elements.
      Another problem you might encounter while working with Active Accessibility is that the function WindowFromAccessibleObject won't work for some controls if the calling thread is initialized with COINIT_MULTITHREADED. This happens because COM disallows interapartment calls to input-synchronous methods from multithreaded apartments. These methods are marked as [input_sync] in the .idl file. Note that these methods can be called from single-threaded apartments.
      Since WindowFromAccessibleObject calls a method that is marked as [input_sync], it doesn't work for UI elements that live outside the calling thread if the calling thread was initialized with CoInitializeEx(COINIT_MULTITHREADED). The function WindowFromAccessibleObject will work only for controls that are supported by OLEACC.DLL.
      A workaround for this is the use of a helper thread. This thread is initialized with CoInitializeEx(NULL, COINIT_APARTMENTTHREADED). In the main thread, if a call to WindowFromAccessibleObject fails, the pointer to IAccessible is marshaled to the helper thread. Then the main thread posts a user message to the helper thread, notifying it that a window handle for the given IAccessible pointer is expected. After this, the main thread waits for an event. The simplified code (without additional verifications) for the main thread is shown in Figure 6. In this function, pAcc is a pointer to IAccessible that represents a window whose handle must be found, and phwnd is the pointer to this handle.
      When the helper thread receives the user message, it unmarshals the pointer to IAccessible, calls the function WindowFromAccessibleObject, and sends an event indicating it has finished. The simplified code for the helper thread is shown in Figure 7.
      In the following example code, the STA_WFAO_PARAMS structure is declared as:

struct STA_WFAO_PARAMS{ IStream *pstm; HWND *phwnd; HRESULT hr;};

The handle to the event, g_STA_hEv, is created as:

g_STA_hEv = CreateEvent ( NULL, FALSE, FALSE, TEXT("STA_ThreadEvent"));

This code assumes that there is just one main (or calling) thread. Additional code that organizes multiple requests in a queue should be added in the case of multiple calling threads.
      The last problem with Active Accessibility that I'll discuss is that sometimes the COM server doesn't process the Active Accessibility function calls performed from a WinEvent callback function while processing certain events. I have encountered this situation several times while working on different applications. Most likely, this is a problem in the server implementation.
      One workaround is to filter WinEvents that cause the problem, as I described in the section on monitoring WinEvents. Another workaround is to call Win32 functions instead of IAccessible methods, if possible. For instance you can call

GetWindowText(hwndMsg, szObjName, sizeof(szObjName));

instead of get_accName. The window handle hwndMsg is the third parameter passed to the WinEvent callback function. The idea here is that calling Win32 APIs is safer than calling IAccessible methods that may have buggy implementations. Although Win32 APIs won't work for completely custom controls, at least your application will not crash or take down the target app. Thus, this approach would work for HWND controls only.

Last Thoughts

      There are quite a few Active Accessibility functions that I have not mentioned. These are listed in Figure 8, along with brief descriptions. More detailed description and samples can be found at https://msdn.microsoft.com/library/default.asp?url=/nhp/Default.asp?contentid=28000544.

Dmitri Klementiev is a software engineer with Microsoft. He has a PhD in math from Moscow Institute of Physics and Technology. Dmitri spent more that 12 years developing mathematical and computer models for control optimization. He can be reached at dklem@microsoft.com.

From the April 2000 issue of MSDN Magazine.