Export (0) Print
Expand All

Access Embedded Objects Using UI Automation

Note Note

This documentation is intended for .NET Framework developers who want to use the managed UI Automation classes defined in the System.Windows.Automation namespace. For the latest information about UI Automation, see Windows Automation API: UI Automation.

This topic shows how Microsoft UI Automation can be used to expose objects embedded within the content of a text control.

Note Note

Embedded objects can include images, hyperlinks, buttons, tables, or ActiveX controls.

Embedded objects are considered children of the UI Automation text provider. This allows them to be exposed through the same UI Automation tree structure as all other user interface (UI) elements. Functionality, in turn, is exposed through the control patterns typically required by the embedded objects control type (for example, since hyperlinks are text-based they will support TextPattern).

 

A sample document with textual content, ("Did You Know?"…) and two embedded objects (a picture of a whale and a text hyperlink), used as a target for the code examples.

Embedded objects in a text container.

The following code example demonstrates how to retrieve a collection of embedded objects from within a UI Automation text provider. For the sample document provided in the introduction, two objects would be returned (an image element and a text element).

Note Note

The image element should have some intrinsic text associated with it that describes the image, typically in its NameProperty (for example, "A blue whale."). However, when a text range spanning the image object is obtained, neither the image nor this descriptive text is returned in the text stream.

///-------------------------------------------------------------------- 
/// <summary> 
/// Starts the target application. 
/// </summary> 
/// <param name="app">
/// The application to start. 
/// </param> 
/// <returns>The automation element for the app main window.</returns> 
/// <remarks> 
/// Three WPF documents, a rich text document, and a plain text document  
/// are provided in the Content folder of the TextProvider project. 
/// </remarks> 
///-------------------------------------------------------------------- 
private AutomationElement StartApp(string app)
{
    // Start application.
    Process p = Process.Start(app);

    // Give the target application some time to start. 
    // For Win32 applications, WaitForInputIdle can be used instead. 
    // Another alternative is to listen for WindowOpened events. 
    // Otherwise, an ArgumentException results when you try to 
    // retrieve an automation element from the window handle.
    Thread.Sleep(2000);

    targetResult.Content =
        WPFTarget +
        " started. \n\nPlease load a document into the target " +
        "application and click the 'Find edit control' button above. " +
        "\n\nNOTE: Documents can be found in the 'Content' folder of the FindText project.";
    targetResult.Background = Brushes.LightGreen;

    // Return the automation element for the app main window. 
    return (AutomationElement.FromHandle(p.MainWindowHandle));
}


...


///-------------------------------------------------------------------- 
/// <summary> 
/// Finds the text control in our target. 
/// </summary> 
/// <param name="src">The object that raised the event.</param>
/// <param name="e">Event arguments.</param>
/// <remarks> 
/// Initializes the TextPattern object and event handlers. 
/// </remarks> 
///-------------------------------------------------------------------- 
private void FindTextProvider_Click(object src, RoutedEventArgs e)
{
    // Set up the conditions for finding the text control.
    PropertyCondition documentControl = new PropertyCondition(
        AutomationElement.ControlTypeProperty,
        ControlType.Document);
    PropertyCondition textPatternAvailable = new PropertyCondition(
        AutomationElement.IsTextPatternAvailableProperty, true);
    AndCondition findControl =
        new AndCondition(documentControl, textPatternAvailable);

    // Get the Automation Element for the first text control found. 
    // For the purposes of this sample it is sufficient to find the  
    // first text control. In other cases there may be multiple text 
    // controls to sort through.
    targetDocument =
        targetWindow.FindFirst(TreeScope.Descendants, findControl);

    // Didn't find a text control. 
    if (targetDocument == null)
    {
        targetResult.Content =
            WPFTarget +
            " does not contain a Document control type.";
        targetResult.Background = Brushes.Salmon;
        startWPFTargetButton.IsEnabled = false;
        return;
    }

    // Get required control patterns 
    targetTextPattern =
        targetDocument.GetCurrentPattern(
        TextPattern.Pattern) as TextPattern;

    // Didn't find a text control that supports TextPattern. 
    if (targetTextPattern == null)
    {
        targetResult.Content =
            WPFTarget +
            " does not contain an element that supports TextPattern.";
        targetResult.Background = Brushes.Salmon;
        startWPFTargetButton.IsEnabled = false;
        return;
    }

    // Text control is available so display the client controls.
    infoGrid.Visibility = Visibility.Visible;

    targetResult.Content =
        "Text provider found.";
    targetResult.Background = Brushes.LightGreen;

    // Initialize the document range for the text of the document.
    documentRange = targetTextPattern.DocumentRange;

    // Initialize the client's search buttons. 
    if (targetTextPattern.DocumentRange.GetText(1).Length > 0)
    {
        searchForwardButton.IsEnabled = true;
    }
    // Initialize the client's search TextBox.
    searchString.IsEnabled = true;

    // Check if the text control supports text selection 
    if (targetTextPattern.SupportedTextSelection ==
        SupportedTextSelection.None)
    {
        targetResult.Content = "Unable to select text.";
        targetResult.Background = Brushes.Salmon;
        return;
    }

    // Edit control found so remove the find button from the client.
    findEditButton.Visibility = Visibility.Collapsed;

    // Initialize the client with the current target selection, if any.
    NotifySelectionChanged();

    // Search starts at beginning of doc and goes forward
    searchBackward = false;

    // Initialize a text changed listener. 
    // An instance of TextPatternRange will become invalid if  
    // one of the following occurs: 
    // 1) The text in the provider changes via some user activity. 
    // 2) ValuePattern.SetValue is used to programatically change  
    // the value of the text in the provider. 
    // The only way the client application can detect if the text  
    // has changed (to ensure that the ranges are still valid),  
    // is by setting a listener for the TextChanged event of  
    // the TextPattern. If this event is raised, the client needs  
    // to update the targetDocumentRange member data to ensure the  
    // user is working with the updated text.  
    // Clients must always anticipate the possibility that the text  
    // can change underneath them.
    Automation.AddAutomationEventHandler(
        TextPattern.TextChangedEvent,
        targetDocument,
        TreeScope.Element,
        TextChanged);

    // Initialize a selection changed listener. 
    // The target selection is reflected in the client.
    Automation.AddAutomationEventHandler(
        TextPattern.TextSelectionChangedEvent,
        targetDocument,
        TreeScope.Element,
        OnTextSelectionChange);
}


...


///-------------------------------------------------------------------- 
/// <summary> 
/// Gets the children of the target selection. 
/// </summary> 
/// <param name="sender">The object that raised the event.</param>
/// <param name="e">Event arguments.</param>
///-------------------------------------------------------------------- 
private void GetChildren_Click(object sender, RoutedEventArgs e)
{
    // Obtain an array of child elements.
    AutomationElement[] textProviderChildren;
    try
    {
        textProviderChildren = searchRange.GetChildren();
    }
    catch (ElementNotAvailableException)
    {
        // TODO: error handling. 
        return;
    }

    // Assemble the information about the enclosing element.
    StringBuilder childInformation = new StringBuilder();
    childInformation.Append(textProviderChildren.Length)
        .AppendLine(" child element(s).");

    // Iterate through the collection of child elements and obtain 
    // information of interest about each. 
    for (int i = 0; i < textProviderChildren.Length; i++)
    {
        childInformation.Append("Child").Append(i).AppendLine(":");
        // Obtain the name of the child control.
        childInformation.Append("\tName:\t")
            .AppendLine(textProviderChildren[i].Current.Name);
        // Obtain the control type.
        childInformation.Append("\tControl Type:\t")
            .AppendLine(textProviderChildren[i].Current.ControlType.ProgrammaticName);

        // Obtain the supported control patterns. 
        // NOTE: For the purposes of this sample we use GetSupportedPatterns().  
        // However, the use of GetSupportedPatterns() is strongly discouraged  
        // as it calls GetCurrentPattern() internally on every known pattern.  
        // It is therefore much more efficient to use GetCurrentPattern() for  
        // the specific patterns you are interested in.
        AutomationPattern[] childPatterns = 
            textProviderChildren[i].GetSupportedPatterns();
        childInformation.AppendLine("\tSupported Control Patterns:");
        if (childPatterns.Length <= 0)
        {
            childInformation.AppendLine("\t\t\tNone");
        }
        else
        {
            foreach (AutomationPattern pattern in childPatterns)
            {
                childInformation.Append("\t\t\t")
                    .AppendLine(pattern.ProgrammaticName);
            }
        }

        // Obtain the child elements, if any, of the child control.
        TextPatternRange childRange = 
            documentRange.TextPattern.RangeFromChild(textProviderChildren[i]);
        AutomationElement[] childRangeChildren = 
            childRange.GetChildren();
        childInformation.Append("\tChildren: \t").Append(childRangeChildren.Length).AppendLine();
    }
    // Display the information about the child controls.
    targetSelectionDetails.Text = childInformation.ToString();
}

The following code example demonstrates how to obtain a text range from an embedded object within a UI Automation text provider. The text range retrieved is an empty range where the starting endpoint follows "… ocean.(space)" and the ending endpoint precedes the closing "." representing the embedded hyperlink (as shown by the image provided in the introduction). Even though this is an empty range, it is not considered a degenerate range because it has a non-zero span.

Note Note

TextPattern can retrieve a text-based embedded object such as a hyperlink; however, a secondary TextPattern will have to be obtained from the embedded object to expose its full functionality.

/// ------------------------------------------------------------------- 
/// <summary> 
/// Obtains a text range spanning an embedded child  
/// of a document control and displays the content of the range. 
/// </summary> 
/// <param name="targetTextElement">
/// The AutomationElment that represents a text control. 
/// </param> 
/// ------------------------------------------------------------------- 
private void GetRangeFromChild(AutomationElement targetTextElement)
{
    TextPattern textPattern =
        targetTextElement.GetCurrentPattern(TextPattern.Pattern)
        as TextPattern;

    if (textPattern == null)
    {
        // Target control doesn't support TextPattern. 
        return;
    }

    // Obtain a text range spanning the entire document.
    TextPatternRange textRange = textPattern.DocumentRange;

    // Retrieve the embedded objects within the range.
    AutomationElement[] embeddedObjects = textRange.GetChildren();

    // Retrieve and display text value of embedded object. 
    foreach (AutomationElement embeddedObject in embeddedObjects)
    {
        if ((bool)embeddedObject.GetCurrentPropertyValue(
            AutomationElement.IsTextPatternAvailableProperty))
        {
           // For full functionality a secondary TextPattern should 
           // be obtained from the embedded object. 
           // embeddedObject must be a child of the text provider.
            TextPatternRange embeddedObjectRange =
                textPattern.RangeFromChild(embeddedObject);
            // GetText(-1) retrieves all text in the range. 
            // Typically a more limited amount of text would be  
            // retrieved for performance and security reasons.
            Console.WriteLine(embeddedObjectRange.GetText(-1));
        }
    }
}
Show:
© 2014 Microsoft