Text Control Pattern and Embedded Objects Overview
This overview describes how Microsoft UI Automation exposes embedded objects, or child elements, in a text document or container.
In UI Automation, an embedded object is any element that has nontextual boundaries; for example, an image, hyperlink, table, or document type such as an Microsoft Excel spreadsheet or Windows Media file (WMF). This differs from the standard definition, where an element is created in one application and embedded or linked in another application. Whether the object can be edited in its original application is irrelevant in the context of UI Automation.
This topic contains the following sections:
- Embedded Objects and the UI Automation Tree
- Exposing Embedded Objects
- Common Scenarios
- Related Topics
Embedded Objects and the UI Automation Tree
Embedded objects are treated as individual elements in the Control view of the UI Automation tree. They are exposed as children of the text container so that they can be accessed through the same model as other controls in UI Automation.
The following example shows a text container (document) with an embedded table and image.
.jpg "Example showing a document with an embedded table and image.")
The UI Automation Content view of the preceding document is shown in the following diagram.
.jpg "UI Automation Content view of the document with embedded objects")
Exposing Embedded Objects
The Text control pattern and TextRange control pattern expose properties and methods that facilitate navigation and querying of embedded objects.
The textual content (or inner text) of a text container and an embedded object, such as a hyperlink or table cell, is exposed as a single, continuous text stream in both the Control view and the Content view of the UI Automation tree; object boundaries are ignored. If a UI Automation client is retrieving the text to recite, interpret, or analyze in some manner, the text range should be checked for special cases, such as a table with textual content or other embedded objects. Call IUIAutomationTextRange::GetChildren to obtain a IUIAutomationElement interface for each embedded object, and then call IUIAutomationTextPattern::RangeFromChild to obtain a text range for each element. This is done recursively until all textual content has been retrieved.
The following diagram shows a text stream with embedded objects and their range spans.
.jpg "Diagram showing a text stream with embedded objects and their range spans.")
To traverse the content of a text range, a series of steps are involved behind the scenes for the IUIAutomationTextRange::Move method to execute successfully.
- The text range is normalized. That is, the text range is collapsed to a degenerate range at the Start endpoint, which makes the End endpoint superfluous. This step is necessary to remove ambiguity in situations where a text range spans TextUnit boundaries.For example, "{The U}RL https://www.microsoft.com is embedded in text" where "{" and "}" are the text range endpoints.
- The resulting range is moved backward in the document range to the beginning of the requested TextUnit boundary.
- The range is moved forward or backward in the document range by the requested number of TextUnit boundaries.
- The range is expanded from a degenerate range state by moving the End endpoint by one requested TextUnit boundary.
The following diagram shows how the IUIAutomationTextRange::ExpandToEnclosingUnit method adjusts a text range after a Move operation.
.jpg "Diagram showing how a text range is adjusted with ExpandToEnclosingUnit.")
Common Scenarios
This section presents examples of common scenarios that involve embedded objects: hyperlinks, images, and tables. In the following examples, the left brace ({) represents the Start endpoint of the text range, and the right brace (}) represents the End endpoint.
HyperLink Example 1: A text range that contains an embedded text hyperlink
The following text range contains an embedded text hyperlink.
{The URL https://www.microsoft.com is embedded in text}.
Calling the IUIAutomationTextRange::GetText, GetEnclosingElement, GetChildren, and IUIAutomationTextPattern::RangeFromChild methods results in the behaviors described in the following table.
| Method called | Result |
|---|---|
| IUIAutomationTextRange::GetText | Returns the string "The URL https://www.microsoft.com is embedded in text". |
| IUIAutomationTextRange::GetEnclosingElement | Returns the innermost UI Automation that encloses the text range, in this case, the AutomationElement that represents the text provider itself. |
| IUIAutomationTextRange::GetChildren | Returns a UI Automation element representing the hyperlink control. |
| IUIAutomationTextPattern::RangeFromChild, where the UI Automation element was returned by the previous IUIAutomationTextRange::GetChildren method. | Returns the range that represents "https://www.microsoft.com". |
HyperLink Example 2: A text range that partially spans an embedded text hyperlink
The following text range partially spans an embedded text hyperlink.
The URL https://{www} is embedded in text.
Calling the IUIAutomationTextRange::GetText, GetEnclosingElement, and GetChildren methods results in the behaviors described in the following table.
| Method called | Result |
|---|---|
| IUIAutomationTextRange::GetText | Returns the string "www". |
| IUIAutomationTextRange::GetEnclosingElement | Returns the innermost UI Automation element that encloses the text range, in this case, the hyperlink control. |
| IUIAutomationTextRange::GetChildren | Returns NULL because the text range does not span the entire URL string. |
HyperLink Example 3: A text range that partially spans the content of a text container
The following text range partially spans the content of a text container. The text container has an embedded text hyperlink that is not part of the text range.
{The URL} https://www.microsoft.com is embedded in text.
Calling the IUIAutomationTextRange::GetText, GetEnclosingElement, and Move methods results in the behaviors described in the following table.
| Method called | Result |
|---|---|
| IUIAutomationTextRange::GetText | Returns the string "The URL". |
| IUIAutomationTextRange::GetEnclosingElement | Returns the innermost UI Automation element that encloses the text range, in this case, the element that represents the text provider itself. |
| IUIAutomationTextRange::Move | Moves the text range span to "http" because the text of the hyperlink is made up of individual words. In this case, the hyperlink is not treated as a single object. The URL {http} is embedded in text. |
Image Example 1: A text range that contains an embedded image
The following text range contains an embedded image of a shuttle.
{The image.jpg "An embedded picture of a shuttle.")
is embedded in text}.
Calling the IUIAutomationTextRange::GetText, GetEnclosingElement, GetChildren, and IUIAutomationTextPattern::RangeFromChild methods results in the behaviors described in the following table.
| Method called | Result |
|---|---|
| IUIAutomationTextRange::GetText | Returns the string "The image is embedded in text". Any ALT text associated with the image is not included in the text stream. |
| IUIAutomationTextRange::GetEnclosingElement | Returns the innermost UI Automation element that encloses the text range, in this case, the element that represents the text provider itself. |
| IUIAutomationTextRange::GetChildren | Returns a UI Automation element representing the image control. |
| IUIAutomationTextPattern::RangeFromChild where the UI Automation element was returned by the previous IUIAutomationTextRange::GetChildren method. | Returns the degenerate range that represents " ". |
Image Example 2: A text range that partially spans the content of a text container
The following text range partially spans the content of a text container. The text ontainer has an embedded image that is not part of the text range.
{The image}is embedded in text.
Calling the IUIAutomationTextRange::GetText, GetEnclosingElement, and Move methods results in the behaviors described in the following table.
| Method called | Result |
|---|---|
| IUIAutomationTextRange::GetText | Returns the string "The image". |
| IUIAutomationTextRange::GetEnclosingElement | Returns the innermost UI Automation element that encloses the text range, in this case, the element that represents the text provider itself. |
| IUIAutomationTextRange::Move with parameters of (TextUnit_Word, 1). | Moves the text range span to "is ". Because only text-based embedded objects are considered part of the text stream, the image in this example does not affect Move or its return value, in this case, 1. |
Table
Table Example 1: Gets the text container from the content of a cell
The following table gets the text container from the content of a cell.
| Cell with Image | Cell with Text |
|---|---|
|
X |
.jpg "A drawing of space and a telescope.") |
Y |
.jpg "A picture of a microscope.") |
Z |
Calling the IUIAutomationGridPattern::GetItem, IUIAutomationTextPattern::RangeFromChild, and IUIAutomationTextRange::GetEnclosingElement methods results in the behaviors described in the following table.
| Method called | Result |
|---|---|
| IUIAutomationGridPattern::GetItem with parameters (0, 0). | Returns the UI Automation element that represents the content of the table cell, in this case, the element is a text control. |
| IUIAutomationTextPattern::RangeFromChild | Returns the range that spans the image . |
| IUIAutomationTextRange::GetEnclosingElement for the object returned by the previous RangeFromChild method. | Returns the UI Automation element representing the table cell. In this case, the element is a text control that supports the TableItem control pattern. |
| IUIAutomationTextRange::GetEnclosingElement for the object returned by the previous GetEnclosingElement method. | Returns the UI Automation element representing the table. |
| IUIAutomationTextRange::GetEnclosingElement for the object returned by the previous GetEnclosingElement method. | Returns the UI Automation element that represents the text provider itself. |
Table Example 2: Gets the text content of a cell
The table in the preceding example gets the text content of a cell.
Calling the IUIAutomationGridPattern::GetItem and RangeFromChild methods results in the behaviors described in the following table.
| Method called | Result |
|---|---|
| IUIAutomationGridPattern::GetItem with parameters (1,1). | Returns the UI Automation element representing the content of the table cell. In this case, the element is a text control. |
| IUIAutomationTextPattern::RangeFromChild where the UI Automation element is the object returned by the previous GetItem method. | Returns "Y". |