Skip to main content
SelectableWordsSegmenter.Tokenize | tokenize method

Calls the provided handler with two iterators that iterate through the selectable words prior to and following a given index into the provided text.

Syntax


selectableWordsSegmenter.tokenize(text, startIndex, handler);

public void Tokenize(
  string text, 
  uint startIndex, 
  SelectableWordSegmentsTokenizingHandler handler
)

Public Sub Tokenize(
  text As String,  
  startIndex As UInteger,  
  handler As SelectableWordSegmentsTokenizingHandler 
)

public:
void Tokenize(
  String^ text, 
  unsigned int startIndex, 
  SelectableWordSegmentsTokenizingHandler^ handler
)

Parameters

text

Type: String [JavaScript] | System.String [.NET] | Platform::String [C++]

Provided text containing words to be selected.

startIndex

Type: Number [JavaScript] | System.UInt32 [.NET] | uint32 [C++]

A zero-based index into text. It must be less than the length of text.

handler

Type: SelectableWordSegmentsTokenizingHandler

The function that receives the iterators.

Remarks

The iterators in SelectableWordSegmentsTokenizingHandler are lazy and evaluate small chunks of text at a time.

The handler is called at most once per call to Tokenize. The handler is not called if there are no selectable words in text.

Requirements (Windows 10 device family)

Device family

Universal, introduced version 10.0.10240.0 [for Xbox, see UWP features that aren't yet supported on Xbox]

API contract

Windows.Foundation.UniversalApiContract, introduced version 1.0

Namespace

Windows.Data.Text
Windows::Data::Text [C++]

Metadata

Windows.Foundation.UniversalApiContract.winmd

Requirements (Windows 8.x and Windows Phone 8.x)

Minimum supported client

Windows 8.1

Minimum supported server

Windows Server 2012 R2

Minimum supported phone

None supported

Namespace

Windows.Data.Text
Windows::Data::Text [C++]

Metadata

Windows.winmd

See also

SelectableWordsSegmenter
SelectableWordSegmentsTokenizingHandler