Was this page helpful?
Your feedback about this content is important. Let us know what you think.
Additional feedback?
1500 characters remaining
Export (0) Print
Expand All
Important This document may not represent best practices for current development, links to downloads and other resources may no longer be valid. Current recommended version can be found here.


Finds the next token in a target string

CStringT Tokenize(
   PCXSTR pszTokens,
   int& iStart
) const;



A string containing token delimiters. The order of these delimiters is not important.


The zero-based index to begin the search.

A CStringT object containing the current token value.

The Tokenize function finds the next token in the target string. The set of characters in pszTokens specifies possible delimiters of the token to be found. On each call to Tokenize the function starts at iStart, skips leading delimiters, and returns a CStringT object containing the current token, which is the string of characters up to the next delimiter character. The value of iStart is updated to be the position following the ending delimiter character, or -1 if the end of the string was reached. More tokens can be broken out of the remainder of the target string by a series of calls to Tokenize, using iStart to keep track of where in the string the next token is to be read. When there are no more tokens the function will return an empty string and iStart will be set to -1.

Unlike the CRT tokenize functions like strtok_s, _strtok_s_l, wcstok_s, _wcstok_s_l, _mbstok_s, _mbstok_s_l, Tokenize does not modify the target string.

//typedef CStringT< TCHAR, StrTraitATL< TCHAR > > CAtlString;
CAtlString str( "%First Second#Third" );
CAtlString resToken;
int curPos= 0;

resToken= str.Tokenize("% #",curPos);
while (resToken != "")
   printf("Resulting token: %s\n", resToken);
   resToken= str.Tokenize("% #",curPos);


Resulting Token: First
Resulting Token: Second
Resulting Token: Third

Community Additions

© 2015 Microsoft