CStringT::Tokenize
Finds the next token in a target string
CStringT Tokenize( PCXSTR pszTokens, int& iStart ) const;
Parameters
- pszTokens
-
A string containing token delimiters. The order of these delimiters is not important.
- iStart
-
The zero-based index to begin the search.
The Tokenize function finds the next token in the target string. The set of characters in pszTokens specifies possible delimiters of the token to be found. On each call to Tokenize the function starts at iStart, skips leading delimiters, and returns a CStringT object containing the current token, which is the string of characters up to the next delimiter character. The value of iStart is updated to be the position following the ending delimiter character, or -1 if the end of the string was reached. More tokens can be broken out of the remainder of the target string by a series of calls to Tokenize, using iStart to keep track of where in the string the next token is to be read. When there are no more tokens the function will return an empty string and iStart will be set to -1.
Unlike the CRT tokenize functions like strtok_s, _strtok_s_l, wcstok_s, _wcstok_s_l, _mbstok_s, _mbstok_s_l, Tokenize does not modify the target string.
//typedef CStringT< TCHAR, StrTraitATL< TCHAR > > CAtlString;
CAtlString str( "%First Second#Third" );
CAtlString resToken;
int curPos= 0;
resToken= str.Tokenize("% #",curPos);
while (resToken != "")
{
printf("Resulting token: %s\n", resToken);
resToken= str.Tokenize("% #",curPos);
};
Output
Resulting Token: First Resulting Token: Second Resulting Token: Third
Quote:
Shouldn't the example check while (curPos > 0) instead of resToken != "" ? What happens if you have a CSV file that has
value,value,,value,value,value
You will never parse the last 3 values.
Consecutive delimeters will be collapsed, so in your example it would correctly return 5 values e.g.:
Code:
CAtlString Input = "value1,value2,,\nvalue3,value4,value5\n";
CAtlString tok;
int pos = 0;
while( (tok=input.Tokenize(", \n", pos)) != "" )
printf("tok=%s\n", tok );
Output:
tok=value1
tok=value2
tok=value3
tok=value4
tok=value5
- 2/25/2010
- Todd Aspeotis
- 2/25/2010
- SamJ__
