GetStringTokenizer

GetStringTokenizer(string, string)

Create a string tokenizer for a given string with a specified delimiter

struct sStringTokenizer GetStringTokenizer(
    string sString,
    string sDelimiter
);

Parameters

sString

The string to be split up into parts (tokens.)

sDelimiter

The delimiter used to split up the string.

Description

This function is to be used in combination with the other functions from x0_i0_stringlib in order to split up a string into parts (tokens).

GetStringTokenizer() requires two arguments: The string that is to be split up into tokens (sString) and the delimiter used to split up the string (sDelimiter). The delimiter MUST be a string containing a single character.

Remarks

Contrary to the description found in x0_i0_stringlib , the DELIMITER MUST be a SINGLE CHARACTER. Otherwise the tokenization process does not work as expected. The reason is that the function x0_i0_stringlib , a token is any substring (including empty substrings!) within the original string enclosed by the specified delimiter (e.g. every token has one delimiter to the left, one to the right and NO delimiters within). Any non-empty original string is treated as if it were enclosed within a pair of (virtual) delimiters to its left and right. Thus, unless the original string is empty, the number of tokens is always one higher than the number of delimiters contained within the original string. A non-empty string with no delimiters therefore consists out of one token, which is equal to the original string. An empty string has no tokens, although an empty token will be returned on request.

Example:

sString = "I|am|sloppy||programmer";
sDelimiter = "|";

Token[[0] | = "I";
Token[1 ]] = "am";
Token[[2] | = "sloppy";
Token[3 | x0_i0_stringlib ]], a token is any substring (including empty substrings!) within the original string enclosed by the specified delimiter (e.g. every token has one delimiter to the left, one to the right and NO delimiters within). Any non-empty original string is treated as if it were enclosed within a pair of (virtual) delimiters to its left and right. Thus, unless the original string is empty, the number of tokens is always one higher than the number of delimiters contained within the original string. A non-empty string with no delimiters therefore consists out of one token, which is equal to the original string. An empty string has no tokens, although an empty token will be returned on request.

Example:

sString = "I|am|sloppy||programmer";
sDelimiter = "|";

Token[[0] | = "I";
Token[1 ]] = "am";
Token[[2] | = "sloppy";
Token[3 ]] = "";
Token[4] = "programmer";

sString contains five tokens and four delimiters.

Known Bugs

Contrary to the description found in the include file, the delimiter must be a single character (see remarks above).

Requirements

#include " x0_i0_stringlib "

Version

???

See Also

functions:

HasMoreTokens

---

author: motu99, editors: Mistress, Kolyana