NAME

tokenize − split string into words

SYNOPSIS

include "sys.m";
sys := load Sys Sys->PATH;
 tokenize: fn(s, delim: string): (int, list of string);

DESCRIPTION

­Tokenize breaks ­s into words separated by characters in delim. The returned tuple holds the number of words and an ordered list of those words (whose ­hd gives the leftmost word from s).

Words are delimited by the maximal sequences of any character from the ­delim string.  ­Tokenize skips delimiter characters at the beginning and end of s, so each element in the returned list has non-zero length.

If ­s is ­nil or contains no words, ­tokenize returns a count of zero and a ­nil list. 

­Delim may be ­nil or the empty string, specifying no delimiter characters.  The resulting word list will be ­nil (if ­s is ­nil or the empty string) or a single-item list with a copy of s.

SEE ALSO

sys-intro(2)

Plan 9  —  June 29, 2000