From: Pieter de Ruiter 
>For my money, Pieter probably needs to write a custom GetToken() function 
that
>applies the same rules to all delimiters.
>
>Cheers,
>Ross
Yes indeed. I already did, see below. The mGetToken() has the same behaviour
as the strtok() in C-language.
>----------
>From: chrisant
>To: kcarr; tsepro
>Subject: RE: GetToken() Bug??
>Date: Monday, 15 December 1997 13:46
> ...
>
>the GetToken command does treat whitespace and non-whitespace characters
>differently, to mimic this.  this is a very useful feature.  GetToken
>also understand quotes.  if you use GetToken('foo "a b c" bar', ' "', 2)
>the result is not 'a', it is 'a b c'.  that is, the double quotes are
>stripped, and everything between them is returned.
Yes it's useful in many cases, like parsing DBF-outputfiles with ';'
delimiters (on the other hand you will specify ";" and not " ;" as delims,
if you don't want to trim every field).
But in my case you can see it's not the case. I've written a macro to
'smart-jump' from an error-outputfile to the source file. It understands
many compileroutput like SAL, C, C++, Basic etc. So it has to understand:
SomeFile.c      : 30, 17 'Undeclared identifier'
 - as well as something like -
SomeFile.c (30, 17) 'Undeclared identifier'
In both cases i'm only interested in the tokens 'SomeFile.c', '30', 17 and
the rest (the message) i'll have to grab with other delims.
See below.
proc mTestToken(string sSearch, string sDelim)
    integer i, nNum = NumTokens(sSearch,sDelim )
    addline("Number of tokens: " + str(nNum))
    for i=1 to nNum
        addline(str(i) + " " + GetToken(sSearch, sDelim,i) + "-")
    endfor
end
proc mTest()
     mTestToken('SomeFile.c    30,12 "Some error"', " :(),")
     mTestToken('SomeFile.c  :  (30,12) "Some error"', " :(),")
     addline("By the way:")
     mTestToken('foo "a b c" bar', ' "')        
end
Output:
-------
Number of tokens: 5
1 SomeFile.c-
2 30-
3 12-
4 "Some-
5 error"-
Number of tokens: 9
1 SomeFile.c-
2 -
3 -
4 -
5 30-
6 12-
7 -
8 "Some-
9 error"-
By the way:
Number of tokens: 7
1 foo-
2 -
3 a-
4 b-
5 c-
6 -
7 bar-
These problems are solved when mNumToken() and mGetToken() is used. The
tokens have the same indices.
proc mTestToken(string sSearch, string sDelim)
    integer i, nNum = mNumTokens(sSearch,sDelim )
    addline("Number of tokens: " + str(nNum))
    for i=1 to nNum
        addline(str(i) + " " + mGetToken(sSearch, sDelim,i) + "-")
    endfor
end
proc mTest()
     mTestToken('SomeFile.c    30,12 "Some error"', " :(),")
     mTestToken('SomeFile.c  :  (30,12) "Some error"', " :(),")
     addline("By the way, not a blank as delim works like:")
     mTestToken('foo "a b c" bar', '"')
end
Number of tokens: 5
1 SomeFile.c-
2 30-
3 12-
4 "Some-
5 error"-
Number of tokens: 5
1 SomeFile.c-
2 30-
3 12-
4 "Some-
5 error"-
By the way, not a blank as delim works like:
Number of tokens: 3
1 foo -
2 a b c-
3  bar-
Here are the functions:
----------------------------------------------------------------
integer proc mNumTokens(string s, string delimiter)
    integer i=0, num=0
    integer n=NumTokens(s, delimiter)
    while i * Origin: apana>>>>>fidonet [sawasdi.apana.org.au] (3:800/846.13)