Re: [EXTERN] Re: Definition of tokens in GP language

[Bill Allombert <Bill.Allombert@math.u-bordeaux.fr>]

On Fri, Oct 24, 2025 at 09:59:38PM +0200, Hong-Phuc Bui wrote:
> Hi Bill,
> 
> I took a look at the function. It is not really long, but I'm not sure about complexity:
> Is it possible to recognize all tokens of GP by using regular express?

Strictly speaking it is not possible using regex due to ambiguities.
For example
3!=6  is  3 != 6   (3 tokens)
while
3!==6 is  3 ! == 6 (4 tokens)
and
3!===6 is  3 ! === 6 (4 tokens)

There are other corner cases, depending how you define a token.

Note that all spaces are removed before parsing so for example
a ! = b  is a != b (3 tokens)

Cheers,
Bill.