Consume token only when it matches a predicate?

[thejohnfreeman]

Is there a way to consume a token only when its image matches a predicate?

[msujew]

@thejohnfreeman You can probably accomplish this using a structure like this:

$.OR([
  {
    GATE: () => predicate(),
    ALT: () => $.CONSUME(<token>)
  },
  {
    GATE: () => !predicate(),
    ALT: EMPTY_ALT
  }
]);

Note that for specific use cases such as "does the image match a predicate" you can probably also use custom token patterns. They're probably more efficient, but they operate on the lexer level, not the parser level.

[msujew]

If it is just a single token using OPTION + GATE may be more concise

I believe that's syntactically different? Even if the GATE yields true, the production inside of the OPTION would still be optional, so it's different than the OR approach which would enforce parsing of the production. I guess it depends on what's exactly needed here. Both options might be valid.

[thejohnfreeman]

The OR approach worked for me. My situation is that I'm parsing someone else's grammar (a stack trace) and there are many "keywords", but I don't want to go through the trouble of writing out tokens for all of them.

[bd82]

I believe that's syntactically different? Even if the GATE yields true, the production inside of the OPTION would still be optional, so it's different than the OR approach

I think you are right and using the OPTIONAL construct may be confusing here.

There may be slight differences in edge cases here.
But these may be syntactically identical if we assume:

• if the predicate yields true --> the OPTION / non-empty branch will parse successfully.

OR

• If true --> non-empty alt
• if false --> skip (empty alt)

OPTION

• If true --> enter OPTION (non-empty alt)
• if false --> skip OPTION (empty alt)

Basically you can think of OPTION as a choice between two alternatives, a non-empty one and an empty one.

[thejohnfreeman]

Right. I should clarify I'm using OR with only one alternative so that it diagnoses an error if it fails to match.

[bd82]

Right. I should clarify I'm using OR with only one alternative so that it diagnoses an error if it fails to match.

Good point I have not considered that difference.