CharLexer

Simple lexer, which treats every byte as a token. This is only used for tests.

struct CharLexer {

string input;

this(string input, LocationBytes startLocation);

template tokenID(string tok);

string tokenName(SymbolID id);

void popFront();

}

Constructors

this this(string input, LocationBytes startLocation): Creates a character based lexer.

Members

Functions

popFront void popFront(): Advances to the next token and updates front and empty.
tokenName string tokenName(SymbolID id): Gets the name for token with ID id.

Structs

Front struct Front: Stores information about the current token.

Templates

tokenID template tokenID(string tok): Gets the internal ID for token with name tok.

Variables

empty bool empty;: True if all tokens are consumed.
front Front front;: Stores information about the current token.
input string input;: Undocumented in source.

Meta

Source

See Implementation

dparsergen core charlexer structs

CharLexer