API Documentation

Lark(grammar::String;options...)

Create a Lark parser based on the Lark grammar expressed in grammar. Options are as follows:

• parser Decides which parser engine to use, "earley" or "lalr". (Default: "lalr") Note: "lalr" requires a lexer. "earley" is not currently implemented.

• lexer Decides whether or not to use a lexer stage

  • "standard": Use a standard lexer

  • "contextual": Stronger lexer (only works with parser="lalr")

  • "auto" (default): Choose for me based on grammar and parser

• transformer - Applies the transformer to every parse tree

• debug - Affects verbosity (default: False)

• keep_all_tokens - Don't automagically remove "punctuation" tokens (default: False)

• postlex - Lexer post-processing (Requires standard lexer. Default: None)

• start - The start symbol (Default: start)

• propagate_positions - Propagates [line, column, end line, end column] attributes into all tree branches.

• lexer_callbacks - Dictionary of callbacks for the lexer. May alter tokens during lexing. Use with caution.

Lark(grammar::IOStream,source;options...)

Create a Lark parser based on the Lark grammar found by reading grammar, whose source is recorded as source. See above for options.

open(grammar_filename;rel_to=nothing,options...)

Open grammar_filename and read in a Lark grammar, returning a Lark parser. If rel_to is not nothing, grammar_filename is relative to rel_to. Options are the same as for Lark(grammar::String;options...).

parse(l::Lark,text;start=nothing,on_error=nothing)

Parse text using parser l, returning a parse tree. If start is not nothing, it identifies the start symbol. on_error is currently ignored.

If text does not conform to the grammar, either UnexpectedToken or UnexpectedCharacters is raised.

UnexpectedCharacters

This exception is raised when the next characters in the input stream cannot be matched with any acceptable tokens.

UnexpectedToken

This exception is raised when a token is encountered in the input stream that is not allowed by the grammar.

@rule s

s is a function rule_name(t,args) where t is an instance of a subtype of Transformer or Visitor. args is an array holding values for each node of the grammar production.

@inline_rule s

s is a function of the form rule_name(t,args...) where rule_name is the name of a grammar rule and args are values for the individual elements of that rule's grammar production. This rule is called by transform(t,tree) and visit(v,tree). t is a subtype of Transformer and v is a subtype of Visitor.

@terminal s

s is a function definition of form terminal_name(t,tok) = ... where t is an instance of a subtype of Transformer or Visitor. tok is a token of type terminal_name. If visit_tokens(::t) is true, this function will be called whenever tokens of type terminal_name are processed.

Transformers visit each node of the tree, and run the appropriate method on it according to the node's data.

The methods (provided by the user) are dispatched according to the type and the node's grammar rule. The returned value replaces the old one in the structure.

They work bottom-up (or depth-first), starting with the leaves and ending at the root of the tree. Transformers can be used to implement map - reduce patterns. Because nodes are reduced from leaf to root, at any point the callbacks may assume the children have already been transformed (if applicable).

Transformer can do anything Visitor can do, but because it reconstructs the tree, it is slightly less efficient.

NOTE: A transformer without methods essentially performs a non-memoized deepcopy.

Transformer_InPlace

Non-recursive Transformer. Changes the tree in-place instead of returning new instances

Transformer_InPlaceRecursive

Recursive. Changes the tree in-place instead of returning new instances.

visit_tokens(t::Transformer)

Should the transformer visit tokens in addition to rules. Setting this to false is an order of magnitude faster. For consistency with Lark Defaults to true. (For processing ignored tokens, use the $lexer_callbacks$ options)

transform(tr::Transformer,tree)

Transform parse tree tree according to rules defined for type tr. Typically tr is a subtype of Transformer.

Visitor

User-defined methods called with an instance of a subtype of Visitor will visit each node of the parse tree in top-down order without altering it or returning a transformed tree.

When called by `visit`, subtypes of `Visitor_Recursive` visit nodes of the parse tree
in bottom-up order.

Subtypes of the `Interpreter` type do not automatically visit children
of the parse tree node when called by `visit`.

visit(v::Visitor,tree)

Visit each node of tree, calling methods defined for v on each node.