Dynamic grammar generator for [Lark parsing toolkit](https://github.com/lark-parser/lark)

# Why?

While writing EBNF by hand is a preferred approach for most use cases (because of rich syntax), it's only viable for static grammars.    
If your grammar is variable (e.g. based on some kind of configuration), it should be built with code on the fly.    
As per [lark#439](https://github.com/lark-parser/lark/issues/439), no API for grammar exists, so building a string is only solution (it's pretty fast though).

**This README implies you are familiar with Lark and Lark EBNF.**    

# Basic usage

```python
from lark_dynamic import *

g = Grammar()

g.DIGIT = g.NONZERO | "0"
g.NONZERO = Range("1", "9")
g.INTEGER = makeBoolVariable(
    "zero_leading_numbers", 
    true=Many(g.DIGIT), 
    false=(g.NONZERO, Some(g.DIGIT))
)
g.FLOAT[2] = Group(g.INTEGER, ".", Some(g.DIGIT)) | (".", Many(g.DIGIT))

g.number = Alias.integer(g.INTEGER) | Alias.float(g.FLOAT)

print(g.generate(zero_leading_numbers=False))
print(g.generate(zero_leading_numbers=True))
```

Which outputs:    
*zero_leading_numbers=False*:    
```
DIGIT: NONZERO | "0"
NONZERO: "1".."9"
INTEGER: ( NONZERO ( DIGIT )* )
FLOAT.2: ( INTEGER "." ( DIGIT )* ) | ( "." ( DIGIT )+ )

number: INTEGER -> integer | FLOAT -> float
```

*zero_leading_numbers=True*    
```
DIGIT: NONZERO | "0"
NONZERO: "1".."9"
INTEGER: ( DIGIT )+
FLOAT.2: ( INTEGER "." ( DIGIT )* ) | ( "." ( DIGIT )+ )

number: INTEGER -> integer | FLOAT -> float
```

# Getting started

To create a grammar, you need to create `Grammar` object:
```python
from lark_dynamic import * # it is recommended to create grammar in a separate file, so * import is fine

g = Grammar() # short name is convenient
```

Then write rules and terminals:

```python
# this is a terminal (uppercase)
g.NUMBER = RegExp(r'[0-9]+')
g.SIGN = Literal("+") | "-" # first (or second) string needs to be wrapped into a Literal to support |
g.test = Maybe(g.SIGN), Many(g.TEST)
```

# Reference

All code snippets use this setup:
```python
from lark_dynamic import *
g = Grammar()
```

## Grammar

Main object, stores rules, terminals, templates and directives

To create a rule, you can use a shorthand:

```python
g.any_lowercase_identifier = ... # rule contents here
```

Same for terminal:

```python
g.ANY_UPPERCASE_IDENTIFIER = ... # terminal contents here
```

To create a template (`name{arg1, arg2} = ...`) you need to use `[]`:

```python
g.template_name[g.arg1, g.arg2] = ... # template contents here
```

Directives have different syntaxes, so no special shorthand is present:

```python
g.make_directive("name", "directive contents here")
```

To build grammar, you need to call `.generate(**context)`:

```python
g.generate(some_variable=True) # variables are discussed below
```

## Variable

Since the purpose of the module is to create grammars dynamically, this class exists.    
It takes a function, which accepts one argument (context) - a dictionary of everything passed to `.generate()`.    
Function sould return anything renderable (e.g. a token or a tuple of tokens):

```python
def integer_term(context):
    if context.get("zero_leading_numbers"):
        return Many(g.DIGIT)
    return g.NONZERO, Some(g.DIGIT)

g.DIGIT = g.NONZERO | "0"
g.NONZERO = Range("1", "9")
g.INTEGER = Variable(integer_term)
g.FLOAT[2] = Group(g.INTEGER, ".", Some(g.DIGIT)) | (".", Many(g.DIGIT))

g.number = Alias.integer(g.INTEGER) | Alias.float(g.FLOAT)
```
*(this is an example from Basic usage section rewritten to use Variable)*

### BoolVariable and makeBoolVariable

Common use case for Variable is to render different content based on a boolean key (such as in Basic usage example).    
`BoolVariable` makes it more convenient:

```python
def integer_term(zln):
    if zln:
        return Many(g.DIGIT)
    return g.NONZERO, Some(g.DIGIT)

g.DIGIT = g.NONZERO | "0"
g.NONZERO = Range("1", "9")
g.INTEGER = BoolVariable(integer_term, key="zero_leading_numbers")
g.FLOAT[2] = Group(g.INTEGER, ".", Some(g.DIGIT)) | (".", Many(g.DIGIT))

g.number = Alias.integer(g.INTEGER) | Alias.float(g.FLOAT)
```

`makeBoolVariable` goes even further, providing arguments for `True` and `False` values:

```python
g.DIGIT = g.NONZERO | "0"
g.NONZERO = Range("1", "9")
g.INTEGER = makeBoolVariable(
    "zero_leading_numbers", 
    true=Many(g.DIGIT), 
    false=(g.NONZERO, Some(g.DIGIT))
)
g.FLOAT[2] = Group(g.INTEGER, ".", Some(g.DIGIT)) | (".", Many(g.DIGIT))

g.number = Alias.integer(g.INTEGER) | Alias.float(g.FLOAT)
```

## Literal

Used for literal strings:

```python
g.hello = Literal("Hello"), Literal("World")
```
yields:

```
hello: "Hello" "World"
```

In most cases you can just use a string. Explicit `Literal(...)` is needed to support `|` (see [Combinators/Option](#option-usage-rules))

### Using `i` flag

```python
g.hello = Literal("Hello").i, Literal("World").i
```
yields:
```
hello: "Hello"i "World"i
```

## Combinators

### Maybe (aka QuestionMark)

Corresponds to `?` combinator in Lark:

```python
g.hello = "Hello", Maybe(","), "World", QuestionMark("!") # QuestionMark is just an alias for Maybe
```
yields:

```
hello: "Hello" (",")? "World" ("!")?
```

### Some (aka Star)

Corresponds to `*` combinator in Lark:

```python
g.hello = "Hello", Maybe(","), "World", Some("!")
```
yields:

```
hello: "Hello" (",")? "World" ("!")*
```

### Many (aka Plus)

Corresponds to `+` combinator in Lark:

```python
g.POSITION = Literal("two number 9s") | "a number 9 large" | "a number 6 with extra dip" | "a number 7" | "two number 45s" | "one with cheese" | "a large soda"

g.order = "I'll have ", g.position, Many(", ", g.position), Maybe(", and ", g.position)
g.position = g.POSITION
```
yields:

```
POSITION: "two number 9s" | "a number 9 large" | "a number 6 with extra dip" | "a number 7" | "two number 45s" | "one with cheese" | "a large soda"

order: "I'll have " position ( ", " position )+ ( ", and " position )?
position: POSITION
```

### Optional (aka Brackets)

Corresponds to `[...]` combinator in Lark:

```python
g.hello = "Hello", Optional(","), "World", ["!"]
```
yields:

```
hello: "Hello" [","] "World" ["!"]
```

Can also be written as `[...]` (as a list, basically), but be aware of [`|` usage rules (below)](#option-usage-rules)

### Option 

Corresponds to `|` combinator in Lark:

```python
g.hello = "Hello", Maybe(","), (Option("World", "dlroW"), ), Some("!") # note Option is placed inside a tuple, (explained below)
```
yields:

```
hello: "Hello" (",")? ("World" | "dlroW") ("!")*
```

Can also be written as `|`:
```python
g.hello = "Hello", Maybe(","), (Literal("World") | "dlroW", ), Some("!") # note Literal (explained below)
```

#### Option usage rules

**`|` priority in Lark is higher than in Python:**

`Maybe(","), Option("World", "dlroW"), Maybe("!")`    
yields    
`(",")? "World" | "dlroW" ("!")?`    
which, in turn, is equal to    
`(","? "World") | ("dlroW" "!"?)`    

You should wrap option with any other combinator (e.g. `Group()` or tuple. Remember that `(x)` is not a tuple) or just use `OptionG`.    
The reason why default `Option` doesn't do this is that wrapping top-level option in parens would break alias creation.

**`|` operator would not work with python built-in types (str, list or tuple)**

You need at least one `Token` (rule, combinator, literal etc.) on either side of `|` for it to work.

### Group (aka Parens)

Corresponds to `(...)` combinator in Lark:

```python
g.hello = "Hello", Group(Maybe(","), "World"), (Some("!"), )
```
yields:

```
hello: "Hello" ((",")? "World") (("!")*)
```

Can also be written as `(...)` (as a tuple, basically), but be aware of [`|` usage rules](#option-usage-rules)    
`Some`, `Many` and `Maybe` also will render in parenthesis by default to avoid combinator collision (e.g. to avoid rendering `(("!")+)?` as `"!"+?` which is invalid).

### SomeSeparated and ManySeparated

Correspond to `content (sep, content)*` and `content (sep, content)+` patterns, respectively:

```python
g.POSITION = Literal("two number 9s") | "a number 9 large" | "a number 6 with extra dip" | "a number 7" | "two number 45s" | "one with cheese" | "a large soda"
g._COMMA = ",", Maybe(g.WS)
g.WS = " "

g.order = "I'll have ", SomeSeparated(g._COMMA, g.position), Maybe(g._COMMA, "and ", g.position)
g.position = g.POSITION
```
yields:

```
POSITION: "two number 9s" | "a number 9 large" | "a number 6 with extra dip" | "a number 7" | "two number 45s" | "one with cheese" | "a large soda"
_COMMA: "," ( WS )?
WS: " "

order: "I'll have " (position ( _COMMA position )*) ( _COMMA "and " position )?
position: POSITION
```

## RegExp

Corresponds to Lark regexp syntax (`/content/flags`):

```python
g.WORD = RegExp(r"\w+") # flags are provided as a second argument (e.g. RegExp(r"\w+", "is")) 
g.some_phrase = g.WORD, Maybe(","), g.WORD, Some("!")
```
yields:
```
WORD: /\w+/

some_phrase: WORD ( "," )? WORD ( "!" )*
```

## Repeat

Corresponds to Lark repeat syntax: `item ~ n` and `item ~ n..m`:

```python
g.three_lemons = Repeat("Lemon", 3)
g.three_to_five_apples = Repeat("Lemon", [3, 5])
```
yields:
```
three_lemons: ( "Lemon" ) ~ 3
three_to_five_apples: ( "Lemon" ) ~ 3..5
```

## Prerendered

If you want to insert a prerendered content into a rule/terminal/template, you can use `Prerendered`:

```python
g.hello = "Hello", Prerendered('((",")? "World")'), Some("!")
```
yields:

```
hello: "Hello" ((",")? "World") ("!")*
```

### Empty

Sometimes (when using Variable) you'll need to render nothing.    
It can be achieved with `Prerendered("")`, or its alias, `Empty`

## Modifiers

Rule and terminal modifiers (such as `?` and `!`) can be used with `Modifier` class (**works only on top level of rule/terminal**):

```python
g.parens = Modifier.INLINE_SINGLE("(", g.word, ")")
g.word = g.WORD
g.WORD = RegExp(r"\w+")
```
yields:
```
WORD: /\w+/

?parens: "(" word ")"
word: WORD
```

# Priority

To use terminal/rule priority, you can use `[priority]` notation (similar to templates, but with a number):

```python
g.DIGIT = g.NONZERO | "0"
g.NONZERO = Range("1", "9")
g.INTEGER = g.NONZERO, Some(g.DIGIT)
g.FLOAT[2] = Group(g.INTEGER, ".", Some(g.DIGIT)) | (".", Many(g.DIGIT))

g.number = Alias.integer(g.INTEGER) | Alias.float(g.FLOAT)
```
yields:
```
DIGIT: NONZERO | "0"
NONZERO: "1".."9"
INTEGER: NONZERO ( DIGIT )*
FLOAT.2: ( INTEGER "." ( DIGIT )* ) | ( "." ( DIGIT )+ )

number: INTEGER -> integer | FLOAT -> float
```