
# Parser Toolkit

PTK - (c) Jérôme Laheurte 2015-2026

## What is PTK ?

PTK is a LR(1) parser "generator" for Python. It is not actually a
"generator" in the sense that it doesn't output source code, using
Python's dynamic nature to build everything it needs at runtime
instead.

This code is licensed under the [GNU LGPL version 3](https://www.gnu.org/copyleft/lesser.html)
or, at your option, any later version

## Why another one of those ?

There are a number of parser generators for Python out there. Most of
them only support LL(1) or PEG. The other ones are either

  - Unmaintained
  - Straight translations from Yacc/Bison, and thus use an ugly syntax
  - All of the above

The main goals of PTK are

  - Clean, compact, Python-friendly syntax
  - Support for "no I/O" style parsing; just feed the parser your tokens
    at your own pace
  - Play nice in 'special' cases, like when the underlying
    'filesystem' is a PyZipFile archive.
  - Don't use hacks like module-level introspection to compensate for
    an ugly design (I'm looking at you PLY). Those tend to produce
    subtle and headache-inducing bugs when running from compiled code.

## Supported platforms

Should work on any platform, using Python 3.7 or later.

## Installation

Using pip::

    $ pip install -U ptk

From source::

    $ wget https://pypi.python.org/packages/source/p/ptk/ptk-%(version)s.tar.gz
    $ tar xjf ptk-%(version)s.tar.bz2; cd ptk-%(version)s
    $ sudo python3 ./setup.py install

## Sample usage

Four-operations integer calculator:

    from ptk.parser import LRParser, production, left_assoc
    from ptk.lexer import ReLexer, token
    import operator

    @left_assoc('+', '-')
    @right_assoc('*', '/')
    class Parser(LRParser, ReLexer):
        @token('[1-9][0-9]*')
        def number(self, tok):
            tok.value = int(tok.value)
        @production('E -> number<n>')
        def litteral(self, n):
            return n
        @production('E -> "-" E<val>', priority='*')
        def minus(self, val):
            return -val
        @production('E -> "(" E<val> ")"')
        def paren(self, val):
            return val
        @production('E -> E<left> "+"<op> E<right>')
        @production('E -> E<left> "-"<op> E<right>')
        @production('E -> E<left> "*"<op> E<right>')
        @production('E -> E<left> "/"<op> E<right>')
        def binaryop(self, left, op, right):
            return {
                '+': operator.add,
                '-': operator.sub,
                '*': operator.mul,
                '/': operator.floordiv
                }[op](left, right)

    parser = Parser()
    while True:
        expr = input('> ')
        print parser.parse(expr)

## Code samples

The *samples* subdirectory in the source tree contains the
aforementioned calculator and a script that generates a skeleton
Python file from a Yacc or Bison grammar file.

## API documentation

The full documentation is hosted [on readthedocs](http://ptk.readthedocs.io/en/latest/).

## Changelog

Version 2.0.0:

- Remove all of the async/deferred stuff
- Use snake_case instead of camelCase (camelCase methods are kept but deprecated)
- parse() now returns the start symbol value
- Needs Python 3.7 at least

Version 1.3.8:

- Add more error handling stuff

Version 1.3.7:

- Add token and production positions in source

Version 1.3.6:

- Drop support for Python versions older than 3.3.

Version 1.3.5:

- Update copyright notices
- Fix some packaging issues

Version 1.3.4:

- Added SkipToken for consumers

Version 1.3.3:

- Fix a number of problems when working with bytes in Python 3
- One couldn't mix asynchronous and synchronous parsers in the same program

Version 1.3.2:

- Fix Python regular expression based lexer (use match instead of search)

Version 1.3.1:

- Fix version number in README.rst

Version 1.3.0:

- Added deferred_lexer and deferred_parser (asynchronous parsing using
  Twisted Deferred objects)
- Asynchronous classes cannot be imported from 'regular' modules
  anymore, import them explicitely from 'ptk.async_lexer' and 'ptk.async_parser'.

Version 1.2.0:

- Production methods cannot have the same name any more. This was
  idiotic to begin with. Inheritance thus works as expected.
- Add AsyncLexer and AsyncLRParser for asynchronous parsing.

Version 1.1.0:

- Added repeat operators ('*', '+', '?') in production syntax.
- Support for more yacc/bison declarations in yacc2py sample (most are
  actually ignored)
