lyn — Python bindings for GNU Lightning
=======================================

Lyn provides Python bindings for GNU Lightning, the light-weight just-in-time
(JIT) compiler that translates to native machine code.

The code is hosted on https://github.com/cslarsen/lyn/ and is installable from
https://pypi.python.org/pypi/lyn/ (although I currently don't update PyPi quite
often).

"Lyn" is the Norwegian word for "lightning".

**NOTE:** This project is currently in extremely early stages!

Status
------

As mentioned above, Lyn is fledgling: I've just spent a few hours on it, so
almost nothing is supported and — in particular — I've only targeted x86-64
opcodes just yet. Some tests are also failing: Min and max 64-bit numbers don't
seem to pass through correctly, so I'm currently working on fixing that issue
first.

In spite of this, I've managed to create a *really* simple program in Python
that is JIT-compiled to native x86-64 machine code: A glorious function that
returns the value of 123! Here's the code::

    from lyn import Lightning, Register

    with Lightning().state() as jit:
        jit.prolog()

        # Actual code
        jit.movi(Register.v0, 123)
        jit.retr(Register.v0)

        # Compile to native code and wrap in a Python-callable function
        func = jit.emit_function(Lightning.word_t)

        print("Function returned %s and that is %s!" % (
            func(), "correct" if func() == 123 else "incorrect"))

Also, I've not been able to compile GNU Lightning with disassembly support, so
I just used Capstone instead (install with ``pip install capstone``)::

    from lyn import *
    import capstone
    import ctypes

    lib = Lightning()
    jit = lib.new_state()

    # A function that returns one more than its integer input
    start = jit.note()
    jit.prolog()
    arg = jit.arg()
    jit.getarg(Register.r0, arg)
    jit.addi(Register.r0, Register.r0, 1)
    jit.retr(Register.r0)
    jit.epilog()
    end = jit.note()

    # Bind function to Python: returns a word (native integer), takes a word.
    incr = jit.emit_function(lib.word_t, [lib.word_t])

    # Sanity check
    assert(incr(1234) == 1235)

    # This part should be obvious to C programmers: We need to read data from raw
    # memory in to a Python iterable.
    length = (jit.address(end) - jit.address(start)).value
    codebuf = ctypes.create_string_buffer(length)
    ctypes.memmove(codebuf, ctypes.c_char_p(incr.address.value), length)
    print("Compiled %d bytes starting at 0x%x" % (length, incr.address))

    def hexbytes(b):
        return "".join(map(lambda x: hex(x)[2:] + " ", b))

    # Capstone is smart enough to stop at the first RET-like instruction.
    md = capstone.Cs(capstone.CS_ARCH_X86, capstone.CS_MODE_64)
    md.syntax = capstone.CS_OPT_SYNTAX_ATT # Change to Intel syntax if you want
    for i in md.disasm(codebuf, incr.address.value):
        print("0x%x %-15s%s %s" % (i.address, hexbytes(i.bytes), i.mnemonic, i.op_str))

    raw = "".join(map(lambda x: "\\x%02x" % x, map(ord, codebuf)))
    print("\nRaw bytes: %s" % raw)

This outputs::

    Compiled 34 bytes starting at 0x10acf4000
    0x10acf4000 48 83 ec 30    subq $0x30, %rsp
    0x10acf4004 48 89 2c 24    movq %rbp, (%rsp)
    0x10acf4008 48 89 e5       movq %rsp, %rbp
    0x10acf400b 48 83 ec 18    subq $0x18, %rsp
    0x10acf400f 48 89 f8       movq %rdi, %rax
    0x10acf4012 48 83 c0 1     addq $1, %rax
    0x10acf4016 48 89 ec       movq %rbp, %rsp
    0x10acf4019 48 8b 2c 24    movq (%rsp), %rbp
    0x10acf401d 48 83 c4 30    addq $0x30, %rsp
    0x10acf4021 c3             retq

    Raw bytes: \x48\x83\xec\x30\x48\x89\x2c\x24[...]

I'm using ctypes for creating the bindings, which comes with some challenges:
GNU Lightning is written in C, and relies heavily on compile-time macros that
define machine specific opcodes, register values and so on.

Because of this, it would be more natural to simply create bindings through a C
extension. On the other hand, though, ctypes makes it possible to ship Lyn as a
platform independent, pure Python source. I'll chew on this for a while, and
we'll see what happens.

Installation
------------

Either::

    $ pip install lyn

or::

    $ python setup.py install

Requirements
------------

You need the following C libraries:

    * GNU Lightning v2.1.0 shared library configured with ``--enable-shared``.
      http://www.gnu.org/software/lightning/

    * Capstone shared library, if you want to run ``examples/disassemble.py``.
      http://www.capstone-engine.org

The required Python modules should be automatically installed with ``python
setup.py install``, but you can also install them manually:

    * capstone, if you want to run ``examples/disassemble.py`` (on PyPi)
    * enum34 (and *not* the ``enum`` module that's on PyPi)
    * six, for Python 3.x compatibility.

Note that I had problems building GNU Lightning on Linux, because it had
problems linking with ``libopcodes``.  This worked for me::

    $ ./configure --enable-shared --disable-disassembler

After installing these libraries, you have to make sure they are in the library search
path.  If you can't instantiate a ``lyn.Lightning`` object, then you can try
adding its containing path to ``LD_LIBRARY_PATH``::

    $ LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/path/containing/liblightning python
    >>> import lyn
    >>> lib = lyn.Lightning()

... or you can set the path explicitly::

    $ python
    >>> import lyn
    >>> lib = lyn.Lightning("/usr/local/lib/liblightning.so")

To test that Capstone is working, you can run the disassembly example::

    $ python examples/disassemble.py

Author and license
------------------

Copyright (C) 2015 Christian Stigen Larsen

Distributed under the LGPL v2.1 or later. You are allowed to change the license
on a particular copy to the LGPL v3.0, the GPL v2.0 or the GPL v3.0.
