# Release 0.1.25 - Major changes to internal representation and some new examples

## Overview

### Summary

The focus of this release is the result of the independent intermediate
node refactoring project I was undertaking. This was to transform the list
oriented data structure for representing a "pure" intermediate form of
code into a more stuctured/generalisable format.

The motivation behind this change was the realisation that implementation
of end user functions (ie `def`) would be very difficult with this more
structured approach. 

### Rationale behind changes

Before this change, pyxie used the following data structures, and code
phases when parsing, analysing, transforming and performing code
generation.

These were not ideal:

* Code phase: Parser - read in the source, parsed it, and created a
  data structure
    * Data structure: pynodes - these represent the concrete python
      syntax, and are generated during the code generation phase. These
      are python objects in a class hierarchy. Once created they are
      analysed and placed into a context to analyse datatypes.
* Code phase: Analysis. - Analyses the code, and decorates the existing
  pynodes with type information to understand the programs data and data
  types. (does not create a new data structure).
* Transform phase: walks the pynode CST, and generates an intermediate
  data structure intended to represent the program in the abstract
  independent of the language used. An independent intermediate form
  if you like.
    * Data structure: independent intermediate  form - This is used
      to model the program in a "pure form" - which isn't constrained
      by the source language, and contains enough information for the
      target language output to be generated. That was the theory. In
      practice it was a nested list of  list of lists ... Not ideal.
      More on this below.
* Code generation phase: This walked the independent intermediate form
  (the lists of lists), and created an output data structure representing
  the concrete final program.
  - The third/final data structure is intended to represent the final
  output language. ie it is intended to be a concrete representation of
  the output language. This final data structure is then capable of
  creating the output. Again, forms a hierarchy and could be called
  "CppNodes" (Though they weren't before this change)

The problem here is that while the pynodes are currently well defined (to
a large extent) and that the CppNodes are pretty well defined (even if
they could be better), the independent intermediate form sucked because
it was just nested lists. This meant in practice the code was fragile,
difficult to change, and increasingly difficult to work with. In the early 
days of Pyxie this simplistic structure made sense. However as data
analysis becomes more complex and tricky. This mirrors the fact that
the same was true in the early days of the parsing side. 

So the focus of this sub-project was to replace the intermediate form,
and tighten up the differences and make clearer in the code base that
we have PyNodes, iiNodes, and CppNodes, where:

* Pynodes - are the current structures, unchanged - note these are
  currently prefixed Py - PyDefStatement for example.
* CppNodes - are map to objects/hierarchy/etc that represents C++ programs,
  but made clearer (slightly). These are now prefixed Cpp rather than the
  previous C_ which *some* of the objects use.
* iiNodes - represent the independent, intermediate nodes. Since iiNodes
  need to be used in source code where there are either PyNodes + iiNodes
  or  CppNodes + iiNodes, they have the prefix "ii" enable differentiation.

Since all 3 of these are actually *models*, the code for these has all
moved to sit below pyxie.models.

At some point there will be a need to restructure the code more generally.
Parsing, transforming to IINodes and code generation are actually all
transforms, and should sit together. That was out of scope for this
already relatively major internal change.

## Changelog

### New

* `doc/Versioning.md` - semantic versioning as it applies to pyxie
* `doc/WIPNOTES/6.Models.md` - start of some docs around the models in use
* Added explicit notes on licensing of pyxie's output.
  (Short version: I view the output as being derived from your code by you)
* Language focussed examples / acceptance tests added:
  `if` `if-else` `pass` `print`  `while-break` `while-continue`
* Change `arduino` profile to support the `Adafruit_NeoPixel` library
* `neopixel` example added

### What's been fixed? / Improved

* Handling if if/elif/else improved/fixed
* added `clean.sh` to `arduino` example
* added `clean.sh` to `servo` example
* added `README.md` to `simplest_function` example (won't work yet)

### Internal Changes

* `bin/pyxie` now pulls in functionality from `pyxie.api` to be clearer
   about what the API is
* added `pyxie/api.py` - public API for embedding pyxie. (Assumes
  ubuntu host) Core contents:
    * `initialise_API(profile="default")` - Call first
    * `set_profile(profile_name)` - Call later
    * `PyxieAPI.parse(filename)` - parse file, output goes to console
    * `PyxieAPI.analyse(filename)` - parse & analyse flle, output to console
    * `PyxieAPI.codegen(filename, result_filename=None)` - parse through
       code generation. Output to console
    * `PyxieAPI.compile(filename, result_filename=None)` - Compile file,
       result ends up as "result_filename"
* `pyxie/core` - changed over to drive the conversion of pynodes to
   cppnodes via iinodes. (aim is to simplify pynodes down the line and
   make iinodes our main location for analysis/etc)
* Minor changes to `.gitignore`
* Minor change to the `Makefile` to make editting simpler for me...
* Update `clib.py` based on changes to `clib`
* `pyxie/codegen/simple_cpp.py` - functionality shifted out to
  `pyxie/models/cppnodes.py` to allow a model driven approach
* `pyxie/model/cppnodes.py` Created. A better code representation
   model for C++. Code generation therefore means transforming from the
   `iiNode`s to `CppNode`s 
* `pyxie/model/iinodes.py` - Introduces `iiNode`s - which are used to
  represent an intermediate version of a program such that it can bridge
  the conversion between languages
* `pyxie/model/pynodes/operators.py` - added `args()` method to a number
   of classes to unify behaviours
* `pyxie/model/transform.py` - Major change - converts `PyNode`s to
  `iiNode` representation rather than json objects.

### Other

* `doc/newsletter/07-20161110.Pyxie-Release-0.1.24.md` corrected release date
* `doc/newsletter/XX-TBD.Pyxie-2017-Goals.md` - Unreleased newsletter.
  Interesting notes, will get reused if appropriate later. Rather moot
  contents now though.
* `doc/newsletter/08-TBD.Focus-Or-Pyxie-Release-X.X.XX.md` Template for
   next newsletter
