# -*- coding: utf-8 -*-
from setuptools import setup

packages = \
['elasticai',
 'elasticai.creator',
 'elasticai.creator.brevitas',
 'elasticai.creator.brevitas.translation_functions',
 'elasticai.creator.examples',
 'elasticai.creator.integrationTests',
 'elasticai.creator.systemTests',
 'elasticai.creator.tests',
 'elasticai.creator.tests.vhdl',
 'elasticai.creator.vhdl',
 'elasticai.creator.vhdl.generator',
 'elasticai.creator.vhdl.generator.functions']

package_data = \
{'': ['*'],
 'elasticai.creator.tests.vhdl': ['vhdFiles/*'],
 'elasticai.creator.vhdl': ['source/*', 'testbench/*']}

install_requires = \
['MonkeyType>=21.5.0,<22.0.0', 'torch>=1.10.0,<2.0.0']

extras_require = \
{'brevitas': ['brevitas>=0.7.0,<0.8.0',
              'onnx>=1.10.2,<2.0.0',
              'onnxoptimizer>=0.2.6,<0.3.0'],
 'examples': ['torchvision>=0.11.2,<0.12.0'],
 'systemtests': ['torchvision>=0.11.2,<0.12.0']}

setup_kwargs = {
    'name': 'elasticai.creator',
    'version': '0.3.3a2',
    'description': 'Design, train and compile neural networks optimized specifically for FPGAs.',
    'long_description': '# Elastic.ai Creator\n\nDesign, train and compile neural networks optimized specifically for FPGAs.\nObtaining a final model is typically a three stage process.\n* design and train it using the layers provided in the `elasticai.creator` package.\n* translate the model to a target representation, e.g. VHDL\n* compile the intermediate representation with a third party tool, e.g. Xilinx Vivado (TM)\n\nThis version currently only supports [Brevitas](https://github.com/Xilinx/brevitas) as a target representation.\n\nThe project is part of the elastic ai ecosystem developed by the Embedded Systems Department of the University Duisburg-Essen. For more details checkout the slides at https://www.researchgate.net/publication/356372207_In-Situ_Artificial_Intelligence_for_Self-_Devices_The_Elastic_AI_Ecosystem_Tutorial.\n\n\n## Table of contents\n\n[[_TOC_]]\n\n## Requirements\n\nThe dependencies of the PyTorch implementation of MTLTranslator are listed below:\n- torch\n- torchvision\n- torchaudio\n- matplotlib\n- numpy\n- scikit-learn\n- brevitas\n- onnx\n- onnxoptimizer\n\n### Poetry\n\n#### Using ElasticAICreator as a dependency\nAdd the following to your `pyproject.toml`\n```toml\n[[tool.poetry.source]]\nname = "elasticai"\nurl = "https://git@git.uni-due.de/api/v4/projects/5522/packages/pypi/simple"\nsecondary = true\n```\nAfterwards you should be able to add the ElasticAICreator to your project like so\n```bash\npoetry add elasticaicreator\n```\n\n#### Developing the ElasticAICreator\nFor easy creating a virtual environment and having a guaranteed working environment you can use the\n[poetry](https://python-poetry.org/) module.\n\n(before installing poetry make sure you have python3.9 installed on your system)\n\npoetry can be installed in the following way:\n```bash\npip install poetry\n```\nif your default python version is not (3.9.*) you need to run the following command first:\n```bash\npoetry env use python3.9\n```\n\nAfter installing poetry you can create an environment and pull all necessary dependencies by just typing the following\ncommand in the project root folder where the ```poetry.lock``` and ```pyproject.toml``` is located:\n\n```bash\npoetry install\n```\n\nThe installed poetry environment can be activated typing the following command in the project root folder:\n```bash\npoetry shell\n```\nYou may consider installing poetry plugin in pycharm and adding the created environment\n\n## Installing\n\nIf you want to use the elastic.ai Creator Translator in your project you can install it via pip. \nTo install from git run the following command: \n```bash\npip install git+https://git.uni-due.de/embedded-systems/artificial-intelligence/toolchain/mtltranslator\n```\nTo build locally in the directory run:\n```bash\npip install  .\n```\n\n## Structure of the Project\n\nThe structure of the project is as follows.\nThe [qtorch](elasticai/creator/qtorch) folder includes our implementation of quantized PyTorch layer.\nThe folder [protocols](elasticai/creator/protocols) contains some general protocols for the models and layers which are also used by multiple translated languages.\nIn the [translator](elasticai/creator/translator) folder there are the modules which can be used for every translation from a pytorch model to a target language.\nThe subfolder [brevitas](elasticai/creator/brevitas) is for the translation to Brevitas.\nEach language we can translate to has folders for unit tests, integration tests and system test. \n\n## QTorch\n\nQTorch is a library of quantized PyTorch layers. \nThe [layers](elasticai/creator/layers.py) file includes all implemented quantized PyTorch layers.\nThese layers are the followings:\n- QConv1d for quantized Conv1d\n- QConv2d for quantized Conv2d\n- QLinear for quantized Linear\n- QLSTMCell for quantized LSTMCell\n- QLSTM for stacking QLSTMCell\n\nFor the quantization we implemented quantizers:\n- Binarize which converts the weights to be -1 or 1\n- Ternarize which converts the weights to be -1 or 0 or 1\n- QuantizeTwoBit use Residual Quantization to quantize to two bits\n- ResidualQuantization converts weights to a bit vector using the residual multilevel binarization method [RebNet, Ghasemzadeh et al. 2018](https://arxiv.org/pdf/1711.01243.pdf)\n\n\nWe wrote tests for the layers which can be found in the [layers_tests](elasticai/creator/qtorch/tests/test_layer.py).\nTo add constraints on the convolutional and linear layers you can use the [constraints](elasticai/creator/constraints.py) and can easily expand it with more constraints.\nWe also implemented blocks of convolution and linear layers consisting of the convolution or linear following a batch normalisation and some activation. \nAlso consider the [tests](elasticai/creator/qtorch/tests/test_block.py) for the blocks.\nFor getting more insight in the QTorch library consider the examples in the [examples](elasticai/creator/examples) folder.\n\n## Users guide \n\nAs a user one want to convert an existing pytorch model to one of our languages.\n1. Add our translator as a dependency in your project.\n2. Instantiate the model. \n3. Optionally you can train it or load some weights. \n4. Input the model in the translation function like shown in the following. \n\nPlease refer to the system test of each language as an example.\n\n## Translating to Brevitas\n\nHow to translate a given PyTorch model consisting of QTorch layers to Brevitas?\nThis is how to translate a given model to a Brevitas model:\n\n```python\nfrom elasticai.creator.brevitas.brevitas_representation import BrevitasRepresentation\n\nconverted_model = BrevitasRepresentation.from_pytorch(qtorch_model).translated_model\n```\nargs:\n- qtorch_model: a pytorch model (supports most of the [QTorch](https://git.uni-due.de/embedded-systems/artificial-intelligence/toolchain/qtorch) layers and some standard pytorch layers)\n\nreturns:\n- converted_model: a Brevitas model\n\nExample usages are shown here: [Brevitas system tests](elasticai/creator/systemTests).\nWe also support to translate a brevitas model to onnx which is shown in the system test.\n\n### Translations\n\nThe following QTorch or PyTorch layers are translated to the corresponding Brevitas layers:\n\n- QConv1d to QuantConv1d\n- QConv2d to QuantConv2d\n- QLinear to QuantLinear\n- Binarize to QuantIdentity\n- Ternarize to QuantIdentity\n- PyTorch MaxPool1d to PyTorch MaxPool1d\n- PyTorch BatchNorm1d to PyTorch BatchNorm1d \n- PyTorch Flatten to PyTorch Flatten\n- PyTorch Sigmoid to PyTorch Sigmoid\n\nYou can find the mappings in [translation_mapping](elasticai/creator/brevitas/translation_mapping.py) and can easily add more PyTorch layers.\n\n### Supported Layers for Brevitas Translation\n\n- QuantConv1d: quantized 1d convolution with weight- and bias-quantization\n- QuantConv2d: quantized 2d convolution with weight- and bias-quantization\n- QuantLinear: quantized linear layer with weight- and bias-quantization\n- QuantIdentity(act_quant=quantizers.BinaryActivation): binary activation layer\n- QuantIdentity(act_quant=quantizers.TernaryActivation): ternary activation layer\n\n### Limitations for Brevitas Translation\n\n- we do not support all QTorch layers in the QTorch repository. Not supported layers are:\n  - Ternarization with more complex thresholds e.g threshold of 0.1 \n  - ResidualQuantization \n  - QuantizeTwoBit \n  - QLSTM\n- we do not support all PyTorch layers, but you can easily add them in the [translation_mapping](elasticai/creator/brevitas/translation_mapping.py).\n\n## General Limitations\n\nBy now we only support Sequential models for our translations.\n\n## Developers introductory Guide and Glossary\n\nNew translation targets should be located in their own folder, e.g. Brevitas for translating from any language to Brevitas.\nWorkflow for adding a new translation:\n1. Obtain a structure, such as a list in a sequential case, which will describe the connection between every component.\n2. Identify and label relevant structures, in the base cases it can be simply separate layers.\n3. Map each structure to its function which will convert it, like for [example](elasticai/creator/brevitas/translation_mapping.py).\n4. Do such conversions.\n5. Recreate connections based on 1.\n\nEach sub-step should be separable and it helps for testing if common functions are wrapped around an adapter.\n\n## Model reporter\nAs part of this repository an utility called [model reporter](elasticai/creator/model_reporter.py) exists which is used to produce a file with the individual identifier from a given dataset. \nIt is used for hardware comparisons, where a small subset of the data is used for comparison. \nExample: \n```\necg-path ground-truth model-prediction\nsinus/08219_107.ecg 0 0 \natrial_fibrillation/07879_392.ecg 1 1\nsinus/08219_573.ecg 0 0\nsinus/04126_477.ecg 0 0\natrial_fibrillation/07162_287.ecg 1 1\natrial_fibrillation/06426_789.ecg 1 1\natrial_fibrillation/05121_664.ecg 1 1\n```\nThis is expected to use a path to help a validator to automatically extract the data from the said path and add the further predictions.\nExample usage:\n```python\nreport = ModelReport(model=loaded_model, data=[labels,inputs,data[1]], is_binary=True, threshold=0.5)\nreport.write_to_csv(path="results.csv")\n```\n\n## Tests\n\nOur implementation is fully tested with unit, integration and system tests.\nPlease refer to the system tests as examples of how to use the Elastic Ai Creator Translator.\nYou can run one explicit test with the following statement: \n\n```python -m unittest discover -p "test_*.py" elasticai/creator/translator/path/to/test.py```\n\nIf you want to run all unit tests for example, give the path to the unit tests:\n\n```python -m unittest discover -p "test_*.py" elasticai/creator/translator/path/to/language/unitTests/```\n\nYou can also run all tests together:\n\n```python -m unittest discover -p "test_*.py" elasticai/creator/translator/path/to/language/```\n\nIf you want to add more tests please refer to the [Test Guidelines](test_guidelines.md).\n\n### Brevitas System Tests\n\nThe [Brevitas system tests](elasticai/creator/systemTests) can be used as example use cases of our implementation.\nWe created tests which check the conversion of a model like we would expect our models will look like.\nIn addition, we also created tests for validating the conversion for trained models or unusual models. \nNote that you have to use your own data set and therefore maybe do some small adaptions by using the training.\n\n### Brevitas Integration Tests\n\nOur  [Brevitas integration tests](elasticai/creator/integrationTests) are focused on testing the conversion of one specific layer. \nWe created for all our supported layers a minimal model with this layer included and test its functionality. \n\n### Brevitas Unit Tests\n\nIn addition to system and integration tests we implemented unit tests. \nThe unit tests of each module is named like the model but starting with "test_" and can be found in the unitTest folder.\nThe Brevitas unit tests can be found [here](elasticai/creator/brevitas/unitTests).\n\n## VHDL Generation\n\nWe follow the VHDL code guidelines of this [book](https://github.com/fabriziotappero/Free-Range-VHDL-book).\n\n## Developers Guide\n* We use [black](https://black.readthedocs.io/en/stable/index.html) for code formatting. For instruction on setup with your IDE please refer to https://black.readthedocs.io/en/stable/integrations/editors.html#editor-integration.\n* importing `*` should be avoided in favor of explicit imports\n\n### Test style Guidelines\n\n#### File IO\nIn general try to avoid interaction with the filesystem. In most cases instead of writing to or reading from a file you can use a StringIO object or a StringReader.\nIf you absolutely have to create files, be sure to use pythons [tempfile](https://docs.python.org/3.9/library/tempfile.html) module and cleanup after the tests.\n\n\n#### Diectory structure and file names\nFiles containing tests for a python module should be located in a test directory for the sake of separation of concerns. \nEach file in the test directory should contain tests for one and only one class/function defined in the module. \nFiles containing tests should be named according to the rubric\n`test_ClassName.py`.\nNext, if needed for more specific tests define a class which is a subclass of unittest.TestCase like [test_brevitas_model_comparison](elasticai/creator/integrationTests/test_brevitas_model_comparison.py) in the integration tests folder. \nThen subclass it, in this class define a setUp method (and possibly tearDown) to create the global environment. \nIt avoids introducing the category of bugs associated with copying and pasting code for reuse. \nThis class should be named similarly to the file name.\nThere\'s a category of bugs that appear if  the initialization parameters defined at the top of the test file are directly used: some tests require the initialization parameters to be changed slightly. \nIts possible to define a parameter and have it change in memory as a result of a test. \nSubsequent tests will therefore throw errors.\nEach class contains methods that implement a test. \nThese methods are named according to the rubric\n`test_name_condition`\n\n#### Unit tests\nIn those tests each functionality of each function in the module is tested, it is the entry point  when adding new functions. \nIt assures that the function behaves correctly independently of others. \nEach test has to be fast, so use of heavier libraries is discouraged.\nThe input used is the minimal one needed to obtain a reproducible output. \nDependencies should be replaced with mocks as needed. \n\n#### Integration Tests\nHere the functions\' behaviour with other modules is tested. \nIn this repository each integration function is in the correspondent folder.\nThen the integration with a single class of the target, or the minimum amount of classes for a functionality, is tested in each separated file.\n\n#### System tests\nThose tests will use every component of the system, comprising multiple classes.\nThose tests include expected use cases and unexpected or stress tests.\n\n#### Adding new functionalities and tests required\nWhen adding new functions to an existing module, add unit tests in the correspondent file in the same order of the module, if a new module is created a new file should be created.\nWhen a bug is solved created the respective regression test to ensure that it will not return.\nProceed similarly with integration tests. \nCreating a new file if a functionality completely different from the others is created e.g. support for a new layer.\nSystem tests are added if support for a new library is added.\n\n#### Updating tests\nIf new functionalities are changed or removed the tests are expected to reflect that, generally the ordering is unit tests -> integration tests-> system tests.\nAlso, unit tests that change the dependencies should be checked, since this system is fairly small the internal dependencies are not always mocked.\n\nreferences: https://jrsmith3.github.io/python-testing-style-guidelines.html',
    'author': 'Department Embedded Systems - University Duisburg Essen',
    'author_email': None,
    'maintainer': None,
    'maintainer_email': None,
    'url': 'https://github.com/es-ude/elastic-ai.creator',
    'packages': packages,
    'package_data': package_data,
    'install_requires': install_requires,
    'extras_require': extras_require,
    'python_requires': '>=3.9,<3.10',
}


setup(**setup_kwargs)
