Metadata-Version: 2.1
Name: dsp-tools
Version: 1.21.0
Summary: A Python library and tools for the DaSCH Service Platform
Home-page: https://github.com/dasch-swiss/dsp-tools
Author: DaSCH - Swiss National Data and Service Center for the Humanities
Author-email: support@dasch.swiss
License: GPLv3
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: GNU General Public License v3 (GPLv3)
Classifier: Operating System :: OS Independent
Requires-Python: >=3.9.0
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: argparse (~=1.4.0)
Requires-Dist: jsonpath-ng (~=1.5.3)
Requires-Dist: jsonschema (~=4.17.0)
Requires-Dist: lxml (~=4.9.1)
Requires-Dist: networkx (~=2.8.8)
Requires-Dist: openpyxl (~=3.0.10)
Requires-Dist: pandas (~=1.5.1)
Requires-Dist: ply (~=3.11)
Requires-Dist: pystrict (~=1.3)
Requires-Dist: pytz (~=2022.6)
Requires-Dist: pyyaml (~=6.0)
Requires-Dist: regex (~=2022.10.31)
Requires-Dist: requests (~=2.28.1)
Requires-Dist: types-pyyaml (~=6.0.12.1)
Requires-Dist: xlrd (~=2.0.1)
Requires-Dist: numpy (~=1.23.4) ; python_version < "3.10"
Requires-Dist: python-dateutil (~=2.8.2) ; python_version >= "2.7" and python_version not in "3.0, 3.1, 3.2, 3.3"
Requires-Dist: six (~=1.16.0) ; python_version >= "2.7" and python_version not in "3.0, 3.1, 3.2, 3.3"
Requires-Dist: urllib3 (~=1.26.12) ; python_version >= "2.7" and python_version not in "3.0, 3.1, 3.2, 3.3, 3.4, 3.5" and python_version < "4"
Requires-Dist: attrs (~=22.1.0) ; python_version >= "3.5"
Requires-Dist: decorator (~=5.1.1) ; python_version >= "3.5"
Requires-Dist: idna (~=3.4) ; python_version >= "3.5"
Requires-Dist: certifi (~=2022.9.24) ; python_version >= "3.6"
Requires-Dist: charset-normalizer (~=2.1.1) ; python_version >= "3.6"
Requires-Dist: et-xmlfile (~=1.1.0) ; python_version >= "3.6"
Requires-Dist: pyrsistent (~=0.19.2) ; python_version >= "3.7"

[![PyPI version](https://badge.fury.io/py/dsp-tools.svg)](https://badge.fury.io/py/dsp-tools)

# DSP-TOOLS - DaSCH Service Platform Tools

dsp-tools is a command line tool that helps you interacting with the DaSCH Service Platform API.  
Go to [Full Documentation](https://docs.dasch.swiss/latest/DSP-TOOLS)


## Information for developers

There is a `Makefile` for all the following tasks (and more). Type `make` to print the available targets. 

For a quick start, use: 
```
pip install pipenv
pipenv install --dev
pipenv run make install
```

This creates a pipenv-environment, installs all dependencies, and installs `dsp-tools` from source.

If you prefer getting around pipenv, use instead:
```bash
make install-requirements
make install
```


## Pipenv

We use pipenv for our dependency management. There are two ways to get started:
 - `pipenv install --dev` installs all dependencies, while giving them the opportunity to update themselves
 - `pipenv install --ignore-pipfile` is used to get a deterministic build in production

This works because there are two files defining the dependencies:
 - `Pipfile` replaces `requirements.txt`, but lists only the core dependencies, ordered in two sections:
   - `[packages]` lists the dependencies used to run the software.
   - `[dev-packages]` lists additional dependencies used for tests and deployment.
 - `Pipfile.lock` enables deterministic builds, by exactly pinning the version of all (sub-) dependencies. 
   This is done automatically, you must not edit `Pipfile.lock`.

The diverse `requirements.txt` files in this repo are only present for backwards compatibility
and for GitHub CI.

If you want to install a new package, install it with `pipenv install package`. This 
 - installs the package (incl. sub-dependencies) in your virtual environment
 - adds the package to the section `[packages]` of `Pipfile`. By default, no versions are pinned
 - adds the pinned versions of package and all sub-dependencies to `Pipfile.lock`

If a package is only needed for development, please install it with `pipenv install package --dev`,
so it gets added to the `[dev-packages]` section of `Pipfile`.

For security reasons, the maintainer regularly executes
 - `pipenv check` to get informed about vulnerabilities
 - `pipenv update` to update `Pipfile.lock` with the latest version of every package
 - `make freeze-requirements` to update the requirement files and `setup.py`

`make freeze-requirements` must also be executed after adding a new dependency. If you prefer working 
without pipenv, you can freeze your requirements with `pip3 freeze > requirements.txt` and update `setup.py`
manually.


### Pipenv setup in PyCharm

 - Go to Add Interpreter > Pipenv Environment
 - Base Interpreter: PyCarm auto-detects one of your system-wide installed Pythons as base interpreter. 
 - Pipenv executable: auto-detected
 - After hitting OK, PyCharm creates a new pipenv environment and installs the dependencies from `Pipfile`

If you already initialized a pipenv-environment via command line, you can add its interpreter in PyCharm,
but this will create the pipenv-environment again.


## Testing

Please note that testing requires launching the complete DSP API stack which is based on docker images. 
Therefore, we recommend installing the [docker desktop client](https://www.docker.com/products).  
To run the complete test suite:
```bash
make test
```


## Code style

When contributing to the project please make sure you use the same code style rules as we do. We use
[autopep8](https://pypi.org/project/autopep8/) and [mypy](https://pypi.org/project/mypy/). The 
configuration is defined in `pyproject.toml` in the root directory of the project.

You can use the configuration with `autopep8 --global-config pyproject.toml [file path]` and 
`mypy --config-file pyproject.toml [file path]`.

If you are using PyCharm we recommend installing autopep8 as external tool. You can then use it with 
right-click on the file > `External Tools` > `autopep8` to reformat files in-place. Due to compatibility 
issues with VSCode, the argument  `--in-place=true` can not be declared in the `pyproject.toml` and 
needs to be passed to the external tool in the PyCharm settings.  
mypy is available as [plugin](https://plugins.jetbrains.com/plugin/11086-mypy).

In VSCode, both mypy and autopep8 can be set up as default linter and formatter through the python extension.

For formatting Markdown files (*.md) we use the default styling configuration provided by PyCharm.


## Publishing

Publishing is automated with GitHub Actions and should _not_ be done manually. Please follow the
[Pull Request Guidelines](https://docs.dasch.swiss/latest/developers/dsp/contribution/#pull-request-guidelines). If done
correctly, when merging a pull request into `main`, the `release-please` action will create or update a pull request for
a release. This pull request will follow semantic versioning and update the change log. Once all desired features are
merged, the release can be executed by merging this release pull request into `main`. This will trigger actions that
create a release on GitHub, on PyPI and the docs.

Please ensure you have only one pull request per feature.


## Publishing manually

Publishing is automated with GitHub Actions and should _not_ be done manually. If you still need to do it, follow the
steps below.

Generate the distribution package. Make sure you have the latest versions of `setuptools` and `wheel` installed:

```bash
python3 -m pip install --upgrade pip setuptools wheel
python3 setup.py sdist bdist_wheel
```

You can install the package locally from the dist:

```bash
python3 -m pip ./dist/some_name.whl
```

Upload package works also with `make`:

```bash
make dist
make upload
```

For local development:

```bash
python3 setup.py develop
```


## Contributing to the documentation

The documentation is a collection of [markdown](https://en.wikipedia.org/wiki/Markdown) files in the `docs` folder.  
After updates of the files, build and check the result with the following command:

```bash
make docs-serve 
```

The documentation is published on https://docs.dasch.swiss/latest/DSP-TOOLS. During the centralized release process of all
components of the DSP software stack, the docs of dsp-tools get built from the main branch to https://docs.dasch.swiss.
