# -*- coding: utf-8 -*-
from setuptools import setup

packages = \
['clinlp', 'clinlp.component', 'clinlp.resources']

package_data = \
{'': ['*']}

install_requires = \
['intervaltree>=3.1.0,<4.0.0', 'spacy>=3.4.4,<4.0.0']

setup_kwargs = {
    'name': 'clinlp',
    'version': '0.1.1',
    'description': 'Performant and production-ready NLP pipelines for clinical text written in Dutch',
    'long_description': '# clinlp\n\n![clinlp](media/clinlp.png)\n\n* :hospital: `clinical` + :netherlands: `nl` + :clipboard: `NLP` = :sparkles: `clinlp`\n* :star: Performant and production-ready NLP pipelines for clinical text written in Dutch\n* :rocket: Open source, created and maintained by the Dutch Clinical NLP community\n* :triangular_ruler: Useful out of the box, but customization highly recommended\n \n\nRead the [principles and goals](#principles-and-goals), futher down :arrow_down:\n\n## Contact and contributing\n\n`clinlp` is very much still being shaped, so if you are enthusiastic about using or contributing to `clinlp`, please don\'t hesitate to get in touch ([email](mailto:analytics@umcutrecht.nl) | [issue](https://github.com/umcu/clinlp/issues/new)). We would be very happy to discuss your ideas and needs, whether its from the perspective of an (end) user, engineer or clinician, and formulate a roadmap with next steps together. \n\n## Getting started\n\n### Installation\n```bash\npip install clinlp\n```\n\n### Example\n```python\nimport clinlp\nimport spacy\n\nnlp = spacy.blank("clinlp")\n\n# Sentences\nnlp.add_pipe(\'clinlp_sentencizer\')\n\n# Entities\nruler = nlp.add_pipe(\'entity_ruler\')\n\nterms = {\n    \'covid_19_symptomen\': [\n        \'verkoudheid\', \'neusverkoudheid\', \'loopneus\', \'niezen\', \'vermoeidheid\',\n        \'keelpijn\', \'hoesten\', \'benauwdheid\', \'kortademigheid\', \'verhoging\', \n        \'koorts\', \'verlies van reuk\', \'verlies van smaak\'\n    ]\n}\n\nfor term_description, terms in terms.items():\n    ruler.add_patterns([{\'label\': term_description, \'pattern\': term} for term in terms])\nimport clinlp\nimport spacy\n\nnlp = spacy.blank("clinlp")\n\n# Sentences\nnlp.add_pipe(\'clinlp_sentencizer\')\n\n# Entities\nruler = nlp.add_pipe(\'entity_ruler\')\n\nterms = {\n    \'covid_19_symptomen\': [\n        \'verkoudheid\', \'neusverkoudheid\', \'loopneus\', \'niezen\', \'vermoeidheid\',\n        \'keelpijn\', \'hoesten\', \'benauwdheid\', \'kortademigheid\', \'verhoging\', \n        \'koorts\', \'verlies van reuk\', \'verlies van smaak\'\n    ]\n}\n\nfor term_description, terms in terms.items():\n    ruler.add_patterns([{\'label\': term_description, \'pattern\': term} for term in terms])\n\n# Qualifiers\nnlp.add_pipe(\'clinlp_context_matcher\')\n\ntext = (\n    "Patiente bij mij gezien op spreekuur, omdat zij vorige maand verlies van "\n    "reuk na covid infectie aangaf. Zij had geen last meer van kortademigheid, "\n    "wel was er nog sprake van hoesten, geen afname vermoeidheid."\n)\n\n\ndoc = nlp(text)\n```\n\nFind information in the doc object:\n\n```python\nfrom spacy import displacy\n\ndisplacy.render(doc, style=\'ent\')\n```\n\n![example_doc_render.png](media/example_doc_render.png)\n\nWith relevant qualifiers:\n\n```python\nfor ent in doc.ents:\n  print(ent, ent.start, ent.end, ent._.qualifiers)\n\n```\n\n* `11` `14` `verlies van reuk` `{\'Temporality.HISTORICAL\'}`\n* `25` `26` `kortademigheid` `{\'Negation.NEGATED\'}`\n* `33` `34` `hoesten` `{}`\n* `37` `38` `vermoeidheid` `{}`\n\n## Documentation\n\n### Introduction\n\n`clinlp` is built on top of spaCy, a widely used library for Natural Language Processing. Before getting started with `clinlp`, it may be useful to read [spaCy 101: Everything you need to know (~10 mins)](https://spacy.io/usage/spacy-101). Main things to know are that spaCy consists of a tokenizer (breaks a text up into small pieces, i.e. words), and various components that further process the text. \n\nCurrently, `clinlp` offers the following components, tailored to Dutch Clinical text, further discussed below: \n\n1. [Tokenizer](#tokenizer)\n2. [Sentence splitter](#sentence-splitter)\n3. [Entity matcher (builtin Spacy)](#entity-matcher)\n4. [Context detection](#context-detection)\n\n### Tokenizer\n\nThe `clinlp` tokenizer is builin the blank model:\n\n```python\nnlp = spacy.blank(\'clinlp\')\n```\n\nIt employs some custom rule based logic, including:\n- Clinical text-specific logic for splitting punctuation, units, dosages (e.g. `20mg/dag` :arrow_right: `20` `mg` `/` `dag`)\n- Custom lists of abbreviations, units (e.g. `pt.`, `zn.`, `mmHg`)\n- Custom tokenizing rules (e.g. `xdd` :arrow_right: `x` `dd`)\n- Regarding [DEDUCE](https://github.com/vmenger/deduce) tags as a single token (e.g. `[DATUM-1]`). \n  - Deidentification is not builtin `clinlp` and should be done as a preprocessing step.\n\n### Sentence splitter\n\nThe sentence splitter can be added as follows:\n\n```python\nnlp.add_pipe(\'clinlp_sentencizer\')\n```\n\nIt is designed to detect sentence boundaries in clinical text, whenever a character that demarks a sentence ending is matched (e.g. newline, period, question mark). It also correctly detects items in an enumerations (e.g. starting with `-` or `*`). \n\n### Entity matcher\n\nCurrently, the spaCy builtin `PhraseMatcher` and `Matcher` can be used for finding (named) entities in text. The first one accepts literal phrases only, that are matched in the tokenized text, while the second only also accepts [spaCy patterns](https://spacy.io/usage/rule-based-matching#adding-patterns). These are not tailored for the clinical domain, but nevertheless useful when a somewhat coherent list of relevant patterns can be generated/obtained.\n\nFor instance, a matcher that helps recognize COVID19 symptoms:\n\n```python\nruler = nlp.add_pipe(\'entity_ruler\')\n\nterms = {\n    \'covid_19_symptomen\': [\n        \'verkouden\', \'neusverkouden\', \'loopneus\', \'niezen\', \n        \'keelpijn\', \'hoesten\', \'benauwd\', \'kortademig\', \'verhoging\', \n        \'koorts\', \'verlies van reuk\', \'verlies van smaak\'\n    ]\n}\n\nfor term_description, terms in terms.items():\n    ruler.add_patterns([{\'label\': term_description, \'pattern\': term} for term in terms])\n```\n\nFor more info, it\'s useful to check out these spaCy documentation pages:\n* [Rule based matching](https://spacy.io/usage/rule-based-matching)\n* [[spaCy API] Matcher](https://spacy.io/api/matcher)\n* [[spaCy API] PhraseMatcher](https://spacy.io/api/phrasematcher)\n\nNote that the `DependencyMatcher` cannot be used, and neither are part of speech tags available, as no good models for determining this information for clinical text exist (yet).  \n\n### Context detection\n\nAfter finding entities, it\'s often useful to qualify these entities, e.g.: are they negated or affirmed, historical or current? `clinlp` currently implements the rule-based [Context algorithm](https://doi.org/10.1016%2Fj.jbi.2009.05.002) for this purpose. This algorithm is fairly accurate, and quite transparent and fast. Better solutions will hopefully be added to `clinlp` in the future. \n\nA set of rules, that checks for negation, temporality, plausibility and experiencer, is loaded by default:\n\n```python\nnlp.add_pipe(\'clinlp_context_matcher\')\n```\n\nA custom set of rules, including different types of qualifiers, can easily be defined. See [`clinlp/resources/psynlp_context_rules.json`](clinlp/resources/psynlp_context_rules.json) for an example, and load it as follows: \n\n```python\nfrom clinlp.component.qualifier import parse_rules\n\ncm = nlp.add_pipe(\'clinlp_context_matcher\', config={\'default_rules\': None})\ncm.add_rules(parse_rules(\'my_custom_rules.json\'))\n```\n\n### Where to go from here\n\nWe hope to extend `clinlp` with new functionality and more complete documentation in the near future. In the meantime, if any questions or problems arise, we recommend:\n\n* Checking the source code \n* Getting in touch ([email](mailto:analytics@umcutrecht.nl) | [issue](https://github.com/umcu/clinlp/issues/new))\n\n## Principles and goals\n\nFunctional:\n\n* Provides NLP pipelines optimized for Dutch clinical text\n  * Performant and production-ready\n  * Useful out-of-the-box, but highly configurable\n* A single place to visit for your Dutch clinical NLP needs\n* (Re-)uses existing components where possible, implements new components where needed\n* Not intended for annotating, training, and analysis — already covered by existing packages\n\nDevelopment: \n\n* Free and open source\n* Targeted towards the technical user\n* Curated and maintained by the Dutch Clinical NLP community\n* Built using the [`spaCy`](https://spacy.io/) framework (`>3.0.0`)\n  * Therefore non-destructive\n* Work towards some level of standardization of components (abstraction, protocols)\n* Follows industry best practices (system design, code, documentation, testing, CI/CD)\n\nOverarching goals:\n\n* Improve the quality of Dutch Clinical NLP pipelines\n* Enable easier (re)use/valorization of efforts\n* Help mature the field of Dutch Clinical NLP\n* Help develop the Dutch Clinical NLP community\n',
    'author': 'UMCU DIT Analytics',
    'author_email': 'analytics@umcutrecht.nl',
    'maintainer': 'None',
    'maintainer_email': 'None',
    'url': 'None',
    'packages': packages,
    'package_data': package_data,
    'install_requires': install_requires,
    'python_requires': '>=3.9,<4.0',
}


setup(**setup_kwargs)
