# -*- coding: utf-8 -*-
from setuptools import setup

package_dir = \
{'': 'src'}

packages = \
['spaczz',
 'spaczz.fuzz',
 'spaczz.matcher',
 'spaczz.pipeline',
 'spaczz.regex',
 'spaczz.similarity']

package_data = \
{'': ['*']}

install_requires = \
['rapidfuzz<1.0.0', 'regex>=2020.6.8,<2021.0.0', 'spacy>=2.2.0,<3.0.0']

extras_require = \
{':python_version < "3.8"': ['importlib_metadata>=1.7.0,<2.0.0']}

entry_points = \
{'spacy_factories': ['spaczz_ruler = spaczz.pipeline:SpaczzRuler']}

setup_kwargs = {
    'name': 'spaczz',
    'version': '0.3.1',
    'description': 'Adds fuzzy matching and additional regex matching support to spaCy.',
    'long_description': '[![Tests](https://github.com/gandersen101/spaczz/workflows/Tests/badge.svg)](https://github.com/gandersen101/spaczz/actions?workflow=Tests)\n[![Codecov](https://codecov.io/gh/gandersen101/spaczz/branch/master/graph/badge.svg)](https://codecov.io/gh/gandersen101/spaczz)\n[![PyPI](https://img.shields.io/pypi/v/spaczz.svg)](https://pypi.org/project/spaczz/)\n[![Read the Docs](https://readthedocs.org/projects/spaczz/badge/)](https://spaczz.readthedocs.io/)\n\n# spaczz: Fuzzy matching and more for spaCy\n\nSpaczz provides fuzzy matching and multi-token regex matching functionality for [spaCy](https://spacy.io/).\nSpaczz\'s components have similar APIs to their spaCy counterparts and spaczz pipeline components can integrate into spaCy pipelines where they can be saved/loaded as models.\n\nFuzzy matching is currently performed with matchers from [RapidFuzz](https://github.com/maxbachmann/rapidfuzz)\'s fuzz module and regex matching currently relies on the [regex](https://pypi.org/project/regex/) library. Spaczz certainly takes additional influence from other libraries and resources. For additional details see the references section.\n\nSpaczz has been tested on Ubuntu 18.04, MacOS 10.15, and Windows Server 2019.\n\n*v0.3.1 Release Notes:*\n- *spaczz now includes an experimental `SimilarityMatcher` that attempts to match search terms based on vector similarity. It requires a a spaCy model with word vectors (e.x. spaCy\'s medium and large English models) to function properly. See the documentation below for usage details.*\n\n*v0.3.0 Release Notes:*\n- *The `FuzzyMatcher` and `RegexMatcher` now return fuzzy ratio and fuzzy count details respectively. The behavior of these two matchers is still the same except they now return lists of tuples of length 4 (match id, start, end, fuzzy details).*\n    - *This change could be breaking in instances where these tuples are unpacked in the traditional spaCy fashion (match id, start, end). Simply include the fuzzy details or a placeholder during unpacking to fix.*\n- *The SpaczzRuler now writes fuzzy ratio and fuzzy count details for fuzzy/regex matches respectively as custom `Span` attributes. These are `spaczz_ent_ratio` and `spaczz_ent_counts` respectively. They return `None` by default.*\n    - *The `spaczz_ent` portion of these attributes is controlled by the `attr` parameter and can be changed if needed. However, the `_ent_ratio` and `_ent_counts` extensions are hard-coded.*\n    - *If, in the rare case, the same match is made via a fuzzy pattern and regex pattern, the span will have both extensions set with their repsective values.*\n- *Fixed a bug where the `attr` parameter in the `SpaczzRuler` did not actually change the name of the custom span attribute.*\n\n*v0.2.0 Release Notes:*\n- *Fuzzy matching is now performed with [RapidFuzz](https://github.com/maxbachmann/rapidfuzz) instead of [FuzzyWuzzy](https://github.com/seatgeek/fuzzywuzzy).*\n    - *RapidFuzz is higher performance with a more liberal license.*\n- *The spaczz ruler now automatically sets a custom, boolean, `Span` attribute on all entities it adds.*\n    - *This is set by the `attr` parameter during `SpaczzRuler` instantiation and defaults to: "spaczz_ent".*\n    - *For example: an entity set by the spaczz ruler will have `ent._.spaczz_ent` set to `True`.*\n- *Spaczz ruler patterns now support optional "id" values like spaCy\'s entity ruler. See [this spaCy documentation](https://spacy.io/usage/rule-based-matching#entityruler-ent-ids) for usage details.*\n- *Automated Windows testing is now part of the build process.*\n\n<h1>Table of Contents<span class="tocSkip"></span></h1>\n<div class="toc"><ul class="toc-item"><li><span><a href="#Installation" data-toc-modified-id="Installation-1">Installation</a></span></li><li><span><a href="#Basic-Usage" data-toc-modified-id="Basic-Usage-2">Basic Usage</a></span><ul class="toc-item"><li><span><a href="#Fuzzy-Matcher" data-toc-modified-id="Fuzzy-Matcher-2.1">Fuzzy Matcher</a></span></li><li><span><a href="#Regex-Matcher" data-toc-modified-id="Regex-Matcher-2.2">Regex Matcher</a></span></li><li><span><a href="#SimilarityMatcher" data-toc-modified-id="SimilarityMatcher-2.3">SimilarityMatcher</a></span></li><li><span><a href="#SpaczzRuler" data-toc-modified-id="SpaczzRuler-2.4">SpaczzRuler</a></span></li><li><span><a href="#Saving/Loading" data-toc-modified-id="Saving/Loading-2.5">Saving/Loading</a></span></li></ul></li><li><span><a href="#Limitations" data-toc-modified-id="Limitations-3">Limitations</a></span></li><li><span><a href="#Future-State" data-toc-modified-id="Future-State-4">Future State</a></span></li><li><span><a href="#Development" data-toc-modified-id="Development-5">Development</a></span></li><li><span><a href="#References" data-toc-modified-id="References-6">References</a></span></li></ul></div>\n\n## Installation\n\nSpaczz can be installed using pip.\n\n\n```python\npip install spaczz\n```\n\n## Basic Usage\n\nSpaczz\'s primary features are fuzzy and regex matchers that function similarily to spaCy\'s phrase matcher, and the spaczz ruler which integrates the fuzzy/regex matcher into a spaCy pipeline component similar to spaCy\'s entity ruler.\n\n### Fuzzy Matcher\n\nThe basic usage of the fuzzy matcher is similar to spaCy\'s phrase matcher except it returns the fuzzy ratio along with match id, start and end information, so make sure to include a variable for the ratio when unpacking results.\n\n\n```python\nimport spacy\nfrom spaczz.matcher import FuzzyMatcher\n\nnlp = spacy.blank("en")\ntext = """Grint Anderson created spaczz in his home at 555 Fake St,\nApt 5 in Nashv1le, TN 55555-1234 in the USA.""" # Spelling errors intentional.\ndoc = nlp(text)\n\nmatcher = FuzzyMatcher(nlp.vocab)\nmatcher.add("NAME", [nlp("Grant Andersen")])\nmatcher.add("GPE", [nlp("Nashville")])\nmatches = matcher(doc)\n\nfor match_id, start, end, ratio in matches:\n    print(match_id, doc[start:end], ratio)\n```\n\n    NAME Grint Anderson 86\n    GPE Nashv1le 82\n\n\nUnlike spaCy matchers, spaczz matchers are written in pure Python. While they are required to have a spaCy vocab passed to them during initialization, this is purely for consistency as the spaczz matchers do not use currently use the spaCy vocab. This is why the `match_id` is simply a string in the above example instead of an integer value like in spaCy matchers.\n\nSpaczz matchers can also make use of on match rules via callback functions. These on match callbacks need to accept the matcher itself, the doc the matcher was called on, the match index and the matches produced by the matcher.\n\n\n```python\nimport spacy\nfrom spacy.tokens import Span\nfrom spaczz.matcher import FuzzyMatcher\n\nnlp = spacy.blank("en")\ntext = """Grint Anderson created spaczz in his home at 555 Fake St,\nApt 5 in Nashv1le, TN 55555-1234 in the USA.""" # Spelling errors intentional.\ndoc = nlp(text)\n\ndef add_name_ent(\n    matcher, doc, i, matches\n):\n    """Callback on match function. Adds "NAME" entities to doc."""\n    # Get the current match and create tuple of entity label, start and end.\n    # Append entity to the doc\'s entity. (Don\'t overwrite doc.ents!)\n    _match_id, start, end, _ratio = matches[i]\n    entity = Span(doc, start, end, label="NAME")\n    doc.ents += (entity,)\n\nmatcher = FuzzyMatcher(nlp.vocab)\nmatcher.add("NAME", [nlp("Grant Andersen")], on_match=add_name_ent)\nmatches = matcher(doc)\n\nfor ent in doc.ents:\n    print((ent.text, ent.start, ent.end, ent.label_))\n```\n\n    (\'Grint Anderson\', 0, 2, \'NAME\')\n\n\nLike spaCy\'s EntityRuler, a very similar entity updating logic has been implemented in the `SpaczzRuler`. The `SpaczzRuler` also takes care of handling overlapping matches. It is discussed in a later section.\n\nUnlike spaCy\'s matchers, rules added to spaczz matchers have optional keyword arguments that can modify the matching behavior. Take the below fuzzy matching example:\n\n\n```python\nimport spacy\nfrom spaczz.matcher import FuzzyMatcher\n\nnlp = spacy.blank("en")\n# Let\'s modify the order of the name in the text.\ntext = """Anderson, Grint created spaczz in his home at 555 Fake St,\nApt 5 in Nashv1le, TN 55555-1234 in the USA.""" # Spelling errors intentional.\ndoc = nlp(text)\n\nmatcher = FuzzyMatcher(nlp.vocab)\nmatcher.add("NAME", [nlp("Grant Andersen")])\nmatches = matcher(doc)\n\n# The default fuzzy matching settings will not find a match.\nfor match_id, start, end, ratio in matches:\n    print(match_id, doc[start:end], ratio)\n```\n\nNext we change the fuzzy matching behavior for the "NAME" rule.\n\n\n```python\nimport spacy\nfrom spaczz.matcher import FuzzyMatcher\n\nnlp = spacy.blank("en")\n# Let\'s modify the order of the name in the text.\ntext = """Anderson, Grint created spaczz in his home at 555 Fake St,\nApt 5 in Nashv1le, TN 55555-1234 in the USA.""" # Spelling errors intentional.\ndoc = nlp(text)\n\nmatcher = FuzzyMatcher(nlp.vocab)\nmatcher.add("NAME", [nlp("Grant Andersen")], kwargs=[{"fuzzy_func": "token_sort"}])\nmatches = matcher(doc)\n\n# The default fuzzy matching settings will not find a match.\nfor match_id, start, end, ratio in matches:\n    print(match_id, doc[start:end], ratio)\n```\n\n    NAME Anderson, Grint 86\n\n\nThe full list of keyword arguments available for fuzzy matching rules includes:\n\n- `fuzzy_func`: Key name of fuzzy matching function to use. All rapidfuzz matching functions with default settings are available. Default is "simple". The included fuzzy matchers are:\n    - "simple" = fuzz.ratio\n    - "partial" = fuzz.partial_ratio\n    - "token_set" = fuzz.token_set_ratio\n    - "token_sort" = fuzz.token_sort_ratio\n    - "partial_token_set" = fuzz.partial_token_set_ratio\n    - "partial_token_sort" = fuzz.partial_token_sort_ratio\n    - "quick" = fuzz.QRatio\n    - "weighted" = fuzz.WRatio\n    - "quick_lev" = fuzz.quick_lev_ratio\n- `ignore_case`: If strings should be lower-cased before fuzzy matching or not. Default is True.\n- `min_r1`: Minimum fuzzy match ratio required for selection during the intial search over doc. This should be lower than min_r2 and "low" in general because match span boundaries are not flexed initially. 0 means all spans of query length in doc will have their boundaries flexed and will be re-compared during match optimization. Lower min_r1 will result in more fine-grained matching but will run slower. Default is 25.\n- `min_r2`: Minimum fuzzy match ratio required for selection during match optimization. Should be higher than min_r1 and "high" in general to ensure only quality matches are returned. Default is 75.\n- `flex`: Number of tokens to move match span boundaries left and right during match optimization. Default is "default".\n\n### Regex Matcher\n\nThe basic usage of the regex matcher is also fairly similar to spaCy\'s phrase matcher. It accepts regex patterns as strings so flags must be inline. Regexes are compiled with the [regex](https://pypi.org/project/regex/) package so approximate fuzzy matching is supported. Due to the supported fuzzy matching the matcher returns the fuzzy count values along with match id, start and end information, so make sure to include a variable for the counts when unpacking results.\n\n\n```python\nimport spacy\nfrom spaczz.matcher import RegexMatcher\n\nnlp = spacy.blank("en")\ntext = """Anderson, Grint created spaczz in his home at 555 Fake St,\nApt 5 in Nashv1le, TN 55555-1234 in the USA.""" # Spelling errors intentional.\ndoc = nlp(text)\n\nmatcher = RegexMatcher(nlp.vocab)\n# Use inline flags for regex strings as needed\nmatcher.add("APT", [r"""(?ix)((?:apartment|apt|building|bldg|floor|fl|suite|ste|unit\n|room|rm|department|dept|row|rw)\\.?\\s?)#?\\d{1,4}[a-z]?"""]) # Not the most robust regex.\nmatcher.add("GPE", [r"(?i)[U](nited|\\.?) ?[S](tates|\\.?)"])\nmatches = matcher(doc)\n\nfor match_id, start, end, counts in matches:\n    print(match_id, doc[start:end], counts)\n```\n\n    APT Apt 5 (0, 0, 0)\n    GPE USA (0, 0, 0)\n\n\nSpaczz matchers can also make use of on match rules via callback functions. These on match callbacks need to accept the matcher itself, the doc the matcher was called on, the match index and the matches produced by the matcher. See the fuzzy matcher usage example for details.\n\nLike the fuzzy matcher, the regex matcher has optional keyword arguments that can modify matching behavior. Take the below regex matching example.\n\n\n```python\nimport spacy\nfrom spaczz.matcher import RegexMatcher\n\nnlp = spacy.blank("en")\ntext = """Anderson, Grint created spaczz in his home at 555 Fake St,\nApt 5 in Nashv1le, TN 55555-1234 in the USA.""" # Spelling errors intentional.\ndoc = nlp(text)\n\nmatcher = RegexMatcher(nlp.vocab)\n# Use inline flags for regex strings as needed\nmatcher.add("STREET", ["street_addresses"], kwargs=[{"predef": True}]) # Use predefined regex by key name.\n# Below will not expand partial matches to span boundaries.\nmatcher.add("GPE", [r"(?i)[U](nited|\\.?) ?[S](tates|\\.?)"], kwargs=[{"partial": False}])\nmatches = matcher(doc)\n\nfor match_id, start, end, counts in matches:\n    print(match_id, doc[start:end], counts)\n```\n\n    STREET 555 Fake St, (0, 0, 0)\n\n\nThe full list of keyword arguments available for regex matching rules includes:\n\n- `partial`: Whether partial matches should be extended to existing span boundaries in doc or not, i.e. the regex only matches part of a token or span. Default is True.\n- `predef`: Whether the regex string should be interpreted as a key to a predefined regex pattern or not. Default is False. The included regexes are:\n    - "dates"\n    - "times"\n    - "phones"\n    - "phones_with_exts"\n    - "links"\n    - "emails"\n    - "ips"\n    - "ipv6s"\n    - "prices"\n    - "hex_colors"\n    - "credit_cards"\n    - "btc_addresses"\n    - "street_addresses"\n    - "zip_codes"\n    - "po_boxes"\n    - "ssn_number"\n\nThe above patterns are the same that the [commonregex](https://github.com/madisonmay/CommonRegex) package provides.\n\n### SimilarityMatcher\n\nThe basic usage of the similarity matcher is similar to spaCy\'s phrase matcher except it returns the vector similarity ratio along with match id, start and end information, so make sure to include a variable for the ratio when unpacking results.\n\nIn order to produce meaningful results from the similarity matcher, a spaCy model with word vectors (e.x. medium or large English models) must be used to initialize the matcher, process the target doc, and process any patterns added.\n\n\n```python\nimport spacy\nfrom spaczz.matcher import SimilarityMatcher\n\nnlp = spacy.load("en_core_web_md")\ntext = "I like apples, grapes and bananas."\ndoc = nlp(text)\n\n#lowering min_r2 from default of 75 to produce matches in this example\nmatcher = SimilarityMatcher(nlp.vocab, min_r2=65)\nmatcher.add("FRUIT", [nlp("fruit")])\nmatches = matcher(doc)\n\nfor match_id, start, end, ratio in matches:\n    print(match_id, doc[start:end], ratio)\n```\n\n    FRUIT apples 72\n    FRUIT grapes 72\n    FRUIT bananas 68\n\n\nPlease note that even for the mostly pure-Python spaczz, this process is currently extremely slow so be mindful of the scope in which it is applied. Enabling GPU support in spaCy ([see here](https://spacy.io/usage#gpu)) should improve the speed somewhat, but I believe the process will still be bottlenecked in the pure-Python search algorithm until I develop a better search algorithm and/or drop the search to lower-level code (e.x. C).\n\nAlso as a somewhat experimental feature, the SimilarityMatcher is not currently part of the SpaczzRuler nor does it have a separate ruler. If you need to add similarity matches to a doc\'s entities you will need to use an on-match callback for the time being. Please see the on-match callback example in the FuzzyMatcher documentation for ideas.\n\nThe full list of keyword arguments available for similarity matching rules includes:\n\n- `min_r1`: Minimum similarity match ratio required for selection during the intial search over doc. This should be lower than min_r2 and "low" in general because match span boundaries are not flexed initially. 0 means all spans of query length in doc will have their boundaries flexed and will be re-compared during match optimization. Lower min_r1 will result in more fine-grained matching but will run slower. Default is 50.\n- `min_r2`: Minimum similarity match ratio required for selection during match optimization. Should be higher than min_r1 and "high" in general to ensure only quality matches are returned. Default is 75.\n- `flex`: Number of tokens to move match span boundaries left and right during match optimization. Default is "default".\n\n### SpaczzRuler\n\nThe spaczz ruler combines the fuzzy matcher and regex matcher into one pipeline component that can update a docs entities similar to spaCy\'s entity ruler.\n\nPatterns must be added as an iterable of dictionaries in the format of *{label (str), pattern(str), type(str), optional kwargs (dict), and optional id (str)}*.\n\nFor example:\n\n*{"label": "ORG", "pattern": "Apple", "type": "fuzzy", "kwargs": {"ignore_case": False}, "id": "TECH"}*\n\nThe spaczz ruler also writes custom `Span` attributes to matches it adds.\n\nWhen instantiated, the spaczz ruler adds three custom span attributes: `spaczz_ent`, `spaczz_ent_ratio`, `spaczz_ent_counts`, which all default to `None`. Any span set by the spaczz ruler will have the `spaczz_ent` set to `True`. If it was a fuzzy match it\'s `spaczz_ent_ratio` value will be set and if it was a regex match it\'s `spaczz_ent_counts` value will be set. In the rare case that the same match is made via a fuzzy pattern and regex pattern, the span will have both extensions set with their repsective values.\n\nThe `spaczz_ent` portion of these attributes is controlled by the spaczz ruler\'s `attr` parameter and can be changed if needed. However, the `_ent_ratio` and `_ent_counts` extensions are hard-coded.\n\n\n```python\nimport spacy\nfrom spaczz.pipeline import SpaczzRuler\n\nnlp = spacy.blank("en")\ntext = """Anderson, Grint created spaczz in his home at 555 Fake St,\nApt 5 in Nashv1le, TN 55555-1234 in the USA.""" # Spelling errors intentional.\ndoc = nlp(text)\n\npatterns = [\n    {"label": "NAME", "pattern": "Grant Andersen", "type": "fuzzy", "kwargs": {"fuzzy_func": "token_sort"}},\n    {"label": "STREET", "pattern": "street_addresses", "type": "regex", "kwargs": {"predef": True}},\n    {"label": "GPE", "pattern": "Nashville", "type": "fuzzy"},\n    {"label": "ZIP", "pattern": r"\\b(?:55554){s<=1}(?:(?:[-\\s])?\\d{4}\\b)", "type": "regex"}, # fuzzy regex\n    {"label": "GPE", "pattern": "(?i)[U](nited|\\.?) ?[S](tates|\\.?)", "type": "regex"}\n]\n\nruler = SpaczzRuler(nlp)\nruler.add_patterns(patterns)\ndoc = ruler(doc)\n\nprint("Fuzzy Matches:")\nfor ent in doc.ents:\n    if ent._.spaczz_ent_ratio:\n        print((ent.text, ent.start, ent.end, ent.label_, ent._.spaczz_ent_ratio))\n\nprint("\\n", "Regex Matches:", sep="")\nfor ent in doc.ents:\n    if ent._.spaczz_ent_counts:\n        print((ent.text, ent.start, ent.end, ent.label_, ent._.spaczz_ent_counts))\n```\n\n    Fuzzy Matches:\n    (\'Anderson, Grint\', 0, 3, \'NAME\', 86)\n    (\'Nashv1le\', 17, 18, \'GPE\', 82)\n\n    Regex Matches:\n    (\'555 Fake St,\', 9, 13, \'STREET\', (0, 0, 0))\n    (\'55555-1234\', 20, 23, \'ZIP\', (1, 0, 0))\n    (\'USA\', 25, 26, \'GPE\', (0, 0, 0))\n\n\n### Saving/Loading\n\nThe SpaczzRuler has it\'s own to/from disk/bytes methods and will accept cfg parameters passed to spacy.load(). It also has it\'s own spaCy factory entry point so spaCy is aware of the SpaczzRuler. Below is an example of saving and loading a spacy pipeline with the small English model, the EntityRuler, and the SpaczzRuler.\n\n\n```python\nimport spacy\nfrom spaczz.pipeline import SpaczzRuler\n\nnlp = spacy.load("en_core_web_sm")\ntext = """Anderson, Grint created spaczz in his home at 555 Fake St,\nApt 5 in Nashv1le, TN 55555-1234 in the USA.""" # Spelling errors intentional.\ndoc = nlp(text)\n\nfor ent in doc.ents:\n    print((ent.text, ent.start, ent.end, ent.label_))\n```\n\n    (\'Anderson\', 0, 1, \'ORG\')\n    (\'Grint\', 2, 3, \'ORG\')\n    (\'spaczz\', 4, 5, \'GPE\')\n    (\'555\', 9, 10, \'CARDINAL\')\n    (\'Fake St\', 10, 12, \'PERSON\')\n    (\'5\', 15, 16, \'CARDINAL\')\n    (\'TN\', 19, 20, \'ORG\')\n    (\'55555-1234\', 20, 23, \'DATE\')\n    (\'USA\', 25, 26, \'GPE\')\n\n\nWhile spaCy does a decent job of identifying that named entities are present in this example, we can definitely improve the matches - particularly with the types of labels applied.\n\nLet\'s add an entity ruler for some rules-based matches.\n\n\n```python\nfrom spacy.pipeline import EntityRuler\n\nentity_ruler = EntityRuler(nlp)\nentity_ruler.add_patterns([\n    {"label": "GPE", "pattern": "Nashville"},\n    {"label": "GPE", "pattern": "TN"}\n])\n\nnlp.add_pipe(entity_ruler, before="ner")\ndoc = nlp(text)\n\nfor ent in doc.ents:\n    print((ent.text, ent.start, ent.end, ent.label_))\n```\n\n    (\'Anderson\', 0, 1, \'ORG\')\n    (\'Grint\', 2, 3, \'ORG\')\n    (\'spaczz\', 4, 5, \'GPE\')\n    (\'555\', 9, 10, \'CARDINAL\')\n    (\'Fake St\', 10, 12, \'PERSON\')\n    (\'5\', 15, 16, \'CARDINAL\')\n    (\'TN\', 19, 20, \'GPE\')\n    (\'55555-1234\', 20, 23, \'DATE\')\n    (\'USA\', 25, 26, \'GPE\')\n\n\nWe\'re making progress, but Nashville is spelled wrong in the text so the entity ruler does not find it, and we still have other entities to fix/find.\n\nLet\'s add a spaczz ruler to round this pipeline out. We will also include the `spaczz_ent` custom attribute in the results to denote which entities were set via spaczz.\n\n\n```python\nspaczz_ruler = nlp.create_pipe("spaczz_ruler") # Works due to spaCy factory entry point.\nspaczz_ruler.add_patterns([\n    {"label": "NAME", "pattern": "Grant Andersen", "type": "fuzzy", "kwargs": {"fuzzy_func": "token_sort"}},\n    {"label": "STREET", "pattern": "street_addresses", "type": "regex", "kwargs": {"predef": True}},\n    {"label": "GPE", "pattern": "Nashville", "type": "fuzzy"},\n    {"label": "ZIP", "pattern": r"\\b(?:55554){s<=1}(?:[-\\s]\\d{4})?\\b", "type": "regex"}, # fuzzy regex\n])\nnlp.add_pipe(spaczz_ruler, before="ner")\ndoc = nlp(text)\n\nfor ent in doc.ents:\n    print((ent.text, ent.start, ent.end, ent.label_, ent._.spaczz_ent))\n```\n\n    (\'Anderson, Grint\', 0, 3, \'NAME\', True)\n    (\'spaczz\', 4, 5, \'GPE\', False)\n    (\'555 Fake St,\', 9, 13, \'STREET\', True)\n    (\'5\', 15, 16, \'CARDINAL\', False)\n    (\'Nashv1le\', 17, 18, \'GPE\', True)\n    (\'TN\', 19, 20, \'GPE\', False)\n    (\'55555-1234\', 20, 23, \'ZIP\', True)\n    (\'USA\', 25, 26, \'GPE\', False)\n\n\nAwesome! The small English model still makes a couple named entity recognition mistakes, but we\'re satisfied overall.\n\nLet\'s save this pipeline to disk and make sure we can load it back correctly.\n\n\n```python\nnlp.to_disk("./example")\nnlp = spacy.load("./example")\nnlp.pipe_names\n```\n\n\n\n\n    [\'tagger\', \'parser\', \'entity_ruler\', \'spaczz_ruler\', \'ner\']\n\n\n\nWe can even ensure all the spaczz ruler patterns are still present.\n\n\n```python\nspaczz_ruler = nlp.get_pipe("spaczz_ruler")\nspaczz_ruler.patterns\n```\n\n\n\n\n    [{\'label\': \'NAME\',\n      \'pattern\': \'Grant Andersen\',\n      \'type\': \'fuzzy\',\n      \'kwargs\': {\'fuzzy_func\': \'token_sort\'}},\n     {\'label\': \'GPE\', \'pattern\': \'Nashville\', \'type\': \'fuzzy\'},\n     {\'label\': \'STREET\',\n      \'pattern\': \'street_addresses\',\n      \'type\': \'regex\',\n      \'kwargs\': {\'predef\': True}},\n     {\'label\': \'ZIP\',\n      \'pattern\': \'\\\\b(?:55554){s<=1}(?:[-\\\\s]\\\\d{4})?\\\\b\',\n      \'type\': \'regex\'}]\n\n\n\n## Limitations\n\nSpaczz is written in pure Python and it\'s matchers do not currently utilize spaCy language vocabularies, which means following it\'s logic should be easy to those familiar with Python. However, this means spaczz components will run slower and likely consume more memory than their spaCy counterparts, especially as more patterns are added and documents get longer. It is therefore recommended to use spaCy components like the EntityRuler for entities with little uncertainty, like consistent spelling errors. Use spaczz components when there are not viable spaCy alternatives.\n\n## Future State\n\n1. API support for adding user-defined regexes to the predefined regex.\n    1. Saving these additional predefined regexes as part of the SpaczzRuler will also be supported.\n2. Entity start/end trimming on the token level to prevent fuzzy matches from starting/ending with unwanted tokens, i.e. spaces/punctuation. Will support similar options as spaCy\'s matcher.\n\nWishful thinking:\n\n1. Having the fuzzy/regex matchers utilize spaCy vocabularies.\n2. Rewrite the fuzzy searching algorithm in Cython to utilize C speed.\n3. Fuzzy/regex matching with token patterns along with phrase patterns.\n\n## Development\n\nPull requests and contributors are welcome.\n\nspaczz is linted with [Flake8](https://flake8.pycqa.org/en/latest/), formatted with [Black](https://black.readthedocs.io/en/stable/), type-checked with [MyPy](http://mypy-lang.org/) (although this could benefit from improved specificity), tested with [Pytest](https://docs.pytest.org/en/stable/), automated with [Nox](https://nox.thea.codes/en/stable/), and built/packaged with [Poetry](https://python-poetry.org/). There are a few other development tools detailed in the noxfile.py, along with Git pre-commit hooks.\n\nTo contribute to spaczz\'s development, fork the repository then install spaczz and it\'s dev dependencies with Poetry. If you\'re interested in being a regular contributor please contact me directly.\n\n\n```python\npoetry install # Within spaczz\'s root directory.\n```\n\nThe only package that will not be installed via Poetry but is used for testing and in doc examples is the spaCy medium English model (en-core-web-md). This will need to be installed separately. The command below should do the trick:\n\n\n```python\npoetry run python -m spacy download("en_core_web_md")\n```\n\n## References\n\n- Spaczz tries to stay as close to [spaCy](https://spacy.io/)\'s API as possible. Whenever it made sense to use existing spaCy code within spaczz this was done.\n- Fuzzy matching is performed using [RapidFuzz](https://github.com/maxbachmann/rapidfuzz).\n- Regexes are performed using the [regex](https://pypi.org/project/regex/) library.\n- The search algorithm for fuzzy matching was heavily influnced by Stack Overflow user *Ulf Aslak*\'s answer in this [thread](https://stackoverflow.com/questions/36013295/find-best-substring-match).\n- Spaczz\'s predefined regex patterns were borrowed from the [commonregex](https://github.com/madisonmay/CommonRegex) package.\n- Spaczz\'s development and CI/CD patterns were inspired by Claudio Jolowicz\'s [*Hypermodern Python*](https://cjolowicz.github.io/posts/hypermodern-python-01-setup/) article series.\n',
    'author': 'Grant Andersen',
    'author_email': 'gandersen.codes@gmail.com',
    'maintainer': None,
    'maintainer_email': None,
    'url': 'https://github.com/gandersen101/spaczz',
    'package_dir': package_dir,
    'packages': packages,
    'package_data': package_data,
    'install_requires': install_requires,
    'extras_require': extras_require,
    'entry_points': entry_points,
    'python_requires': '>=3.7,<4.0',
}


setup(**setup_kwargs)
