wildgram tokenizes english text into "wild"-grams (tokens of varying word count)
that match closely to the the natural pauses of conversation. I originally built
it as the first step in an abstraction pipeline for medical language: since
medical concepts tend to be phrases of varying lengths, bag-of-words or bigrams
doesn't really cut it.

Wildgrams works by measuring the size of the noise (stopwords, punctuation, and
whitespace) and breaks up the text against noise of a certain size
(it varies slightly depending on the noise).
Some examples:
"rats, bats, and vats" -> ["rats","bats", "vats"]
"I dreamed a dream in time gone by" -> ["i dreamed","dream", "time gone"]

Because this is originally for a medical abstraction, some of the stop words include
words like "denied", "describe" which tend to signify a change
in topic in medical notes. Some of these words are separated into their own topics
(since they have clinical meaning), while others are removed entirely.
Numbers (0.1, 1.0, 3), "denied", etc. are an example of the former.

Words with digits that have other punctuation or characters
(e.g. potentially ids like CS123123 or time like 12:30)
are left alone as potentially part of a larger phrase.

The full list of words that get removed entirely is in the global constant STOPWORDS in wildgram.py
The full list of words/regex patterns that are separated into their own chunk can be found in TOPICWORDS in wildgram.py

Two examples:
"patient denies consuming alcohol" -> ["patient", "denies", "consuming alcohol"]
"patient describes consuming alcohol" -> ["patient", "consuming alcohol"]

Also note that it doesn't strictly tokenize each token like so:
"I dreamed a dream in time gone by" -> [("I","dreamed"),("dream"), ("time","gone")]

Final note: I do not include "of" in the stop word list, because there are quite a few
medical concepts that have of in the middle (e.g. "shortness of breath").

Updates for 0.0.6 Version:
There are now 3 new parameters to the wildgram function, all with defaults:
topicwords, stopwords, and include1gram.
topicwords: default is the list in TOPICWORDS. Can be replaced with a custom list.
stopwords: default is the list in STOPWORDS. Can be replaced with a custom list.
include1gram: boolean, default is False. True will include both the full
wildgram phrase as well as all individual tokens in the phrase, e.g.:
"I dreamed a dream in time gone by" -> ["I dreamed", "I", "dream","time gone", "time", "gone"]
Any 1-gram tokens immediately follow the wildgram token so it's still a semi-ordered list.
In the future, I may make this default to True. Currently in the process of testing
the quality of the annotation matching if you include 1grams vs. exclude it. 

Example code:

```python
from wildgram import wildgram
tokens, ranges = wildgram("and was beautiful")
#tokens -- the wildgram tokens
#ranges -- a list of tuples, the ith tuple has the start and end indexes for the ith wildgram
print(tokens, ranges) #["beautiful"], [(8, 17)]
tokens, ranges = wildgram("was a beautiful day", include1gram=True)
#tokens -- the wildgram tokens
#ranges -- a list of tuples, the ith tuple has the start and end indexes for the ith wildgram
print(tokens, ranges) #["beautiful day", "beautiful", "day"]
```
That's all folks!
