SoMaJo
======

SoMaJo is a rule-based tokenizer and sentence splitter that implements
tokenization guidelines for German and English. It has a strong focus
on web and social media texts (it was originally created as the
winning submission to the `EmpiriST 2015 shared task
<https://sites.google.com/site/empirist2015/>`_ on automatic
linguistic annotation of computer-mediated communication / social
media) and is particularly well-suited to perform tokenization on all
kinds of written discourse, for example chats, forums, wiki talk
pages, tweets, blog comments, social networks, SMS and WhatsApp
dialogues. Of course it also works on more formal texts.

More detailed documentation is available `here
<https://github.com/tsproisl/SoMaJo>`_.
