# NLP Data (Corpora, Lexica)

Corpora are collections of documents like conversation transcripts, books, e-mails, tweets, etc. The most valuable corpora have labels for things like the syntax tree, parts of speech, sentiment, "in-reply-to" (for dialog), number of favorites (for tweets) etc

Lexica are collections of words like dictionaries, thesauruses, N-gram collections, or vocabularies (lists of words)


## Lists and Compilations

- [niderhoff list of NLP datasets](https://github.com/niderhoff/nlp-datasets)

## Corpora

- [keyword extraction datasets](https://github.com/zelandiya/keyword-extraction-datasets)
- [Cornell movie dialog corpus](https://github.com/suriyadeepan/datasets/tree/master/seq2seq/cornell_movie_corpus)
- [Manually Annotated Sub-Corpus (MASC) for American English](http://www.anc.org/data/masc/downloads/data-download/)
- [PCAP packet logs](http://pen-testing.sans.org/holiday-challenge/2013)
- Twitter [Bullying Traces Data Set](http://research.cs.wisc.edu/bullying/data.html)
- [twitter corpora and lexica](http://saifmohammad.com/WebPages/lexicons.html)
- [Google NGrams (up to 5-Grams) for 15% of all books up to 2012](http://storage.googleapis.com/books/ngrams/books/datasetsv2.html)
- [Music Match Dataset](http://labrosa.ee.columbia.edu/millionsong/musixmatch#getting)

## Lexica

- [All Top Level Domains (for URI regexes)](https://domainpunch.com/tlds/)
### Password Cracking Dictionaries

**!!BE CAREFUL!!**

[word lists for password cracking](https://hashcat.net/forum/thread-1236.html) 
[Crack Station 15GB password list](https://crackstation.net/buy-crackstation-wordlist-password-cracking-dictionary.htm) <--donate at your own risk


## Trained Models

[Official Google News Word2Vec Code & Data](https://code.google.com/archive/p/word2vec/)
[GitHub Mirror of Google News Word2Vec Code & Data](https://github.com/mmihaltz/word2vec-GoogleNews-vectors)
Download Google Drive 1.5GB Word2Vec Model[GoogleNews-vectors-negative300.bin.gz](https://drive.google.com/uc?export=download&id=0B7XkCwpI5KDYNlNUTTlSS21pQmM)