Language processing


“the trophy doesn’t fit in the suitcase because it is too big” - geoff hinton “the trophy doesn’t fit in the suitcase because it is too small” - geoff hinton

Trying to get algorithms to make sense of ambiguity of human language, we begin to appreciate just how much we take for granted. We hardly notice the tiny feats of disambiguation our brains do when reading sentences like the ones above.

you shall know the nature of a word by the company it keeps

What kinds of language-oriented tasks might we be interested in?

This chapter is about applications of machine learning to natural language processing. like ml, NLP is a nebulous term with several precise definitions and most have something to do wth making sense from text. This chapter will take a broad view of NLP

“Deep Learning waves have lapped at the shores of computational linguistics for several years now, but 2015 seems like the year when the full force of the tsunami hit the major Natural Language Processing (NLP) conferences.” -Dr. Christopher D. Manning, Dec 2015

quote/link from:

word vectors

word vectors are a rep such that geometric preserved in emeddings. reverse king queen

cover tf-idf in detail (link to it fromm tsne chapter). link to t-sne chapter from here

since then, there have been a number of writings which have tried to interpret these word vectors. gender binary

tf-idf examples -> t-SNE examples

aparrish generative poetry


tf-idf -> t-SNE LSA + LDA -> t-SNE

RNNs annotating?

word2vec chapter


Mario RNN

attention + DRAW the trophy can’t fit into the suitcase because it’s too big (it = trophy) the trophy can’t fit into the suitcase because it’s too small (it = suitcase)

colah word2vec

metal + NLP

lda2vec Chris Moody hybrid word2vec and LDA

historical word embeddings



Language modeling a billion words

demystifying word2vec

keras word2vec

hotel reviews: rejecting gender binary

Voynich Manuscript: word vectors and t-SNE visualization of some patterns

kcimc synonyms + antonyms

CNN sentence classificaiton

Chris Olah Word2Vec + tSNE

paragraph vectors:

text_analytics_on_mpp doc2vec from newsgroups

Harvard NLP Stanford NLP



Feature extraction and word embeddings

Organizing, retrieving documents

NLP tasks

Speculative NLP tasks

db extract wikipedia





dimensionality reduction

text representations

applications of text representations

word vectors

paragraph vectors / skip-thoughts

NLP tasks



sebastian ruder blog: