Cheat sheet

Part 4 · Semantics & Word Embeddings — Cheat Sheet

Four illustrated pages — lexical relations, similarity and WordNet, distributional semantics, and Word2Vec.

← Read the full post (Part 4)Updated June 2026

Part 4 · Semantics & Word Embeddings — Cheat Sheet — printable cheat sheet

Or read the searchable version below.

Page 1 · Lexical relations

Page 1 of 4 of the semantics cheatsheet. Ten cards covering: one lemma multiple meanings (bank — financial institution / river edge); semantic relationships overview (synonymy, antonymy, polysemy, homonymy, metonymy, hyponymy around word meaning); synonyms (big/large, sofa/couch, begin/start); synonyms are not perfect clones (buy/purchase, child/kid); antonyms (gradable hot/cold, complementary alive/dead, converse buy/sell); homonyms / homographs / homophones (bat, lead, two/too); metonymy and systematic polysemy (White House → institution, school → building/organization); hyponymy and hypernymy (animal → dog → poodle); hierarchical lexical database (entity → animal → canine → dog); WordNet works at the sense level (bass — fish / low voice). — Core word-sense relations, ambiguity, and WordNet basics.

Page 2 · Similarity and WordNet

Page 2 of 4. Ten cards: WordNet hierarchy for bass (entity → organism → person → entertainer → musician → singer → bass singer); similarity as a distance metric (red/blue/green markers between 0 and 1); two families of similarity algorithms (thesaurus-based vs distributional); path-based similarity (currency / coin / nickel / dime / United States coin / standard); information-content similarity (entity → animal → dog → poodle, more specific → more information); Dekang Lin similarity (formula sim(c₁,c₂) = 2·IC(LCS) / (IC(c₁)+IC(c₂))); why thesaurus-only methods struggle (scale, low-resource, new words, slang); distributional meaning (WordNet vs corpus-based vector learning); distributional similarity intuition (tesguino — drink, corn, bottle, alcoholic, beverage, similar to beer); term-document matrix reminder. — Taxonomies, semantic similarity, and the bridge to distributional meaning.

Page 3 · Distributional semantics

Page 3 of 4. Ten cards: term-context matrix (rows = target words, columns = nearby context words); context windows (window size 2 around apricot/pineapple/digital); similarity from context (similar rows mean similar meanings); why raw co-occurrence counts mislead (the/of/in/and dominate); pointwise mutual information PMI formula; applying PMI to a matrix (raw counts → PMI matrix); PPMI and smoothing (PPMI = max(PMI,0), add-k smoothing); linear vs syntactic context; dependency-based co-occurrence vectors (drink with subject-of, object-of, modified-by, with/using); PMI in dependency relations (drink → wine vs drink → it). — Term-context matrices, PMI, and dependency-based context.

Page 4 · Word2Vec

Page 4 of 4. Eight cards: Word2Vec overview (input target word → hidden layer V×d → output context words); training samples (sliding window over 'the quick brown fox jumps over the lazy dog'); hidden layer equals embedding (matrix V×d, each row is a dense word vector); skip-gram vs CBOW (predict context from target vs predict target from context); word-vector analogies (king − man + woman ≈ queen; France:Paris :: Germany:Berlin; walk:walked :: talk:talked); advantages of Word2Vec (dense representations, less sparsity, captures semantic relations, helps downstream NLP); limitations of Word2Vec (one vector per word, weak on word order, domain mismatch, depends on data quality); Anything2Vec extensions (Doc2Vec, FastText, GloVe, Med2Vec, Author2Vec, Concept2Vec). — From context prediction to dense word embeddings.