Skip to content

Improve Word2Vec and GloVe slides with interactive resources - #15

Merged
zzcoolj merged 1 commit into
zzcoolj:mainfrom
avenirlumen:feature/improve-word2vec-glove-slides
Feb 11, 2026
Merged

zzcoolj merged 1 commit into
zzcoolj:mainfrom
avenirlumen:feature/improve-word2vec-glove-slides

Conversation

@avenirlumen

Copy link
Copy Markdown
Contributor

Summary

This PR enhances the word embeddings session (Day 1 Session 2) with improved explanations and curated interactive resources.

What's Added

New Interactive Demo Section

  • wevi link: Watch Word2Vec weight matrices update in real-time during training (by Xin Rong)

Expanded GloVe Explanation (3 new slides)

  1. Co-occurrence Ratios Insight: The ice/steam table from the original paper showing how ratios encode semantics
  2. Derivation of Objective: Step-by-step explanation of WHY dot products ≈ log counts (the exponential converts differences → ratios)
  3. GloVe Objective Function: Details on the weighted squared error and the weighting function f(x)

Deep Connection Slide

  • Levy & Goldberg (2014) result: Word2Vec Skip-gram with Negative Sampling implicitly factorizes a shifted PMI matrix
  • Shows that Word2Vec and GloVe are mathematically related

Interactive Resources Section

New slide with 5 curated learning tools:

Tool What it does
wevi Watch Word2Vec weights update live
Imperial College Demo Train your own model, see PCA clustering
TensorFlow Projector Explore pre-trained embeddings in 3D
Lena Voita's Course Best mathematical explanations
Dive into Deep Learning Interactive textbook with code

Updated References

  • Added Levy & Goldberg (2014) "Neural Word Embedding as Implicit Matrix Factorization"
  • Added Xin Rong (2014) "word2vec Parameter Learning Explained"

Stats

  • +235 lines (6 new slides)
  • No breaking changes to existing structure

…urces

Key improvements:
- Add interactive wevi demo link (watch Word2Vec training live)
- Add 3 new slides for GloVe:
  - Co-occurrence ratios insight (ice/steam table from original paper)
  - Derivation of objective function (why dot product = log counts)
  - GloVe objective function details with weighting function
- Add 'Deep Connection' slide (Levy & Goldberg 2014: Word2Vec ≈ PMI matrix factorization)
- Add Interactive Resources section with 5 curated learning tools:
  - wevi (weight visualization)
  - Imperial College demo (PCA clustering)
  - TensorFlow Projector (3D exploration)
  - Lena Voita's NLP course (best math explanations)
  - Dive into Deep Learning (runnable code)
- Update references to include Levy & Goldberg and Xin Rong papers
@zzcoolj
zzcoolj merged commit 45eb7a3 into zzcoolj:main Feb 11, 2026
1 check failed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants