AI Foundations Playgrounds · Intermediate

Embedding Layer Playground

Watch words become vectors: token lookup, embedding dimensions and positional encoding, step by step.

Embedding lookup Positional encoding Vector dimensions

Words are converted into numerical vectors (lists of numbers) that capture semantic meaning.

Each number represents a different aspect of meaning

Example: "king" → [0.2, -0.5, 0.8, ...]

Similar words have similar vector values. Distance between vectors measures semantic closeness.

Closer vectors = more similar meanings

"cat" and "dog" vectors are closer than "cat" and "car"

Each dimension captures different semantic features like gender, size, or category learned from text.

Higher dimensions = more nuanced representations

Common sizes: 128, 256, 512, or 768 dimensions

Learned Relationships: Models discover patterns like "king - man + woman = queen"

Context Awareness: Same word can have different embeddings based on surrounding text

Transfer Learning: Pre-trained embeddings capture general language understanding

Visualization: High-dimensional vectors projected to 2D/3D for human interpretation

Tokenized Words