Neural-network building blocks · MLD-0357

Embedding

A vector representation learned or constructed for an item, token, or concept.

Expanded guide · editorial review 2026-11-01. Original definitions, examples, notes and citations are preserved below. New examples are illustrative, not benchmark results.

Prerequisites

How it works

An embedding represents an item as a vector. Its training objective determines which relationships the geometry captures. Word embeddings can be learned from neighboring words; sentence and image embeddings use other objectives. The same word can have a fixed vector in one system and a context-dependent representation in another.

Worked example

In an illustrative two-dimensional space, vectors (1, 0) and (2, 0) have the same direction and cosine similarity 1. Vector (0, 1) is orthogonal to both and has cosine similarity 0. Real learned dimensions are not usually interpretable as named human attributes.

A useful equation

cosine(a, b) = dot(a, b) / (norm(a) norm(b)), for nonzero vectors. For normalized vectors, ranking by cosine is equivalent to ranking by dot product.

What to test

For a search system, label relevant document-query pairs and inspect nearest neighbors. Include paraphrases, negation, specialized vocabulary and multilingual queries before treating proximity as useful relevance.

Common pitfall

Similarity is model- and task-dependent. Nearby vectors need not represent equivalent facts, and high similarity does not prove a retrieved passage answers the question.

Sources for the expanded explanation

Editorial synthesis checked against the linked primary material; no independent replication is claimed.

Original example

Represent a document for similarity search.

Original caveat

Nearby vectors are meaningful only relative to the embedding objective.

Context and aliases

Level: Intermediate. Stages: Training · Inference · Theory.

Tags: DL · SLM · LLM · Vision · NLP.

Aliases: None recorded.

Related concepts

Concepts linking here

Technology connections

Original references

Open original record, export or annotate →