Tokenization and text representation · MLD-0212
Contextual embedding
A representation whose value depends on an item's surrounding context.
Original reference page · detailed expansion pending. Original definitions, examples, notes and citations are preserved below. New examples are illustrative, not benchmark results.
Original example
Give bank different representations in river and finance sentences.
Original caveat
The same token can have different vectors across layers and contexts.
Context and aliases
Level: Intermediate. Stages: Data · Training · Inference.
Tags: NLP · SLM · LLM.
Aliases: None recorded.
Related concepts
Concepts linking here
Original references
- Hugging Face: chat templates ↗ · Reference link
- Transformers glossary ↗ · Consulted
- Hugging Face: tokenization algorithms ↗ · Consulted
- Original reference URL ↗