How Large Language Models Generate Coherent Text

Part 1: Tokens and the Embedding Space

Computers cannot read letters or words directly. Text is chopped into sub-word chunks called tokens, each assigned a numerical identifier and placed into a multi-thousand-dimensional semantic space where words with related meanings sit near each other.

Part 2: The Transformer Self-Attention Mechanism

1. 1. Query, Key, Value Projections: Every token generates Query, Key, and Value vectors.
2. 2. Attention Matrix: The dot-product of Queries and Keys measures how much attention each word owes every other word in the prompt.
3. 3. Softmax Normalization: Scores are normalized into attention weights summing to 1.0.
4. 4. Feed-Forward Transformation: Tokens pass through dense multi-layer perceptrons to synthesize abstract contextual facts.
5. 5. Sampling: The model outputs a probability distribution over the entire vocabulary and samples the next token.

Part 3: Hallucinations, Grounding & AEO Verification

Because pure LLMs predict plausibility rather than ground truth, modern systems pair foundation models with retrieval-augmented generation (RAG) and active recall verification gates.