25.4K
GELearn more at https://aimlcompanion.ai
Part 1 - LLM Architecture and inferences
1. Tokenization: Text is split into sub-words/bytes, then mapped to token IDs the model can process.
2. Embeddings: Each token ID becomes a dense vector encoding semantic meaning and context potential.
3. Next Token Prediction: The model predicts the most probable next token, one step at a time.
4. Temperature: Scales logits -> low = deterministic, high = creative but risky.
5. Top-K / Top-P: Restricts sampling to likely tokens to avoid nonsense outputs.
6. KV Cache: Stores past attention keys/values so generation doesn’t recompute history.
7. Beam Search: Explores multiple token sequences in parallel and picks the best overall path.
8. Context Window: The maximum number of tokens the model can attend to at once.
9. RoPE: Injects relative position info directly into attention using rotations, not embeddings.
10. Flash Attention: Memory-efficient attention via tiling + recomputation, enabling longer contexts.
11. Self-Attention: Tokens attend to each other using Query, Key, Value projections.
12. Multi-Head Attention: Multiple attention spaces learn different relationships in parallel.
13. Causal Masking: Prevents the model from seeing future tokens during generation.
14. Transformer Block: Attention + MLP + residuals + layer norm = one reasoning step.
15. Softmax: Converts raw logits into a probability distribution over the vocabulary.
LLMs don’t think, they compress patterns from massive data & predict the next token extremely well.
#genai #artificalintelligence #generativeai
@genieincodebottle
