Transformer Self-Attention
You are a Engineering Manager on-call. A production issue might involve Transformer Self-Attention. How do you diagnose and mitigate?
Answers use simple, clear English.
Audio N/AQuick interview answer
Mitigate first, then root-cause. Check symptoms against: Confusing encoder-only (BERT) vs decoder-only (GPT) vs encoder-decoder (T5)..
Detailed answer
Mitigate first, then root-cause. Check symptoms against: Confusing encoder-only (BERT) vs decoder-only (GPT) vs encoder-decoder (T5).. Validate with: Know masking in causal LMs; KV-cache for inference speed.. Context: Queries/Keys/Values compute attention weights; each token aggregates context. Multi-head captures different relations. Positional encodings add order. Core: Queries/Keys/Values compute attention weights; each token aggregates context. Multi-head captures different relations. Positional encodings add order. Real-time example: In 'The animal didn't cross the street because it was tired', attention links 'it'→'animal'. Pros: Parallel training; long-range dependencies better than RNNs. Cons: O(n²) memory/time in sequence length. Common mistakes: Confusing encoder-only (BERT) vs decoder-only (GPT) vs encoder-decoder (T5). Best practices: Know masking in causal LMs; KV-cache for inference speed. Audience level: Engineering Manager.
Full explanation
Queries/Keys/Values compute attention weights; each token aggregates context. Multi-head captures different relations. Positional encodings add order.
Real example & use case
In 'The animal didn't cross the street because it was tired', attention links 'it'→'animal'.
Pros & cons
Pros: Parallel training; long-range dependencies better than RNNs. Cons: O(n²) memory/time in sequence length.
Common mistakes
Confusing encoder-only (BERT) vs decoder-only (GPT) vs encoder-decoder (T5).
Best practices
Know masking in causal LMs; KV-cache for inference speed.
Follow-up questions
- How would you test Transformer Self-Attention?
- What metrics prove Transformer Self-Attention is healthy in prod?
- How does Transformer Self-Attention change at 10× traffic?