Good folks from Universitat Politècnica de Catalunya, University of Groningen, and Meta have released "A Primer on the Inner Workings of Transformer-based Language Models."
text
Good folks from Universitat Politècnica de Catalunya, University of Groningen, and Meta have released "A Primer on the Inner Workings of Transformer-based Language Models."
new_line
new_line
They don't make survey papers like they used to, but this is an exciting new survey on Transformer LM interpretability!
text
They don't make survey papers like they used to, but this is an exciting new survey on Transformer LM interpretability!
new_line
new_line
This comprehensive survey provides a technical deep dive into:
text
This comprehensive survey provides a technical deep dive into:
new_line
new_line
• Transformer architecture components (attention, FFN, residual stream)
text
• Transformer architecture components (attention, FFN, residual stream)
new_line
• Methods for localizing model behavior:
text
• Methods for localizing model behavior:
new_line
  - Input attribution (gradient & perturbation-based)
text
  - Input attribution (gradient & perturbation-based)
new_line
  - Component importance (logit attribution, causal interventions)
text
  - Component importance (logit attribution, causal interventions)
new_line
• Information decoding techniques:
text
• Information decoding techniques:
new_line
  - Probing, linear feature analysis
text
  - Probing, linear feature analysis
new_line
  - Sparse autoencoders for disentangling features
text
  - Sparse autoencoders for disentangling features
new_line
• Key insights on model internals:
text
• Key insights on model internals:
new_line
  - Attention mechanisms (induction heads, copy suppression)
text
  - Attention mechanisms (induction heads, copy suppression)
new_line
  - FFN neuron behaviors 
text
  - FFN neuron behaviors 
new_line
  - Residual stream properties
text
  - Residual stream properties
new_line
  - Multi-component emergent behaviors
text
  - Multi-component emergent behaviors
new_line
new_line
The paper offers a unified notation and connects insights across different areas of interpretability research. It's a must-read for anyone working on understanding large language models!
text
The paper offers a unified notation and connects insights across different areas of interpretability research. It's a must-read for anyone working on understanding large language models!
new_line
new_line
Some fascinating technical highlights:
text
Some fascinating technical highlights:
new_line
- Detailed breakdowns of attention head circuits (e.g., IOI task)
text
- Detailed breakdowns of attention head circuits (e.g., IOI task)
new_line
- Analysis of factual recall mechanisms 
text
- Analysis of factual recall mechanisms 
new_line
- Overview of polysemanticity and superposition
text
- Overview of polysemanticity and superposition
new_line
- Discussion of grokking as circuit emergence
text
- Discussion of grokking as circuit emergence
new_line
new_line
What interpretability insights do you find most intriguing?
text
What interpretability insights do you find most intriguing?
Good folks from Universitat Politècnica de Catalunya, University of Groningen, and Meta have released "A Primer on the Inner Workings of Transformer-based Language Models."

They don't make survey papers like they used to, but this is an exciting new survey on Transformer LM interpretability!

This comprehensive survey provides a technical deep dive into:

• Transformer architecture components (attention, FFN, residual stream)
• Methods for localizing model behavior:
  - Input attribution (gradient & perturbation-based)
  - Component importance (logit attribution, causal interventions)
• Information decoding techniques:
  - Probing, linear feature analysis
  - Sparse autoencoders for disentangling features
• Key insights on model internals:
  - Attention mechanisms (induction heads, copy suppression)
  - FFN neuron behaviors 
  - Residual stream properties
  - Multi-component emergent behaviors

The paper offers a unified notation and connects insights across different areas of interpretability research. It's a must-read for anyone working on understanding large language models!

Some fascinating technical highlights:
- Detailed breakdowns of attention head circuits (e.g., IOI task)
- Analysis of factual recall mechanisms 
- Overview of polysemanticity and superposition
- Discussion of grokking as circuit emergence

What interpretability insights do you find most intriguing?
👀
John6666