💥 Today in Interpretability & Analysis of LMs: Fine-Tuning Enhances Existing Mechanisms: A Case Study on Entity Tracking by 
text
💥 Today in Interpretability & Analysis of LMs: Fine-Tuning Enhances Existing Mechanisms: A Case Study on Entity Tracking by 
@nikhil07prakash
mention
nikhil07prakash
text
@tamarott
mention
tamarott
text
@TalHaklay
mention
TalHaklay
text
@belinkov
mention
belinkov
text
@davidbau
mention
davidbau
new_line
new_line
Fine-tuning is commonly used to improve LM’s capabilities, but its impact on model-internal mechanisms remains poorly understood.
text
Fine-tuning is commonly used to improve LM’s capabilities, but its impact on model-internal mechanisms remains poorly understood.
new_line
new_line
This work evaluates the impact of fine-tuning from a mechanistic perspective, using entity tracking in fine-tuned LLaMA 7B variants as a test bench.
text
This work evaluates the impact of fine-tuning from a mechanistic perspective, using entity tracking in fine-tuned LLaMA 7B variants as a test bench.
new_line
new_line
Authors use path patching to highlight how fine-tuned models largely employ the same circuits as their pre-trained counterparts to solve entity tracking. Desiderata-based Component Masking (DCM) is used to discern the function of circuit components, finding that even the functionality of the circuit components remains consistent after fine-tuning.
text
Authors use path patching to highlight how fine-tuned models largely employ the same circuits as their pre-trained counterparts to solve entity tracking. Desiderata-based Component Masking (DCM) is used to discern the function of circuit components, finding that even the functionality of the circuit components remains consistent after fine-tuning.
new_line
new_line
Where do the gains stem from, then? Using Cross-Model Activation Patching (CMAP), authors show the benefits of fine-tuning are largely derived from an improved ability of circuit components to encode important task-relevant information rather than an overall functional rearrangement. Interestingly, fine-tuned activations are compatible with the base model despite no explicit constraint during representation learning.
text
Where do the gains stem from, then? Using Cross-Model Activation Patching (CMAP), authors show the benefits of fine-tuning are largely derived from an improved ability of circuit components to encode important task-relevant information rather than an overall functional rearrangement. Interestingly, fine-tuned activations are compatible with the base model despite no explicit constraint during representation learning.
new_line
new_line
🌐 Website: 
text
🌐 Website: 
http://finetuning.baulab.info/
http://finetuning.baulab.info/
link
new_line
🤖 Model: 
text
🤖 Model: 
https://huggingface.co/nikhil07prakash/float-7b
nikhil07prakash/float-7b
model
resource
https://huggingface.co/nikhil07prakash/float-7b
new_line
📄 Paper: 
text
📄 Paper: 
Fine-Tuning Enhances Existing Mechanisms: A Case Study on Entity
  Tracking (2402.14811)
https://huggingface.co/papers/2402.14811
2402.14811
paper
resource
https://huggingface.co/papers/2402.14811
new_line
new_line
🔍 All daily picks: 
text
🔍 All daily picks: 
https://huggingface.co/collections/gsarti/daily-picks-in-interpretability-and-analysis-of-lms-65ae3339949c5675d25de2f9
https://huggingface.co/collections/gsarti/daily-picks-in-interpretability-and-analysis-of-lms-65ae3339949c5675d25de2f9
link
💥 Today in Interpretability & Analysis of LMs: Fine-Tuning Enhances Existing Mechanisms: A Case Study on Entity Tracking by @nikhil07prakash @tamarott @TalHaklay @belinkov @davidbau

Fine-tuning is commonly used to improve LM’s capabilities, but its impact on model-internal mechanisms remains poorly understood.

This work evaluates the impact of fine-tuning from a mechanistic perspective, using entity tracking in fine-tuned LLaMA 7B variants as a test bench.

Authors use path patching to highlight how fine-tuned models largely employ the same circuits as their pre-trained counterparts to solve entity tracking. Desiderata-based Component Masking (DCM) is used to discern the function of circuit components, finding that even the functionality of the circuit components remains consistent after fine-tuning.

Where do the gains stem from, then? Using Cross-Model Activation Patching (CMAP), authors show the benefits of fine-tuning are largely derived from an improved ability of circuit components to encode important task-relevant information rather than an overall functional rearrangement. Interestingly, fine-tuned activations are compatible with the base model despite no explicit constraint during representation learning.

🌐 Website: http://finetuning.baulab.info/
🤖 Model: https://huggingface.co/nikhil07prakash/float-7b
📄 Paper: https://huggingface.co/papers/2402.14811

🔍 All daily picks: https://huggingface.co/collections/gsarti/daily-picks-in-interpretability-and-analysis-of-lms-65ae3339949c5675d25de2f9
👍
vladbogo Alexandro14
❤️
clem