"A Closer Look at the Limitations of Instruction Tuning" is a new paper that explores the efficacy and limitations of Instruction Tuning (IT) in Large Language Models (LLMs) for conversational agents. The authors conduct a series of experiments using both LoRA fine-tuning (LFT) and standard full-parameter fine-tuning (SFT) across various LLMs and IT datasets. 
text
"A Closer Look at the Limitations of Instruction Tuning" is a new paper that explores the efficacy and limitations of Instruction Tuning (IT) in Large Language Models (LLMs) for conversational agents. The authors conduct a series of experiments using both LoRA fine-tuning (LFT) and standard full-parameter fine-tuning (SFT) across various LLMs and IT datasets. 
new_line
new_line
The key findings are:
text
The key findings are:
new_line
* LoRA fine-tuning (LFT) preserves the pre-training token distribution while SFT doesn't. This indicates that using LFT, post fine-tuning the model still heavily relies on the pre-training and doesn't acquire new information.
text
* LoRA fine-tuning (LFT) preserves the pre-training token distribution while SFT doesn't. This indicates that using LFT, post fine-tuning the model still heavily relies on the pre-training and doesn't acquire new information.
new_line
* Dataset scaling is ineffective for LFT - experiments show that scaling the dataset size 52x or even 326x doesn't improve the performance.
text
* Dataset scaling is ineffective for LFT - experiments show that scaling the dataset size 52x or even 326x doesn't improve the performance.
new_line
* LoRA fine-tuning mainly enhances response initiation and style without substantial knowledge enhancement.
text
* LoRA fine-tuning mainly enhances response initiation and style without substantial knowledge enhancement.
new_line
* Full-parameter fine-tuning tends to degrade LLM knowledge base and increase hallucination occurrences.
text
* Full-parameter fine-tuning tends to degrade LLM knowledge base and increase hallucination occurrences.
new_line
* Popular other methods and adjustments fail to significantly outperform simple LoRA fine-tuned models in terms of conversational quality and accuracy.
text
* Popular other methods and adjustments fail to significantly outperform simple LoRA fine-tuned models in terms of conversational quality and accuracy.
new_line
new_line
Congrats to the authors 
text
Congrats to the authors 
@Sreyan88
mention
Sreyan88
 and others for their work!
text
 and others for their work!
new_line
new_line
Paper: 
text
Paper: 
A Closer Look at the Limitations of Instruction Tuning (2402.05119)
https://huggingface.co/papers/2402.05119
2402.05119
paper
resource
https://huggingface.co/papers/2402.05119
"A Closer Look at the Limitations of Instruction Tuning" is a new paper that explores the efficacy and limitations of Instruction Tuning (IT) in Large Language Models (LLMs) for conversational agents. The authors conduct a series of experiments using both LoRA fine-tuning (LFT) and standard full-parameter fine-tuning (SFT) across various LLMs and IT datasets. 

The key findings are:
* LoRA fine-tuning (LFT) preserves the pre-training token distribution while SFT doesn't. This indicates that using LFT, post fine-tuning the model still heavily relies on the pre-training and doesn't acquire new information.
* Dataset scaling is ineffective for LFT - experiments show that scaling the dataset size 52x or even 326x doesn't improve the performance.
* LoRA fine-tuning mainly enhances response initiation and style without substantial knowledge enhancement.
* Full-parameter fine-tuning tends to degrade LLM knowledge base and increase hallucination occurrences.
* Popular other methods and adjustments fail to significantly outperform simple LoRA fine-tuned models in terms of conversational quality and accuracy.

Congrats to the authors @Sreyan88 and others for their work!

Paper: https://huggingface.co/papers/2402.05119
👍
Hopkinx osanseviero ioanacroi kaleinaNyan clem Felladrin RamaniD sonalkum TheDetective Sreyan88 Arjun4707 TerraNull ta012
65dbf975c5f46bf23bd5afd2
/avatars/e23e00fa020ae3f3f304fbf9fb2dc244.svg
mohd khazli aswad
khazli
user
6454aff9273f649830234978
https://cdn-avatars.huggingface.co/v1/production/uploads/6454aff9273f649830234978/cvVV08YHJpJx9xWVZqgVW.jpeg
Victor Nogueira
Felladrin
user