🎭 You picked an LLM for your work but then you find out it hallucinates! 🤖
text
🎭 You picked an LLM for your work but then you find out it hallucinates! 🤖
new_line
new_line
🤔 Your first thought might be to fine-tune it on more training data.... but should you? 🛠️
text
🤔 Your first thought might be to fine-tune it on more training data.... but should you? 🛠️
new_line
new_line
📜 This is what 
text
📜 This is what 
@Google
mention
Google
 is exploring in the paper "Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?" 🕵️‍♂️
text
 is exploring in the paper "Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?" 🕵️‍♂️
new_line
new_line
📘 When LLMs undergo supervised fine-tuning with new factual knowledge not present in their initial training data, there is a risk they might "hallucinate" or produce factually incorrect information. 🚨
text
📘 When LLMs undergo supervised fine-tuning with new factual knowledge not present in their initial training data, there is a risk they might "hallucinate" or produce factually incorrect information. 🚨
new_line
new_line
🔍 The paper investigates how fine-tuning LLMs with new facts influences their ability to leverage pre-existing knowledge and the extent to which they generate errors. 📊
text
🔍 The paper investigates how fine-tuning LLMs with new facts influences their ability to leverage pre-existing knowledge and the extent to which they generate errors. 📊
new_line
new_line
⚙️Technical Setup:
text
⚙️Technical Setup:
new_line
new_line
🔧 Approach: They introduce a system named SliCK (this stands for Sampling-based Categorization of Knowledge, don't even bother understanding how) to categorize knowledge into four levels (HighlyKnown, MaybeKnown, WeaklyKnown, and Unknown) based on how well the model's generated responses agree with known facts. 🗂️
text
🔧 Approach: They introduce a system named SliCK (this stands for Sampling-based Categorization of Knowledge, don't even bother understanding how) to categorize knowledge into four levels (HighlyKnown, MaybeKnown, WeaklyKnown, and Unknown) based on how well the model's generated responses agree with known facts. 🗂️
new_line
new_line
📝 Experimental Setup: The study uses a controlled setup focusing on closed-book QA, adjusting the proportion of fine-tuning examples that introduce new facts versus those that do not. 🧪
text
📝 Experimental Setup: The study uses a controlled setup focusing on closed-book QA, adjusting the proportion of fine-tuning examples that introduce new facts versus those that do not. 🧪
new_line
new_line
👉 Here is the gist of the findings:
text
👉 Here is the gist of the findings:
new_line
new_line
🚸 LLMs struggle to integrate new factual knowledge during fine-tuning, and such examples are learned slower than those consistent with the model's pre-existing knowledge. 🐢
text
🚸 LLMs struggle to integrate new factual knowledge during fine-tuning, and such examples are learned slower than those consistent with the model's pre-existing knowledge. 🐢
new_line
new_line
📈 As LLMs learn from examples containing new knowledge, their propensity to hallucinate increases. 👻
text
📈 As LLMs learn from examples containing new knowledge, their propensity to hallucinate increases. 👻
new_line
new_line
⏱️ Early stopping during training can mitigate the risks of hallucinations by minimizing exposure to unlearned new facts. 🛑
text
⏱️ Early stopping during training can mitigate the risks of hallucinations by minimizing exposure to unlearned new facts. 🛑
new_line
new_line
🧠 Training LLMs mostly with known examples leads to better utilization of pre-existing knowledge, whereas examples introducing new knowledge increase the risk of generating incorrect information. 🏗️
text
🧠 Training LLMs mostly with known examples leads to better utilization of pre-existing knowledge, whereas examples introducing new knowledge increase the risk of generating incorrect information. 🏗️
new_line
new_line
📄 Paper: 
text
📄 Paper: 
Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations? (2405.05904)
https://huggingface.co/papers/2405.05904
2405.05904
paper
resource
https://huggingface.co/papers/2405.05904
 📚
text
 📚
🎭 You picked an LLM for your work but then you find out it hallucinates! 🤖

🤔 Your first thought might be to fine-tune it on more training data.... but should you? 🛠️

📜 This is what @Google is exploring in the paper "Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?" 🕵️‍♂️

📘 When LLMs undergo supervised fine-tuning with new factual knowledge not present in their initial training data, there is a risk they might "hallucinate" or produce factually incorrect information. 🚨

🔍 The paper investigates how fine-tuning LLMs with new facts influences their ability to leverage pre-existing knowledge and the extent to which they generate errors. 📊

⚙️Technical Setup:

🔧 Approach: They introduce a system named SliCK (this stands for Sampling-based Categorization of Knowledge, don't even bother understanding how) to categorize knowledge into four levels (HighlyKnown, MaybeKnown, WeaklyKnown, and Unknown) based on how well the model's generated responses agree with known facts. 🗂️

📝 Experimental Setup: The study uses a controlled setup focusing on closed-book QA, adjusting the proportion of fine-tuning examples that introduce new facts versus those that do not. 🧪

👉 Here is the gist of the findings:

🚸 LLMs struggle to integrate new factual knowledge during fine-tuning, and such examples are learned slower than those consistent with the model's pre-existing knowledge. 🐢

📈 As LLMs learn from examples containing new knowledge, their propensity to hallucinate increases. 👻

⏱️ Early stopping during training can mitigate the risks of hallucinations by minimizing exposure to unlearned new facts. 🛑

🧠 Training LLMs mostly with known examples leads to better utilization of pre-existing knowledge, whereas examples introducing new knowledge increase the risk of generating incorrect information. 🏗️

📄 Paper: https://huggingface.co/papers/2405.05904 📚
👍
Ksgk-fy DynaTechSystems Rybens osanseviero lamhieu kimleang123 moock
😎
tasmani DynaTechSystems osanseviero
❤️
DynaTechSystems osanseviero
664a23d0cf5fd472d77ad034
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/PKzQSYUHCgFFVYPeeNGYy.png
DynaTech Systems
DynaTechSystems
user
5f5ac23710b2753d9000c864
/avatars/a5d3be1c3e9df8a367922d7aa3214cca.svg
Kevin Amiri
kevin009
user