Eigenvalues to the rescue? 🛟🤔
text
Eigenvalues to the rescue? 🛟🤔
new_line
new_line
I found out about this paper thanks to 
text
I found out about this paper thanks to 
@gsarti
mention
gsarti
 's post from last week; I got curious, so I want to post my take on it. 🤗
text
 's post from last week; I got curious, so I want to post my take on it. 🤗
new_line
new_line
The paper proposes a new metric called EigenScore to detect LLM hallucinations. 📄
text
The paper proposes a new metric called EigenScore to detect LLM hallucinations. 📄
new_line
new_line
Their idea is that given an input question, they generate K different answers, take their internal embedding states, calculate a covariance matrix with them, and use it to calculate an EigenScore.
text
Their idea is that given an input question, they generate K different answers, take their internal embedding states, calculate a covariance matrix with them, and use it to calculate an EigenScore.
new_line
new_line
We can think of the EigenScore as the mean of the eigenvalues of the covariance matrix of the embedding space of the K-generated answers.
text
We can think of the EigenScore as the mean of the eigenvalues of the covariance matrix of the embedding space of the K-generated answers.
new_line
new_line
❓But why eigenvalues?
text
❓But why eigenvalues?
new_line
new_line
Well, if the K generations have similar semantics, the sentence embeddings will be highly correlated, and most eigenvalues will be close to 0.
text
Well, if the K generations have similar semantics, the sentence embeddings will be highly correlated, and most eigenvalues will be close to 0.
new_line
new_line
On the other hand, if the LLM hallucinates, the K generations will have diverse semantics, and the eigenvalues will be significantly different from 0.
text
On the other hand, if the LLM hallucinates, the K generations will have diverse semantics, and the eigenvalues will be significantly different from 0.
new_line
new_line
The idea is pretty neat and shows better results when compared to other methods like sequence probabilities, length-normalized entropy, and other uncertainty quantification-based methods.
text
The idea is pretty neat and shows better results when compared to other methods like sequence probabilities, length-normalized entropy, and other uncertainty quantification-based methods.
new_line
new_line
💭 What I'm personally missing from the paper is that they don't compare their results with other methods like LLM-Eval and SelfcheckGPT. They do mention that EigenScore is much cheaper to implement than SelfcheckGPT, but that's all on the topic.
text
💭 What I'm personally missing from the paper is that they don't compare their results with other methods like LLM-Eval and SelfcheckGPT. They do mention that EigenScore is much cheaper to implement than SelfcheckGPT, but that's all on the topic.
new_line
new_line
Paper: 
text
Paper: 
INSIDE: LLMs' Internal States Retain the Power of Hallucination
  Detection (2402.03744)
https://huggingface.co/papers/2402.03744
2402.03744
paper
resource
https://huggingface.co/papers/2402.03744
Eigenvalues to the rescue? 🛟🤔

I found out about this paper thanks to @gsarti 's post from last week; I got curious, so I want to post my take on it. 🤗

The paper proposes a new metric called EigenScore to detect LLM hallucinations. 📄

Their idea is that given an input question, they generate K different answers, take their internal embedding states, calculate a covariance matrix with them, and use it to calculate an EigenScore.

We can think of the EigenScore as the mean of the eigenvalues of the covariance matrix of the embedding space of the K-generated answers.

❓But why eigenvalues?

Well, if the K generations have similar semantics, the sentence embeddings will be highly correlated, and most eigenvalues will be close to 0.

On the other hand, if the LLM hallucinates, the K generations will have diverse semantics, and the eigenvalues will be significantly different from 0.

The idea is pretty neat and shows better results when compared to other methods like sequence probabilities, length-normalized entropy, and other uncertainty quantification-based methods.

💭 What I'm personally missing from the paper is that they don't compare their results with other methods like LLM-Eval and SelfcheckGPT. They do mention that EigenScore is much cheaper to implement than SelfcheckGPT, but that's all on the topic.

Paper: https://huggingface.co/papers/2402.03744
❤️
gsarti adamm-hf EquinoxElahin samusenps clem zeusfsx linoyts Nikhil7280 Dlbk Parkerlambert123 dillfrescott Citaman shubhamg2208
🤗
osanseviero Dlbk gsarti adamm-hf dillfrescott Citaman
👍
AnaDP Esquare ashwinlimaye dillfrescott Pent anse10rville