So, I have this idea to (potentially) improve uncertainty quantification for LLM hallucination detection.
text
So, I have this idea to (potentially) improve uncertainty quantification for LLM hallucination detection.
new_line
new_line
The premise is that not all output tokens of a generated response share the same importance. Hallucinations are more dangerous in the form of a noun, date, number, etc.
text
The premise is that not all output tokens of a generated response share the same importance. Hallucinations are more dangerous in the form of a noun, date, number, etc.
new_line
new_line
The idea is to have a "token selection" layer that filters the output token probabilities sequence. Then, we use only the probabilities of the relevant tokens to calculate uncertainty quantification metrics.
text
The idea is to have a "token selection" layer that filters the output token probabilities sequence. Then, we use only the probabilities of the relevant tokens to calculate uncertainty quantification metrics.
new_line
new_line
The big question is how we know which tokens are the relevant ones. 🤔
text
The big question is how we know which tokens are the relevant ones. 🤔
new_line
new_line
My idea is to use the output sequence (decoded one) and use an NLP model (it doesn't need to be a fancy one) to do entity recognition and part-of-speech tagging to the output sequence and then do uncertainty quantification only on the entities that we have set as relevant (nouns, dates, numbers, etc).
text
My idea is to use the output sequence (decoded one) and use an NLP model (it doesn't need to be a fancy one) to do entity recognition and part-of-speech tagging to the output sequence and then do uncertainty quantification only on the entities that we have set as relevant (nouns, dates, numbers, etc).
new_line
new_line
What are your thoughts? Have you seen anyone try this before? 
text
What are your thoughts? Have you seen anyone try this before? 
new_line
new_line
Curious to see if anyone has tried this before and know if this would have an impact on the correlation with human-annotated evaluations.
text
Curious to see if anyone has tried this before and know if this would have an impact on the correlation with human-annotated evaluations.
So, I have this idea to (potentially) improve uncertainty quantification for LLM hallucination detection.

The premise is that not all output tokens of a generated response share the same importance. Hallucinations are more dangerous in the form of a noun, date, number, etc.

The idea is to have a "token selection" layer that filters the output token probabilities sequence. Then, we use only the probabilities of the relevant tokens to calculate uncertainty quantification metrics.

The big question is how we know which tokens are the relevant ones. 🤔

My idea is to use the output sequence (decoded one) and use an NLP model (it doesn't need to be a fancy one) to do entity recognition and part-of-speech tagging to the output sequence and then do uncertainty quantification only on the entities that we have set as relevant (nouns, dates, numbers, etc).

What are your thoughts? Have you seen anyone try this before? 

Curious to see if anyone has tried this before and know if this would have an impact on the correlation with human-annotated evaluations.
❤️
gsarti osanseviero cnmoro clem santiviquez
629a173153a72d997d3f57d0
https://cdn-avatars.huggingface.co/v1/production/uploads/1657144463525-629a173153a72d997d3f57d0.jpeg
Santiago Viquez
santiviquez
user
5e7749883d77a72421292d07
https://cdn-avatars.huggingface.co/v1/production/uploads/5e7749883d77a72421292d07/M4AmBReZk_otxCIG3o0bL.jpeg
Gabriele Sarti
gsarti
user