How talkative is your chatbot about your internal data? 😬
text
How talkative is your chatbot about your internal data? 😬
new_line
new_line
As more chatbots get deployed in production, with access to internal databases, we need to make sure they don't leak private information to anyone interacting with them.
text
As more chatbots get deployed in production, with access to internal databases, we need to make sure they don't leak private information to anyone interacting with them.
new_line
new_line
The Lighthouz AI team therefore introduced the Chatbot Guardrails Arena to stress test models and see how well guarded your private information is.
text
The Lighthouz AI team therefore introduced the Chatbot Guardrails Arena to stress test models and see how well guarded your private information is.
new_line
Anyone can try to make models reveal information they should not share 😈
text
Anyone can try to make models reveal information they should not share 😈
new_line
(which is quite fun to do for the strongest models)!
text
(which is quite fun to do for the strongest models)!
new_line
new_line
The votes will then be gathered to create an Elo ranking of the safest models with respect to PII. 
text
The votes will then be gathered to create an Elo ranking of the safest models with respect to PII. 
new_line
new_line
In the future, with the support of the community, this arena could inform safety choices that company make, when choosing models and guardrails on their resistance to adversarial attacks.
text
In the future, with the support of the community, this arena could inform safety choices that company make, when choosing models and guardrails on their resistance to adversarial attacks.
new_line
It's also a good way to easily demonstrate the limitations of current systems!
text
It's also a good way to easily demonstrate the limitations of current systems!
new_line
new_line
Check out the arena: 
text
Check out the arena: 
https://huggingface.co/spaces/lighthouzai/guardrails-arena
lighthouzai/guardrails-arena
space
resource
https://huggingface.co/spaces/lighthouzai/guardrails-arena
new_line
Learn more in the blog: 
text
Learn more in the blog: 
https://huggingface.co/blog/arena-lighthouz
https://huggingface.co/blog/arena-lighthouz
link
How talkative is your chatbot about your internal data? 😬

As more chatbots get deployed in production, with access to internal databases, we need to make sure they don't leak private information to anyone interacting with them.

The Lighthouz AI team therefore introduced the Chatbot Guardrails Arena to stress test models and see how well guarded your private information is.
Anyone can try to make models reveal information they should not share 😈
(which is quite fun to do for the strongest models)!

The votes will then be gathered to create an Elo ranking of the safest models with respect to PII. 

In the future, with the support of the community, this arena could inform safety choices that company make, when choosing models and guardrails on their resistance to adversarial attacks.
It's also a good way to easily demonstrate the limitations of current systems!

Check out the arena: https://huggingface.co/spaces/lighthouzai/guardrails-arena
Learn more in the blog: https://huggingface.co/blog/arena-lighthouz
🚀
osanseviero m-ric zelus82 julien-c