🔍 Today's pick in Interpretability & Analysis of LMs: Gradient-Based Language Model Red Teaming  by N. Wichers, C. Denison and 
text
🔍 Today's pick in Interpretability & Analysis of LMs: Gradient-Based Language Model Red Teaming  by N. Wichers, C. Denison and 
@beirami
mention
beirami
new_line
new_line
This work proposes Gradient-Based Red Teaming (GBRT), a red teaming method for automatically generating diverse prompts inducing an LM to output unsafe responses. 
text
This work proposes Gradient-Based Red Teaming (GBRT), a red teaming method for automatically generating diverse prompts inducing an LM to output unsafe responses. 
new_line
new_line
In practice, prompts are learned by scoring LM responses with a safety-trained probing classifier, and back-propagating through frozen classifier and LM to update the prompt.
text
In practice, prompts are learned by scoring LM responses with a safety-trained probing classifier, and back-propagating through frozen classifier and LM to update the prompt.
new_line
new_line
Authors experiment with variants of GBRT aimed at inducing realistic prompts in an efficient way, and GBRT prompts are more likely to generate unsafe responses than those found by established RL-based red teaming methods. Moreover, these attacks are shown to succeed even when the LM has been fine-tuned to produce safer outputs. 
text
Authors experiment with variants of GBRT aimed at inducing realistic prompts in an efficient way, and GBRT prompts are more likely to generate unsafe responses than those found by established RL-based red teaming methods. Moreover, these attacks are shown to succeed even when the LM has been fine-tuned to produce safer outputs. 
new_line
new_line
📄 Paper: 
text
📄 Paper: 
In-Context Language Learning: Architectures and Algorithms (2401.12973)
https://huggingface.co/papers/2401.12973
2401.12973
paper
resource
https://huggingface.co/papers/2401.12973
new_line
💻 Code: 
text
💻 Code: 
https://github.com/google-research/google-research/tree/master/gbrt
https://github.com/google-research/google-research/tree/master/gbrt
link
🔍 Today's pick in Interpretability & Analysis of LMs: Gradient-Based Language Model Red Teaming  by N. Wichers, C. Denison and @beirami

This work proposes Gradient-Based Red Teaming (GBRT), a red teaming method for automatically generating diverse prompts inducing an LM to output unsafe responses. 

In practice, prompts are learned by scoring LM responses with a safety-trained probing classifier, and back-propagating through frozen classifier and LM to update the prompt.

Authors experiment with variants of GBRT aimed at inducing realistic prompts in an efficient way, and GBRT prompts are more likely to generate unsafe responses than those found by established RL-based red teaming methods. Moreover, these attacks are shown to succeed even when the LM has been fine-tuned to produce safer outputs. 

📄 Paper: https://huggingface.co/papers/2401.12973
💻 Code: https://github.com/google-research/google-research/tree/master/gbrt
❤️
clem osanseviero adamm-hf beirami 9voltfan2009
👍
neovalle noobmldude