The TRL v0.13 release is 🔥! My highlight are the new process reward trainer to train models similar to o1 and tool call support:
text
The TRL v0.13 release is 🔥! My highlight are the new process reward trainer to train models similar to o1 and tool call support:
new_line
new_line
🧠 Process reward trainer: Enables training of Process-supervised Reward Models (PRMs), which reward the quality of intermediate steps, promoting structured reasoning. Perfect for tasks like stepwise reasoning.
text
🧠 Process reward trainer: Enables training of Process-supervised Reward Models (PRMs), which reward the quality of intermediate steps, promoting structured reasoning. Perfect for tasks like stepwise reasoning.
new_line
new_line
🔀 Model merging: A new callback leverages mergekit to merge models during training, improving performance by blending reference and policy models - optionally pushing merged models to the Hugging Face Hub.
text
🔀 Model merging: A new callback leverages mergekit to merge models during training, improving performance by blending reference and policy models - optionally pushing merged models to the Hugging Face Hub.
new_line
new_line
🛠️ Tool call support: TRL preprocessing now supports tool integration, laying the groundwork for agent fine-tuning with examples like dynamic temperature fetching in prompts.
text
🛠️ Tool call support: TRL preprocessing now supports tool integration, laying the groundwork for agent fine-tuning with examples like dynamic temperature fetching in prompts.
new_line
new_line
⚖️ Mixture of judges: The new AllTrueJudge combines decisions from multiple binary judges for more nuanced evaluation.
text
⚖️ Mixture of judges: The new AllTrueJudge combines decisions from multiple binary judges for more nuanced evaluation.
new_line
new_line
Read the release notes and other resources here 👇
text
Read the release notes and other resources here 👇
new_line
Release: 
text
Release: 
https://github.com/huggingface/trl/releases/tag/v0.13.0
https://github.com/huggingface/trl/releases/tag/v0.13.0
link
new_line
Mergekit: 
text
Mergekit: 
https://github.com/arcee-ai/mergekit
https://github.com/arcee-ai/mergekit
link
new_line
Mixture of judges paper: 
text
Mixture of judges paper: 
The Perfect Blend: Redefining RLHF with Mixture of Judges (2409.20370)
https://huggingface.co/papers/2409.20370
2409.20370
paper
resource
https://huggingface.co/papers/2409.20370
The TRL v0.13 release is 🔥! My highlight are the new process reward trainer to train models similar to o1 and tool call support:

🧠 Process reward trainer: Enables training of Process-supervised Reward Models (PRMs), which reward the quality of intermediate steps, promoting structured reasoning. Perfect for tasks like stepwise reasoning.

🔀 Model merging: A new callback leverages mergekit to merge models during training, improving performance by blending reference and policy models - optionally pushing merged models to the Hugging Face Hub.

🛠️ Tool call support: TRL preprocessing now supports tool integration, laying the groundwork for agent fine-tuning with examples like dynamic temperature fetching in prompts.

⚖️ Mixture of judges: The new AllTrueJudge combines decisions from multiple binary judges for more nuanced evaluation.

Read the release notes and other resources here 👇
Release: https://github.com/huggingface/trl/releases/tag/v0.13.0
Mergekit: https://github.com/arcee-ai/mergekit
Mixture of judges paper: https://huggingface.co/papers/2409.20370
🔥
prithivMLmods John6666 timrbula TornButter Daemontatox Yukkkop
❤️
MrOvkill