To demonstrate that it was possible, I performed a "trapezoid" gradient merge of a Llama 3 8B model onto Llama 3.1 8B Instruct, favoring the L3.1 model at the ends in order to preserve coherence and limiting the influence of the L3 model to at most 0.1 weight. Tested to 16k context length.
text
To demonstrate that it was possible, I performed a "trapezoid" gradient merge of a Llama 3 8B model onto Llama 3.1 8B Instruct, favoring the L3.1 model at the ends in order to preserve coherence and limiting the influence of the L3 model to at most 0.1 weight. Tested to 16k context length.
new_line
https://huggingface.co/grimjim/Llama-Nephilim-Metamorphosis-v2-8B
grimjim/Llama-Nephilim-Metamorphosis-v2-8B
model
resource
https://huggingface.co/grimjim/Llama-Nephilim-Metamorphosis-v2-8B
To demonstrate that it was possible, I performed a "trapezoid" gradient merge of a Llama 3 8B model onto Llama 3.1 8B Instruct, favoring the L3.1 model at the ends in order to preserve coherence and limiting the influence of the L3 model to at most 0.1 weight. Tested to 16k context length.
https://huggingface.co/grimjim/Llama-Nephilim-Metamorphosis-v2-8B
😎
John6666 TuringsSolutions