🦙🦙 LLaMA Duo project update 
text
🦙🦙 LLaMA Duo project update 
new_line
new_line
Last time, I gave a brief introduction about LLaMA Duo project with 
text
Last time, I gave a brief introduction about LLaMA Duo project with 
@sayakpaul
mention
sayakpaul
 . It is a simple toolset to aligning sLLM with service LLM with coverage dataset 👉🏻 (
text
 . It is a simple toolset to aligning sLLM with service LLM with coverage dataset 👉🏻 (
https://huggingface.co/posts/chansung/708646454991943
https://huggingface.co/posts/chansung/708646454991943
link
).
text
).
new_line
- coverage dataset is what we believe to be the most important/desired (instruction, response) pairs. In system thinking, each instruction could be an analogy of a function from traditional programming. We make unit tests and measure the coverage % for all the features/functions. Similarly, we need to ensure if our fine-tuned model could handle what % of given instructions from coverage dataset satisfactory (hence coverage dataset).
text
- coverage dataset is what we believe to be the most important/desired (instruction, response) pairs. In system thinking, each instruction could be an analogy of a function from traditional programming. We make unit tests and measure the coverage % for all the features/functions. Similarly, we need to ensure if our fine-tuned model could handle what % of given instructions from coverage dataset satisfactory (hence coverage dataset).
new_line
new_line
We have tested it with "Coding" category of data from 
text
We have tested it with "Coding" category of data from 
https://huggingface.co/datasets/HuggingFaceH4/no_robots
HuggingFaceH4/no_robots
dataset
resource
https://huggingface.co/datasets/HuggingFaceH4/no_robots
 dataset. It has about 300 SFT training data points under Coding category. After fine-tuning Gemma 7B model on that, the result was very poor. LLaMA Duo's evaluation tool gave < 20% of metrics in similarity and preciseness on the test split. 
text
 dataset. It has about 300 SFT training data points under Coding category. After fine-tuning Gemma 7B model on that, the result was very poor. LLaMA Duo's evaluation tool gave < 20% of metrics in similarity and preciseness on the test split. 
new_line
new_line
So, we used LLaMA Duo's synthetic data generation tool to generate 60k data points that looks similar to the original dataset. We first created ~10k synthetic data points, then created 50k more based on the synthetic dataset itself. 
text
So, we used LLaMA Duo's synthetic data generation tool to generate 60k data points that looks similar to the original dataset. We first created ~10k synthetic data points, then created 50k more based on the synthetic dataset itself. 
new_line
new_line
After fine-tuning Gemma 7B on the 60k synthetic dataset, the evaluation result went up to 80~90% high. Also, when testing out the model in UI, it tends to give good responses. 
text
After fine-tuning Gemma 7B on the 60k synthetic dataset, the evaluation result went up to 80~90% high. Also, when testing out the model in UI, it tends to give good responses. 
new_line
new_line
It is a good showcase to transition from service LLM to sLLM or having a backup sLLM for service LLM failure scenarios. I am going to expand this experiments on all categories of no_robots dataset. It will roughly generate > 100k data points. 
text
It is a good showcase to transition from service LLM to sLLM or having a backup sLLM for service LLM failure scenarios. I am going to expand this experiments on all categories of no_robots dataset. It will roughly generate > 100k data points. 
new_line
new_line
Here are some links:
text
Here are some links:
new_line
- LLaMA Duo project repo: 
text
- LLaMA Duo project repo: 
https://github.com/deep-diver/llamaduo
https://github.com/deep-diver/llamaduo
link
new_line
- 60k Coding synthetic dataset: 
text
- 60k Coding synthetic dataset: 
https://huggingface.co/datasets/chansung/merged_ds_coding
chansung/merged_ds_coding
dataset
resource
https://huggingface.co/datasets/chansung/merged_ds_coding
new_line
- Fine-tuned Gemma 7B model: 
text
- Fine-tuned Gemma 7B model: 
https://huggingface.co/chansung/coding_llamaduo_60k_v0.2
chansung/coding_llamaduo_60k_v0.2
model
resource
https://huggingface.co/chansung/coding_llamaduo_60k_v0.2
🦙🦙 LLaMA Duo project update 

Last time, I gave a brief introduction about LLaMA Duo project with @sayakpaul . It is a simple toolset to aligning sLLM with service LLM with coverage dataset 👉🏻 (https://huggingface.co/posts/chansung/708646454991943).
- coverage dataset is what we believe to be the most important/desired (instruction, response) pairs. In system thinking, each instruction could be an analogy of a function from traditional programming. We make unit tests and measure the coverage % for all the features/functions. Similarly, we need to ensure if our fine-tuned model could handle what % of given instructions from coverage dataset satisfactory (hence coverage dataset).

We have tested it with "Coding" category of data from https://huggingface.co/datasets/HuggingFaceH4/no_robots dataset. It has about 300 SFT training data points under Coding category. After fine-tuning Gemma 7B model on that, the result was very poor. LLaMA Duo's evaluation tool gave < 20% of metrics in similarity and preciseness on the test split. 

So, we used LLaMA Duo's synthetic data generation tool to generate 60k data points that looks similar to the original dataset. We first created ~10k synthetic data points, then created 50k more based on the synthetic dataset itself. 

After fine-tuning Gemma 7B on the 60k synthetic dataset, the evaluation result went up to 80~90% high. Also, when testing out the model in UI, it tends to give good responses. 

It is a good showcase to transition from service LLM to sLLM or having a backup sLLM for service LLM failure scenarios. I am going to expand this experiments on all categories of no_robots dataset. It will roughly generate > 100k data points. 

Here are some links:
- LLaMA Duo project repo: https://github.com/deep-diver/llamaduo
- 60k Coding synthetic dataset: https://huggingface.co/datasets/chansung/merged_ds_coding
- Fine-tuned Gemma 7B model: https://huggingface.co/chansung/coding_llamaduo_60k_v0.2
👍
Tonic peterschmidt85 chansung adamm-hf juyongjiang
🔥
hllj chansung adamm-hf Hiraishin juyongjiang
➕
cyb3rh4v0k juyongjiang
🚀
cyb3rh4v0k juyongjiang