GPU Poor POV: Building a RAG which solves specific task.
text
GPU Poor POV: Building a RAG which solves specific task.
new_line
new_line
Everyone loves benchmarks.
text
Everyone loves benchmarks.
new_line
They are great because we have standarized approach,  competitive feeling. But if you are in specific area, trying  to implement some LLM/RAG use case, these benchmarks  cannot exactly reflect on the data that you have to deal with.
text
They are great because we have standarized approach,  competitive feeling. But if you are in specific area, trying  to implement some LLM/RAG use case, these benchmarks  cannot exactly reflect on the data that you have to deal with.
new_line
new_line
I built RAG system on bunch of niche procedures/regulation etc,  which can be finally deployed as an virtual assistant to minimize the effort in searching through a lot of documentations manually. 
text
I built RAG system on bunch of niche procedures/regulation etc,  which can be finally deployed as an virtual assistant to minimize the effort in searching through a lot of documentations manually. 
new_line
new_line
Tested a lot of different methods/models/pretrains, finetunes and whats interesting is that, final solution which was scored by human feedback is based on relatively low param models, with multitask ability 
text
Tested a lot of different methods/models/pretrains, finetunes and whats interesting is that, final solution which was scored by human feedback is based on relatively low param models, with multitask ability 
new_line
Something like:
text
Something like:
new_line
new_line
https://huggingface.co/BAAI/llm-embedder
BAAI/llm-embedder
model
resource
https://huggingface.co/BAAI/llm-embedder
new_line
new_line
LLMs help summarize the chunk version of knowledge base found, does not require the model with high number of params, because tradeoff between inference time and accuracy has to be made. Some lightweight models have ability to perform certain task based on instructions,  so eg. qwen 7b or mistral 7b (not moe one), realized a task really nicely. And what is more important is that in overall we are able to deploy a RAG system in smaller tasks, in specific area. They can be used by people who need it, give additive value and positive feedback, which IMO is what is all of the building process about. 
text
LLMs help summarize the chunk version of knowledge base found, does not require the model with high number of params, because tradeoff between inference time and accuracy has to be made. Some lightweight models have ability to perform certain task based on instructions,  so eg. qwen 7b or mistral 7b (not moe one), realized a task really nicely. And what is more important is that in overall we are able to deploy a RAG system in smaller tasks, in specific area. They can be used by people who need it, give additive value and positive feedback, which IMO is what is all of the building process about. 
new_line
new_line
Have a great day and think about problem which your models have to solve <3
text
Have a great day and think about problem which your models have to solve <3
GPU Poor POV: Building a RAG which solves specific task.

Everyone loves benchmarks.
They are great because we have standarized approach,  competitive feeling. But if you are in specific area, trying  to implement some LLM/RAG use case, these benchmarks  cannot exactly reflect on the data that you have to deal with.

I built RAG system on bunch of niche procedures/regulation etc,  which can be finally deployed as an virtual assistant to minimize the effort in searching through a lot of documentations manually. 

Tested a lot of different methods/models/pretrains, finetunes and whats interesting is that, final solution which was scored by human feedback is based on relatively low param models, with multitask ability 
Something like:

https://huggingface.co/BAAI/llm-embedder

LLMs help summarize the chunk version of knowledge base found, does not require the model with high number of params, because tradeoff between inference time and accuracy has to be made. Some lightweight models have ability to perform certain task based on instructions,  so eg. qwen 7b or mistral 7b (not moe one), realized a task really nicely. And what is more important is that in overall we are able to deploy a RAG system in smaller tasks, in specific area. They can be used by people who need it, give additive value and positive feedback, which IMO is what is all of the building process about. 

Have a great day and think about problem which your models have to solve <3
👍
100rab RASMUS victor merve osanseviero tomaarsen samusenps clem Citaman adamm-hf sethuiyer arpitingle Ji-Ha
❤️
merve samusenps akashicmarga clem adamm-hf TuringsSolutions sbrandeis
🤗
victor merve clem adamm-hf TuringsSolutions s3nh
6032802e1f993496bc14d9e3
https://cdn-avatars.huggingface.co/v1/production/uploads/6032802e1f993496bc14d9e3/w6hr-DEQot4VVkoyRIBiy.png
Omar Sanseviero
osanseviero
user
6527e89a8808d80ccff88b7a
https://cdn-avatars.huggingface.co/v1/production/uploads/6527e89a8808d80ccff88b7a/CuGNmF1Et8KMQ0mCd1NEJ.jpeg
Not Lain
not-lain
user