🌊 Released #LaVague, fullly open-source AI pipeline to turn natural language into browser actions!
text
🌊 Released #LaVague, fullly open-source AI pipeline to turn natural language into browser actions!
new_line
new_line
In less than 150 lines of code (RAG with local embedding + Zephyr-7b-Gemma locally or Mixtral on HF Inference API), it generates #Selenium code from user query. In this GIF you can see it follow user instructions to command a browser to browse HF website! 
text
In less than 150 lines of code (RAG with local embedding + Zephyr-7b-Gemma locally or Mixtral on HF Inference API), it generates #Selenium code from user query. In this GIF you can see it follow user instructions to command a browser to browse HF website! 
new_line
new_line
Try it on Colab: colab.research.google.com/github/dhuynh95/LaVague/blob/main/LaVague.ipynb
text
Try it on Colab: colab.research.google.com/github/dhuynh95/LaVague/blob/main/LaVague.ipynb
new_line
GitHub: github.com/dhuynh95/LaVague
text
GitHub: github.com/dhuynh95/LaVague
new_line
new_line
Pretty exciting how it becomes possible to create an AI assistant that could perform actions for us, such as logging on gov accounts, fill forms, or pull personal information!
text
Pretty exciting how it becomes possible to create an AI assistant that could perform actions for us, such as logging on gov accounts, fill forms, or pull personal information!
new_line
new_line
It was quite fun to hack in the weekend using open-source tools, from 
text
It was quite fun to hack in the weekend using open-source tools, from 
@huggingface
mention
huggingface
 local embedding with transformers for local inference or HF Inference API, to RAG with @llama_index, through 
text
 local embedding with transformers for local inference or HF Inference API, to RAG with @llama_index, through 
@MistralAI
mention
MistralAI
 Mixtral model! 
text
 Mixtral model! 
new_line
new_line
Some challenges: to make it run on Colab for the #GPU Poors, I first resorted to 
text
Some challenges: to make it run on Colab for the #GPU Poors, I first resorted to 
@huggingface
mention
huggingface
 Inference API with Mixtral as it was the only model good enough (gemma-7b did not make it and refused to produce code). But after some experimentations, I managed to make it work a local Zephyr-7b-Gemma so that people could run this assistant fully locally! 
text
 Inference API with Mixtral as it was the only model good enough (gemma-7b did not make it and refused to produce code). But after some experimentations, I managed to make it work a local Zephyr-7b-Gemma so that people could run this assistant fully locally! 
new_line
new_line
Because I used an off-the-shelf model, I had to improve performance with few-shot learning and Chain Of Thought, which managed to generate appropriate code! 
text
Because I used an off-the-shelf model, I had to improve performance with few-shot learning and Chain Of Thought, which managed to generate appropriate code! 
new_line
new_line
I hope this project will herald a new dawn where transparent, private and local AI assistants help automate menial but critical tasks, such as helping fill taxes, book accomodation, or research information for us.
text
I hope this project will herald a new dawn where transparent, private and local AI assistants help automate menial but critical tasks, such as helping fill taxes, book accomodation, or research information for us.
new_line
🌊 Released #LaVague, fullly open-source AI pipeline to turn natural language into browser actions!

In less than 150 lines of code (RAG with local embedding + Zephyr-7b-Gemma locally or Mixtral on HF Inference API), it generates #Selenium code from user query. In this GIF you can see it follow user instructions to command a browser to browse HF website! 

Try it on Colab: colab.research.google.com/github/dhuynh95/LaVague/blob/main/LaVague.ipynb
GitHub: github.com/dhuynh95/LaVague

Pretty exciting how it becomes possible to create an AI assistant that could perform actions for us, such as logging on gov accounts, fill forms, or pull personal information!

It was quite fun to hack in the weekend using open-source tools, from @huggingface local embedding with transformers for local inference or HF Inference API, to RAG with @llama_index, through @MistralAI Mixtral model! 

Some challenges: to make it run on Colab for the #GPU Poors, I first resorted to @huggingface Inference API with Mixtral as it was the only model good enough (gemma-7b did not make it and refused to produce code). But after some experimentations, I managed to make it work a local Zephyr-7b-Gemma so that people could run this assistant fully locally! 

Because I used an off-the-shelf model, I had to improve performance with few-shot learning and Chain Of Thought, which managed to generate appropriate code! 

I hope this project will herald a new dawn where transparent, private and local AI assistants help automate menial but critical tasks, such as helping fill taxes, book accomodation, or research information for us.

👍
osanseviero kramp ejsellers vladbogo mexicanamerican 0xKrypt0r Dlbk qnguyen3 julien-c pranay-j victor diwank iamrobotbear samusenps bruceunx George-Blaze superLk timnon
❤️
osanseviero VictorSanh Guilherme34 Dlbk julien-c victor clem diwank bonsaielectric iamrobotbear samusenps z3ymd kgawron bruceunx Tavi31 superLk sbrandeis
🤯
fffiloni victor clem iamrobotbear Tavi31
657fbd30a575d54a1ea75442
https://cdn-avatars.huggingface.co/v1/production/uploads/657fbd30a575d54a1ea75442/X-PDAOj-OwNGbnZ2WHpfe.jpeg
bruce hu
bruceunx
user
5dd96eb166059660ed1ee413
https://cdn-avatars.huggingface.co/v1/production/uploads/5dd96eb166059660ed1ee413/NQtzmrDdbG0H8qkZvRyGk.jpeg
Julien Chaumond
julien-c
user
62f4ac43567dbf9a39f75474
https://cdn-avatars.huggingface.co/v1/production/uploads/1661497922734-62f4ac43567dbf9a39f75474.jpeg
Daniel Huynh
dhuynh95
user
63cb49e71b705cc951e8ab56
/avatars/9af57ddd73fc94d02499d54478d0cfeb.svg
Brian King
iamrobotbear
user
65f1287473f2e1e29157d80b
/avatars/6556f74fc6c3d314cd05ded359a5fd5f.svg
Shiv Kumar
Shivkumar27
user