Haystack can now see 👀
text
Haystack can now see 👀
new_line
new_line
The latest release of the Haystack OSS LLM framework adds a long-requested feature: image support!
text
The latest release of the Haystack OSS LLM framework adds a long-requested feature: image support!
new_line
new_line
📓 Notebooks below 
text
📓 Notebooks below 
new_line
new_line
This isn't just about passing images to an LLM. We built several features to enable practical multimodal use cases.
text
This isn't just about passing images to an LLM. We built several features to enable practical multimodal use cases.
new_line
new_line
What's new?
text
What's new?
new_line
🧠 Support for multiple LLM providers: OpenAI, Amazon Bedrock, Google Gemini, Mistral, NVIDIA, OpenRouter, Ollama and more (support for Hugging Face API coming 🔜)
text
🧠 Support for multiple LLM providers: OpenAI, Amazon Bedrock, Google Gemini, Mistral, NVIDIA, OpenRouter, Ollama and more (support for Hugging Face API coming 🔜)
new_line
🎛️ Prompt template language to handle structured inputs, including images
text
🎛️ Prompt template language to handle structured inputs, including images
new_line
📄 PDF and image converters
text
📄 PDF and image converters
new_line
🔍 Image embedders using CLIP-like models
text
🔍 Image embedders using CLIP-like models
new_line
🧾 LLM-based extractor to pull text from images
text
🧾 LLM-based extractor to pull text from images
new_line
🧩 Components to build multimodal RAG pipelines and Agents
text
🧩 Components to build multimodal RAG pipelines and Agents
new_line
new_line
new_line
I had the chance of leading this effort with 
text
I had the chance of leading this effort with 
@sjrhuschlee
mention
sjrhuschlee
 (great collab).
text
 (great collab).
new_line
new_line
📓 Below you can find two notebooks to explore the new features:
text
📓 Below you can find two notebooks to explore the new features:
new_line
󠁯•󠁏󠁏 Introduction to Multimodal Text Generation 
text
󠁯•󠁏󠁏 Introduction to Multimodal Text Generation 
https://haystack.deepset.ai/cookbook/multimodal_intro
https://haystack.deepset.ai/cookbook/multimodal_intro
link
new_line
󠁯•󠁏󠁏 Creating Vision+Text RAG Pipelines 
text
󠁯•󠁏󠁏 Creating Vision+Text RAG Pipelines 
https://haystack.deepset.ai/tutorials/46_multimodal_rag
https://haystack.deepset.ai/tutorials/46_multimodal_rag
link
new_line
new_line
(🖼️ image by 
text
(🖼️ image by 
@bilgeyucel
mention
bilgeyucel
 )
text
 )
Haystack can now see 👀

The latest release of the Haystack OSS LLM framework adds a long-requested feature: image support!

📓 Notebooks below 

This isn't just about passing images to an LLM. We built several features to enable practical multimodal use cases.

What's new?
🧠 Support for multiple LLM providers: OpenAI, Amazon Bedrock, Google Gemini, Mistral, NVIDIA, OpenRouter, Ollama and more (support for Hugging Face API coming 🔜)
🎛️ Prompt template language to handle structured inputs, including images
📄 PDF and image converters
🔍 Image embedders using CLIP-like models
🧾 LLM-based extractor to pull text from images
🧩 Components to build multimodal RAG pipelines and Agents


I had the chance of leading this effort with @sjrhuschlee (great collab).

📓 Below you can find two notebooks to explore the new features:
󠁯•󠁏󠁏 Introduction to Multimodal Text Generation https://haystack.deepset.ai/cookbook/multimodal_intro
󠁯•󠁏󠁏 Creating Vision+Text RAG Pipelines https://haystack.deepset.ai/tutorials/46_multimodal_rag

(🖼️ image by @bilgeyucel )
❤️
sjrhuschlee sweatSmile baiall bilgeyucel
👍
John6666 whitebill sweatSmile