Researchers from Auburn University and the University of Alberta have explored the limitations of Vision Language Models (VLMs) in their recently published paper titled "Vision language models are blind." (
text
Researchers from Auburn University and the University of Alberta have explored the limitations of Vision Language Models (VLMs) in their recently published paper titled "Vision language models are blind." (
Vision language models are blind (2407.06581)
https://huggingface.co/papers/2407.06581
2407.06581
paper
resource
https://huggingface.co/papers/2407.06581
)
text
)
new_line
new_line
Key Findings:🔍
text
Key Findings:🔍
new_line
VLMs, including GPT-4o, Gemini-1.5 Pro, Claude-3 Sonnet, and Claude-3.5 Sonnet, struggle with basic visual tasks.
text
VLMs, including GPT-4o, Gemini-1.5 Pro, Claude-3 Sonnet, and Claude-3.5 Sonnet, struggle with basic visual tasks.
new_line
Tasks such as identifying where lines intersect or counting basic shapes are challenging for these models.
text
Tasks such as identifying where lines intersect or counting basic shapes are challenging for these models.
new_line
The authors noted, "The shockingly poor performance of four state-of-the-art VLMs suggests their vision is, at best, like of a person with myopia seeing fine details as blurry, and at worst, like an intelligent person that is blind making educated guesses"​(Vision Language Models Are Blind; 2024)​.
text
The authors noted, "The shockingly poor performance of four state-of-the-art VLMs suggests their vision is, at best, like of a person with myopia seeing fine details as blurry, and at worst, like an intelligent person that is blind making educated guesses"​(Vision Language Models Are Blind; 2024)​.
new_line
new_line
Human-like Myopia? 👓
text
Human-like Myopia? 👓
new_line
VLMs may have a blind spot similar to human myopia.
text
VLMs may have a blind spot similar to human myopia.
new_line
This limitation makes it difficult for VLMs to perceive details.
text
This limitation makes it difficult for VLMs to perceive details.
new_line
Suggests a potential parallel between human and machine vision limitations.
text
Suggests a potential parallel between human and machine vision limitations.
new_line
new_line
Technical Details: 🔧
text
Technical Details: 🔧
new_line
The researchers created a new benchmark called BlindTest.
text
The researchers created a new benchmark called BlindTest.
new_line
BlindTest consists of simple visual tasks to evaluate VLMs low-level vision capabilities.
text
BlindTest consists of simple visual tasks to evaluate VLMs low-level vision capabilities.
new_line
Four VLMs were assessed using BlindTest.
text
Four VLMs were assessed using BlindTest.
new_line
Many shortcomings were revealed in the models ability to process basic visual information.
text
Many shortcomings were revealed in the models ability to process basic visual information.
new_line
new_line
Learn More: 🖼️
text
Learn More: 🖼️
new_line
For a deeper dive into this research, check out the project page: 
text
For a deeper dive into this research, check out the project page: 
https://vlmsareblind.github.io/
https://vlmsareblind.github.io/
link
new_line
Researchers from Auburn University and the University of Alberta have explored the limitations of Vision Language Models (VLMs) in their recently published paper titled "Vision language models are blind." (https://huggingface.co/papers/2407.06581)

Key Findings:🔍
VLMs, including GPT-4o, Gemini-1.5 Pro, Claude-3 Sonnet, and Claude-3.5 Sonnet, struggle with basic visual tasks.
Tasks such as identifying where lines intersect or counting basic shapes are challenging for these models.
The authors noted, "The shockingly poor performance of four state-of-the-art VLMs suggests their vision is, at best, like of a person with myopia seeing fine details as blurry, and at worst, like an intelligent person that is blind making educated guesses"​(Vision Language Models Are Blind; 2024)​.

Human-like Myopia? 👓
VLMs may have a blind spot similar to human myopia.
This limitation makes it difficult for VLMs to perceive details.
Suggests a potential parallel between human and machine vision limitations.

Technical Details: 🔧
The researchers created a new benchmark called BlindTest.
BlindTest consists of simple visual tasks to evaluate VLMs low-level vision capabilities.
Four VLMs were assessed using BlindTest.
Many shortcomings were revealed in the models ability to process basic visual information.

Learn More: 🖼️
For a deeper dive into this research, check out the project page: https://vlmsareblind.github.io/

🔥
loganbolton