Hey !
text
Hey !
new_line
text
new_line
I'm working on a small-scale multi-drone control system and I'm looking for an open-source VLM that can run in real time on a Jetson Orin. If anyone knows a model or is personally interested in this kind of edge robotics problem, I'd love pointers.
text
I'm working on a small-scale multi-drone control system and I'm looking for an open-source VLM that can run in real time on a Jetson Orin. If anyone knows a model or is personally interested in this kind of edge robotics problem, I'd love pointers.
new_line
new_line
What I'm trying to solve : 
text
What I'm trying to solve : 
new_line
new_line
I have 4 simultaneous video streams coming from four drones (grayscale, 320×320 ). I can feed the model either:
text
I have 4 simultaneous video streams coming from four drones (grayscale, 320×320 ). I can feed the model either:
new_line
• a 2×2 mosaic frame, or
text
• a 2×2 mosaic frame, or
new_line
• 4 separate frames as a batch.
text
• 4 separate frames as a batch.
new_line
Along with this, I provide a short text instruction describing the mission state.
text
Along with this, I provide a short text instruction describing the mission state.
new_line
new_line
What I need from the model :
text
What I need from the model :
new_line
new_line
A single structured JSON command representing the next action for the swarm controller. Something like (not decided yet):
text
A single structured JSON command representing the next action for the swarm controller. Something like (not decided yet):
new_line
new_line
{
  "action": "move_forward",
  "confidence": 0.87,
  "reason": "front corridor detected, no obstacles in drone_2 and drone_4 views"
}
```
{
  "action": "move_forward",
  "confidence": 0.87,
  "reason": "front corridor detected, no obstacles in drone_2 and drone_4 views"
}
```
code_fence
new_line
new_line
So I need a VLM that can:
text
So I need a VLM that can:
new_line
• handle multi-image or mosaic image input
text
• handle multi-image or mosaic image input
new_line
• run efficiently on a Jetson Orin (ideally INT4/INT8 friendly, TensorRT-compatible)
text
• run efficiently on a Jetson Orin (ideally INT4/INT8 friendly, TensorRT-compatible)
new_line
• generate stable JSON outputs based on visual + textual context
text
• generate stable JSON outputs based on visual + textual context
new_line
new_line
I would really appreciate suggestions, or even just thoughts on what architectures make sense here. 
text
I would really appreciate suggestions, or even just thoughts on what architectures make sense here. 
new_line
new_line
Models like 
text
Models like 
https://huggingface.co/openbmb/MiniCPM-V-4_5
openbmb/MiniCPM-V-4_5
model
resource
https://huggingface.co/openbmb/MiniCPM-V-4_5
, 
text
, 
https://huggingface.co/dustnehowl/nanoVLM
dustnehowl/nanoVLM
model
resource
https://huggingface.co/dustnehowl/nanoVLM
, and 
text
, and 
https://huggingface.co/Qwen/Qwen3-VL-8B-Instruct
Qwen/Qwen3-VL-8B-Instruct
model
resource
https://huggingface.co/Qwen/Qwen3-VL-8B-Instruct
 look promising, but I'm still exploring what’s actually viable on-device.
text
 look promising, but I'm still exploring what’s actually viable on-device.
new_line
new_line
Happy to share benchmarks or test anything people want to throw at this problem. The multi-drone video + action JSON setup is niche but potentially useful to others building edge-deployed agents.
text
Happy to share benchmarks or test anything people want to throw at this problem. The multi-drone video + action JSON setup is niche but potentially useful to others building edge-deployed agents.
Hey !
 
I'm working on a small-scale multi-drone control system and I'm looking for an open-source VLM that can run in real time on a Jetson Orin. If anyone knows a model or is personally interested in this kind of edge robotics problem, I'd love pointers.

What I'm trying to solve : 

I have 4 simultaneous video streams coming from four drones (grayscale, 320×320 ). I can feed the model either:
• a 2×2 mosaic frame, or
• 4 separate frames as a batch.
Along with this, I provide a short text instruction describing the mission state.

What I need from the model :

A single structured JSON command representing the next action for the swarm controller. Something like (not decided yet):

```
{
  "action": "move_forward",
  "confidence": 0.87,
  "reason": "front corridor detected, no obstacles in drone_2 and drone_4 views"
}
```

So I need a VLM that can:
• handle multi-image or mosaic image input
• run efficiently on a Jetson Orin (ideally INT4/INT8 friendly, TensorRT-compatible)
• generate stable JSON outputs based on visual + textual context

I would really appreciate suggestions, or even just thoughts on what architectures make sense here. 

Models like https://huggingface.co/openbmb/MiniCPM-V-4_5, https://huggingface.co/dustnehowl/nanoVLM, and https://huggingface.co/Qwen/Qwen3-VL-8B-Instruct look promising, but I'm still exploring what’s actually viable on-device.

Happy to share benchmarks or test anything people want to throw at this problem. The multi-drone video + action JSON setup is niche but potentially useful to others building edge-deployed agents.
👀
John6666
6640bbd0220cfa8cbfdce080
https://cdn-avatars.huggingface.co/v1/production/uploads/6640bbd0220cfa8cbfdce080/wiAHUu5ewawyipNs0YFBR.png
John Smith
John6666
user
6717841e71416a26cbc6b704
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/Gg11psZiZ4VziEKoFm6LG.png
Quentin
Qvelard
user