Okay this is insane... WebGPU-accelerated semantic video tracking, powered by DINOv3 and Transformers.js! 🤯
text
Okay this is insane... WebGPU-accelerated semantic video tracking, powered by DINOv3 and Transformers.js! 🤯
new_line
Demo (+ source code): 
text
Demo (+ source code): 
https://huggingface.co/spaces/webml-community/DINOv3-video-tracking
webml-community/DINOv3-video-tracking
space
resource
https://huggingface.co/spaces/webml-community/DINOv3-video-tracking
text
new_line
new_line
This will revolutionize AI-powered video editors... which can now run 100% locally in your browser, no server inference required (costs $0)! 😍
text
This will revolutionize AI-powered video editors... which can now run 100% locally in your browser, no server inference required (costs $0)! 😍
new_line
new_line
How does it work? 🤔
text
How does it work? 🤔
new_line
1️⃣ Generate and cache image features for each frame
text
1️⃣ Generate and cache image features for each frame
new_line
2️⃣ Create a list of embeddings for selected patch(es)
text
2️⃣ Create a list of embeddings for selected patch(es)
new_line
3️⃣ Compute cosine similarity between each patch and the selected patch(es)
text
3️⃣ Compute cosine similarity between each patch and the selected patch(es)
new_line
4️⃣ Highlight those whose score is above some threshold
text
4️⃣ Highlight those whose score is above some threshold
new_line
new_line
... et voilà! 🥳
text
... et voilà! 🥳
new_line
new_line
You can also make selections across frames to improve temporal consistency! This is super useful if the object changes its appearance slightly throughout the video.
text
You can also make selections across frames to improve temporal consistency! This is super useful if the object changes its appearance slightly throughout the video.
new_line
new_line
Excited to see what the community builds with it!
text
Excited to see what the community builds with it!
Okay this is insane... WebGPU-accelerated semantic video tracking, powered by DINOv3 and Transformers.js! 🤯
Demo (+ source code): https://huggingface.co/spaces/webml-community/DINOv3-video-tracking 

This will revolutionize AI-powered video editors... which can now run 100% locally in your browser, no server inference required (costs $0)! 😍

How does it work? 🤔
1️⃣ Generate and cache image features for each frame
2️⃣ Create a list of embeddings for selected patch(es)
3️⃣ Compute cosine similarity between each patch and the selected patch(es)
4️⃣ Highlight those whose score is above some threshold

... et voilà! 🥳

You can also make selections across frames to improve temporal consistency! This is super useful if the object changes its appearance slightly throughout the video.

Excited to see what the community builds with it!
🔥
John6666 Tak2Hu g-ronimo PANZERBERO K-ai-Innovations manchere carstenli subinc fffiloni SisterHell nenadg Peramanathan MidnightDarling smvrnn wolfofbackstreet eyesss 4qrab justinchuby SharpAI GordonM rajofearth gnovelli pavel-ai carvierdotdev Adrianweb frablock condr-at
🚀
K-ai-Innovations Norod78 Peramanathan
👍
xynic Javedalam PrinceZaman
❤️
Croc-Prog-HF carvierdotdev
6463395539359568c63b3288
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/9E9qcp9qZ0x5Vnjr4JUJe.png
Sam Savage 
Samthedataman
user
69458eb85f7b0d886e1061a0
/avatars/ed888051525d8f2e7722b959d3ef35c1.svg
Darion Henry
Deeronx2
user
63830983b88af0bb8e5e4a69
https://cdn-avatars.huggingface.co/v1/production/uploads/63830983b88af0bb8e5e4a69/Wn-yzVZEKgz-AhDLwvFaf.jpeg
Marcos isidio da silva junior
UltraMarkoRJ
user