9 Multimodal Chain-of-Thought methods
text
9 Multimodal Chain-of-Thought methods
new_line
new_line
How Chain-of-Thought (CoT) prompting can unlock models' full potential across images, video, audio and more? Finding special multimodal CoT techniques is the answer.
text
How Chain-of-Thought (CoT) prompting can unlock models' full potential across images, video, audio and more? Finding special multimodal CoT techniques is the answer.
new_line
new_line
Here are 9 methods of Multimodal Chain-of-Thought (MCoT). Most of them are open-source:
text
Here are 9 methods of Multimodal Chain-of-Thought (MCoT). Most of them are open-source:
new_line
new_line
1. KAM-CoT -> 
text
1. KAM-CoT -> 
KAM-CoT: Knowledge Augmented Multimodal Chain-of-Thoughts Reasoning (2401.12863)
https://huggingface.co/papers/2401.12863
2401.12863
paper
resource
https://huggingface.co/papers/2401.12863
new_line
This lightweight framework combines CoT prompting with knowledge graphs (KGs) and achieves 93.87% accuracy
text
This lightweight framework combines CoT prompting with knowledge graphs (KGs) and achieves 93.87% accuracy
new_line
new_line
2. Multimodal Visualization-of-Thought (MVoT) -> 
text
2. Multimodal Visualization-of-Thought (MVoT) -> 
Imagine while Reasoning in Space: Multimodal Visualization-of-Thought (2501.07542)
https://huggingface.co/papers/2501.07542
2501.07542
paper
resource
https://huggingface.co/papers/2501.07542
new_line
Lets models generate visual reasoning traces, using a token discrepancy loss to improve visual quality
text
Lets models generate visual reasoning traces, using a token discrepancy loss to improve visual quality
new_line
new_line
3. Compositional CoT (CCoT) -> 
text
3. Compositional CoT (CCoT) -> 
Compositional Chain-of-Thought Prompting for Large Multimodal Models (2311.17076)
https://huggingface.co/papers/2311.17076
2311.17076
paper
resource
https://huggingface.co/papers/2311.17076
new_line
Uses scene graph (SG) representations generated by the LMM itself to improve performance on compositional and general multimodal benchmarks
text
Uses scene graph (SG) representations generated by the LMM itself to improve performance on compositional and general multimodal benchmarks
new_line
new_line
4. URSA -> 
text
4. URSA -> 
URSA: Understanding and Verifying Chain-of-thought Reasoning in
  Multimodal Mathematics (2501.04686)
https://huggingface.co/papers/2501.04686
2501.04686
paper
resource
https://huggingface.co/papers/2501.04686
text
new_line
Brings System 2-style thinking to multimodal math reasoning, using a 3-module CoT data synthesis process with CoT distillation, trajectory-format rewriting and format unification
text
Brings System 2-style thinking to multimodal math reasoning, using a 3-module CoT data synthesis process with CoT distillation, trajectory-format rewriting and format unification
new_line
new_line
5. MM-Verify -> 
text
5. MM-Verify -> 
MM-Verify: Enhancing Multimodal Reasoning with Chain-of-Thought
  Verification (2502.13383)
https://huggingface.co/papers/2502.13383
2502.13383
paper
resource
https://huggingface.co/papers/2502.13383
new_line
Introduces a verification mechanism with MM-Verifier and MM-Reasoner that implements synthesized high-quality CoT data for multimodal reasoning
text
Introduces a verification mechanism with MM-Verifier and MM-Reasoner that implements synthesized high-quality CoT data for multimodal reasoning
new_line
new_line
6. Duty-Distinct CoT (DDCoT) -> 
text
6. Duty-Distinct CoT (DDCoT) -> 
DDCoT: Duty-Distinct Chain-of-Thought Prompting for Multimodal Reasoning
  in Language Models (2310.16436)
https://huggingface.co/papers/2310.16436
2310.16436
paper
resource
https://huggingface.co/papers/2310.16436
new_line
Divides the reasoning responsibilities between LMs and visual models, integrating the visual recognition capabilities into the joint reasoning process
text
Divides the reasoning responsibilities between LMs and visual models, integrating the visual recognition capabilities into the joint reasoning process
new_line
new_line
7. Multimodal-CoT from Amazon Web Services -> 
text
7. Multimodal-CoT from Amazon Web Services -> 
Multimodal Chain-of-Thought Reasoning in Language Models (2302.00923)
https://huggingface.co/papers/2302.00923
2302.00923
paper
resource
https://huggingface.co/papers/2302.00923
new_line
A two-stage framework separates rationale generation from answer prediction, allowing the model to reason more effectively using multimodal inputs
text
A two-stage framework separates rationale generation from answer prediction, allowing the model to reason more effectively using multimodal inputs
new_line
new_line
8. Graph-of-Thought (GoT) -> 
text
8. Graph-of-Thought (GoT) -> 
Beyond Chain-of-Thought, Effective Graph-of-Thought Reasoning in Large
  Language Models (2305.16582)
https://huggingface.co/papers/2305.16582
2305.16582
paper
resource
https://huggingface.co/papers/2305.16582
new_line
This two-stage framework models reasoning as a graph of interconnected ideas, improving performance on text-only and multimodal tasks
text
This two-stage framework models reasoning as a graph of interconnected ideas, improving performance on text-only and multimodal tasks
new_line
new_line
More in the comments👇
text
More in the comments👇
9 Multimodal Chain-of-Thought methods

How Chain-of-Thought (CoT) prompting can unlock models' full potential across images, video, audio and more? Finding special multimodal CoT techniques is the answer.

Here are 9 methods of Multimodal Chain-of-Thought (MCoT). Most of them are open-source:

1. KAM-CoT -> https://huggingface.co/papers/2401.12863
This lightweight framework combines CoT prompting with knowledge graphs (KGs) and achieves 93.87% accuracy

2. Multimodal Visualization-of-Thought (MVoT) -> https://huggingface.co/papers/2501.07542
Lets models generate visual reasoning traces, using a token discrepancy loss to improve visual quality

3. Compositional CoT (CCoT) -> https://huggingface.co/papers/2311.17076
Uses scene graph (SG) representations generated by the LMM itself to improve performance on compositional and general multimodal benchmarks

4. URSA -> https://huggingface.co/papers/2501.04686 
Brings System 2-style thinking to multimodal math reasoning, using a 3-module CoT data synthesis process with CoT distillation, trajectory-format rewriting and format unification

5. MM-Verify -> https://huggingface.co/papers/2502.13383
Introduces a verification mechanism with MM-Verifier and MM-Reasoner that implements synthesized high-quality CoT data for multimodal reasoning

6. Duty-Distinct CoT (DDCoT) -> https://huggingface.co/papers/2310.16436
Divides the reasoning responsibilities between LMs and visual models, integrating the visual recognition capabilities into the joint reasoning process

7. Multimodal-CoT from Amazon Web Services -> https://huggingface.co/papers/2302.00923
A two-stage framework separates rationale generation from answer prediction, allowing the model to reason more effectively using multimodal inputs

8. Graph-of-Thought (GoT) -> https://huggingface.co/papers/2305.16582
This two-stage framework models reasoning as a graph of interconnected ideas, improving performance on text-only and multimodal tasks

More in the comments👇
👀
John6666 suryakiran786 nosuyo dantezxcd on3dj Takugen alexa4cg sourishwicon
64838b28c235ef76b63e4999
https://cdn-avatars.huggingface.co/v1/production/uploads/64838b28c235ef76b63e4999/ZhQCYoU3vps71Ag7Jezj6.jpeg
Ksenia Se
Kseniase
user