🧠 Introducing Ellora Recipe #6: Execution-Aware World Model for Qwen3-4B-Thinking
text
🧠 Introducing Ellora Recipe #6: Execution-Aware World Model for Qwen3-4B-Thinking
new_line
new_line
Teaching LLMs to understand not just what code does, but HOW it executes at runtime!
text
Teaching LLMs to understand not just what code does, but HOW it executes at runtime!
new_line
new_line
Inspired by Meta's CWM (Code World Model) research, this LoRA adapter adds execution awareness to Qwen3-4B-Thinking-2507. The model learns to predict variable states, trace program execution step-by-step, and debug code by understanding runtime behavior.
text
Inspired by Meta's CWM (Code World Model) research, this LoRA adapter adds execution awareness to Qwen3-4B-Thinking-2507. The model learns to predict variable states, trace program execution step-by-step, and debug code by understanding runtime behavior.
new_line
new_line
🔍 Key Innovation:
text
🔍 Key Innovation:
new_line
We combine Qwen3's native thinking capabilities with real Python execution traces captured via sys.settrace(). The model is trained using GRPO with a custom reward function that scores execution prediction accuracy.
text
We combine Qwen3's native thinking capabilities with real Python execution traces captured via sys.settrace(). The model is trained using GRPO with a custom reward function that scores execution prediction accuracy.
new_line
new_line
📊 Training Approach:
text
📊 Training Approach:
new_line
  - Hybrid Magpie-style code generation
text
  - Hybrid Magpie-style code generation
new_line
  - Real execution tracing for ground truth
text
  - Real execution tracing for ground truth
new_line
  - Self-supervised learning (no manual annotations!)
text
  - Self-supervised learning (no manual annotations!)
new_line
  - 298 training samples with execution traces
text
  - 298 training samples with execution traces
new_line
new_line
✨ What it does:
text
✨ What it does:
new_line
  - Predicts variable states at each line of code
text
  - Predicts variable states at each line of code
new_line
  - Explains execution flow with thinking tags
text
  - Explains execution flow with thinking tags
new_line
  - Helps debug by understanding runtime behavior
text
  - Helps debug by understanding runtime behavior
new_line
  - Works as a "neural debugger"
text
  - Works as a "neural debugger"
new_line
new_line
  🎯 Results:
text
  🎯 Results:
new_line
  - 20% overall accuracy on execution prediction
text
  - 20% overall accuracy on execution prediction
new_line
  - 33.3% mean state accuracy
text
  - 33.3% mean state accuracy
new_line
  - Trained on Qwen3-4B-Thinking (262K context, 4B params)
text
  - Trained on Qwen3-4B-Thinking (262K context, 4B params)
new_line
new_line
  🔗 Links:
text
  🔗 Links:
new_line
Model: 
text
Model: 
https://huggingface.co/codelion/Qwen3-4B-execution-world-model-lora
codelion/Qwen3-4B-execution-world-model-lora
model
resource
https://huggingface.co/codelion/Qwen3-4B-execution-world-model-lora
new_line
Dataset: 
text
Dataset: 
https://huggingface.co/datasets/codelion/execution-world-model-dataset
codelion/execution-world-model-dataset
dataset
resource
https://huggingface.co/datasets/codelion/execution-world-model-dataset
new_line
GitHub Recipe: 
text
GitHub Recipe: 
https://github.com/codelion/ellora
https://github.com/codelion/ellora
link
new_line
Notebook: 
text
Notebook: 
https://github.com/codelion/ellora/blob/main/Ellora_Recipe_6_Execution_World_Model_Thinking_LoRA.ipynb
https://github.com/codelion/ellora/blob/main/Ellora_Recipe_6_Execution_World_Model_Thinking_LoRA.ipynb
link
new_line
new_line
Part of the Ellora project - standardized LoRA recipes for enhancing LLM capabilities. All recipes use self-supervised data generation and work with existing infrastructure (PEFT, LoRAX, vLLM).
text
Part of the Ellora project - standardized LoRA recipes for enhancing LLM capabilities. All recipes use self-supervised data generation and work with existing infrastructure (PEFT, LoRAX, vLLM).
new_line
new_line
#LLM #LoRA #CodeGeneration #WorldModel #Qwen #AI #MachineLearning
text
#LLM #LoRA #CodeGeneration #WorldModel #Qwen #AI #MachineLearning
🧠 Introducing Ellora Recipe #6: Execution-Aware World Model for Qwen3-4B-Thinking

Teaching LLMs to understand not just what code does, but HOW it executes at runtime!

Inspired by Meta's CWM (Code World Model) research, this LoRA adapter adds execution awareness to Qwen3-4B-Thinking-2507. The model learns to predict variable states, trace program execution step-by-step, and debug code by understanding runtime behavior.

🔍 Key Innovation:
We combine Qwen3's native thinking capabilities with real Python execution traces captured via sys.settrace(). The model is trained using GRPO with a custom reward function that scores execution prediction accuracy.

📊 Training Approach:
  - Hybrid Magpie-style code generation
  - Real execution tracing for ground truth
  - Self-supervised learning (no manual annotations!)
  - 298 training samples with execution traces

✨ What it does:
  - Predicts variable states at each line of code
  - Explains execution flow with thinking tags
  - Helps debug by understanding runtime behavior
  - Works as a "neural debugger"

  🎯 Results:
  - 20% overall accuracy on execution prediction
  - 33.3% mean state accuracy
  - Trained on Qwen3-4B-Thinking (262K context, 4B params)

  🔗 Links:
Model: https://huggingface.co/codelion/Qwen3-4B-execution-world-model-lora
Dataset: https://huggingface.co/datasets/codelion/execution-world-model-dataset
GitHub Recipe: https://github.com/codelion/ellora
Notebook: https://github.com/codelion/ellora/blob/main/Ellora_Recipe_6_Execution_World_Model_Thinking_LoRA.ipynb

Part of the Ellora project - standardized LoRA recipes for enhancing LLM capabilities. All recipes use self-supervised data generation and work with existing infrastructure (PEFT, LoRAX, vLLM).

#LLM #LoRA #CodeGeneration #WorldModel #Qwen #AI #MachineLearning
❤️
codelion noobmldude RunaB manliketg
🚀
codelion John6666 woodywu
👍
xynic codelion
🔥
codelion
➕
codelion