xtools-at/Ministral-3-3B-Medical-Reasoning

Ministral3 3B Medical Reasoning

A fine-tuned model for medical question answering and step-by-step clinical reasoning using a subset of the Medical-o1-Reasoning-SFT dataset.

The model is built on unsloth/Ministral-3-3B-Reasoning-2512, a fine-tuned version of Ministral 3 3B. The LoRA fine-tune adds medical reasoning examples to this existing model rather than training a new foundation model from scratch.

This is an experimental medical reasoning model, not a clinically validated system. Its outputs may contain incorrect diagnoses, fabricated details, or misleading explanations. It is intended for research and evaluation, not as a substitute for qualified medical judgment.

Model details

Property	Details
Model type	LoRA adapter
Architecture lineage	Ministral-3-3B-Reasoning-2512
Fine-tuning base	unsloth/Ministral-3-3B-Reasoning-2512
Fine-tuning method	Supervised fine-tuning (SFT) with LoRA
Primary task	Medical question answering and reasoning
Training dataset	xtools-at/medical-o1-reasoning-SFT-chat
Training sample count	4,000 randomly selected examples
Training duration	1 epoch
Training precision	16-bit
Maximum training sequence length	4,096 tokens
Training hardware	NVIDIA A100, 40 GB VRAM
License declared for this release	Apache-2.0

Intended use

Potential research and experimentation use cases include:

• Answering medical knowledge questions.
• Reasoning about symptoms, laboratory findings, and clinical vignettes.
• Exploring differential diagnoses and possible follow-up investigations.
• Evaluating how a small language model adapts to medical reasoning examples.
• Prototyping medical NLP workflows in controlled, non-clinical environments.

The model has not been established as suitable for patient-facing diagnosis, treatment recommendations, emergency triage, or autonomous clinical decision-making.

Model lineage

This fine-tune uses the following model lineage:

1. mistralai/Ministral-3-3B-Base-2512 — the upstream model family.
2. unsloth/Ministral-3-3B-Reasoning-2512 — the immediate fine-tuning base.
3. This model — a LoRA fine-tune of the immediate base on medical reasoning data.

Because the immediate base is already a customized fine-tune, the resulting behavior reflects both its prior training and the additional medical fine-tuning. The relative contribution of each stage has not been independently measured.

Training dataset and preprocessing

Training used a cleaned, randomly selected subset of FreedomIntelligence/medical-o1-reasoning-SFT, a dataset designed for medical question answering and reasoning.

The dataset was preprocessed before fine-tuning. Thinking traces were incorporated into the training data, and every training example includes a thinking trace. This ensures that the model is consistently trained on medical questions paired with reasoning-oriented responses rather than a mixture of examples with and without thinking traces.

Preprocessing and training data

• Source dataset: FreedomIntelligence/medical-o1-reasoning-SFT
• Data preparation: Cleaned before use
• Reasoning traces: Incorporated into the training examples
• Trace coverage: 100% of training examples
• Sampling: 4,000 randomly selected examples
• Training duration: 1 epochs
• Language: English
• Maximum sequence length: 4,096 tokens
• Evaluation split: Additional 5% of the sample data

The fine-tuning objective exposes the model to medical reasoning traces alongside the associated questions and answers, encouraging it to learn patterns of structured reasoning when addressing medical problems.

Reasoning behavior

This fine-tune is intended to improve performance on medical questions that benefit from multi-step reasoning, including interpreting symptoms and laboratory findings, considering plausible diagnoses, and identifying relevant follow-up investigations.

Although every training example includes a thinking trace, this does not guarantee that the model will reason correctly, produce faithful explanations, or achieve better diagnostic accuracy than its base model. Reasoning quality and medical accuracy require separate evaluation.

The presence of training traces also does not establish that generated explanations faithfully represent the internal processes responsible for the model's predictions.

Training configuration

Parameter	Value
Method	LoRA supervised fine-tuning
GPU	NVIDIA A100 (40 GB)
Precision	16-bit
Epochs	1
Learning rate	2e-4
Learning-rate scheduler	Linear
Optimizer	AdamW 8-bit
LoRA rank (r)	16
LoRA alpha	32
LoRA dropout	0.02
Target modules	All modules, as configured in the training run
Per-device batch size	4
Gradient accumulation steps	8
Effective batch size	32
Weight decay	0.002
Maximum sequence length	4,096 tokens

Reported training results

Metric	Value
Final training loss	1.454
Evaluation loss	1.432

These are the losses reported by the training run. They indicate the model's token-prediction performance under the training and evaluation setup, but do not independently establish medical accuracy, diagnostic reliability, or superiority over the base model.

The evaluation set was small. No independent medical benchmark results, clinician review, or comparison against the immediate base model are reported here.

Usage

Required base Mistral Reasoning system prompt (always set this):

# HOW YOU SHOULD THINK AND ANSWER

First draft your thinking process (inner monologue) until you arrive at a response. Format your response using Markdown, and use LaTeX for any mathematical equations. Write both your thoughts and the response in the same language as the input.

Your thinking process must follow the template below:[THINK]Your thoughts or/and draft, like working through an exercise on scratch paper. Be as casual and as long as you want until you are confident to generate the response to the user.[/THINK]Here, provide a self-contained response.

You can append additional instructions to the base system prompt:

You are an assistant for medical reasoning research. Analyze the clinical information provided, identify the most plausible explanation, and explain the relevant medical reasoning. State important uncertainties and avoid presenting an unverified conclusion as a confirmed diagnosis.

Example user prompt:

A 48-year-old woman has been experiencing intermittent nausea, decreased appetite, and increasing weakness and fatigue. She has a history of kidney stones and an elevated calcium level of 12.9 mg/dL, along with elevated nocturnal blood pressure. Given these symptoms and laboratory findings, what abnormality is most likely to be found upon further hormonal evaluation?

This example is an evaluation prompt, not a claim that the model's answer has been clinically verified.

Limitations and safety

• No clinical validation: The model has not been validated for clinical use or shown to improve patient outcomes.
• Potential hallucinations: Explanations may sound convincing while containing incorrect medical facts or conclusions.
• Limited training sample: Fine-tuning on a small number of examples does not guarantee coverage of medical specialties, uncommon conditions, or diverse patient populations.
• Inherited behavior: The customized base model may influence response style, instruction following, and safety behavior.
• No demonstrated generalization: Performance on held-out examples from the same dataset may not transfer to real-world clinical cases.
• No established calibration: Confidence expressed in generated answers should not be interpreted as a reliable measure of diagnostic certainty.
• No benchmark claims: The reported losses are not a substitute for external evaluation against medical QA benchmarks or clinician-adjudicated cases.

Do not use the model as the sole basis for diagnosis, treatment, medication decisions, or emergency care. Outputs must be independently checked against reliable medical references and reviewed by qualified healthcare professionals where appropriate.

Licensing and attribution

This release declares the Apache-2.0 license. Users should review the licenses and terms associated with the immediate base model, upstream Qwen model, and training dataset, and verify that their intended use complies with all applicable terms.

Please cite and acknowledge the Medical-o1-Reasoning-SFT dataset and the upstream models when using this adapter in research or derivative work.