OpenMed

OpenMed Qwen3-1.7B WSP Clinical

OpenMed on Hugging Face OpenMed SDK on GitHub License Apache-2.0 No training

OpenMed's clinical weight edit of Qwen3-1.7B: no training, a few attention heads edited by hand, and the model answers medical exam questions in the format they ask for. On the 1,273 MedQA-USMLE test questions it scores 52.7%, against 45.1% for Qwen3-1.7B as released (paired difference +7.6 points, 95% CI +5.3 to +9.9, McNemar p = 6.5e-11). No gradient descent, no fine-tuning, no training data: a frontier model read the weights in a Python kernel and changed them directly, following "Sparks of Weight-Space Programming" (MIT CSAIL).

Developed by OpenMed
Base model Qwen/Qwen3-1.7B @ 70d244cc86ccca08cf5af4e1e306ecf908b1ad5e
What changed The output projections (o_proj) of three attention heads (layer 21 head 9, layer 23 head 5, layer 24 head 10)
Method Weight-space programming (Takehi, Teoh, Zhang and Khattab, MIT CSAIL, 2026): a frontier LLM reads the weights in a Python kernel and edits them directly
Editor of the clinical edit a frontier LLM, 12 forward passes, 34 Python runs, 47 turns
Practice data the editor could use training splits only: MedQA-USMLE train (10,178) and 10,000 MedMCQA train questions
License Apache-2.0 (see Credits for the inputs and their licenses)
Intended use research on model editing and evaluation; not for clinical use

About OpenMed

OpenMed builds open models and open-source tools for clinical and biomedical text that run on your own hardware.

Results

Same model architecture, tokenizer, chat template and sampling for every row; only weight values differ. One sample per question with seed 0 (MedQA also with seeds 1 and 2). Thinking on, temperature 0.6, top_p 0.95, top_k 20, presence penalty 1.5, at most 32,768 new tokens (the paper's evaluation protocol). The prompt is OpenMed's fixed MedQA exam prompt (below), and an answer is graded from the text after </think>: its last Answer: X line, X in A to D. Answers cut at the token limit and answers with no gradable line count as wrong, as in the paper.

Checkpoint MedQA (seed 0) MedQA, median of 3 seeds (range) MedMCQA MMLU medical MedQA answers cut at 32,768 tokens MedQA answers with no gradable line
OpenMed WSP Clinical 1.7B (this model) 52.7% 53.0% (52.7% to 53.8%) 46.3% 71.9% 0.0% 0.1%
Random patch, same size, random direction (seed 1) 45.8% n/a 43.7% 66.8% 0.0% 13.7%
Random patch, same size, random direction (seed 2) 44.9% n/a 42.6% 63.0% 0.0% 13.8%
Qwen3-1.7B as released 45.1% n/a 42.1% 64.1% 0.0% 14.7%

Paired tests on the same questions (seed 0; exact McNemar on the questions only one checkpoint got right; bootstrap 95% CI over questions, 10,000 resamples):

Comparison Benchmark n Before After Difference (95% CI) Only before right / only after right McNemar p
Qwen3-1.7B -> WSP Clinical 1.7B MedQA-USMLE (4 options) 1,273 45.1% 52.7% +7.6 pts (+5.3 to +9.9) 63 / 160 6.5e-11
Qwen3-1.7B -> WSP Clinical 1.7B MedMCQA 1,000 42.1% 46.3% +4.2 pts (+1.7 to +6.6) 58 / 100 0.001
Qwen3-1.7B -> WSP Clinical 1.7B MMLU 1,089 64.1% 71.9% +7.8 pts (+5.4 to +10.3) 52 / 137 5.1e-10
random patch 1 -> WSP Clinical 1.7B MedQA-USMLE (4 options) 1,273 45.8% 52.7% +6.9 pts (+4.6 to +9.2) 72 / 160 7.5e-09
random patch 1 -> WSP Clinical 1.7B MedMCQA 1,000 43.7% 46.3% +2.6 pts (+0.1 to +5.1) 63 / 89 0.042
random patch 1 -> WSP Clinical 1.7B MMLU 1,089 66.8% 71.9% +5.0 pts (+2.9 to +7.2) 51 / 106 1.4e-05
random patch 2 -> WSP Clinical 1.7B MedQA-USMLE (4 options) 1,273 44.9% 52.7% +7.8 pts (+5.5 to +10.1) 69 / 168 1.0e-10
random patch 2 -> WSP Clinical 1.7B MedMCQA 1,000 42.6% 46.3% +3.7 pts (+1.1 to +6.3) 66 / 103 0.005
random patch 2 -> WSP Clinical 1.7B MMLU 1,089 63.0% 71.9% +8.9 pts (+6.6 to +11.3) 42 / 139 2.5e-13

Format-agnostic score (secondary)

Also accepting Answer: <B>, Answer: (b), Answer: <letter>B and \boxed{B} after </think>. The gap between this and the main score shows how much of a checkpoint's score is lost to the format of its final line rather than to the choice itself.

Checkpoint MedQA MedMCQA MMLU medical
OpenMed WSP Clinical 1.7B (this model) 52.7% 46.3% 71.9%
Random patch, same size, random direction (seed 1) 52.7% 47.8% 73.2%
Random patch, same size, random direction (seed 2) 52.1% 47.4% 71.3%
Qwen3-1.7B as released 52.2% 47.1% 72.2%

Nothing else broke (general benchmarks, one sample, seed 0)

Checkpoint MATH-500 MATH-500 cut at 32,768 tokens IFEval (strict, prompt level)
OpenMed WSP Clinical 1.7B (this model) 92.2% 0.6% 71.0%
Qwen3-1.7B as released 91.2% 0.6% 71.2%

What the edit does

OpenMed's fixed exam prompt ends with "finish with one last line exactly like: Answer: ". Under the evaluation's sampling, Qwen3-1.7B often copies the placeholder instead of filling it in and ends with Answer: <B> or Answer: <letter>, which no grader reads as a letter. The editor traced the copy to three attention heads (layer 21 head 9, layer 23 head 5, layer 24 head 10): at the final Answer: they attend to the prompt's < and write the direction that raises the logit of <. The presence penalty makes it worse, because the four letters have usually appeared in the thinking already while < has not.

The edit removes that direction from each of these heads' output columns: W_h <- W_h - r r^T W_h, with r the output direction of the token ' <' (W_h = the head's columns of o_proj). We checked it independently: the main output direction of every edited tensor's change lines up with the output direction of < (eval/direction_check.json).

Tensor Shape bfloat16 entries changed Relative change sha256 (edited, bfloat16 bytes)
model.layers.21.self_attn.o_proj.weight 2048 x 2048 224,129 0.0072 8cc60bc6f63d3dc792cc67a8fb4710b82fd2f823c7d22455b71274928475a64f
model.layers.23.self_attn.o_proj.weight 2048 x 2048 232,604 0.0130 bb893c4afcc418f964825549a43efabb32375859bca61ae66c072df73eda699d
model.layers.24.self_attn.o_proj.weight 2048 x 2048 233,656 0.0126 0b40da5ad79c1b837843f00121ecc1f8b656ebcd4ae38b63ed617ff94386808c

690,389 bfloat16 entries differ from the starting checkpoint (Qwen/Qwen3-1.7B @ 70d244cc). edit/manifest.json and edit/edit.npz hold the edit as an exact diff (indices and bfloat16 bits), with the sha256 of every tensor before and after; rebuilding the tensors from the diff reproduces these hashes.

Quick start

from vllm import LLM, SamplingParams
from transformers import AutoTokenizer


def main():
    repo = "OpenMed/Qwen3-1.7B-WSP-Clinical"
    tok = AutoTokenizer.from_pretrained(repo)
    llm = LLM(model=repo, dtype="bfloat16", max_model_len=34816, generation_config="vllm")

    # a MedQA-USMLE practice question (train split, id medqa_train_4541; the key is B)
    question = ("A 23-year-old male with a homozygous CCR5 mutation is found to be immune to HIV infection. "
                "The patient's CCR5 mutation interferes with the function of which viral protein?")
    options = {"A": "Reverse transcriptase", "B": "gp120", "C": "gp41", "D": "pp17"}
    user = ("Which option is the best answer to this clinical question?\n\n" + question + "\n\n"
            + "\n".join(f"{k}. {v}" for k, v in options.items())
            + "\n\nReason as much as you need, then finish with one last line exactly like: Answer: <letter>")
    prompt = tok.apply_chat_template([{"role": "user", "content": user}], tokenize=False,
                                     add_generation_prompt=True, enable_thinking=True)
    sp = SamplingParams(temperature=0.6, top_p=0.95, top_k=20, presence_penalty=1.5, max_tokens=32768, seed=0)
    text = llm.generate([prompt], sp)[0].outputs[0].text
    print(text.split("</think>")[-1].strip())


if __name__ == "__main__":  # vLLM starts worker processes
    main()

How it was made

  1. The harness. We rebuilt the paper's setup, whose prompts and rules are published but whose code is not: a persistent CPU Python kernel over the weights (no GPU, no internet, no gradients; save_edit refuses the final norm, the embeddings and whole-tensor rescaling) and a forward-pass tool (generations with the evaluation's sampling, paired original vs edited with fixed seeds, and teacher-forced reads of hidden states, MLP activations and attention). The task text is the paper's, with the target changed from math to clinical questions.
  2. The editor. A frontier LLM had a budget of 12 forward passes and 40 Python runs. It could use only training-split practice questions; the MedQA, MedMCQA and MMLU test sets in the tables were never shown to it. It diagnosed the failures, localized them, tested strengths and confirmed its edit on fresh practice questions before submitting.
  3. Evaluation and control. Every checkpoint ran with the same vLLM 0.33.0 server and settings on one GPU. The random-patch control changes the same rows and columns of the same tensors by the same amount (the same singular values) in random directions; two seeds.

Files

  • The weights, config, tokenizer and chat template: Qwen3-1.7B's release files with the edited tensors (every other tensor is unchanged).
  • edit/: the clinical edit as an exact, hash-checked diff from the starting checkpoint.
  • eval/: per-question results of every checkpoint on every benchmark (id, finish reason, tokens, extracted answer, grade) with no question text, the scores, the first 20 generated texts per benchmark for this model, and receipt.json (forward passes, Python runs, turns and wall time of the edit run).

Limitations

  • Small model, one sample per question (MedQA with three seeds). The gains are relative to the same model and sampling and say nothing about clinical safety.
  • Most of the gain is the model now writing a letter where it used to copy the prompt's <letter> placeholder; the format-agnostic table shows how much is format and how much is the choice itself.
  • The fixed prompt literally says Answer: <letter>; with a prompt that shows no placeholder the edit has less to fix.
  • The edit lowers the probability of < everywhere these heads write, not only in the answer line.
  • Research only. Not for clinical use, diagnosis or treatment.

Credits

  • Method: "Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights" by Rikiya Takehi, Jayden Teoh, Alex Zhang and Omar Khattab (MIT CSAIL, 2 Nov 2026), https://rikiyat.github.io/blog/weight-space-programming/; logs and diffs at https://github.com/RikiyaT/lm_understand-logs. OpenMed rebuilt their harness from the published prompts and rules and pointed it at clinical questions.
  • Base model: Qwen/Qwen3-1.7B by the Qwen team, Apache-2.0.
  • Data: MedQA-USMLE (GBaker/MedQA-USMLE-4-options, CC-BY-4.0), MedMCQA (openlifescienceai/medmcqa, Apache-2.0), MMLU (cais/mmlu, MIT), MATH-500 (HuggingFaceH4/MATH-500), IFEval (google/IFEval, Apache-2.0).
  • Hugging Face's ML Intern reproduction of the paper's math edit showed the method reproduces; ML Intern did not take part in this model.

Citation

@misc{takehi2026weightspace,
  title  = {Sparks of Weight-Space Programming: LLMs can natively read \& write neural network weights},
  author = {Rikiya Takehi and Jayden Teoh and Alex Zhang and Omar Khattab},
  year   = {2026},
  month  = {nov},
  url    = {https://rikiyat.github.io/blog/weight-space-programming/}
}
Downloads last month
784
Safetensors
Model size
2B params
Tensor type
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for OpenMed/Qwen3-1.7B-WSP-Clinical

Finetuned
Qwen/Qwen3-1.7B
Finetuned
(1,422)
this model

Datasets used to train OpenMed/Qwen3-1.7B-WSP-Clinical

Collection including OpenMed/Qwen3-1.7B-WSP-Clinical

Article mentioning OpenMed/Qwen3-1.7B-WSP-Clinical

Evaluation results