You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

This is an experimental Anima derivative. Access is subject to the CircleStone Labs Non-Commercial License v1.2 and the upstream Qwen license.

Temporary Anima Qwen3.5 2B Adapter

Updated September 12, 2026: the default adapter is the completed English/Chinese/Japanese joint text-distillation checkpoint (job 7753, epoch 2, step 6880), including both trained embedding tables. It follows the three-epoch Japanese embedding stage (job 7690), initialized from the merged final image-stage adapter (job 7437).

The six-layer dense adapter maps Qwen3.5-2B language-model hidden states (2048 dimensions) to Anima conditioning (1024 dimensions). Queries use learned embeddings indexed by the accompanying expanded T5 tokenizer. There is no T5 encoder and this is not a fixed-query adapter.

Files

adapter/anima-qwen35-2b-multilingual-en-zh-ja-joint2ep.safetensors: default inference export, FP32, with one merged 59,888-row embed.weight table.
adapter/anima-qwen35-2b-multilingual-en-zh-ja-joint2ep.split.safetensors: exact native training checkpoint, including split embedding tables and tokenizer identity buffers.
tokenizer/: the required expanded T5 tokenizer and vocabulary manifest. Use this bundle, not the original T5 tokenizer or mT5.
adapter/adapter_config.json: current architecture and file selection.
MANIFEST.json: SHA256 checksums and publication verification.
qwen3.5-2b-base/: unchanged Qwen3.5-2B-Base snapshot at revision b1485b2fa6dfa1287294f269f5fb618e03d52d7c.

The old adapter/anima-qwen35-2b-dense-adapter-stage2.safetensors remains available for compatibility. Its original instructions and configuration are under legacy/. Do not use the expanded tokenizer with that old checkpoint.

Anima DiT, VAE, DiT LoRA, datasets, captions, images, optimizer states, and credentials are not included in this update.

Architecture And Vocabulary

Source dimension 2048; target/model dimension 1024; six layers; 16 attention heads; target-side self-attention enabled.
Logical vocabulary: 32,100 legacy IDs plus 27,788 added IDs, totaling 59,888. The first new ID is 32,100.
The native checkpoint retains the 32,128-row physical legacy table. Its unused final 28 rows are omitted from the merged inference table; the new table is appended after legacy row 32,099.
The export preserves every usable embedding and all other adapter weights exactly. Native and merged adapters were compared with synthetic boundary IDs and produced identical outputs in CPU FP32 verification.
The adapter and both embedding tables were jointly updated during this two-epoch text-distillation stage; Qwen and the fixed teacher remained frozen. No DiT, images, or latents were used in this stage.

Loading The Default Inference Export

Use the Anima LLMAdapter implementation from the compatible training/inference code. Resize its embedding table before strict loading. This example does not download or execute Python code from this repository.

import json
from pathlib import Path

import torch
from huggingface_hub import snapshot_download
from safetensors.torch import load_file
from tokenizers import Tokenizer
from transformers import T5TokenizerFast

from library.anima_models import LLMAdapter

root = Path(snapshot_download(
    "SumomoLee/tmp-anima-qwen35-adapter",
    allow_patterns=[
        "adapter/anima-qwen35-2b-multilingual-en-zh-ja-joint2ep.safetensors",
        "adapter/adapter_config.json",
        "tokenizer/*",
    ],
))
tokenizer_file = root / "tokenizer/tokenizer.json"
t5_tokenizer = T5TokenizerFast(
    tokenizer_file=str(tokenizer_file), local_files_only=True,
)
# Preserve the exact saved normalizer, including its CJK handling.
t5_tokenizer.backend_tokenizer.normalizer = (
    Tokenizer.from_file(str(tokenizer_file)).normalizer
)
vocab = json.loads((root / "tokenizer/anima_vocab.json").read_text())
assert len(t5_tokenizer) == int(vocab["extended_vocab_size"]) == 59888

adapter = LLMAdapter(
    source_dim=2048, target_dim=1024, model_dim=1024,
    num_layers=6, num_heads=16, self_attn=True,
)
adapter.embed = torch.nn.Embedding(59888, 1024)
adapter.load_state_dict(load_file(str(
    root / "adapter/anima-qwen35-2b-multilingual-en-zh-ja-joint2ep.safetensors"
)), strict=True)
adapter.t5_tokenizer = t5_tokenizer
adapter.requires_grad_(False).eval()

Use the unchanged Qwen tokenizer/backbone for source text. Feed its language-model hidden states and the expanded T5 IDs into the adapter, passing both attention masks. Preserve the inference pipeline's padding and sequence-length conventions; conditioning length is not inherently fixed at 512. FP32 adapter attention is recommended when the rest of the model uses BF16.

The native .split.safetensors file requires an expanded-vocabulary-aware loader with SplitT5Embedding; it is not directly loadable into an ordinary nn.Embedding.

Training

Completed stage: English/Chinese/Japanese joint embedding + adapter text distillation, 2 epochs and 6,880 optimizer updates on 8 B300 GPUs.
Per-GPU batch size 512; global batch size 4,096; maximum sequence length 1,024. Length-aware microbatching allows up to 256 full-length sequences per microbatch.
Peak learning rates: adapter 4e-6, legacy embeddings 2e-6, added embeddings 2e-5; warmup 100 steps. AdamW, betas (0.9, 0.95), weight decay 0, gradient clipping 1.
4,603,207 source training rows; English repeated twice. Per epoch: 5,024,282 English, 4,603,072 Chinese, and 4,458,910 Japanese exposures (14,086,264 total; English 35.67%).
Artist-caption additions use 146,953 matched English/Chinese records from the local SDXL300 selection. The full translation export was not used, and these artist additions do not include Japanese counterparts.
Japanese short-text supervision uses the previously verified same-record Chinese teacher bridge. The fixed teacher is the merged final job-7437 adapter; the student starts from the completed job-7690 Japanese embedding stage.
Recorded training-stage elapsed time: 41,556 seconds (approximately 11 hours 33 minutes).
W&B training run.

Status And Limitations

This is an experimental, temporary release, not an official CircleStone Labs or Qwen release. English, Chinese, and Japanese are training languages; completion and export checks do not establish image-generation quality or language parity. Earlier qualitative findings belong to previous checkpoints and should not be treated as evaluations of this release.

The checkpoint inherits the merged final job-7437 image-stage adapter through job 7690; job 7753 then updates its embeddings and dense adapter using text distillation. No paired DiT weights or DiT LoRA are included in this update. Do not reapply the already-merged job-7437 adapter LoRA to this checkpoint. Compatibility and quality with a chosen DiT state require separate evaluation.

Earlier checkpoints remain available. The previous image-step1200 release instructions and configuration are archived in legacy/README-image-step1200.md and legacy/adapter_config-image-step1200.json; earlier text-stage instructions remain in legacy/README-joint3ep.md.

Licenses And Attribution

The Qwen3.5 files are distributed under Apache-2.0; see qwen3.5-2b-base/LICENSE.

The adapter is a derivative of Anima and is distributed under the CircleStone Labs Non-Commercial License v1.2; see LICENSE.md and NOTICE before use or redistribution. Existing license files and repository access settings are unchanged.

Downloads last month: -; Downloads are not tracked for this model. How to track

Model tree for SumomoLee/tmp-anima-qwen35-adapter

Base model

Qwen/Qwen3.5-2B-Base

Finetuned

(71)

this model