Qwen3-8B-FengGe-SFT

๐Ÿ“Œ Overview

Qwen3-8B-FengGe-SFT is a LoRA fine-tuned conversational model based on Qwen/Qwen3-8B, trained to mimic the distinctive speaking style (โ€œๅณฐๅ“ฅโ€) using the Zhoulifeng Streaming Dataset.

๐Ÿ“š Training data: https://huggingface.co/datasets/hzb29/Zhoulifeng-Streaming-Dataset

The model is designed for informal, conversational Chinese dialogue with a strong personality style, emphasizing long-form, free-flowing responses.


๐Ÿง  Base Model

  • Model: Qwen/Qwen3-8B
  • Fine-tuning method: LoRA

๐Ÿ“‰ Training Loss

training_loss


๐Ÿš€ Inference

Load with Hugging Face

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "boqiny/Qwen3-8B-FengGe-SFT"

tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(model_name, trust_remote_code=True)

prompt = "ๅณฐๅ“ฅๆˆ‘ไผผไนŽๆ€ปๆ˜ฏ็ˆฑไธŠไธไผšๅ–œๆฌขๆˆ‘็š„ๅฅณ็”Ÿ๏ผŒๆœ‰็š„ๅนฒ่„†ไธๆญ็†๏ผŒ่ฟ™ๆ˜ฏๅ‘ฝ่ฟ็š„ๅฎ‰ๆŽ’่ฟ˜ๆ˜ฏๆ€งๅŽ‹ๆŠ‘"

inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(
    **inputs,
    max_new_tokens=256,
    repetition_penalty=1.3
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

โš ๏ธ Note: repetition_penalty=1.3 is important to reduce looping/repetitive outputs.


๐Ÿ’ฌ Example Generations

Example 1

User:
ๅณฐๅ“ฅๆˆ‘ไผผไนŽๆ€ปๆ˜ฏ็ˆฑไธŠไธไผšๅ–œๆฌขๆˆ‘็š„ๅฅณ็”Ÿ๏ผŒๆœ‰็š„ๅนฒ่„†ไธๆญ็†๏ผŒ่ฟ™ๆ˜ฏๅ‘ฝ่ฟ็š„ๅฎ‰ๆŽ’่ฟ˜ๆ˜ฏๆ€งๅŽ‹ๆŠ‘

Assistant:
ไฝ ้ƒฝๆฒกๆœ‰ๆŽฅ่งฆ่ฟ‡็š„ๅฏน่ฑกไฝ ๆ€Žไนˆๅฐฑ็Ÿฅ้“ไบบๅฎถ่ฎจๅŽŒไฝ ไนŸๅ–œๆฌขไธไธŠไฝ ่‡ชๅทฑๅ‘€๏ผŸๅฐฑๆ˜ฏๅ› ไธบไธ็Ÿฅ้“ๅŽŸๅ› ๆ‰่ƒฝ่ฎฉๅคง่„‘่ƒกๆ€ไนฑๆƒณไบ†ใ€‚ๅๆญฃไฝ ่ฆๆ˜ฏๆœ‰ๆœบไผš็œŸๆญฃ่ง้ข่ฏ•่ฏ•ๆ‰็Ÿฅ้“ๅ…ทไฝ“็š„็ป“ๆžœๅˆฐๅบ•ๆ˜ฏๆ€Žไนˆๅ›žไบ‹ใ€‚ๆœ‰ๆ—ถๅ€™็”ท็”Ÿๅœจไธ€่ตทๅŽๅ•Š่Š่Šๅคฉๆ‹‰็€ๆ‰‹ๆ•ฃๆญฅ็‰ตๆ‰‹ๆ‚่…ฐๆŠฑๅคง่…ฟ่ทณ่ˆžๅ”ฑๆญŒๅ–้…’ๅƒ้ฅญ้€›่ก—้€›ๅ•†ๅœบ็œ‹็”ตๅฝฑๅ‘ๅพฎไฟกๅˆทๆŠ–้Ÿณ่ง†้ข‘็›ดๆ’ญๆ‰“ๆธธๆˆๅฅ่บซ้”ป็‚ผ่ฟๅŠจ็™ปๅฑฑ็ˆฌๅฑฑ้œฒ่ฅ่นฆ่ฟชๅนฟๅœบ่ก—่พน้—ฒๆธธๅคœๅบ—้…’ๅงๆก‘ๆ‹ฟๆŒ‰ๆ‘ฉ่ถณ็–—KTVๆญŒๅŽ…้…’ๅบ—ๆฐ‘ๅฎฟๅ†œๅฎถไน็ƒง็ƒคๅ›ญๆธธๆณณ้ฆ†ๆธฉๆณ‰ๆฑ ๆฐดไนๅ›ญๅŠจ็‰ฉๅ›ญๆธธไนๅœบๅ…ฌๅ›ญ่Šฑๅ›ญ็ƒๅœบ่ถณ็ƒๅœบ็ฏฎ็ƒๅœบ็พฝๆฏ›็ƒๅœบ็ฝ‘็ƒๅœบๅœฐ่‡ช่กŒ่ฝฆ่ต›้“้ฉฌ็ƒ่ต›ๅฐ„็ฎญ้ถๆ ‡ๅฐ„ๅ‡ปๅœบ้ฆ†ๆๆ–—ๆ ผๆŒก็ซžๆŠ€่ง’ๆ‹ณๅ‡ปๆ‘”่ทค็ฉบๆˆ˜ๅฏนๆŠ—่กจๆผ”้ญ”ๅนป้ญ”ๆœฏๆ‚ๆŠ€็‰น่‰บ็ญ‰้ƒฝ่ƒฝไฝ“้ชŒไธ€ๆฌกไธคๅฃๅญๅœจไธ€่ตทๆ—ถๅฟซไนๆ—ถๅ…‰่ถŠ้•ฟ่ถŠๅฅฝ๏ผŒๅœจไธ€ๅ—็š„ๆ—ถ้—ด่ถŠๅคšๅฐฑ่ถŠไธๅฎนๆ˜“ๅˆ†ๅผ€็ฆปๅผ€ๅฝผๆญคใ€‚็›ธๅๅฆ‚ๆžœไฝ ไธ€็›ดๅ•้ฃžๅญคๅ†›ๅฅ‹ๆˆ˜็‹ฌ่‡ชไธ€ไบบๅฝขๅ•ๅฝฑๅชๅญค็‹ฌ็ปˆ่€ไธ€่พˆๅญไนŸๆฒก่ฐˆๆˆๆ‹็ˆฑๆ€ŽไนˆๅŠžๅ‘ข๏ผŸ่ฟ˜่ƒฝๅนฒ็‚นๅˆซ็š„ไบ‹ๆŒบๅผ€ๅฟƒ็š„๏ผŒๆ‰พไธ€็พค็”ทๅฎถไบบ็ป„ๅ›ขๅ‡บๅŽป็Žฉไธ€ๅœˆไนŸๅพˆไธ้”™ใ€‚


Example 2

User:
ๅณฐๅ“ฅ๏ผŒไฝ ๆ˜ฏๆ€Žไนˆๅšๅˆฐ่งฃ็ญ”ไธ–้—ดไธ‡็‰ฉๆœ€ๅŽๅพ—ๅ‡บ่ฟ™ๆ˜ฏๅฅฝไบ‹็š„็ป“่ฎบ๏ผŒ่ƒฝๆ•™ๆ•™ๆˆ‘ๅ—

Assistant:
้‚ฃๅฐฑๆ˜ฏไฝ ็š„ไปทๅ€ผ่ง‚ๅฐฑๆ˜ฏ่ฟ™ๆ ทๅญ่€Œๅทฒใ€‚ๆœ‰ไบ›ไบบๅฏ่ƒฝ่ฎคไธบ่ฟ™ๆ ทไธๅฅฝ้‚ฃๆ ทไธ่กŒ๏ผŒไฝ†ๆˆ‘่ง‰ๅพ—ๅฅฝ็š„ๅœฐๆ–นๅคชๅคšไบ†ใ€‚ๅช่ฆ่ƒฝๅคŸๆปก่ถณไบบ็ฑป็š„ๅŸบๆœฌ้œ€ๆฑ‚ๅฐฑๅฏไปฅ๏ผŒๆฒกๅฟ…่ฆ่ฟฝๆฑ‚ไธ€ไบ›้ซ˜็ฒพๅฐ–็š„ไธœ่ฅฟ๏ผŒๆฒกๆœ‰ๅฟ…่ฆๅš้‚ฃ็ง่ถ…่ถŠๆ—ถไปฃ็š„ไบ‹ๅ„ฟไนŸไธๅฟ…่ฆๆˆไธบๅไบบๅฏŒ่ฑช็ง‘ๅญฆๅฎถๅ•ฅ็š„๏ผŒๅช่ฆๆ˜ฏๅฎ‰ๅฎ‰็จณ็จณ็š„็”Ÿๆดปๅฐฑๅฅฝใ€‚ๆ‰€ไปฅ่ฏดๆˆ‘็š„่ง‚็‚นไนŸๆ˜ฏๅŸบไบŽๆˆ‘่‡ชๅทฑ็”Ÿๆดป็ป้ชŒไน‹ไธŠๅฝขๆˆ็š„่ฟ™ไนˆไธช็œ‹ๆณ•ใ€‚ๆฒกๆœ‰ไป€ไนˆ็‰นๅˆซ็š„ๅŽŸๅ› ๅชๆ˜ฏๅ› ไธบๆˆ‘ๆ›ดๅ–œๆฌข่ฟ™ๆ ท็š„็”Ÿๆดปๆ–นๅผ็ฝขไบ†๏ผŒๅฐฑ่ฟ™ไนˆ็ฎ€ๅ•ใ€‚


๐Ÿ“ฆ Training with LLaMA-Factory

This model was trained using LLaMA-Factory with the following configuration:

### model
model_name_or_path: Qwen/Qwen3-8B
trust_remote_code: true

### method
stage: sft
do_train: true
finetuning_type: lora
lora_rank: 32
lora_target: all

### dataset
dataset: zhoulifeng_streaming
template: qwen3_nothink
cutoff_len: 2048

### train
per_device_train_batch_size: 1
gradient_accumulation_steps: 8
learning_rate: 5.0e-5
num_train_epochs: 2.0
lr_scheduler_type: cosine
warmup_ratio: 0.1
bf16: true

โš ๏ธ Limitations

  • Strong stylistic bias toward โ€œๅณฐๅ“ฅโ€ conversational tone
  • May generate long, repetitive enumerations without proper decoding settings
  • Not suitable for factual QA or safety-critical applications

๐Ÿ“œ License

Please follow the license of the base model:
๐Ÿ‘‰ Qwen/Qwen3-8B

Downloads last month
10
Safetensors
Model size
8B params
Tensor type
BF16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for boqiny/Qwen3-8B-FengGe-SFT

Finetuned
Qwen/Qwen3-8B
Adapter
(2,506)
this model

Dataset used to train boqiny/Qwen3-8B-FengGe-SFT