code_repo_finetuning / requirements.txt
tensense's picture
Upload folder using huggingface_hub
4e909c7 verified
# Core dependencies - 必须先安装
torch>=2.1.0
transformers>=4.51.0 # 升级:Qwen3需要4.51.0+
accelerate>=0.25.0
peft>=0.7.0
deepspeed>=0.12.0
bitsandbytes>=0.41.0
# Data processing
datasets>=2.15.0
numpy>=1.24.0
pandas>=2.0.0
scipy>=1.10.0 # 用于稀疏矩阵操作
scikit-learn>=1.3.0
# Code analysis
# tree-sitter>=0.20.0 # 可选:用于语法解析优化,AST已足够
# tree-sitter-python>=0.20.0 # 可选
gitpython>=3.1.0
# Evaluation metrics
rouge-score>=0.1.2
nltk>=3.8.0
sacrebleu>=2.3.0
# Utilities
pyyaml>=6.0
tqdm>=4.66.0
sentencepiece>=0.1.99
# Optional for better performance (可选,编译失败不影响运行)
# flash-attn>=2.3.0 # 注释掉:需要CUDA编译,可选安装
triton>=2.1.0
# Visualization
tensorboard>=2.15.0
matplotlib>=3.7.0
# Development tools
ipython>=8.12.0
jupyter>=1.0.0