🇨🇳 Gitee Code Dataset - The Missing Piece of the Stack
text
🇨🇳 Gitee Code Dataset - The Missing Piece of the Stack
new_line
https://huggingface.co/datasets/nyuuzyou/gitee-code
nyuuzyou/gitee-code
dataset
resource
https://huggingface.co/datasets/nyuuzyou/gitee-code
new_line
new_line
Gitee is not included in the Software Heritage archive, meaning it is currently missing from datasets like The Stack. This release fills that massive gap, serving as the largest Chinese code dataset and one of the largest code corpuses overall.
text
Gitee is not included in the Software Heritage archive, meaning it is currently missing from datasets like The Stack. This release fills that massive gap, serving as the largest Chinese code dataset and one of the largest code corpuses overall.
new_line
text
new_line
- 819,472,785 files from 3,105,923 repositories
text
- 819,472,785 files from 3,105,923 repositories
new_line
- 536 GB compressed Parquet storage
text
- 536 GB compressed Parquet storage
new_line
- 554 programming languages
text
- 554 programming languages
new_line
- Extensive quality filtering: Removed vendor code, artifacts, and generated files
text
- Extensive quality filtering: Removed vendor code, artifacts, and generated files
new_line
- Rich Chinese language understanding: High volume of Chinese comments and docs
text
- Rich Chinese language understanding: High volume of Chinese comments and docs
new_line
new_line
Huge thanks to Hugging Face for the storage grant that made hosting this (and all my other datasets) possible!
text
Huge thanks to Hugging Face for the storage grant that made hosting this (and all my other datasets) possible!
new_line
new_line
I have also already dropped several other new code datasets and rolled out QoL improvements for older ones. I will be dropping posts on those throughout the week.
text
I have also already dropped several other new code datasets and rolled out QoL improvements for older ones. I will be dropping posts on those throughout the week.
🇨🇳 Gitee Code Dataset - The Missing Piece of the Stack
https://huggingface.co/datasets/nyuuzyou/gitee-code

Gitee is not included in the Software Heritage archive, meaning it is currently missing from datasets like The Stack. This release fills that massive gap, serving as the largest Chinese code dataset and one of the largest code corpuses overall.
 
- 819,472,785 files from 3,105,923 repositories
- 536 GB compressed Parquet storage
- 554 programming languages
- Extensive quality filtering: Removed vendor code, artifacts, and generated files
- Rich Chinese language understanding: High volume of Chinese comments and docs

Huge thanks to Hugging Face for the storage grant that made hosting this (and all my other datasets) possible!

I have also already dropped several other new code datasets and rolled out QoL improvements for older ones. I will be dropping posts on those throughout the week.
🔥
unmodeled-tyler John6666 hanzceo calculatortamer WillHeld
🤯
calculatortamer
686c460ba3fc457ad14ab6f8
https://cdn-avatars.huggingface.co/v1/production/uploads/686c460ba3fc457ad14ab6f8/QFa608PO_fSX8WCmi_T1s.jpeg
Tyler Williams
unmodeled-tyler
user