Huggingface Pytorch_model.bin download corrupted

🤗Hub
1

I uploaded a model to huggingface, it has a pytorch_model.bin that is 12.2gb large, the sha256 is the same for website and locally but when i download it again, it is 11.7gb sha256 is different and its unopenable. I can confirm this behavior happens on a direct download, in kobold-ai colab and kobold-ai local. I couldn’t find anyone else having this issue, any ideas?

1 Like
2

I have also faced this issue multiple times…

possible solutions:

Git LFS (Large File Storage):

  • Hugging Face uses Git LFS for large files. Ensure you have Git LFS installed and configured properly. You can check by running git lfs install and then cloning the repository again.

    • Hugging Face provides a faster transfer tool called hf_transfer. You can enable it by installing the huggingface_hub library and setting the environment variable HF_HUB_ENABLE_HF_TRANSFER=1. This might help with large file downloads.
    • Instead of downloading directly, try using the Hugging Face Python library:

python

from huggingface_hub import hf_hub_download
hf_hub_download(repo_id="your_repo_id", filename="pytorch_model.bin")

Replace "your_repo_id" with the actual repository ID.

1 Like