Zero-Shot Image Classification
OpenCLIP
English
clip
biology
CV
images
animals
species
taxonomy
rare species
endangered species
evolutionary biology
multimodal
knowledge-guided
Instructions to use imageomics/bioclip with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- OpenCLIP
How to use imageomics/bioclip with OpenCLIP:
import open_clip model, preprocess_train, preprocess_val = open_clip.create_model_and_transforms('hf-hub:imageomics/bioclip') tokenizer = open_clip.get_tokenizer('hf-hub:imageomics/bioclip') - Notebooks
- Google Colab
- Kaggle
Anomaly in pybioclip’s custom classifier
#6
by rays45 - opened
I have been working on pybioclip’s custom classifier on zebras. For testing purposes, I started with a very small dataset of just 10 images of zebras [containing 25 zebras : 18 plain zebras and 7 grevy’s].
- First, I ran the YOLOv8 to predict the bounding boxes [which it does correctly].
- Then on each bounding box, I applied the pybioclip’s custom classifier with 2 different set of labels [ In bioclip, the common name for plain zebra is half-maned zebra ]
- So I tried with labels : [“Grevy’s Zebra”, “Plain Zebra”] : With this configuration, the pybioclip classifies each zebra correctly [18 plains, 7 grevy’s]
- Then with labels : [“Grevy’s Zebra”, “Half-Maned Zebra”] : With this configuration, the pybioclip classifies the plains zebra correctly, but nearly half the grevy’s zebra are misclassified as plains/half-maned. [21 plains, 4 grevy’s]
The data and results can be found here: https://drive.google.com/drive/folders/16DJx6wHZIg07PpomJEyVyNdupTZdaZVV?usp=sharing
I am interested to know why and how this is occurring. Thank you!