TinyGuard1.1-22M

A small (22.7M parameters) English text classifier for basic security filtering. It labels a message as one of four classes: safe, spam, phishing or prompt_injection.

Results

Metric Score
Accuracy 0.9865
Macro F1 0.9850
Weighted F1 0.9865
Class Precision Recall F1 Support
safe 0.9906 0.9915 0.9910 2115
spam 0.9845 0.9878 0.9861 900
phishing 0.9732 0.9689 0.9710 900
prompt_injection 0.9922 0.9911 0.9917 900

Comparison vs TinyGuard1-22M

Metric TinyGuard1-22M TinyGuard1.1-22M Fark
Accuracy 0.9669 0.9865 +0.0196
Macro F1 0.9696 0.9850 +0.0154
Phishing Recall 0.9255 0.9689 +0.0434
Safe F1 0.9698 0.9910 +0.0212
Spam F1 0.9758 0.9861 +0.0103
Phishing F1 0.9458 0.9710 +0.0252
Prompt Injection F1 0.9870 0.9917 +0.0047

Usage

from transformers import pipeline
clf = pipeline("text-classification", model="Kuantum-ai1/TinyGuard1.1-22M")
print(clf("Congratulations! You won a free prize, click here to claim it."))

Training data

Same public datasets as v1.0, plus augmentation (typo, spacing, homoglyph, case) to balance classes.

Class Examples (after augmentation)
safe 21148
phishing 9000
spam 9000
prompt_injection 9000

Training setup

Fine-tuning with PyTorch on one T4 GPU: max length 320, batch size 64, learning rate 2e-5, 5 epochs, AdamW (weight decay 0.01), cosine schedule with 10% warmup, label smoothing 0.05, class weights, fp16. Best checkpoint selected by validation macro F1.

License

Apache-2.0

Downloads last month
51
Safetensors
Model size
22.7M params
Tensor type
F32
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for Kuantum-ai1/TinyGuard1.1-22M

Finetuned
(26)
this model

Datasets used to train Kuantum-ai1/TinyGuard1.1-22M

Collection including Kuantum-ai1/TinyGuard1.1-22M