distilbert-classn-LinearAlg-finetuned-span-width-4

This model is a fine-tuned version of dslim/distilbert-NER on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.9139
• Accuracy: 0.7937
• F1: 0.7904
• Precision: 0.8022
• Recall: 0.7937

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 2
• eval_batch_size: 2
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 4
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 25
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1	Precision	Recall
5.0288	0.6849	50	2.5823	0.0714	0.0442	0.0409	0.0714
5.0268	1.3699	100	2.5627	0.0238	0.0132	0.0094	0.0238
4.9407	2.0548	150	2.5352	0.0556	0.0482	0.0468	0.0556
4.9127	2.7397	200	2.5053	0.0714	0.0461	0.0403	0.0714
4.8177	3.4247	250	2.4931	0.0794	0.0443	0.0329	0.0794
4.7255	4.1096	300	2.4508	0.1111	0.1076	0.1194	0.1111
4.6022	4.7945	350	2.4190	0.1032	0.0814	0.0855	0.1032
4.5423	5.4795	400	2.3782	0.1429	0.1250	0.1246	0.1429
4.2497	6.1644	450	2.2977	0.1270	0.1066	0.1300	0.1270
4.1307	6.8493	500	2.0833	0.3016	0.2806	0.3279	0.3016
3.3278	7.5342	550	1.8698	0.3889	0.3504	0.3621	0.3889
2.9272	8.2192	600	1.6661	0.4921	0.4868	0.5582	0.4921
2.3628	8.9041	650	1.4250	0.5794	0.5722	0.5898	0.5794
1.8072	9.5890	700	1.2825	0.6190	0.6167	0.6262	0.6190
1.3522	10.2740	750	1.1679	0.6905	0.6925	0.7317	0.6905
1.0691	10.9589	800	1.0539	0.7460	0.7416	0.7598	0.7460
0.7661	11.6438	850	1.0198	0.7381	0.7390	0.7530	0.7381
0.5899	12.3288	900	0.9670	0.7540	0.7510	0.7725	0.7540
0.4413	13.0137	950	0.9887	0.7381	0.7364	0.7648	0.7381
0.3106	13.6986	1000	0.9224	0.7698	0.7645	0.7737	0.7698
0.2357	14.3836	1050	0.9266	0.7778	0.7700	0.7801	0.7778
0.211	15.0685	1100	0.9692	0.7460	0.7440	0.7610	0.7460
0.1573	15.7534	1150	0.9402	0.7540	0.7507	0.7589	0.7540
0.113	16.4384	1200	0.8856	0.8016	0.7982	0.8124	0.8016
0.0977	17.1233	1250	0.8920	0.8016	0.7952	0.8065	0.8016
0.0826	17.8082	1300	0.8878	0.7937	0.7912	0.8036	0.7937
0.0578	18.4932	1350	0.8937	0.8016	0.7973	0.8114	0.8016
0.0572	19.1781	1400	0.8905	0.7937	0.7905	0.7966	0.7937
0.0604	19.8630	1450	0.9128	0.7857	0.7819	0.7882	0.7857
0.0283	20.5479	1500	0.9102	0.8016	0.7979	0.8108	0.8016
0.0402	21.2329	1550	0.9044	0.8016	0.7985	0.8095	0.8016
0.0317	21.9178	1600	0.9108	0.8016	0.7982	0.8106	0.8016
0.0253	22.6027	1650	0.9134	0.8016	0.7998	0.8186	0.8016
0.0332	23.2877	1700	0.9136	0.7937	0.7913	0.7991	0.7937
0.0144	23.9726	1750	0.9141	0.7937	0.7904	0.8022	0.7937
0.0205	24.6575	1800	0.9139	0.7937	0.7904	0.8022	0.7937

Framework versions

• Transformers 4.48.3
• Pytorch 2.5.1+cu124
• Datasets 3.3.1
• Tokenizers 0.21.0