Text Classification
Transformers
PyTorch
ONNX
English
roberta
text-classfication
int8
Intel® Neural Compressor
neural-compressor
PostTrainingStatic
Eval Results (legacy)
text-embeddings-inference
Instructions to use INC4AI/roberta-base-mrpc-int8-static-inc with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use INC4AI/roberta-base-mrpc-int8-static-inc with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-classification", model="INC4AI/roberta-base-mrpc-int8-static-inc")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForSequenceClassification tokenizer = AutoTokenizer.from_pretrained("INC4AI/roberta-base-mrpc-int8-static-inc") model = AutoModelForSequenceClassification.from_pretrained("INC4AI/roberta-base-mrpc-int8-static-inc", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -35,6 +35,8 @@ The original fp32 model comes from the fine-tuned model [roberta-base-mrpc](http
|
|
| 35 |
|
| 36 |
The calibration dataloader is the train dataloader. The default calibration sampling size 300 isn't divisible exactly by batch size 8, so the real sampling size is 304.
|
| 37 |
|
|
|
|
|
|
|
| 38 |
### Test result
|
| 39 |
|
| 40 |
- Batch size = 8
|
|
|
|
| 35 |
|
| 36 |
The calibration dataloader is the train dataloader. The default calibration sampling size 300 isn't divisible exactly by batch size 8, so the real sampling size is 304.
|
| 37 |
|
| 38 |
+
Embedding module **roberta.embeddings.token_type_embeddings** is fallbacked to fp32 due to *Unexpected exception RuntimeError('Expect weight, indices, and offsets to be contiguous.')*
|
| 39 |
+
|
| 40 |
### Test result
|
| 41 |
|
| 42 |
- Batch size = 8
|