| --- |
| model-index: |
| - name: opt-350m |
| results: |
| - task: |
| type: text-generation |
| dataset: |
| name: Wikitext |
| type: wikitext |
| metrics: |
| - type: perplexity (BASELINE) |
| value: 25.420656250994373 |
| - type: perplexity (BASIC) |
| value: 25.56288418958359 |
| --- |
| This is a d-Matrix functional reference of the OPT-350M model. |
| The reference provides the following functional *configurations*: |
| Configuration | Explanation |
| :-- | :-- |
| **`BASELINE`** | a reference functionally equivalent to the original model |
| **`BASIC`** | all linear algebraic operands quantized to `MXINT8-64` |
|
|
|
|
| ### Usage |
|
|
| Install d-Matrix [Dmx_Compressor](https://github.com/d-matrix-ai/dmx-compressor) first. |
| ```sh |
| pip install dmx_compressor |
| ``` |
|
|
| The following is an example model and its evaluation. |
|
|
| ```sh |
| git clone https://github.com/EleutherAI/lm-evaluation-harness |
| cd lm-evaluation-harness |
| pip install -e . |
| ``` |
|
|
| ```python |
| from dmx.compressor.modeling import DmxModel |
| import lm_eval |
| from lm_eval.models.huggingface import HFLM |
| |
| lm_eval.api.registry.register_model("hf", HFLM) |
| model_args = "pretrained=d-matrix/opt-350m,trust_remote_code=True" |
| |
| lm = lm_eval.api.registry.get_model("hf").create_from_arg_string(model_args, {"batch_size": 1}) |
| |
| # Transform the model with DMX |
| lm._model = DmxModel.from_torch(lm._model) |
| |
| eval_results = lm_eval.evaluate(lm, lm_eval.tasks.get_task_dict(["wikitext"])) # Assign desired task, i.e. "wikitext" |
| ``` |
