Mitra-v2 regressor weights and model card
Browse files- README.md +70 -0
- config.json +1 -0
- model.safetensors +3 -0
README.md
ADDED
|
@@ -0,0 +1,70 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
pipeline_tag: tabular-regression
|
| 4 |
+
---
|
| 5 |
+
|
| 6 |
+
# Mitra-v2 Regressor
|
| 7 |
+
|
| 8 |
+
Mitra-v2 regressor is a tabular foundation model that is pre-trained on purely synthetic datasets sampled from a mix of random regressors, including the new Hybrid SCM prior. It is the second generation of the Mitra regressor ([autogluon/mitra-regressor](https://huggingface.co/autogluon/mitra-regressor)), pre-trained with a 10x longer context, three times as many features, and an improved optimizer, and it replaces the scalar mean-squared-error head of Mitra-v1 with a 1,000-bin distributional head. On the TabArena and TALENT benchmarks it delivers state-of-the-art accuracy at the level of TabFM and EXAONE Tabular, while surpassing TabPFN-3 by a wide margin. The classification model is at [autogluon/mitra-classifier-2](https://huggingface.co/autogluon/mitra-classifier-2), and the inference and fine-tuning code with our evaluation results is at [autogluon/mitra-finetune](https://huggingface.co/autogluon/mitra-finetune).
|
| 9 |
+
|
| 10 |
+
## Architecture
|
| 11 |
+
|
| 12 |
+
Mitra-v2 is based on a 12-layer 2D Transformer of 76.7 M parameters (attention across rows and across columns), pre-trained by incorporating an in-context learning paradigm. Regression is cast as classification over 1,000 target bins: the model predicts a distribution over bins and the point prediction is the mean of that distribution. Apart from this head the architecture is unchanged from Mitra-v1.
|
| 13 |
+
|
| 14 |
+
## Usage
|
| 15 |
+
|
| 16 |
+
To use Mitra-v2 regressor, install AutoGluon and the `mitra-finetune` package by running:
|
| 17 |
+
|
| 18 |
+
```sh
|
| 19 |
+
pip install uv
|
| 20 |
+
uv pip install "autogluon.tabular[mitra]>=1.6" "tabarena>=0.1.0"
|
| 21 |
+
uv pip install git+https://huggingface.co/autogluon/mitra-finetune
|
| 22 |
+
```
|
| 23 |
+
|
| 24 |
+
A minimal example showing how to fine-tune and predict with the Mitra-v2 regressor using the same recipe as our reported results (50-step fine-tuning with 8-fold bagging). The recipe fine-tunes and bags eight copies of the model and requires a CUDA GPU; each `predict` call runs one bagged fine-tune:
|
| 25 |
+
|
| 26 |
+
```python
|
| 27 |
+
import pandas as pd
|
| 28 |
+
from sklearn.model_selection import train_test_split
|
| 29 |
+
from sklearn.datasets import fetch_california_housing
|
| 30 |
+
from sklearn.metrics import root_mean_squared_error
|
| 31 |
+
from huggingface_hub import snapshot_download
|
| 32 |
+
from mitra_finetune import MitraFinetune
|
| 33 |
+
|
| 34 |
+
# Load dataset
|
| 35 |
+
housing_data = fetch_california_housing()
|
| 36 |
+
X = pd.DataFrame(housing_data.data, columns=housing_data.feature_names)
|
| 37 |
+
y = pd.Series(housing_data.target, name="target")
|
| 38 |
+
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
|
| 39 |
+
X_train, y_train = X_train.iloc[:2000], y_train.iloc[:2000] # small subsample for a quick example
|
| 40 |
+
|
| 41 |
+
# Download the Mitra-v2 regressor weights
|
| 42 |
+
ckpt_dir = snapshot_download("autogluon/mitra-regressor-2")
|
| 43 |
+
|
| 44 |
+
# Fine-tune and predict
|
| 45 |
+
model = MitraFinetune(checkpoint_dir=ckpt_dir, problem_type="regression")
|
| 46 |
+
model.fit(X_train, y_train)
|
| 47 |
+
pred = model.predict(X_test)
|
| 48 |
+
print("RMSE:", root_mean_squared_error(y_test, pred))
|
| 49 |
+
```
|
| 50 |
+
|
| 51 |
+
The `mitra-finetune` package is required for regression: stock AutoGluon's Mitra regressor expects the scalar head of Mitra-v1, whereas these weights carry the 1,000-bin head, which the package wires into AutoGluon at fit time.
|
| 52 |
+
|
| 53 |
+
## License
|
| 54 |
+
|
| 55 |
+
This project is licensed under the Apache-2.0 License.
|
| 56 |
+
|
| 57 |
+
## Reference
|
| 58 |
+
|
| 59 |
+
Mitra-v2 Technical Report (Amazon, 2026). The arXiv link will be added once the report is public.
|
| 60 |
+
|
| 61 |
+
```
|
| 62 |
+
@article{zhang2025mitra,
|
| 63 |
+
title={Mitra: Mixed synthetic priors for enhancing tabular foundation models},
|
| 64 |
+
author={Zhang, Xiyuan and Maddix, Danielle C and Yin, Junming and Erickson, Nick and Ansari, Abdul Fatir and Han, Boran and Zhang, Shuai and Akoglu, Leman and Faloutsos, Christos and Mahoney, Michael W and others},
|
| 65 |
+
journal={arXiv preprint arXiv:2510.21204},
|
| 66 |
+
year={2025}
|
| 67 |
+
}
|
| 68 |
+
```
|
| 69 |
+
|
| 70 |
+
Amazon Science blog: [Mitra: Mixed synthetic priors for enhancing tabular foundation models](https://www.amazon.science/blog/mitra-mixed-synthetic-priors-for-enhancing-tabular-foundation-models)
|
config.json
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
{"dim": 512, "dim_output": 1000, "n_layers": 12, "n_heads": 4, "task": "REGRESSION"}
|
model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:8c90b8f281a793f9feb19f8fc3c5529c58ac844dd7ffc690a0fdbb96fae0967c
|
| 3 |
+
size 306776952
|