Tabular Regression
Safetensors
yft commited on
Commit
d5cd701
·
verified ·
1 Parent(s): 865c400

Mitra-v2 regressor weights and model card

Browse files
Files changed (3) hide show
  1. README.md +70 -0
  2. config.json +1 -0
  3. model.safetensors +3 -0
README.md ADDED
@@ -0,0 +1,70 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ pipeline_tag: tabular-regression
4
+ ---
5
+
6
+ # Mitra-v2 Regressor
7
+
8
+ Mitra-v2 regressor is a tabular foundation model that is pre-trained on purely synthetic datasets sampled from a mix of random regressors, including the new Hybrid SCM prior. It is the second generation of the Mitra regressor ([autogluon/mitra-regressor](https://huggingface.co/autogluon/mitra-regressor)), pre-trained with a 10x longer context, three times as many features, and an improved optimizer, and it replaces the scalar mean-squared-error head of Mitra-v1 with a 1,000-bin distributional head. On the TabArena and TALENT benchmarks it delivers state-of-the-art accuracy at the level of TabFM and EXAONE Tabular, while surpassing TabPFN-3 by a wide margin. The classification model is at [autogluon/mitra-classifier-2](https://huggingface.co/autogluon/mitra-classifier-2), and the inference and fine-tuning code with our evaluation results is at [autogluon/mitra-finetune](https://huggingface.co/autogluon/mitra-finetune).
9
+
10
+ ## Architecture
11
+
12
+ Mitra-v2 is based on a 12-layer 2D Transformer of 76.7 M parameters (attention across rows and across columns), pre-trained by incorporating an in-context learning paradigm. Regression is cast as classification over 1,000 target bins: the model predicts a distribution over bins and the point prediction is the mean of that distribution. Apart from this head the architecture is unchanged from Mitra-v1.
13
+
14
+ ## Usage
15
+
16
+ To use Mitra-v2 regressor, install AutoGluon and the `mitra-finetune` package by running:
17
+
18
+ ```sh
19
+ pip install uv
20
+ uv pip install "autogluon.tabular[mitra]>=1.6" "tabarena>=0.1.0"
21
+ uv pip install git+https://huggingface.co/autogluon/mitra-finetune
22
+ ```
23
+
24
+ A minimal example showing how to fine-tune and predict with the Mitra-v2 regressor using the same recipe as our reported results (50-step fine-tuning with 8-fold bagging). The recipe fine-tunes and bags eight copies of the model and requires a CUDA GPU; each `predict` call runs one bagged fine-tune:
25
+
26
+ ```python
27
+ import pandas as pd
28
+ from sklearn.model_selection import train_test_split
29
+ from sklearn.datasets import fetch_california_housing
30
+ from sklearn.metrics import root_mean_squared_error
31
+ from huggingface_hub import snapshot_download
32
+ from mitra_finetune import MitraFinetune
33
+
34
+ # Load dataset
35
+ housing_data = fetch_california_housing()
36
+ X = pd.DataFrame(housing_data.data, columns=housing_data.feature_names)
37
+ y = pd.Series(housing_data.target, name="target")
38
+ X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
39
+ X_train, y_train = X_train.iloc[:2000], y_train.iloc[:2000] # small subsample for a quick example
40
+
41
+ # Download the Mitra-v2 regressor weights
42
+ ckpt_dir = snapshot_download("autogluon/mitra-regressor-2")
43
+
44
+ # Fine-tune and predict
45
+ model = MitraFinetune(checkpoint_dir=ckpt_dir, problem_type="regression")
46
+ model.fit(X_train, y_train)
47
+ pred = model.predict(X_test)
48
+ print("RMSE:", root_mean_squared_error(y_test, pred))
49
+ ```
50
+
51
+ The `mitra-finetune` package is required for regression: stock AutoGluon's Mitra regressor expects the scalar head of Mitra-v1, whereas these weights carry the 1,000-bin head, which the package wires into AutoGluon at fit time.
52
+
53
+ ## License
54
+
55
+ This project is licensed under the Apache-2.0 License.
56
+
57
+ ## Reference
58
+
59
+ Mitra-v2 Technical Report (Amazon, 2026). The arXiv link will be added once the report is public.
60
+
61
+ ```
62
+ @article{zhang2025mitra,
63
+ title={Mitra: Mixed synthetic priors for enhancing tabular foundation models},
64
+ author={Zhang, Xiyuan and Maddix, Danielle C and Yin, Junming and Erickson, Nick and Ansari, Abdul Fatir and Han, Boran and Zhang, Shuai and Akoglu, Leman and Faloutsos, Christos and Mahoney, Michael W and others},
65
+ journal={arXiv preprint arXiv:2510.21204},
66
+ year={2025}
67
+ }
68
+ ```
69
+
70
+ Amazon Science blog: [Mitra: Mixed synthetic priors for enhancing tabular foundation models](https://www.amazon.science/blog/mitra-mixed-synthetic-priors-for-enhancing-tabular-foundation-models)
config.json ADDED
@@ -0,0 +1 @@
 
 
1
+ {"dim": 512, "dim_output": 1000, "n_layers": 12, "n_heads": 4, "task": "REGRESSION"}
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8c90b8f281a793f9feb19f8fc3c5529c58ac844dd7ffc690a0fdbb96fae0967c
3
+ size 306776952