Nums AI has released Causilo, a pretrained tabular foundation model that currently holds the highest Elo score among single models for both classification and regression tasks on TabArena.
The code is open source under the Apache-2.0 license, and the pretrained weights are available on Hugging Face. A scikit-learn interface allows immediate integration. Commercial deployment, production use, or hosted API access require a separate agreement with Nums AI.
How it works
Causilo operates as an in-context learning model. Calling fit does not update the underlying pretrained weights. Instead, the system stores training rows as context to predict query rows in a single forward pass. The model was pretrained exclusively on synthetic data, with no training on TabArena datasets.
Inputs accept NumPy arrays or pandas DataFrames, handling categorical features and missing values. Classification supports up to ten classes. Regression returns mean predictions by default, though version 1.0.1 adds median and quantile outputs derived from 999 native quantiles.
Architecture details
Nums AI describes the network in three phases: refinement, compression, and in-context learning.
Features group into sets of three. Each value embeds with 16 learned sine and cosine frequencies. Missing values receive their own learned vector.
Two column stages summarise each feature group. In each stage, 128 latent slots read only the training rows and pass that summary to every row. A row stage between the two column stages allows feature groups to interact through four latent tokens. The system uses cross-attention instead of full self-attention, which keeps costs linear in feature count.
A pooling block compresses each row into a fixed 512-dimensional vector. Labels attach to the training rows. A 12-layer prediction block lets query rows attend to those labeled rows. Query rows cannot change the training context or each other.
By default, eight ensemble members share the same weights. Each cycles through none, rank2gaussian, robust, or power normalization, with seeded feature and class permutations.
Benchmark performance
Nums AI used the official TabArena pipeline: 51 datasets and 816 Full splits, with eight estimators and seed 42. A TabArena maintainer re-ran the full evaluation and confirmed the overall Elo of 1794.
| Task | Causilo Elo | Next best single model | Causilo improvability |
|---|---|---|---|
| Overall | 1792.9 | TabFM, 1764.4 | 0.0684 |
| Classification | 1771.8 | EXAONE Tabular, 1758.8 | 0.0875 |
| Regression | 2032.6 | TabFM, 1992.8 | 0.0125 |
Competitors include Google Research’s TabFM, LG AI Research’s EXAONE Tabular, and Prior Labs’ TabPFN-3 (1636.2 overall).
Several points clarify these numbers:
- The top positions exclude system entries. When included, a maintainer re-run placed Causilo third of 88 overall.
- On improvability, TabFM leads overall and on classification. Causilo leads on regression.
- Elo confidence intervals at the top overlap, meaning the lead over TabFM and EXAONE Tabular is narrow.
- Nums AI also lists Xiaomi-TabLDM and Amazon’s Mitra-v2 behind Causilo. Neither model appears in the benchmark files within Causilo’s repository.
ScoringBench results
ScoringBench scores regression models with proper scoring rules such as CRPS, alongside RMSE and R². Nums AI submitted Causilo 1.0.1 on 101 datasets with five folds each, capped at 3,000 samples. The system reports that Causilo ranks first by CRPS, R², and RMSE. The ScoringBench maintainer independently checked the results before committing them.
Speed and memory
Nums AI reran three models on one H100 80 GB GPU, allocating eight CPU cores per job.
| Model | Fit (s per 1k rows) | Predict (s per 1k rows) | GPU memory (GiB) |
|---|---|---|---|
| Causilo | 2.504 | 0.251 | 8.15 |
| TabICLv2 | 3.449 | 0.303 | 8.37 |
| TabPFN-3 | 4.18 | 0.686 | 0.88 |
In this test, Causilo is fastest on both fit and predict. TabPFN-3 uses far less GPU memory. Setting use_kv_cache=True moves context work into fit, using more memory to speed up repeated predictions.
Installation
Causilo requires Python 3.10 to 3.12 and PyTorch 2.13 or newer. The first fit downloads the checkpoint automatically.
# pip install causilo
from causilo import CausiloClassifier, CausiloRegressor
clf = CausiloClassifier(n_estimators=8, random_state=42)
clf.fit(X_train, y_train)
proba = clf.predict_proba(X_test)
reg = CausiloRegressor()
reg.fit(X_train, y_train)
bands = reg.predict(X_test, output_type="quantiles", quantiles=[0.05, 0.5, 0.95])A demo Space is available on Hugging Face for testing.
What it means
For data scientists, the primary change is a shift away from heavy fine-tuning. The model accepts training data as context, meaning users can plug in new datasets without retraining weights. This lowers the barrier for applying foundation models to specific, small-scale tabular problems. However, the requirement for a separate commercial license limits immediate adoption by companies needing production-grade reliability.
