NVIDIA Releases Kumo Tabular: Open Tabular Foundation Models That Predict New Rows in a Single Forward Pass

NVIDIA has released Kumo Tabular, a new family of tabular foundation models capable of classifying and regressing data by predicting new rows in a single forward pass. The system accepts labeled rows as context and generates predictions without requiring training, hyperparameter tuning, or feature engineering.

The release includes Small, Medium, and Large variants ranging from 28 million to 215 million parameters. These models are distributed through NVIDIA’s open-source structured-data-models (SDM) library.

Is it deployable?

Yes. The weights are available under the OpenMDW-1.1 license, which permits commercial use. The SDM code operates under Apache-2.0 and requires Python 3.11 or later alongside PyTorch 2.7+. Documentation indicates examples are targeted at CUDA GPUs.

What the SDM Library Adds

SDM is a GPU-native library designed for structured-data foundation models and preprocessing. Alongside Kumo Tabular, the library ships TabICLv2, Google’s TabFM, and KumoRelational for multi-table data. All models share a single in-context learning interface built on a TableTensor container. The library also handles preprocessing, ensembling, and many-class prediction.

How Kumo Tabular Works

Kumo Tabular functions as a Transformer built around the structure of a table. It employs column, row, and in-context attention mechanisms as introduced in TabICL and TabPFN.

The pipeline consists of three stages:

• Cell embedding: Numerical and categorical values pass through learned Fourier features with separate weights for each type. Missing values require no imputation.
• Row embedding: Column attention uses induced self-attention so cost grows linearly with rows. Row attention, with rotary positions, learns feature interactions. Four learnable [CLS] tokens compress each row.
• In-context learning: A final Transformer runs over row embeddings. Context rows attend to each other, while query rows attend only to context rows.

Because the context never sees the queries, its keys and values are computed once and reused. The head outputs class probabilities or 999 quantiles for regression, providing a point prediction plus an uncertainty estimate.

One detail matters at scale. Softmax attention spreads thin as the number of keys grows. Kumo Tabular scales each query by a temperature that grows with the log of the key count. The coefficient is learned per attention head, ensuring attention stays sharp on larger tables.

Trained Only on Artificial Tables

Kumo Tabular is pretrained entirely on synthetic tables sampled from Structural Causal Models (SCMs). A random causal graph links hidden variables through linear maps, small neural networks, trees, or Gaussian processes. The generator also injects messy, real-world patterns including missing values, high-cardinality categories, heavy-tailed targets, and conflicting duplicate rows.

Training ran in three stages similar to TabICLv2. Context grew from 1,024 rows to 60,000 rows with up to 100 columns. The Small, Medium, and Large variants saw approximately 35 million, 71 million, and 137 million artificial tables respectively. Classification and regression are trained as separate models. NVIDIA says the training recipe and data generators will be released soon.

Benchmarks

With default settings, Kumo Tabular ranks first overall on TabArena with an Elo of 1950. The NVIDIA team reports it runs 17x faster than LimiX-2 on a single RTX 6000 Pro. All three sizes sit on the accuracy and inference-time Pareto front.

• BeyondArena: First place with an Elo of 1418 and an Improvability score of 7.78%.
• TALENT: Top overall ranking with average ranks of 6.67 for accuracy, 3.98 for log-loss, and 4.22 for RMSE.
• ScoringBench: Large and Medium rank first and second on average rank.

Kumo Tabular vs Its Closest Competitors

The license row is the real differentiator. TabPFN-3, LimiX-2, and TabFM weights carry non-commercial terms. Kumo Tabular and TabICLv2 are the permissive options, and Kumo Tabular leads the benchmarks NVIDIA reports.

Getting Started

Install the library and pass a DataFrame through TableTensor, following the model card instructions:

# pip install structured-data-models
from sklearn.datasets import load_breast_cancer
import sdm

df = load_breast_cancer(as_frame=True).frame
table = sdm.TableTensor.from_pandas(
    df=df,
    stypes=sdm.infer_stypes(df, overrides={"target": "categorical"}),
    device="cuda",
)
model = sdm.models.KumoTabular(task="classification", device="cuda")
probs = model(
    x_context=table[:300].drop_columns("target"),
    y_context=table[:300, "target"],
    x_query=table[300:].drop_columns("target"),
    num_estimators=8,
)

The size argument accepts “small”, “medium”, or “large”, and defaults to large.

What it means

The commercial license is the primary advantage for businesses. TabPFN-3, LimiX-2, and TabFM restrict commercial use of their weights. Kumo Tabular allows commercial deployment without a paid license. The models also run within the same GPU-native library as TabICLv2, TabFM, and KumoRelational, simplifying infrastructure for teams already using the SDM stack.