NVIDIA AI Releases Nemotron 3 Embed: An Open Embedding Collection Whose 8B Checkpoint Ranks #1 on RTEB

Embedding models decide which passages an agent ever sees. NVIDIA released Nemotron 3 Embed model to work on that layer. It targets production-scale RAG, agentic retrieval, code retrieval, and agent memory.

What is Nemotron 3 Embed?

The model collection includes three open checkpoints. Nemotron-3-Embed-8B-BF16 is the accuracy-first option. Nemotron-3-Embed-1B-BF16 carries the same design into a smaller footprint. Nemotron-3-Embed-1B-NVFP4 is the Blackwell-optimized 4-bit path.

All three are transformer encoders trained with bidirectional attention masking. The final embedding comes from average pooling over token-level representations. Maximum sequence length is 32,768 tokens on every checkpoint.

Each model was evaluated across 34 languages. All three carry the OpenMDW License Agreement, version 1.1 (OpenMDW-1.1). Notably, the bases are Mistral models. The 8B is built with Ministral-3-8B-Instruct-2512. Both 1B variants use Ministral-3-3B-Instruct-2512.

Performance

Nemotron-3-Embed-8B-BF16 ranks #1 overall on RTEB (as of July 17 2026), the Retrieval Embedding Benchmark. Evaluation covers its 16 public tasks. Every figure below is average NDCG@10, at model sequence length 4096.

Two gaps are worth noting. The 1B gains 10.4 RTEB points over llama-nemotron-embed-vl-1b-v2, the prior-generation baseline. Separately, NVFP4 costs 0.38 RTEB points against its BF16 parent, or 99.5% retention.

How the 1B Model was Built?

Those 1B scores come from a compression pipeline, not a smaller training run. The parent was nemotron-3-embed-3b, pruned and distilled across two iterative rounds.

First, the 3B parent was pruned to 2B using NVIDIA ModelOpt mcore_minitron Neural Architecture Search (NAS). The search covers hidden width, FFN size, attention heads, and depth. It then picks the best candidate from the top-10 Pareto front. A 50k in-domain calibration corpus scored those candidates.

Next, the 2B model was distilled from the fine-tuned 8B embedding teacher. Distillation combined cosine distance loss (COS) and mean squared error (MSE) loss. The data blend was multilingual and in-domain. Finally, the same procedure repeated to produce the 1.14B checkpoint.

The NVFP4 Serving Tradeoff

Compression then continues into the serving format. Quantization hit weights and activations of linear layers only, targeting the NVFP4 data type. The research team used nvidia-modelopt v0.45.0. Quantization-Aware Distillation (QAD) followed, primarily to recover accuracy on long inputs.

Calibration used 512 samples: 256 queries and 256 passages from abisee/cnn_dailymail. QAD training used 20k samples.

The rsesearch team reports NVFP4 on Blackwell delivers up to 2x higher throughput than BF16. It retains 99%+ of BF16 retrieval accuracy. The NVFP4 card also documents dynamic embedding sizes. You can slice the 2048-d vector from the start to 1024 or 512 dimensions. Re-normalize afterward.

Interactive Explainer: The Five-Stage Retrieval Path

Before touching code, watch the path run. It animates prefixing, bidirectional encoding, average pooling, L2 normalization, and dot-product scoring. Scores come from each card’s published expected output.

Deployment Matrix

As that walkthrough implies, the checkpoints do not share runtime paths.

Alongside the checkpoints, NVIDIA research team released an optimized NIM microservice for the 1B model. The Rust-based NIM matches or outperforms the vLLM checkpoint on GB200 and RTX PRO 6000. NVIDIA tested input sequence lengths of 256 and 1024. Separately, NVIDIA NeMo AutoModel recipes cover fine-tuning and distillation.

Using It in Code

With those paths in mind, prefixes come first. Queries take query: and documents take passage: . Embeddings are L2-normalized, so dot product equals cosine similarity.

# pip install --upgrade "transformers>=5.2.0" "sentence-transformers>=5.4.1"
import torch
from sentence_transformers import SentenceTransformer

QUERIES = ["How can someone reduce exposure to pollen during allergy season?"]
DOCUMENTS = ["People with pollen allergy can reduce exposure by staying indoors "
             "on dry, windy days, avoiding early-morning outdoor activity, and "
             "going outside after rain when pollen levels are lower."]

model = SentenceTransformer(
    "nvidia/Nemotron-3-Embed-8B-BF16",
    device="cuda",
    model_kwargs={"dtype": torch.bfloat16,
                  # use "sdpa" if FlashAttention-2 is unavailable
                  "attn_implementation": "flash_attention_2"},
    processor_kwargs={"padding_side": "left"},
)
model.max_seq_length = 32768

q = model.encode_query(QUERIES, batch_size=1, convert_to_tensor=True)
d = model.encode_document(DOCUMENTS, batch_size=1, convert_to_tensor=True)
print(model.similarity(q, d))  # card's published q[3]/d[3] score: 0.8008

encode_query and encode_document read the saved prompts. So you never add prefixes by hand. For serving, /v2/embed applies them from input_type instead:

vllm serve nvidia/Nemotron-3-Embed-1B-NVFP4 \
  --max-model-len 4096 \
  --max-num-batched-tokens 4096 \
  --max-cudagraph-capture-size 4096

import numpy as np, requests

def embed(input_type: str, texts: list[str]) -> np.ndarray:
    r = requests.post(
        "http://localhost:8000/v2/embed",
        json={"model": "nvidia/Nemotron-3-Embed-1B-NVFP4",
              "input_type": input_type,          # "query" or "document"
              "texts": texts,
              "embedding_types": ["float"],
              "truncate": "END"},
        timeout=120,
    )
    r.raise_for_status()
    return np.array(r.json()["embeddings"]["float"], dtype=np.float32)

scores = embed("query", QUERIES) @ embed("document", DOCUMENTS).T

Use Cases With Examples

• Multilingual enterprise search: A support team indexes Hindi, Japanese, and English tickets together. Because retrieval is cross-lingual, a German query can surface a Japanese resolution note.
• Code retrieval: Training included coir_apps, coir_cosqa, synthetic_text2sql, and SWE-bench. Natural-language-to-code lookup is therefore closer to in-distribution.
• Agent memory: The 32,768-token limit lets an agent embed long conversation summaries without aggressive chunking.
• Cost-tiered RAG: Serve 1B-NVFP4 for high-volume recall, and route hard queries to the 8B. Because widths differ, this needs two indexes.

Key Takeaways

• Nemotron-3-Embed-8B-BF16 ranks #1 on RTEB at 78.46 avg NDCG@10.
• Three open checkpoints span 8B BF16, 1B BF16, and 1B NVFP4.
• NVFP4 retains 99%+ of BF16 accuracy at up to 2x Blackwell throughput.
• The 1B came from ModelOpt NAS pruning plus COS+MSE distillation from the 8B.
• All checkpoints use OpenMDW-1.1 and support 32,768-token inputs.

Check out the NVIDIA launch post on Hugging Face, Nemotron 3 Embed collection, 8B-BF16 card, 1B-BF16 card and 1B-NVFP4 card. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.

Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us

The post NVIDIA AI Releases Nemotron 3 Embed: An Open Embedding Collection Whose 8B Checkpoint Ranks #1 on RTEB appeared first on MarkTechPost.