How Hugging Face Inference Endpoints, Jobs, and Buckets Power Search on Papers with Code

Papers with Code now embeds over 110,000 current papers sourced from arXiv and Daily Papers to power its search engine. This system relies on a hybrid architecture combining PostgreSQL full-text search with dense vector embeddings.

Searching for research differs from standard text searches. A useful engine must find an exact arXiv identifier or title, yet also understand a query like “small language models for code generation” even if those words do not appear together in the document. It must handle incomplete titles, typos, and cold model services without delay.

The solution uses a hybrid search approach. Keyword search locates exact mentions, while vector search identifies semantically similar terms. Rerankers, or cross-encoders, can improve results but add latency and overhead.

The architecture uses three specific Hugging Face services:

• Hugging Face Jobs provides burstable GPU compute for embedding the paper corpus.
• Hugging Face Storage Buckets offers durable storage for data handoffs between databases, experiments, and jobs.
• Hugging Face Inference Endpoints delivers low-latency embeddings for live queries and incremental updates.

The system separates offline corpus building from online search. Expensive batch work runs as Jobs. Durable artifacts live in a Bucket. Only the small query-embedding step sits on the request path behind a protected Inference Endpoint. If that endpoint is cold, busy, or unhealthy, the system immediately falls back to full-text retrieval. This separation keeps the system both powerful and fast.

Start with a strict embedding contract

Embedding pipelines often fail in subtle ways. A model revision might change, prompts could mix up, vectors might truncate differently, or an updated abstract may no longer match its stored vector.

The team avoids this by treating the embedding format as a versioned API. Every paper is encoded as:

normalized title + "\n\n" + normalized abstract

For each vector generation, the system records:

• the model repository and exact revision;
• the output dimension;
• the input-format version;
• whether the input is a query or a document;
• the normalization method;
• a content hash for the source title and abstract.

Production generation uses

Qwen/Qwen3-Embedding-0.6B

, pinned to an exact revision, producing 256-dimensional L2-normalized vectors. Newer models like Qwen3 allow two new features:

• Users can specify a dynamic embedding size to trade quality for speed and storage costs. Qwen models call this “MRL” (Matryoshka Representation Learning). The team chose a size of 256 to ensure fast search.
• Users can provide an instruction prompt. Qwen embedding models support a

  document

  prompt for embedding papers and a

  query

  prompt for live user searches.

This contract follows an embedding from export, through GPU inference, into PostgreSQL, and finally into online retrieval.

Jobs turn a database snapshot into a vector corpus

Full-corpus embedding is a classic batch workload. It needs a GPU for a short period, benefits from high throughput, and should not consume resources between runs. Hugging Face Jobs fits this shape well. A Job is defined by a command, a hardware flavor, and optionally a Docker image, and can run uv scripts with dependencies declared inline.

The corpus build starts by exporting the latest version of every paper from a repeatable-read PostgreSQL snapshot. The exporter streams rows rather than loading the catalog into memory, writes bounded JSONL shards, and creates a manifest containing row counts and SHA-256 checksums.

The team syncs that immutable run directory to a private Storage Bucket and mounts the Bucket directly (using hf-mount) into an

l4x1

Job (an NVIDIA L4 GPU with 24GB of VRAM). From the worker’s perspective it is simply a filesystem:

hf jobs uv run \
  --flavor l4x1 \
  --timeout 6h \
  --volume hf://buckets/OWNER/pwc-paper-embeddings:/bucket \
  embed_papers_job.py \
  --input /bucket/runs/RUN_ID/input \
  --output /bucket/runs/RUN_ID/output \
  --model Qwen/Qwen3-Embedding-0.6B \
  --revision MODEL_REVISION \
  --dimensions 256 \
  --allow-matryoshka

The worker:

• verifies the input manifest and every shard checksum;
• loads the pinned model revision;
• sorts texts by length to reduce padding;
• calls

  encode_document

  in batches;

• reduces the batch size automatically if the GPU runs out of memory;
• truncates the Matryoshka representation to 256 dimensions and normalizes it;
• writes float16 Parquet shards atomically; and
• records throughput, package versions, hardware, peak VRAM, row counts, and output checksums.

Each completed shard has its own marker, so a restarted Job can skip verified work. This is useful for a large corpus: retrying should just resume work rather than overwriting existing embeddings.

In a 5,000-paper pilot, the Qwen Job encoded about 75 papers per second at 1024 dimensions on an L4 GPU. The same pass could be deterministically materialized at 512 and 256 dimensions, allowing comparison of storage and retrieval trade-offs without paying for more inference.

Buckets are the connective tissue

Storage Buckets are mutable, S3-like object storage on the Hub, optimized for AI workloads. They can be accessed through

hf://buckets/...

paths and mounted read-write in Jobs without building a separate storage integration.

For this project, the Bucket is more than a place to put vectors. It is the boundary between three systems with different lifecycles:

• the production database exports source records;
• ephemeral Jobs consume those records and produce vectors;
• the importer validates the results before touching the search index.

The team organizes artifacts under immutable run prefixes:

runs/<run-id>/
├── input/
│   ├── manifest.json
│   └── papers-*.jsonl
└── output/
    ├── manifest.json
    ├── embeddings-*.parquet
    └── embeddings-*.complete.json

Buckets themselves are intentionally mutable, so immutability is an application-level rule: a run ID is never overwritten, and every artifact is covered by a manifest and checksum.

This gives several useful properties:

• Reproducibility: the team can trace a database generation back to an exact corpus snapshot, model revision, and set of artifacts.
• Safe retries: Jobs can resume from completed shards in the same run prefix.
• Cheap experiments: several models or dimensions can reuse one verified input snapshot.
• Controlled rollout: importing a generation does not activate it. The team first validates coverage and builds its index.
• Simple rollback: the previous generation and its artifacts remain available until the new one is proven stable.

Only after the importer rechecks schemas, checksums, dimensions, normalization, unique paper IDs, and current content hashes do the vectors load into PostgreSQL. The team then builds a separate HNSW index for the new generation and atomically marks it active only when every eligible current paper is covered. HNSW is the graph-based algorithm that enables fast vector search.

Inference Endpoints put semantic search on the request path

Batch embeddings solve the document side of retrieval. A user query still needs to be embedded at request time using the same model contract.

The team deploys the pinned model as an authenticated Inference Endpoint backed by Text Embeddings Inference (TEI). The endpoint accepts the query text and returns a normalized 256-dimensional vector using the model’s

query

prompt. Users could also leverage vLLM or SGLang here.

The API performs a cosine-distance search over the active pgvector generation:

SELECT paper_id,
       embedding <=> CAST(:query_vector AS halfvec(256)) AS distance
FROM paper_embeddings
WHERE generation_id = :active_generation
ORDER BY embedding <=> CAST(:query_vector AS halfvec(256))
LIMIT 50;

The HNSW index keeps this lookup fast. On the 5,000-paper pilot, the 256-dimensional Qwen index achieved 0.9955 Recall@20 against exact search, with 1.31 ms p50 and 2.21 ms p95 HNSW lookup latency. The table and index used about 27% of the storage of the 1024-dimensional version while retaining essentially the same ANN recall in that test.

The Endpoint is configured with a maximum of one replica and can scale to zero when idle. This is a useful cost lever, as the team is not paying when there is no usage. However, this means cold starts must be part of the application design rather than treated as an exceptional event, as it takes time for the endpoint to spin up and serve traffic.

The query client therefore has deliberately strict behavior:

• a one-second production timeout;
• a non-blocking concurrency limit;
• response dimension, finiteness, and norm validation;
• a short cache keyed by the query and embedding generation;
• a circuit breaker after repeated failures; and
• no raw query text in logs, only a normalized fingerprint.

If the endpoint is scaling up, times out, returns a malformed vector, or has no concurrency available, the team skips the semantic branch immediately. Users still receive lexical results instead of waiting for an unreliable dependency.

Inference Endpoints works really reliably, and includes a nice dashboard so you can quickly see key analytics.

Hybrid retrieval is stronger than either branch alone

For every query, the lexical branch retrieves up to 50 candidates using weighted PostgreSQL full-text search. The semantic branch retrieves up to 50 candidates from pgvector.

The team combines their ranks using weighted reciprocal rank fusion (RRF):

RRF is simple and robust