Microsoft Foundry now hosts curated Hugging Face models

The platform and managed compute

Microsoft Foundry is a platform for building and operating agentic AI applications. It offers the widest model selection on any cloud, including options from Microsoft, OpenAI, Anthropic, Meta, Mistral, DeepSeek, Hugging Face, and others. These models span frontier, open-source, and custom weights, all accessible through a single endpoint and a single set of SDKs in Python, C#, JavaScript, and Java.

On top of those models sits the Foundry Agent Service. This includes multi-agent orchestration with built-in memory, knowledge grounding through Foundry IQ, and a catalog of connectable tools via agentic protocols. Agents can work with enterprise data once they are running. Foundry provides end-to-end tracing, real-time monitoring, continuous evaluations, and a prompt optimizer that improves agent behavior based on eval results. These observability and quality loops are part of the platform.

Developers also get access to content safety filters, task-adherence guardrails, an AI Red Teaming Agent for adversarial testing, unified RBAC, private networking, and Azure Policy integration directly within the platform.

Alongside pay-per-token and provisioned throughput, Foundry Managed Compute is the third deployment option in Foundry. It is a managed GPU platform-as-a-service for open-source and custom models.

You deploy a model instance described by the things that matter to your workload — parameter count, context length, and whether you want to optimize for latency or throughput. Foundry handles the GPU topology underneath, whether the instance lands on one accelerator or several. You think and plan in model terms.

Microsoft takes care of the machine. Container updates, runtime upgrades, and security patches happen automatically on the supported runtimes — vLLM, SGLang, TensorRT-LLM, NIM, TEI, llama.cpp — without redeploying your model. Model configuration, deployment behavior, and routing stay with you.

That consistency carries through the developer surface. Pay-per-token, provisioned throughput, and Managed Compute share a single endpoint, the same SDKs, the same authentication, the same observability, and a single bill.

Open-source models integrate with Foundry Agents the same way frontier models do. You can mix model types in a single agent without a separate integration path.

Managed Compute offers global deployments with the broadest capacity and best pricing, and Data Zone deployments for residency and sovereignty.

Same code, same workflow. Quota is aligned to accelerator families, so a plan built on the H100 family today carries forward as new hardware generations come online.

Why Hugging Face

Hugging Face is the public square of open AI. It hosts 15 million builders, 400,000 organizations, and over 3 million open models published. New frontier capabilities — agentic coding, video segmentation, speech, embeddings — land weekly. It is the GitHub of open models. The community publishes weights, writes model cards, compares evaluations, and pulls models for experimentation.

Open models have closed the gap with proprietary models on benchmark after benchmark. They unlock things proprietary endpoints can not.

• State-of-the-art is now open. Leading open-weight models are competitive with the top closed frontier models on the most widely used benchmarks.
• Deep customization. Full weights make it possible to fine-tune, distill, quantize, and adapt with LoRA. You tailor models to your domain, your data, and your latency and cost targets.
• Your model, your hosting. Weights run in your tenant on infrastructure you control, behind your inference endpoint, with your identity and network boundaries.
• Cost shaping. Pay for accelerators by the hour, scale to zero when idle, and right-size GPUs to the specific model. This is useful for steady, high-volume, or latency-sensitive workloads where per-token pricing is harder to predict.
• Version control. Pin a specific model version, evaluate it, deploy it, and move forward or roll back on your own release cadence.

The catch has always been the operational layer. Discovery, license review, security screening, runtime selection, GPU sizing, image building, CVE patching, and standing the model up behind an enterprise-grade endpoint are the hurdles. Hugging Face, by itself, is not an enterprise serving platform. Hugging Face models on Foundry is that operational layer, run by Microsoft.

Hugging Face Models on Foundry

The Hugging Face Collection brings a curated subset of models directly into the Foundry Model Catalog.

• Refreshed weekly. Trending models from the Hugging Face ecosystem are added continuously as the community publishes them.
• Every modality. Text, vision, audio, and multimodal: LLMs and VLMs for chat and agents, ASR and speech translation, embeddings, segmentation, image generation.
• Safetensors only, no untrusted code. Every model in the Collection is security-screened and ships in the SafeTensors weight format. There are no

  trust_remote_code

  execution paths unless rigorously reviewed.

• The right runtime for the model. vLLM and SGLang for LLMs, TensorRT-LLM and NIM where applicable, TEI for embeddings, llama.cpp for CPU. Foundry picks the engine that matches the model.

From your side, an open-weight model in the Hugging Face Collection looks and behaves like any other model in the Foundry Model Catalog. Every model in the Collection has been put through a multi-stage publishing pipeline before it ever shows up there.

The Curation Pipeline

Hugging Face and Microsoft work together to bring the most popular open-weight models from the Hugging Face ecosystem to Microsoft Foundry. They are production-ready for enterprise environments through a systematic curation process.

• Identify trending models in the Hugging Face ecosystem. This is based on community signals, partner requests, and customer demand. The team selects candidates for enterprise readiness.
• Screen for compliance and security. Model licenses are reviewed against Microsoft’s enterprise distribution policy. License metadata is captured and preserved on the catalog model card. Repositories are inspected for

  trust_remote_code

  patterns and custom executable code. Any model that would require executing third-party Python at load time is either remediated or excluded.

• Build, scan, and publish runtimes. Microsoft builds inference container images on supported runtimes (vLLM, SGLang, TensorRT-LLM, NIM, TEI, llama.cpp). They scan them for CVEs, and sign and publish them to a Microsoft-managed container registry.
• Upload weights to secure Azure storage. Model weights are pulled from Hugging Face once, validated against the published model card, and stored in Microsoft-managed Azure storage in the regions where the model is served.
• Validate and publish to the catalog. Every model + runtime + accelerator combination is tested for API conformance (chat completions, embeddings, rerank, etc.) and performance (latency, throughput, time-to-first-token, inter-token decode time). The validated model — with its templates, runtime images, and weights — is published to the Foundry Model Catalog with a one-click deploy path onto Managed Compute.

Because weights are pre-staged in Azure storage and runtime images live in a Microsoft-managed registry, your deployments won’t need outbound network access to Hugging Face Hub. You can deploy to production inside a private network.

Model Runtimes

Hugging Face models on Foundry are powered by a versatile collection of community-built, open-source inference runtimes. Each is selected and tuned for Foundry Managed Compute and matched to the model architectures it serves best. Across all runtimes, the systematic curation process means new versions and patches land on Foundry quickly. Existing model deployments are upgraded automatically without requiring you to redeploy.

• vLLM — the default high-throughput serving engine for open large language models, tuned for production GPU workloads. Because Hugging Face is a direct contributor to vLLM, any model in the Transformers library can run on vLLM out of the box. When a new model lands on Hugging Face, it can be served on Foundry the same day, with no waiting on a custom integration.

• SGLang — a serving engine for language and multi-modal models, with strong support for structured outputs (JSON, regex, grammar-constrained generation) that agentic and tool-using workloads depend on. Hugging Face and the SGLang team have built a Transformers backend integration for SGLang. Any model in the Transformers library runs on SGLang out of the box and reaches Foundry the same day it lands on Hugging Face.

• Text Embeddings Inference (TEI) — the runtime for embedding, reranker, and sequence-classification models. Accelerator-specific images ship with kernels compiled for each GPU and CPU family Foundry supports. This keeps the embedding hot path lean for RAG and semantic-search workloads.

• llama.cpp — the CPU and small-GPU path for GGUF-quantized models. It is useful for cost-optimized deployments, smaller models, and CPU-only regions. It has the same OpenAI-compatible API as vLLM and SGLang.

• TensorRT-LLM and NIM — used on NVIDIA hardware where NVIDIA’s optimized kernels and Triton-based serving deliver meaningfully better latency or throughput for specific model families.

• hf-serve — Hugging Face’s own multi-model inference server. It is used for model architectures outside the LLM and embedding fast paths (vision, audio, segmentation, and other Transformers-native pipelines) so the Collection can cover every modality with a consistent serving layer.

Deploying and Scoring an Open-Weight Model

The Hugging Face Collection in the Foundry Model Catalog is where you start. Deployment is five steps.

• Browse the catalog and pick a model. The deploy wizard also surfaces the model id, deployment template id, and

  acceleratorType

  you’ll need if you’re scripting the deploy via SDK or REST.

• Choose a deployment template. This is latency- vs throughput-optimized, accelerator family, context length, and quantization.
• Configure instance count. Scale throughput by adding model instances.
• Deploy. From the portal, CLI, SDK, or REST.
• Score via the unified Foundry endpoint with the SDK you already use.

Deployment Templates

A deployment template is the unit of choice in step 2. It is a named, versioned asset that pins the runtime, the accelerator family and count, the context length, and the runtime-specific tuning needed to serve the model well. Picking a template is the only knob you turn for how you want this model to run.

Source Read original →