Robbyant, the embodied-AI unit inside Ant Group, has released LingBot-Vision. This is a collection of self-supervised Vision Transformers designed for dense spatial perception. The weights are available under Apache-2.0 on Hugging Face in four sizes: ViT-giant, ViT-large, ViT-base, and ViT-small. A technical report and inference code are also included.
Most vision foundation models train for semantic invariance. They learn to answer what is in an image while discarding fine-grained spatial structure. Object boundaries, contours, and depth discontinuities are lost. Robots and other physically embodied systems depend on these details. LingBot-Vision inverts that priority. It treats boundaries as a native pretraining signal rather than a downstream output. The payoff is a 1B-parameter backbone that matches or surpasses models up to 7× larger on dense spatial tasks, including the 7B DINOv3.
What is LingBot-Vision?
LingBot-Vision is a self-supervised pretrained encoder for spatially structured downstream tasks. The flagship ViT-g/16 has roughly 1.1B parameters. It is trained with a new objective called masked boundary modeling on a curated corpus of about 161M images. This pool was selected from 2B web images. There are no human labels, no external edge detectors, and no pretrained backbone to bootstrap from.
The training is notably economical. The corpus is an order of magnitude smaller than DINOv3’s LVD-1689M. The model consumes less than a third of DINOv3’s training samples. The encoder outputs dense patch-token features intended for frozen readouts. For deployment at smaller budgets, the flagship is distilled into ViT-L (300M), ViT-B (86M), and ViT-S students. These lead dense prediction within their size classes.
How Masked Boundary Modeling Works
The method builds on the DINO/iBOT self-distillation paradigm. A teacher, an EMA copy of the student, generates online targets. The student recovers them from masked views.
Standard masked image modeling hides patches at random. It ignores what each patch depicts. A flat interior patch is cheap to recover from its neighbors. A patch straddling an object boundary carries structure that context alone cannot supply. Boundaries are the least redundant, most informative regions of an image. Random masking treats them like everything else.
LingBot-Vision closes that gap with two ideas.
Boundary-forcing. The teacher predicts a dense boundary field online. It identifies the boundary-bearing tokens B. These are forced into the student’s masked set on top of the random mask M. The combined mask is M⁺ = M ∪ B. Masked tokens are then routed by geometry. Boundary tokens receive an explicit geometric target in addition to the semantic self-distillation target. Interior masked tokens keep the standard semantic objective alone. This routing matters because a semantic target is inherently ambiguous exactly where two regions meet. The geometric target is well-posed precisely where conventional masked modeling is weakest. This allows semantic and geometric representations to co-emerge rather than compete.
Categorical boundary field. Boundaries are modeled as line segments lifted into a dense field. Every nearby pixel stores an attribute vector a(p) = (d, θ, φ¹, φ²). This records its distance to the nearest segment and three angles that locate it. Directly regressing this field in a teacher–student loop collapses. The fix is to discretize each channel into K = 32 bins. This recasts boundary prediction as per-pixel classification. It lets the boundary branch inherit the same centering and sharpening machinery that stabilizes modern self-distillation.
The categorical form has an elegant side effect. Under the classical a-contrario null hypothesis of “no structure,” boundary orientations are uniformly distributed. That null is now literally the uniform distribution over bins. Deviation from uniformity is evidence of a real boundary. A parameter-free Number-of-False-Alarms (NFA) test validates every decoded segment at no extra cost. The teacher exploits this at each iteration. It decodes candidate segments from its own field prediction. It keeps only the NFA-validated survivors. It re-renders them into the target field. Unsupported structure never becomes a teaching signal.
The full objective sums four terms:
L = L_DINO + λᵢ · L_iBOT + λᵦ · L_bnd + λₖ · L_KoLeo
Benchmarks and Performance
All dense results below use frozen features with a single linear layer. Performance is attributable to the representation rather than a decoder.
On NYU-Depth v2, LingBot-Vision posts the best RMSE of the entire comparison (0.296). It beats the 7B DINOv3 (0.309) with roughly 7× fewer parameters. It also beats the 2B V-JEPA 2.1 (0.307). On KITTI it is the best model below 2B parameters. On semantic segmentation it is on par with the distilled DINOv3 ViT-H+. It is 1.3 mIoU behind on ADE20K. It matches on Cityscapes. It is ahead on VOC12. It improves over the same-size DINOv2 by 4+ mIoU on all three benchmarks. The only remaining gap is to the DINOv3 family itself (2.4 mIoU on ADE20K to the 7B model). Its dense strength comes from distillation and dedicated dense-feature objectives.
Video object segmentation uses training-free label propagation over frozen features. LingBot-Vision reaches 70.0 J&F on DAVIS-2017 and 73.5 on YouTube-VOS. It matches DINOv3 ViT-H+ (71.1 / 74.0) and the 7B DINOv3 (71.1 / 74.1). It is the best among all remaining models at any scale. The boundary tokens themselves are stable enough to be tracked through video by plain cosine similarity of frozen features. There is no temporal supervision.
The trade-off is image-level recognition. ImageNet-1K linear probing reaches 86.32 and k-NN 83.39. It trails DINOv3-7B. That model spends its capacity on image-level invariance. The advantages also survive distillation. The 0.3B ViT-L student matches the 7B DINOv3 on NYUv2 depth (0.310 vs. 0.309) with about 23× fewer parameters.
Use Cases and How to Load It
The frozen patch tokens serve several dense workloads directly. Depth estimation reads geometry straight from the features. Semantic segmentation benefits from feature transitions that land exactly on object contours. Video object segmentation works through cosine-similarity token matching. The encoder also serves as the initialization for downstream depth-completion training.
Loading a backbone follows the official repository:
git clone https://github.com/robbyant/lingbot-vision.git
cd lingbot-vision
conda create -n lingbot-vision python=3.10 -y
conda activate lingbot-vision
python -m pip install -r requirements.txt
python -m pip install -e .
import torch
from lingbot_vision import load_pretrained_backbone, extract_patch_tokens, load_image
device = "cuda" if torch.cuda.is_available() else "cpu"
dtype = torch.bfloat16 if device == "cuda" else torch.float32
# Downloads model.pt from Hugging Face on first use.
backbone, embed_dim = load_pretrained_backbone(
variant="small", # giant | large | base | small; defaults to large
device=device,
dtype=dtype,
)
img_norm, _, _ = load_image(
"examples/example.png",
size=512,
patch_size=backbone.patch_size,
mode="square",
)
patch_tokens, patch_grid = extract_patch_tokens(backbone, img_norm, device, dtype)
print(patch_tokens.shape, patch_grid, embed_dim)
# torch.Size([1, 1024, 384]) (32, 32) 384
The variant argument selects the size and defaults to large. Output patch_tokens has shape [B, H*W, C]. Requirements are Python ≥ 3.10 and PyTorch ≥ 2.0. A GPU is recommended for the larger backbones.
LingBot-Depth 2.0: The Downstream Payoff
To show what a spatial-perception-native encoder buys downstream, Robbyant released LingBot-Depth 2.0. This model takes the frozen features from LingBot-Vision and trains a depth head. It achieves state-of-the-art results on NYU-Depth v2 and KITTI without using any depth supervision during pretraining. The approach proves that learning boundaries first makes depth estimation easier.
