For creators and artists building the next generation of physical AI, the era of relying solely on internet text is over. The race to teach machines to manipulate the real world demands a new kind of fuel: high-fidelity data capturing actual physical interaction. Unlike the vast oceans of public text that trained language models, the data required for robotics is scarce, messy, and incredibly difficult to generate at scale. This gap is spawning a critical infrastructure business, where the bottleneck is no longer just computing power or model architecture, but the labour-intensive process of feeding robots reliable information about how to move.
XDOF, a startup that has just emerged from stealth, is positioning itself to solve this specific bottleneck. The company aims to construct the entire data pipeline—from collection tools to annotation systems—serving the frontier labs and robotics firms that lack the resources to build them in-house. To fund this ambitious operation, XDOF has secured $70 million in investment from Thrive Capital, Spark Capital, a16z, Lux, and WndrCo. The team, led by CEO and co-founder Philippe Wu, currently employs around 60 people and reports working with 20 clients, including several top-tier AI labs, though they have not disclosed their identities.
“We’ve already seen some of the downfalls of falling a little bit behind in the language model race … you don’t want to be in this type of situation where you pursue this technology too late, and everyone is in this boat where physical AI is the next frontier.”
Wu’s perspective on the necessity of this data layer was forged during his PhD studies at UC Berkeley. He encountered a classic catch-22: researchers could not train foundation models for robotics because the requisite large-scale datasets did not yet exist. “We didn’t have large-scale data to work with,” Wu explained. “There was this chicken-and-egg problem — we first needed to actually collect data before we could even ask how to train a foundation model for robotics.”
To address this, Wu and Fred Shentu, who now serves as CTO, developed a project called GELLO. This low-cost teleoperation system allowed human operators to control robotic arms remotely, generating the training data needed for machine learning. The approach proved so effective that it became a highly cited paper in the field, with many subsequent teams adopting similar hardware for data collection. Recognising the commercial potential, Wu, Shentu, and co-founder Nemo Jin, the Chief Operating Officer, launched XDOF in October 2024 to formalise this data ecosystem.
While data provision alone can be a dead-end venture, XDOF is building a self-reinforcing loop by also offering data cleaning, tooling, and annotation services. As a proof of concept, the company is partnering with UC Berkeley’s AI Research lab to release ABC, a dataset they claim is the largest of its kind ever assembled for academic use. The collection comprises 130,000 trajectories of robot manipulation data, 300 hours of simulation footage, and 100 hours of evaluation metrics. David McAllister, a Berkeley PhD student who helped organise the release, noted the typical impact of such open resources: “We’ve seen in language, image generation, and other fields, that when models and data are released, the community achieves things that you wouldn’t necessarily have expected.”
Using this new resource, the team has already trained robots to perform complex tasks such as folding T-shirts, flattening boxes, and loading AirPods into their cases.
Unlimited degrees of freedom
XDOF plans to operate across three tiers of a data pyramid. The most valuable layer consists of teleoperation data gathered directly on the specific robot being deployed. The second tier involves teleoperated robots collecting more general data, similar to the GELLO system. The final tier focuses on “egocentric” data—footage of humans performing everyday tasks—which XDOF intends to capture using custom-built wearable sensors.
Wu emphasises that hardware design is fundamental to data quality. “Your camera choice is going to affect the quality of your data — which is going to affect how your hand-tracking algorithm performs,” he said. “If you don’t design the hardware well from the start, the data you collect might have very specific problems that you didn’t anticipate.”
To execute this vision, the company plans to deploy armies of teleoperators and egocentric data operators globally. This labour-intensive model naturally raises the question of why major labs do not handle this production internally. The answer lies in the sheer scale required: “You need a warehouse of hundreds of thousands of square feet with hundreds of robots,” Wu noted. “You need to maintain these robots, calibrate their physical parameters, and properly train operators.”
The operational overhead demands a level of focus, capital, and scale that most research labs prefer to outsource, creating a clear market opportunity for XDOF. The company’s name itself reflects its ambition; it is a play on the robotics term “degrees of freedom,” which measures the independent motions a robot can execute. While a human arm has seven degrees of freedom and Figure.AI’s latest humanoid robot boasts thirty, the “X” in XDOF represents Wu’s goal: “Arbitrary degrees of freedom, unlimited degrees of freedom.”
Key takeaways
- XDOF has raised $70 million to solve the critical shortage of high-fidelity physical interaction data required for training capable robots.
- The company is releasing ABC, a massive new dataset containing 130,000 manipulation trajectories and hundreds of hours of simulation and evaluation data.
- XDOF is positioning itself as a specialised infrastructure partner, handling the labour-intensive and capital-heavy task of data collection and annotation for frontier AI labs.
- The startup’s strategy relies on a three-tier data pyramid, ranging from direct teleoperation to custom wearable sensors for egocentric data.
Stay ahead of AI. Get the most important stories delivered to your inbox — no spam, no noise.
