A private chef arrived at my apartment wearing a baseball cap with a blocky camera attached to the brim. He was there to cook a three-course meal, but he was also there to record the entire experience as training data for robots.
I signed up for the free lunch in exchange for letting Shift capture the footage. Chef Ollie unpacked his supplies and began preparing a gazpacho. The hat camera tracked his hand movements with the goal of building a robot capable of culinary expertise.
This is part of a growing boom in egocentric data collection. Companies record human hand movements from a first-person point of view to teach machines. Shift made waves earlier this year by offering free cleaning services in New York City under a similar arrangement. The company expanded in San Francisco by offering free home-cooked meals. If I ever buy a skilled humanoid to coexist with at home, I hope its knife-wielding skills are up to par.
Shift is a division of the German startup Microagi, which focuses on deploying robots in real life. “The goal of Shift is twofold. First, allow people to join the AI economy,” says Microagi CEO Bercan Kilic. “The other part is to handhold the entire society and governments together, from today until the abundance.”
In the short term, Kilic sees this as a way to build a marketplace where contractors can make some cash recording videos and contributing to the improvement of AI models. Looking at the horizon, he believes impending tech innovations may usher in a new age of widespread wealth.
This year, I spent dozens of hours with an iPhone strapped to my head. I recorded my hand movements for DoorDash while scrambling eggs for breakfast and recorded my fingers tying shoelaces for another data collection startup. Shift offers a similar gig work setup for people who want to record and sell their own egocentric data. While my payouts have been skimpy, I was previously successful in monetizing my physical movements on other platforms. I felt curious about what it would feel like to be on the other side of the camera.
Free Lunch
The uniform for Shift chefs blends elements of a restaurant worker’s outfit with cyborg qualities. Ollie wore a white shirt and pants with a brown apron wrapped around his body. The camera attached to his hat was roughly the size of a GoPro, with a white wire running through the top and down his back. The wire was attached to a smartphone, where the recordings from this free lunch were stored. It jiggled a bit as he cooked.
I have never had a private chef before in my cramped apartment, so I felt a bit weird just standing by in the kitchen as he prepared a gazpacho appetizer and started chopping trumpet mushrooms. The finesse and dexterity displayed with his knife skills go far beyond what my rudimentary chops could help train a robot to cook. My sense of awkwardness only increased as I remembered that everything was being recorded. I should have scrubbed that oven more before it was immortalized as robot training data. Even if the robots do not mind a bit of mess, my human ego still prevails.
The apartment smelled amazing as Ollie got started on the main course, baked salmon with a creamy zucchini sauce, and I began to loosen up. Similar to how I respond when people use Meta’s AI glasses to record in public, I initially froze under the weight of being perceived so directly by technology. But as I chomped on the crispy salmon skin at my dinner table, I considered these recordings from a broader perspective. My kitchen would be just a single drop in the vast ocean of robot training data. I was beginning to get over myself and give in to the humanoid future.
Ollie finished the meal with a beautifully-piped tiramisu and immediately got started cleaning up the kitchen. What good would a humanoid robot be if it could cook you some Michelin-star-level meal but left you with a pile of crusty pans soaking in the sink? Collecting cleaning data can be just as critical as collecting cooking data.
Chef Bot-Yardee
Kilic says Shift plans to branch out and eventually add other free services, or at least offer them at discounted rates, in exchange for recording the experience. While the boom of large language models was powered by scraping the internet for text data, no comparable, vast repository of egocentric video data exists for training robots. That is why the inside of your home, alongside how to navigate it while completing tasks, is so valuable that a company would be willing to front you a private chef for a day.
He is optimistic that we will soon be living with humanoids. “By this time next year we will have quite good home robots that are at reasonable prices,” he says. “It won’t do all the tasks, but it will do most tasks quite well. It will be like vacuum robots in 2020.”
I am not worried about these bots gaining consciousness, but I am anxious about the ramifications of physically embodying AI models further and giving them tools. While a Roomba may smear dog poop all over the carpet when it messes up while you are at work, a nascent, knife-wielding chef-bot could endanger the dog, or even you, if it glitched out.
I am also skeptical that these robots, even if they eventually live up to the hype, will truly usher in an age of abundance. Yes, the Industrial Revolution changed aspects of the world for the better, but wealth inequality is worse than ever, and homelessness remains near record highs in the US. A revolution in robotics has the potential to improve baseline living conditions, without eliminating poverty.
Wealthy early adopters who are willing to spend thousands next year on home bots will have my apartment’s kitchen as well as chef Ollie to thank, in small part, for their humanoid-crafted meals. Until then, I have an abundance of leftover tiramisu in the fridge to enjoy.
