Inside the Robot Data Factory: How Mecka Sensorizes Humans to Train Robots

Cold open

Josh Gao: I very much love the philosophy of Hitchhiker's — just how absurd the world could be, and how absurd the future could be. In this world that is full of robots, I would love to believe that there are going to be crazy entrepreneurs and people who believe that form factors like this would be interesting. I think the shape and size of robotics will look something like a world where we come up with wacky robots that can do anything you haven't even thought of yet.

Intro

Sonya Huang: Today I'm delighted to have Josh from Mecka on the show. Josh is founder and CEO of Mecka AI, which is building a data and deployment company for robotics. I, and I think the whole ecosystem, think that robotics is about to enter a golden age. And unlike LLMs, where we had so much free human data from the internet, robotics is fundamentally data-constrained. Your position in the ecosystem as the leading data factory — bringing robotic data and physical data in distribution for models — is a really, really interesting perch. So I'm excited to use this conversation to chat about how egocentric data and data collection work in robotics, what the actual operations and economics of running a data factory are, and your overall thoughts on robotics research and deployment. Thanks for joining the show.

Josh Gao: Amazing. Thanks for having me.

Main Conversation

Sonya Huang: To start off, for those who aren't familiar, what is Mecka, and how are you looking to impact the world of robotics?

Josh Gao: Mecka is, like you said, the data and deployment company for physical AI. What that means is that today we create, we process, and ultimately we license a lot of the data that goes into training these robotics foundation models.

The big problem across robotics is that we don't have the luxury of the cold start that LLMs had with the internet. Fundamentally, robotics data is three things: it's pixels, it's actions, and it's logic. And we don't have access to a large amount of this curated, processed, clean data to train these models to understand what it is to be in the physical world.

So a lot of our job is, one, to create the different types of hardware — sensorizing and wearables for humans; two, the operations to go collect and partner with individual contributors and businesses across tasks and environments; and then ultimately to post-process and own the quality gates that make these datasets useful and teach robots about the world.

Sonya Huang: Without having been given the benefit of a giant, free internet corpus — many trillions of tokens, trillions of trillions of tokens — do you think robotics, having to pay for and operationally collect all the data that goes into the mix, is even a feasible challenge? Is it economically feasible without the gift of the internet?

Josh Gao: I think we still have the luxury of a lot of things that were given to us by the internet, so we aren't starting from zero. You can see it across LLMs and LLM data: pre-training data was always driven by the labs, largely through web crawlers and things like that. It didn't become a large market. Today in robotics, we do see pre-training as quite a large market, and that can take several forms. It could be that we're teaching video models how to generate more physics-grounded realities of what the world looks like, and we get to leverage all the work that has gone into LLMs and different types of world models before to make that possible.

Is it possible and scalable to get to that volume of data? I think that ultimately comes down to what commercial use case you can unlock at which level of scale. Our strong belief is that along that journey, we'll be able to unlock a lot of the simple pick-and-place tasks at the beginning of what we're creating, and then move toward that long tail — which is going to be many orders of magnitude larger than it is today.

Sonya Huang: I did a little homework for this. Your website is very whimsical. It says that you are teaching robots about carbon life. What does that mean?

Josh Gao: I actually made the website the night before we did our first tweet announcement or something like that. There's an idea in science fiction that when robotics and AI have fully generalized and solved all the tasks humans can do, the final frontier is going to be organic. The final frontier is going to be biological. You see this in Asimov. You see this in Ender's Game. Even in Hitchhiker's, at the very end, the planet was this giant organic computer trying to process and understand what the meaning of life was, or what the question was. In the same vein for us: the human brain runs on just 20 watts. We are self-healing. Along the journey of teaching robots to fully generalize, we will slowly show them what carbon life is made of.

Sonya Huang: You come from an untraditional background for robotics. I think you started in the fintech world and the sneaker world, if I'm not mistaken. Tell me about how you and your co-founder found your way to robotics.

Josh Gao: It was quite an interesting journey. Before this, I was building essentially a Toast for cloud kitchens. That meant we did everything from back-of-house order management to — at one point in my fourth year of university — skipping class to go deploy and install kiosks in rural Rochester, New York, where we had to stay up all night replacing front-of-house systems for restaurants.

My co-founder built a stablecoin. They ultimately sold to Coinbase. He's this very obsessive personality — he was top 1,000 in four competitive games, including League of Legends. A big part of what brought us together: one, we're both Canadian, and two, we both have this big science-fiction grounding. So the belief was that payments was a hard and difficult problem, but whatever we worked on next, we wanted it to be something exciting — something where the future was going to be incredible. And robotics seemed like a great space to get into.

Sonya Huang: I remember walking into your office and immediately seeing the mechatronic robots and the Kawasaki Corleo on the wall. The inner child in you must be very happy about what you're doing these days.

Josh Gao: Yeah, absolutely.

Sonya Huang: Coming from a non-robotics background, what do you think your advantages are in building this business, and where has it presented challenges?

Josh Gao: The advantage is that when we entered the space, we had to go do this full discovery process. We reached out to hundreds of folks in the space whom we appreciated. We read every paper we could get our hands on. We went to visit the different robot labs. What we saw in late 2024, roughly, was that GPT was taking off and robotics was still evolving out of the traditional-robotics era. It seemed obvious to us, at the very least, that data and compute would become very important across the industry.

Moving toward that, we got to read papers like Ego4D with a fresh lens and say, hey, there are about 4,000 hours of this human demonstration data. And when you actually went in to look at the data, you'd realize some of the datasets were the daily lives of folks — bowling, for example. There were very broad use cases and potential applications at the 4,000-hour scale. Coming in with a fresh lens, we were able, from a first-principles approach, to have a general intuition of what would be important, and then iterate very closely with people who were much smarter than us, who had different views of what the future would be, and stay close enough to iterate on what the product would become.

Sonya Huang: Interesting. It seems like robotics has changed so quickly as a field that coming from the field can be both an advantage and a disadvantage, because people come from different eras of what's important in the training mix. For our audience, say a word on what egocentric data is. You really started your company making a bet on it — obviously it's much more distributed now — but what is egocentric data, and why did you decide to bet the company on it at the beginning?

Josh Gao: Egocentric data is video data of humans demonstrating and doing tasks from a first-person POV. The high-level idea was that, in the gap between human data and robotics data, you could close a lot of the vision gap: robots experience the world from POV, so can we also show them what it's like to interact with the world from first person?

Why we bet on it: when we explored the space, most robotics data at the time was teleoperation — humans controlling robots to do demonstrations. It was very hard to scale. And from what we saw across GPT and the principles of scaling, we had to find ways to scale up data much more significantly. Some of our early intuitions were that humans and robots share a lot of similarities. Humanoids were just starting to take off. Humans and humanoids look the same. Why couldn't we turn humans into the kind of fleet-style learning that Tesla has, so they could all contribute to understanding and grounding the physics of the world?

Sonya Huang: And what does that look like? Are you mounting an iPhone on somebody's forehead and letting them go about their day? What does it look like in practice?

Josh Gao: When we first started, our very earliest pilot was a mobile app. We got all the Chinese aunties and uncles to record themselves cooking, cleaning, doing tasks. That obviously evolved a lot. We tried a number of things — we did UMI data, we did all sorts at the beginning — and that's evolved quite a bit into all the custom hardware we now have to produce, and all the partnerships we have across different industries. Ultimately we're sensorizing humans; that's the broader category here.

Sonya Huang: How important is it to get force data from the sensors? One thing I've always thought about is that in self-driving, you don't really need force data as much. Whereas with robotics, especially manipulation, you really care about how hard you're squeezing that door handle or picking up that egg. It seems to me that video doesn't quite capture that — you might approximate it, but it doesn't really capture it. How important is capturing force data, and how do you think about getting it?

Josh Gao: Some of the substitutes for force data we've seen are approaches like wrist cameras — on both humans and robots — where you can see the contact time between the hand and the objects you interact with. That is obviously kind of a hacky way to get to how many newtons of force it takes to pull open a chair or twist a bottle cap.

I think force will be incredibly important. It's actually one of the products we've been building out. When you iterate with many different robotics foundation labs, you realize that to close this gap, we have to create new hardware. So we've been developing things like gloves for humans to wear, and in those gloves, collecting information on how much force it takes to do your daily life and different tasks.

Sonya Huang: I'd love to spend a bit of time on the actual operations of a data factory. So much money has gone into the LLM data factories, and so much money is starting to be poured — and I think will be poured — into the robot data factories. The way they work is mysterious and intriguing to me. So I'd love to ask you about the operations, if that's okay.

Josh Gao: Yeah, let's do it.

Sonya Huang: How does it work? If I want to make an extra buck, do I just sign up on your website, put a camera on my forehead, and start sending you data?

Josh Gao: We actually have a lot of different approaches on the operator side. One is a direct-to-consumer approach: through mobile apps, can you sign up from anywhere in the world, start contributing, register which profession you're in, and get streamed a sequence of tasks to do — cleaning your bedroom, cooking your breakfast, cleaning up afterwards? That's a very lightweight way to get the diversity of this pre-training data.

The second approach: there is so much more beyond the home to collect. So we had to build out a B2B engine and the muscle to recruit businesses across this incredible taxonomy of all physical human work — bakeries, a PCB manufacturing line, everything.

Sonya Huang: Podcasting.

Josh Gao: Podcasting. There's not too much hand work to make a quick buck here — we should have equipped ourselves.

That's how contributors come into the mix. But the actual data engine for the data factory is much more sophisticated, and much more painful, than folks expect. A lot of what we have to distribute into the world didn't exist until we designed and produced the hardware. Tactile gloves are a fantastic example: there is nothing off the shelf that you can purchase and wear that reliably tells you how much force you're applying day to day and turns that into ready datasets for these robotics foundation models. So we have to run that hardware iteration loop — three or four product cycles, the engineering muscle to think and iterate on what's going to be useful — and then scale production into the world.

That becomes the logistics of distributing into thousands of homes and thousands of businesses, collecting, and uploading. We've had to set up 100-gigabit upload lines in a lot of the places we operate just to sustain the hundreds of petabytes of data we deal with at this point. Then you have to process it, make sure your quality gates are incredible, and then do all the computer vision to make it useful.

Sonya Huang: Let's say you decide, okay, in my training data there aren't enough podcasters — or, I don't know, donut bakers. Do you literally go knock on donut bakers' doors to sign them up to the platform? How does that work?

Josh Gao: We'll generally track a heat map across different jobs, industries, and the ways we think about diversity. And if that heat map says, oh, not enough Sonyas podcasting, we have to go get that. That's actually why we're here today. This is our secret mission. That's why there are cameras everywhere. No.

Sonya Huang: You're never automating me out of this. No — I'm kidding. Please do. Interesting. I've heard there are a lot of quality issues in robot data collection generally, and that a lot of the hours of robot data that get sent in end up not being accepted. Why is that, and what are you doing about it?

Josh Gao: This lesson came originally from the different types of robot data. There have been very large-scale robot data factories spun up as large warehouses repetitively doing tasks, trying to scale up teleoperation data. Some of the learnings there came from the research: there's some threshold of diversity beyond which you oversaturate how much robot data you need, or how much data in general you need.

Across human data, quality is actually quite a moving target. As we grow, the edge cases you have to deal with in the real world are very extreme, and you will always see something new. So for us it's been: can you think about the heuristics to catch a lot of different things as data is processed? Can you build engines to escalate very quickly? Ultimately we learn new things about quality and diversity that weren't true even a week ago. So it's the nimbleness to move very quickly and adapt your engine to catch that.

Sonya Huang: What's an example of something you'd catch?

Josh Gao: I think fraud is an underestimated one across data industries in general. We're creating data that doesn't exist yet, versus filtering through the internet. And so you come across very creative —

Sonya Huang: People are sending generated videos, or what?

Josh Gao: Very creative people. At one point we had folks recording YouTube videos — like "POV: working at McDonald's" — and trying to submit them as contributions.

Sonya Huang: Wow. Never underestimate how entrepreneurial people can be. That's amazing.

Josh Gao: Very much. These have been part of the painful lessons. Yes, you have to create a lot of the gates, but you also have to be there where collection is happening. You have to go see how it's done. You have to figure out an incentive model that actually incentivizes people to make good contributions.

Sonya Huang: How do you think about your right to earn a margin long term? I could imagine a mega-competitor coming to Sonya and saying, "Sonya, I really want your podcast data." I could imagine that the margin you're able to own as an intermediary ends up getting squeezed.

Josh Gao: I think the contributor payment is actually not the largest factor in the end-to-end cost of making sure this data is useful. Even though that baseline is always going to be there — and maybe it becomes more and more competitive — there will be ways to differentiate and make sure your data is verifiably high quality at very large scale, with the right hardware, the operations, and the software.

When we think about these products, we do want to scale them up and mature them. Like any product that begins to scale, you want to optimize it, but over time you may have different products that evolve. So when you think about egocentric data of podcasting as simply one of your SKUs, you realize there are going to be different types of data that become important. Maybe we move from the Chinese restaurant on the side of the road to Michelin-star restaurants over time. Can you get closer and closer to what it means to be Federer, rather than an average tennis player? These products will change, and how they're created will change too.

Sonya Huang: Really interesting. What scale of data has already been collected, do you think, and how much do you think will be collected over the next few years?

Josh Gao: I think we've finally entered a world where we're in the millions of hours of data. Coming from Ego4D at 4,000 hours two years ago, that's a huge step up. But comparatively — think about autonomous driving. Tesla spent over a decade solving for two degrees of freedom, acceleration and direction, with millions of cars on the road. They have a lot more than a few million hours of video data at this point. So relatively, we're still many orders of magnitude away from where we have to be.

Sonya Huang: On one hand, that sounds like opportunity for you — a lot of revenue for the robot data factories. On the other hand, as somebody who invests in robot foundation models, I look at that and say, oh shoot, are we ever going to get to the promised land? Because I've done the same math: a car has two or three degrees of freedom. A robot has —

Josh Gao: Even dozens in the hand.

Sonya Huang: Exactly. Combine that with probably needing force data, and it's just a way harder environment to operate in. I get to: you might need billions, trillions, quadrillions of hours. Is there even enough GDP in the world to support the cost of bringing that in? What do you think?

Josh Gao: I think it's about unlocking commercial use cases as we go. It doesn't have to be binary — one day we've unlocked robotics or we haven't. It's going to be a slow creep up into the world.

A great way to get there: with some of this base of the pyramid built up, we're seeing generalization, we're seeing scaling laws at play. That gives people the courage to scale up. And with that courage, can we, one, get some generalizability in some of these environments? Can we then also collect the in-domain, in-task data that closes the gap in post-training? Maybe I can collect data from exactly this room, exactly this podcast demonstration, to close the gap to what is in a lot of cases four nines of reliability in these real-world environments.

Sonya Huang: That's a helpful framework. It's not a binary of getting from zero to one. We're going to increasingly get robots that can do an increasing set of capabilities.

Josh Gao: And we will unlock a lot of tasks along the way.

Sonya Huang: One curiosity, though: the data that's most plentiful right now seems to be home data. Everyone can film that; it's pretty easy to get. But the robot in the home seems like one of the most difficult environments to deploy in. So I'm curious which applications you think will hit first in robotics, because where the data is most readily available seems almost the opposite of where robotics might be easiest to deploy.

Josh Gao: The home is going to be one of the environments we have to be most diligent about, by far. A lot of the benchmarking and evals have just started to mature across robotics — making sure the robot doesn't fall on the baby in the home, for example. I think the initial tasks will be much more constrained, and very goal-oriented, where you can hill-climb on tasks that have existing SOPs for what it means to do that job.

Sonya Huang: Like a laundry-folding robot.

Josh Gao: It could be laundry folding. We spent about a year and a half plus trying to solve laundry folding, and I think it absolutely will get solved in the near term. A lot of the pick-and-place and packaging tasks you see on assembly lines — the manipulation tasks that couldn't be solved by traditional robotics — we will get there within the next year or two.

Sonya Huang: That's a good transition to deployment. I'd love to talk about robot deployment as a concept. You've said that useful robots can already be deployed today. Where do you think that's the case?

Josh Gao: I think people underestimate how diverse robotics can be. Humanoids are incredibly popular, and they'll be a very popular terminal state of robotics as well. But beyond that, there are so many different form factors that exist in robotics today. Look at hotels in China or Southeast Asia, for example, where robots are doing services, doing cleaning, delivering from the front desk to your hotel room door after your Uber Eats arrives. That exists. Many of these bots are widely deployed across the world today, and they're just starting to get popularized across the US and North America.

I think some of the first bots that have to get deployed will be the industrial cleaning bots, for example — a lot of the thematic robotics that are out there — and then slowly stepping up from there, where you start to solve early manipulation tasks along that spectrum, all the way up until we get to the humanoid solving any task out of the box in the future.

Sonya Huang: You mentioned Tesla and FSD earlier. When do you think we'll get to the point where there's an FSD-like flywheel, and when we get there, how do you see your role as a company evolving?

Josh Gao: When will we get there? I think we're a while away, but it'll be a slow transition. As you slowly solve some of these commercial tasks, you'll also finally get some of these flywheels — the in-deployment data where you get the failure cases and the correction data. That will be absolutely important in starting up the robot flywheel. How far off it is depends on that spectrum, and on saturating the different types of tasks and domains that aren't solved yet. So it's quite a ways away.

When we do get there, I believe Mecka will look very different. One, we're not fully subscribed to only egocentric data. We believe that a lot of different SKUs — the entire data soup — are important. Will we be producing the Michelin-star data at some point in the future? Will we be teaching some edge case about the real world that the robots haven't gotten to yet? If we believe the world is incredibly vast and rich in detail, there's going to be no shortage of things to teach.

And even in that world, we have this angle where we believe Mecka will very much contribute to the deployment of robots. In some of these environments, in closing the gap of being able to do that task in that domain, can we be the platform where we collect the domain data, we post-train, we support the enterprises, and white-glove them into what it means to be in a world filled with robotics? We see Mecka maturing into that.

Sonya Huang: I'd love to get your predictions on how deployment and diffusion are going to play out in the physical world. If I can analogize to what's happened in the LLM world, there are a few different ways deployment has happened. There are vertical-specific application companies. There are the foundation model companies themselves scaling up go-to-market. And then there are all the FDE, deploy-code, service-code — whatever you want to call them — armies of people helping enterprises adopt AI at scale. Those seem like the broad vectors disseminating tokens across the economy. What do you think the analog will be in the physical world?

Josh Gao: We believe very much in both the vertical and the horizontal across robotics. The full verticals — very task-specific, all the way from hardware to models to data — are tackling it. We also believe in an ecosystem with vast and different shapes and sizes of hardware and robotics, and different types of open models in the world.

I think it will look something like a modern robotics version of a systems integrator. In the traditional version, you're installing the product; it's incredibly competitive, with tight margins. There's now a world where you can install and help make that product more data-driven and productive over time. Can the bot you installed last week do something it didn't think was possible a few days ago? Can it help you expand the scope of what your business is capable of? That's the milestone we're aiming for.

Sonya Huang: Okay, but you'd tell me, as an investor — give me some investing advice — you'd invest in both the general and the vertical robots, then?

Josh Gao: Yeah.

Sonya Huang: What are your favorite vertical robot categories?

Josh Gao: I think the best ones will be the industrial bots to start. Those already have a lot of traditional robotics in the mix and are task-constrained, like we talked about.

Sonya Huang: Like data-center maintenance robots.

Josh Gao: Yeah. You just don't have to expect a baby to appear and think about not falling on it. The world is much more constrained than in the wild.

Other categories we're excited about are spaces that are feasible and possible early on. Some of the approaches in hotels, for example — where you could start delivering from the front desk to the hotel rooms and have some manipulation to hand off — are exciting to us. And then places where humans simply aren't capable. A joke we make at some of our company all-hands: if one day Mecka was able to help deploy a robot that mines an asteroid, we've probably done a pretty good job. It's a job and a task that completely doesn't exist, that humans can't do, that we have no idea how to do yet — but it seems inevitable.

Sonya Huang: I love that. It reminds me — I'm not thinking interplanetary yet, but have you heard of Gecko Robotics, what they do to help maintain ships and things like that? It's really cool.

Josh Gao: Absolutely.

Sonya Huang: Can you say a word on whether you're willing to share publicly the slate of hardware you've developed, and where the labs are currently most interested in collecting bespoke data?

Josh Gao: Some of the iteration came from when we started with the mobile app. We were quickly able to scale that, and there were a lot of virtues in that product. The downside was that you don't get as much rich information about the world as you could. So some of the things we had to develop very early on were about, essentially, Waymo-fying people. In this sensorization, can you build things like baseball caps equipped with different cameras and sensors? We build a central hub that connects to all these different accessories — the tactile gloves, external cameras, and more — all of which is trying to capture how you interact with the world day to day, and hopefully clone that behavior.

Sonya Huang: Interesting. What is EgoVerse?

Josh Gao: Professor Danfei Xu and Ryan Punamiya are some of the earliest cohort of researchers who believed in egocentric data. We had the privilege of being an early contributor to that consortium of folks who think scaling up human data will be important for robotics. So, one, we created and contributed a large amount of different types of egocentric data across businesses, industries, and home environments. We shared all of the action trajectory data — the post-processing, the computer vision, and the quality gates. And we contribute all the platform and web tooling: the data loading, the infrastructure.

Their vision is that, ideally, we can bring together a lot of the robotics industry — because hopefully we can lift everybody up in this wave and share the tooling, the data, and the lessons, just because we are so far behind as an industry.

Sonya Huang: Is it fair to say the robotics industry has kind of converged on a training data mix that's internet-scale data from YouTube and the like, plus a bunch of egocentric data, plus UMI, plus teleop? Maybe there was a triangle of how much data of each — I think Jim Fan had a really nice visual on this at a Sequoia event. Is that the mix you see? And among your customers, has everybody converged on a very similar mix?

Josh Gao: It's fun to see. Even a year ago, many folks would argue, "Oh no, it's only UMI data that's going to work," or "It's only egocentric data that's going to work." And you slowly realize, to your point, that all of this is essentially a data pot of soup, and every ingredient in that pot is going to be important at some point.

What we're seeing is a convergence on what that recipe might be today. It looks like some version of a million hours of human data across egocentric and different modalities, some tens of thousands of hours of robot data, and then some last-mile teleoperation data — in-domain, in-distribution data, even just a couple of hours of demonstration. We're seeing that.

But the reason we don't subscribe aggressively to any one religion here is that things like tactile data haven't even entered the mix yet. If there's a belief that these models will ultimately need to learn how much force it takes to apply to a door handle, that will have to come into the data pot of soup. Simulation will come into the data pot of soup. They all will be important to converge on.

Sonya Huang: One of the objections I hear most frequently to the types of data being collected: you could watch millions of hours of tennis data and you still wouldn't know how to beat Roger Federer. I'm curious what you think of that, and what it means for where we might be headed in terms of how robots learn.

Josh Gao: I love the analogy of AlphaGo. When you talk about robots playing tennis, you've got to talk about robots playing Go. With Lee Sedol and all the work DeepMind did at the very beginning, it was large-scale behavior cloning of different experts playing Go. The next step was the self-play version: with the rules of the game, can you simulate all the different ways until you come up with the magic of Move 37 — which was so far out of scope of what humans could have thought of in that moment?

For Go, the rules of the game are the entire circumference of the world you operate in. In robotics, you have the incredibly rich detail of the physical world, and we don't have a world simulator that can recreate that today. So to close that gap, a lot of the egocentric data today is, one, helping us ground the basic physics and the basic behavior cloning you have to get to. It's also contributing to creating what a world simulator might look like. Somewhere along the way, I think we'll be able to have robots playing tennis like Roger Federer.

Sonya Huang: Do you think they'll mostly be doing self-play in simulation or in the real world?

Josh Gao: I could see a hybrid. If we can get simulation right — and we don't have to fully reconstruct the world in all its detail right away — I see self-play both in the physical world and in simulation.

Sonya Huang: You used the phrase "world simulator" a couple of times. What are your thoughts on what's happening with world models — which are being used somewhat interchangeably with video models as a term right now — and their contribution to the robot training data mix?

Josh Gao: The potential to have what we just talked about in terms of self-play — to even just dream what it means to drop a cup and have it shatter without actually doing it — is incredibly valuable. I see the value in leveraging a lot of the work from the internet and YouTube in pixel prediction, and if we can translate that back into what the action trajectories would be, it could accelerate us quite a bit.

Sonya Huang: Interesting. You mentioned we've converged on this pyramid of a data mix. Do you think this pyramid will scale — if we just scale it up, does it take us to the promised land? To me the promised land is — people like saying "GPT-3 moment for robotics." I no longer like that. I just want to know when a robot will actually fold my clothes or cook a meal for me. Do you think the current data mix gets us there, or are new breakthroughs in algorithms and data going to be needed?

Josh Gao: I think if we scale it up some order of magnitude in that ratio, we will solve a lot of tasks along the way. Some of those could be in more constrained environments — business operations tasks with very specific SOPs. But the ideal world where we have a drop-in robot that solves any task just by looking at it, that can play tennis like Roger Federer out of the box, in all the crazy complexities of actual real-world scenarios — the weather, for example — I think that's some way off, and I don't think this is what it will look like.

Sonya Huang: What will it look like?

Josh Gao: We still have so much to add into the mix. How can we start introducing other modalities? That's going to be tactile data. I like the approach of a lot of models converging toward an omni-model approach, where you can feed it any type of input as the observation — whether it's force, pixels, even smell in a very extreme case — and then have some output, whether it's action, voice, or different types of controls.

Sonya Huang: Are there even chips that can sense smell?

Josh Gao: It's pretty early. The olfactory approach is very early, but I like to imagine there's going to be an approach in the world where that gets unlocked.

Sonya Huang: The results of GPT-6 Astra are generalizing incredibly well to robot tasks. I'm curious what you make of that and what it means for all these pure-play robotics companies. Are they ultimately just going to get clobbered by the AGI labs? Is the bitter lesson going to steamroll robotics as well? What do you think?

Josh Gao: It takes a lot of courage to scale up data with imperfect information, and I think a lot of these labs have the DNA to take on that courage early. What we're seeing — and what's exciting — is that with the scale-up of robotics data, and even just adding it into the pre-training mixes of the main LLMs and foundation models, you can start to generalize across a lot of different domains. I'm excited about the omni-model approaches where you can introduce many different types of observation, whether that's force or action trajectories, and allow it to generalize.

I think there are also going to be a lot of arguments for being much more vertical than a fully generalized model that can do anything. Somewhere along that time horizon, the model that can do everything is going to be very good at a lot of things. But some of the specialized, more targeted robotics foundation models might become excellent — four nines of capability and reliability — in some of these tasks.

Sonya Huang: What I heard, reading between the lines, is that the fact that the latest foundation models are great at robotics is not actually that surprising, in part because there's now quite a bit of robot data in the training mix. Is that fair?

Josh Gao: It's actually quite expected when you think about it.

Sonya Huang: Very interesting. I'd love to close by going back to your sci-fi roots. You started this company because you could imagine a version of the world with robots everywhere. Paint a picture of what you think that world looks like.

Josh Gao: The funniest moment was probably when you visited our New York office, and in our main conference room we have the Kawasaki Corleo hung up on the wall.

Sonya Huang: I love that.

Josh Gao: Which is —

Sonya Huang: The robot panther-dog-horse, whatever it is.

Josh Gao: The panther-dog-horse. Exactly. It's pretty much a robot panther-dog-horse, and it's the most ridiculous video and trailer you've ever seen.

Sonya Huang: My favorite part is that you guys convinced Anas that it was actually you and your mom on it.

Josh Gao: Yeah, there's a clip where this fantastical Asian lady and her son are riding through a green meadow over a little river, and the Corleo — this horse-dog-panther robot — is jumping over the river. We told Anas this was us when we were kids, and that we'd generated it.

But what's wonderful is how absurd it is. I very much love the philosophy of Hitchhiker's — just how absurd the world could be, and how absurd the future could be. In this world that is full of robots, I would love to believe that there are going to be crazy entrepreneurs and people who believe that form factors like this would be interesting. I think the shape and size of robotics will look something like a world where we come up with wacky robots that can do anything you haven't even thought of yet.

Sonya Huang: I love that.

Josh Gao: Like mining asteroids.

Sonya Huang: Order of magnitude — how many robots will there be in the world in a decade?

Josh Gao: Many, many times more than there are people.

Sonya Huang: Many orders of magnitude more?

Josh Gao: Than there are people, yes. In a decade.

Sonya Huang: A hundred billion-plus robots.

Josh Gao: Yeah.

Sonya Huang: That's really cool. A hundred billion panther-dog-horses.

Josh Gao: Panther-dog-horses — at least another order of magnitude more than that.

Sonya Huang: Love it. Josh, I think what you're building is operationally really impressive, and it's very cool how you're scaling up the data collection efforts to enable this robotics golden age and the sci-fi future you've dreamed of since you were a kid. Thank you for joining the pod today to tell us both how you're building the business and how you see the future. This was really fun.

Josh Gao: Thank you so much for having us. Thank you, Sonya. We actually brought you a present.

Sonya Huang: I love presents.

Josh Gao: This is kind of a full-circle moment for us.

Sonya Huang: Is this what I think it is?

Josh Gao: Who knows? You're going to have to open it up on the podcast.

Sonya Huang: This is amazing. I could be one of those YouTubers who does unboxings. Look — I love this. The Mecka stickers are an A-plus. Okay, I think this is what I think it is. Let's see.

Josh Gao: We're actually going to come up with a series of Mecka figurines. They're all like giant robots — human-piloted stuff. That'll be fun.

Sonya Huang: I'm so excited. Let's see if this is working today. I love it. Oh my gosh. This is the —

Josh Gao: This is the —

Sonya Huang: This is the Kawasaki. Oh, I love this. This is the robot dog-panther-horse.

Josh Gao: This is me and my mom, on the panther-horse. That's right.

Sonya Huang: We're either going to put it in the podcast room or in my office. This is incredible. I'm so happy.

Josh Gao: Do it. So absurd.

Sonya Huang: I'm so happy. Thank you.