alt Hacker NewsGemini Robotics 2 brings whole body intelligence to robots
Comments
While Anthropic and Open AI get 80% of the attention here, it's impressive to see how much Google is doing: near frontier model, fast models, open weight models, image generation, video generation, music generation, robotics, etc.
These robots look slow and not very fluid in their motions, but LLMs like ChatGPT also looked very dumb initially. If progress is as fast as LLMs , this could have massive applications in a few years.
I've kind of given up on humanoid robotics, because of how bad the actuators are. There has been no innovation in robotic actuators since Honda's Asimo. There's just no way that someone wants a 80kg wobbling tin can in their home or workplace.
My bet is that the final robotic revolution will use genetically modified human/animal bodies with replaced brains. You'll have to stretch your ethics a bit, but if you grow a bear genetically modified in a way that it has no consciousness or thought, it'll make a much better construction worker than any humanoid robot. You'll just need to wire it up with neuralink and then control it via LLM. Fast animals can be used to deliver packages, and giraffes for warehouses.
Can anyone that works on this technology provide an honest assessment of where this technology actually stands? How much instrumentation is actually required, what the interaction quality is, how much trouble do humanoids have with in the wild daily tasks like turning doorknobs, recovering from falls, avoiding knocking into things, etc.
Robotics with AI is the real revolution.
Today, humans convert their labor to capital. Capital holders need labor (humans) to acquire more capital. When the price of inference for these robots becomes less than the price of labor then capital holders don’t need labor.
Obviously, AI impacts non-manual labor too, but a significant portion of the world population does manual labor.
As a 47-year-old man with no kids, I have to admit I'm looking forward to the possibility of having a robot help take care of me in my old age.
A lot of these household tasks could be automated without the need for humanoid robots. Roombas can clean your floor, automated rubbish disposal can be integrated into houses and flats, grocery delivery and storage could be done via an automated intake, cataloguing, and storage system. Humanoids are just creepy and you can't really trust them.
Could you sleep easily knowing that you have one in your house? What if you oppose the political views of its creators?
Running a full LLM to actuate a robot is way too heavy, the minimum latency they can get down to is probably 1-2 seconds even with powerful GPUs, which is not very useful for practical robotics applications.
Machine vision should definitely be handled by ML, but motion actuation should be relegated in the realm of traditional PID style linear/nonlinear control. Again, the tech is cool, but the practical usefulness of using a full LLM as a controller will probably run into hardware limitations.
Still can't make a page scroll smoothly though.
I don’t think this is where the robotics revolution is going to happen. The big one will be using AI to design and build highly specialized robots to automate mining and manufacturing.
Why do we want to add more bodies on a finite plane of existence with an infinitely-expanding base (humans)?
We have a finite plane of existence (Earth), with an infinitely-expanding consumer base (humans), and we want to add robotic competition to that finite plane of existence? The data center AI is more appealing (I guess) since it doesn't compete literally shoulder-to-shoulder with me. Why do we want dexterous robots when we have humans?
Relevant: A San Francisco company is advertising humanoid robot housecleaning services[1]. However those 'bots require local supervision and a human controller watching through its video feed.
Presumably that company could use the Gemini Robotics product to eliminate the remote controller.
All VLM, VLA models. I wonder if these architectures will reach prod before Yann LeCunn's world model JEPA startup even gets off the ground in Europe.
No mention of whether the robot's models run locally or in the cloud. Ie. Does it require an internet connection?
For a robot to be in my house it would have to run locally, there's no way I'm allowing one that runs in the cloud to operate my washing machine for example.
This is incredible. Seems like a big achievement.
The only thing that bothers me is what happens when robots finally automated "all the mundane day-to-day tasks" including our jobs, what is there left to do for common folks who are not geniuses working at Google/Anthropic/ChatGPT or occupy the C-Suite of these companies?
Fingers crossed this inspires a Chinese lab to release some weights so people can benefit.
I think the biggest snag with home robots will be that if it is strong enough to install your air conditioners, it's strong enough to break your ribs.
So the robots will need to be weak, so even weak people can overpower it. But then it loses a ton of it's most promising abilities.
I don't mean this like a "rogue robot" situation. I mean it like the robot gets confused, or someone sees a walking $10M lawsuit in their home.
I loved the video. Also, I had a lot of fun watching it while imagining I was living in 2051 and this whole progression had ran its course. It's impossible to predict the future, but certain events can negate whole branches of possibility. This one video in particular prunes the "we never figured out robotic motion and thus all the science fiction scenarios built on that premise never came to happen" subtree.
If they can't even get LLM/agentic coding right, I have little hope for a literal intelligent robot coming out of Gemini... but I wish them luck.
I'm asked to prove I'm not a bot if I want to watch the top video with sound.
anyone notice the jokes they put in?
* bot: "im nervous. is that a microphone?"
* bot: "great job, you're a machine!" (said to another bot, a machine)
* bot: clicks "i am [not] a robot" button on a computer
Is anyone building non-humanoid robots? Feels like humanoid is limiting the creativity of building a truly useful robot in today's age
I believe that LLMs represent the future of robotics in this field. Of course, I don’t want a robot uprising, but if our goal is simply to increase the utility of robots in manufacturing (within the necessary constraints), then LLMs are the only powerful path forward.
> It can better detect when humans are nearby, trigger safety tool calls and bring the robot to a safe stop if someone approaches too closely.
If the robots stop when humans are too close, wouldn't that mean that robots for close interaction or handling of humans need a whole other level of control?
And absolutely nothing in your company charter says the military won't be allowed to buy it. Y'all think we've never seen a Terminator movie before, huh.
What would be the hallucination equivalent of such robots?
Paint it yellow and ask why it is hiding a nuclear battery
I always wonder why these robot forms don't have cameras and distance sensors on every single joint, processing costs I guess?
Is the fundamental challenge for AI directed movement in 3D space the continuity of the environment? Vs LLM's which train on discrete and limited token sets.
> we are teaching robots intelligent whole-body control, fine dexterity, *and teamwork*...
That's the most Skynet thing ever.
Can someone that uses these models provide some tips or docs on how best to get to use these at home to test around? Is the best thing a virtual environment?
Do these learn?
It would be cool to have a robot that can be taught to drive the same way we might teach a teenager to drive.
We're getting closer
Video has too many cuts, similar to kungfu movies where the actor doesn't know how to fight.
Given the rapid advancements in AI. It is not what we thought it will, what it is, what we didn't wanted it to be, but it is great for being what it is.
What's the point of "releasing it"? It only makes sense in the labs, and what would calling your APIs give "me" as a researcher, other than a baseline to beat? mah the model is real, but it feels 100% internal
What does this mean for intrinsic, the moonshot they brought back into the fold in feb?
have you seen that NEO robot's hands, wild
Deepbody
Its almost like Google selling Boston Dynamics was a bad idea.
[dead]
[flagged]
[dead]
[dead]
I'm a researcher at Deepmind that contributed to these models. (And the opinions here are my own)
Just want to say, Deepmind is a great place to work and the only (Edit: one the few unique labs!) lab where you can move from large frontier models (Gemini), frontier open models (Gemma), robotics (what you see here), science (weather, biology, more) and basically any other topic related to intelligence. It's really an incredible place to be, with incredible people. Consider joining! And thank you for the enthusiasm here.