AI Update: Voice Commands & Chaining Tasks

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
2 min read
AI Update: Voice Commands & Chaining Tasks
May 31, 2024
2 min read
AI Update: Voice Commands & Chaining Tasks
May 31, 2024

We have previously developed an autonomous model that can merge many tasks into a single goal-conditioned neural network. However, when multi-task models are small (<100M parameters), adding data to fix one task’s behavior often adversely affects behaviors on other tasks. Increasing the model parameter count can mitigate this forgetting problem, but also take longer to train, which slows down our ability to find out what demonstrations we should gather to improve robot behavior. 

How do we iterate quickly on the data while building a generalist robot that can do many tasks with a single neural network? We want to decouple our ability to quickly improve task performance from our ability to merge multiple capabilities into a single neural network. To accomplish this, we’ve built a voice-controlled natural language interface to chain short-horizon capabilities across multiple small models into longer ones. With humans directing the skill chaining, this allows us to accomplish the long-horizon behaviors shown in this video:

Although humans can do long horizon chores trivially, chaining multiple autonomous robot skills in a sequence is hard because the second skill has to generalize to all the slightly random starting positions that the robot finds itself in when the first skill finishes. This compounds with every successive skill - the third skill has to handle the variation in outcomes of the second skill, and so forth.

From the user perspective, the robot is capable of doing many natural language tasks and the actual number of models controlling the robot is abstracted away. This allows us to merge the single-task models into goal-conditioned models over time. Single-task models also provide a good baseline to do shadow mode evaluations: comparing how a new model’s predictions differ from an existing baseline at test-time. Once the goal-conditioned model matches single-task model predictions well, we can switch over to a more powerful, unified model with no change to the user workflow.

Directing robots with this high-level language interface offers a new user experience for data collection. Instead of using VR to control a single robot, an operator can direct multiple robots with high level language and let the low-level policies execute low-level actions to realize those high-level goals. Because high-level actions are sent infrequently, operators can even control robots remotely, as shown below:

Note that the above video is not completely autonomous; humans are dictating when robots should switch tasks. Naturally, the next step after building a dataset of vision-to-natural language command pairs is to automate the prediction of high level actions using vision-language models like GPT-4o, VILA, and Gemini Vision.

Stay tuned! 
Eric Jang

Less than 1 min read
Podcast: 1X CEO, Bernt Børnich on the Venture Europe Podcast
May 2, 2024
Less than 1 min read
Podcast: 1X CEO, Bernt Børnich on the Venture Europe Podcast
May 2, 2024

In the latest episode of the Venture Europe Podcast, Bernt Børnich, CEO of 1X, sits down with host Calin Fabri to explore the evolving world of humanoid robotics.

Bernt shares his journey from a curious child dismantling kitchen gadgets to founding and leading 1X. He gives insight into the development of NEO, 1X’s next-generation android designed to assist with everyday tasks at home. He discusses the importance of designing safe, compliant humanoids capable of working alongside people in their daily environments. 

Bernt also discusses 1X's strategic expansion, with AI development centered in San Francisco Bay and a new manufacturing facility built in Norway. 

Throughout the episode, he explores the technical and ethical challenges of integrating androids into society, aiming to create an abundant supply of labor.

Listen on Apple Podcast

Listen on Google Podcast

Listen on Amazon Music

2 min read
Scaling NEO Production: 1X builds in-house manufacturing facility
April 2, 2024
2 min read
Scaling NEO Production: 1X builds in-house manufacturing facility
April 2, 2024

MOSS; NORWAY: 1X is currently developing its own production facility, actuator manufacturing, and robot assembly facility in Moss, Norway, right next to our campus and engineering team. This decision is more than just a matter of convenience—it's a commitment to keep building a vertically integrated company where every component of EVE and NEO is designed and produced in-house.

“The close proximity of both the actuator manufacturing, robot assembly, and testing site offers great advantages, especially for our team of creative engineers, brimming with fresh, yet untested ideas. Being adjacent to the manufacturing and assembly process allows them to quickly understand the practical aspects of transforming their creative concepts into feasible, efficient-to-manufacture products, says VP of Manufacturing Operations & Engineering, Csaba Hartmann. 

The manufacturing team consists of diverse professionals, including specialized manufacturing engineers and mechanical designers, process engineers, automation experts, quality engineers, supply chain experts, safety officers, and others. Each member plays a role in designing, trialing, and rolling out our large-scale manufacturing initiatives, contributing to enhancing scalability, rapid iterations, and safety at every stage of the manufacturing and assembly process. 

“Enabling teams that work side by side with each other and thus can easily get and act on feedback, is crucial for us to evolve and improve our products rapidly”, says Hartmann.

All 1X androids are designed with a safety-first mindset, featuring gearless motors and a soft exterior. Our commitment to safety extends beyond design, incorporating measures throughout the assembly process to ensure products are built to specs: thorough testing, quality control, and precise assembly processes.

We’re adopting quality control measures inspired by the automotive industry. We conduct thorough Design Failure Mode and Effects Analysis (DFMEA) on each assembly component to proactively identify and mitigate potential safety risks. 

“Our quality team interprets the results of the DFMEA and PFMEA and then defines the rigorous checks for the assembly process to ensure no safety aspect is overlooked,” says Hartmann. 

The assembly process includes rigorous checks of critical quality parameters to ensure no safety aspect is overlooked. Precision in the use of testing and assembly tools is emphasized to maintain high standards of accuracy. All components, especially motors, undergo extensive testing at multiple stages of assembly to validate their performance and reliability.

"At 1X, we prioritize scalable, cost-efficient manufacturing by integrating engineering expertise and rigorous quality control. Our approach leverages advanced technologies and carefully selected materials to enhance production efficiency. Committed to scalability, we ensure every process is optimized for cost-effectiveness and growth", says 1X CEO Bernt Børnich.

Join us

If you find this work interesting, we’d like to call attention to a few roles that we are hiring for to accelerate our mission toward creating an abundant supply of labor via safe intelligent androids:

We also have other open roles across mechanical, electrical, and software disciplines. Follow 1x_tech on X for more updates, and join us in living in the future.

Less than 1 min read
NEO Featured in NVIDIA GTC Keynote
March 18, 2024
Less than 1 min read
NEO Featured in NVIDIA GTC Keynote
March 18, 2024

March 18th, NVIDIA GTC, San Jose, California.

1X, a leading innovator in the humanoid robotics space, had its next-generation android, NEO, included in a demonstration of robots completing a variety of tasks during the keynote address by NVIDIA founder and CEO Jensen Huang at GTC, a global AI conference running through March 21 in San Jose, CA. 

Robots using the new NVIDIA foundation models will make major advances in being able to understand natural language and emulate movements by observing human actions — quickly learning coordination, dexterity, and other skills to navigate and interact with the real world.

Follow 1X on X and LinkedIn to stay updated on news and events.

CNN: Decoding humanoid robots
March 18, 2024
CNN: Decoding humanoid robots
March 18, 2024
Less than 1 min read
1X Attends NVIDIA GTC
March 12, 2024
Less than 1 min read
1X Attends NVIDIA GTC
March 12, 2024

1X will be attending the NVIDIA GTC Conference on March 18th. Our involvement signifies 1X's dedication to advancing in the field of Embodied AI, showcasing our latest developments, and engaging with the global AI community.

The NVIDIA GTC Conference is renowned for being a pivotal event that gathers innovators, researchers, and industry leaders worldwide to explore the latest advancements in AI, machine learning, and related technologies. Attendees can look forward to a program full of insightful talks, dynamic workshops, and demonstrations.

For more information about the conference or to register:
NVIDIA GTC Conference Official Page
Conference Program

We look forward to connecting with professionals to share our passion for AI and robotics at the event. See you at NVIDIA GTC.

Less than 1 min read
Opening new HQ in Sunnyvale
March 11, 2024
Less than 1 min read
Opening new HQ in Sunnyvale
March 11, 2024

As of March 1st, 1X  has expanded our operations with the establishment of dual headquarters in Sunnyvale, California, and Moss, Norway. 

The Sunnyvale office will include a new branch of our studio for AI data collection and support the expansion of our AI team. The new HQ will be staffed by the AI team, the customer success team, service technicians, and a significant fleet of NEO and EVE androids.

Follow 1X on X and LinkedIn to stay updated on news and events.

IEEE: What’s going on behind the scenes with 1X’s end-to-end autonomy
February 12, 2024
IEEE: What’s going on behind the scenes with 1X’s end-to-end autonomy
February 12, 2024
AI Update: All Neural Networks. All Autonomous. All 1X Speed.
February 8, 2024
AI Update: All Neural Networks. All Autonomous. All 1X Speed.
February 8, 2024

1X's mission is to provide an abundant supply of physical labor via safe, intelligent androids. Our environments are designed for humans, so we design our hardware to take after the human form for maximum generality. To make the best use of this general-purpose hardware, we also pursue the maximally general approach to autonomy: learning motor behaviors end-to-end from vision using neural networks.

We deployed this system on EVE for patrolling tasks in 2023, and are now excited to share some of the new capabilities our androids have learned purely end-to-end from data:

Every behavior you see in the above video is controlled by a single vision-based neural network that emits actions at 10Hz. The neural network consumes images and emits actions to control the driving, the arms, gripper, torso, and head. The video contains no teleoperation, no computer graphics, no cuts, no video speedups, no scripted trajectory playback. It's all controlled via neural networks, all autonomous, all 1X speed.

To train the ML models that generate these behaviors, we have assembled a high-quality, diverse dataset of demonstrations across 30 EVE robots. We use that data to train a “base model” that understands a broad set of physical behaviors, from cleaning to tidying homes to picking up objects to interacting socially with humans and other robots. We then fine-tuned that model into a more specific family of capabilities (e.g. a model for general door manipulation and another for warehouse tasks) and then fine-tuned those models further to align the behavior with solving specific tasks (e.g. open this specific door). This strategy allows us to onboard new skills in just a few minutes of data collection and training on a desktop GPU.

All of the capabilities shown in the video were trained by our android operators. They represent a new generation of "Software 2.0 Engineers'' who express robot capabilities through data instead of writing code. Our ability to teach our robots short mobile manipulation skills is no longer constrained by the number of AI engineers, so this creates a lot of flexibility in what our androids can do for our customers.

Join Us!

If you find this work interesting, we’d like to call attention to two roles that we are hiring for to accelerate our mission toward general-purpose physically embodied intelligence:

Over the last year we’ve built out a data engine for solving general-purpose mobile manipulation tasks in a completely end-to-end manner. We’ve convinced ourselves that it works, so now we're hiring AI researchers in the SF Bay Area to scale it up to 10x as many robots and teleoperators. We're looking for experts in imitation learning, reinforcement learning, large-scale training, and skills relevant to scaling up deployments of autonomous vehicles. You'll be working in a fast-paced team of generalists that ship features to our fleet on a 24-hour release cycle. The work is a mix of pioneering new learning algorithms and fixing speed bottlenecks in our data flywheel. We are relentless in simplifying algorithms and infrastructure as much as possible. 

We're also hiring android operators in both our Oslo and Mountain View offices to collect data, train models with that data, and evaluate those models. Unlike most data collection jobs, our teleoperators are empowered to train their own models to automate their own tasks and think deeply about how data maps to learned robot behavior. If you want to experience what it is like to live in a real-life "Westworld", we'd love for you to apply.

We also have other open roles across mechanical, electrical, and software disciplines that make the foundation possible to ship all of this cutting-edge ML technology. Follow 1x_tech on X for more updates, and join us in living in the future.

NEO Featured in NVIDIA GTC Keynote

Follow 1X on Social Media

No items found.
1X
@1x_tech
1X
@1x_tech
By 2030, 85 million jobs could be unfilled. That’s more than 3x the population of Scandinavia. If humanity is going to keep progressing, humans need support.
No items found.
Eric Jang
@ericjang11
Eric Jang
@ericjang11

My talk at UPenn @GRASPlab is a summary of my worldviews in AI and robotics: humanoid form factor, consumer over enterprise, end2end deep learning, farm2table data. If this roadmap excites you, we're hiring on the 1X AI team! http://1x.tech/careers

No items found.
1X
@1x_tech
1X
@1x_tech
1X’s mission is to create an abundant supply of physical labor through androids that work alongside humans. We're excited to share our latest progress on teaching EVEs general-purpose skills. The following is all autonomous, all 1X speed, all controlled with a single set of neural network weights.
No items found.
1X
@1x_tech
1X
@1x_tech

NVIDIA mentions 1X as a leading innovator in the humanoid industry, and features our android NEO in the keynote address by NVIDIA founder and CEO Jensen Huang at #GTC24.

No items found.
1X
@1x_tech
1X
@1x_tech

ANNOUNCEMENT: We just opened our new headquarters in Sunnyvale, California.

A selection of our open positions

CNC Programmer and Operation Specialist
Moss, Norway