ChatGPT: Creating Robot-Friendly Conversational AI with Covariant

Covariant is building ChatGPT for robots The UC Berkeley spinout says its new AI platform can help robots think more like peopleCovariant this week announced the launch of RFM-1 (Robotics Foundation Model 1). “We at Covariant have already deployed lots of robots at warehouses with success. “We do like a lot of the work that is happening in the more general purpose robot hardware space,” says Chen. “ChatGPT for robots” isn’t a perfect analogy, but it’s a reasonable shorthand (especially in light of the founders’ connection to OpenAI). Chen says the company expects the new RFM-1 platform will work with a “majority” of the hardware on which Covariant software is already deployed.

Covariant is revolutionizing the world of robotics with their latest innovation, ChatGPT. This AI platform, developed by the UC Berkeley spinout, is set to transform the way robots think and function.

Peter Chen, the co-founder and CEO of Covariant, recently announced the launch of their Robotics Foundation Model 1, or RFM-1 for short. He describes the platform as “a large language model (LLM) for robot language.”

The development of RFM-1 was made possible through a vast collection of data gathered from the deployment of Covariant’s Brain AI platform. With the consent of their customers, the startup has been building a massive database specifically designed for robots.

Chen explains that the ultimate goal of RFM-1 is to power the billions of robots that will inevitably become a part of our daily lives. “We at Covariant have already successfully deployed robots in warehouses, but our vision extends far beyond that. We want to empower robots in all industries, from manufacturing and food processing to recycling and even in people’s homes.”

This platform comes at a time when more and more robotics companies are discussing the future of “general-purpose” systems. With the recent surge in humanoid robotics firms like Agility, Figure, 1X, and Apptronik, the conversation has been brought to the forefront. Humanoid robots are known for their adaptability, much like their human counterparts. However, the true question lies in the strength of their on-board AI and software systems.

Currently, Covariant’s software is primarily used on industrial robotic arms for tasks such as bin picking. However, they are not limiting themselves to this specific type of robot and promise to work with a variety of hardware.

“We admire the work being done in the general-purpose robot hardware space,” says Chen. “It’s at the intersection of the intelligence and hardware inflection points that we will see even more progression in robot applications. But, many of these technologies are not yet fully developed, especially on the hardware side. It’s challenging to advance beyond the stage of fancy videos. How many people have actually interacted with a humanoid robot in person? This speaks to the maturity of the technology.”

Although Covariant is hesitant to compare their platform to human capabilities, they do use human reasoning as a point of reference for how RFM-1 operates within a robot’s decision-making process. According to their press release, the platform “gives robots the human-like ability to reason, making it the first time that Generative AI has successfully provided commercial robots with a deeper understanding of language and the physical world.”

It’s essential to be cautious when making comparisons to abstract or philosophical concepts and their actual effectiveness in real-world situations. Claiming to provide robots with a “human-like ability to reason” can hold various meanings for different individuals. In this case, the focus is on the system’s ability to process real-life data accurately and determine the best course of action to complete a task.

This approach is vastly different from traditional robotic systems that are programmed to perform one job over and over again. While these single-purpose robots thrive in highly structured environments, such as automotive assembly lines, even the slightest deviations can cause them to malfunction. If an object is not placed correctly on a conveyor belt or there is a change in lighting, the robot’s capabilities are impacted. This can be a significant hurdle when trying to get the robot to work with a new task, material, or part.

In these cases, programmers would typically have to step in and reprogram the robot. This often requires someone from outside the factory floor, resulting in a significant drain on time and resources. Covariant offers a solution for this issue with their RFM-1 platform. As they like to say, it’s “ChatGPT for robots” – a shorthand description that is fitting, considering the founders’ connections to OpenAI.

From the customer’s perspective, the platform resembles current consumer-facing generative AI systems. It presents as a text field where users can input commands using either typing or voice input. For example, typing “pick up the apple” would prompt the system to use its training data (shape, color, size, etc.) to identify the closest matching object in front of it.

RFM-1 then generates simulations to determine the best course of action, similar to how our brains work out potential outcomes before making a decision. During a demonstration, the system accurately responded to inputs like “pickup the red object” and even more complex commands like “pick up what you put on your feet before putting on your shoes.” This latter input caused the robot to correctly pick up an apple and a pair of socks, respectively.

There is a lot of promise surrounding this platform, with many big ideas being discussed. Covariant’s founders have an impressive pedigree, with Chen studying AI at Berkeley under Pieter Abbeel, the company’s co-founder and Chief Scientist. Abbeel was also an early employee at OpenAI and joined the fledgling company in 2016, only one month after Chen. Covariant was founded the following year.

According to Chen, the new RFM-1 platform should be compatible with the majority of hardware currently using Covariant’s software. This is a significant achievement and showcases the potential impact of RFM-1 on the robotics industry.

Avatar photo
Dylan Williams

Dylan Williams is a multimedia storyteller with a background in video production and graphic design. He has a knack for finding and sharing unique and visually striking stories from around the world.

Articles: 834

Leave a Reply

Your email address will not be published. Required fields are marked *