Stay informed with free updates

OpenAI, in collaboration with renowned designer Jony Ive, faces a range of technical challenges as they work towards unveiling a groundbreaking artificial intelligence device next year.

Led by Sam Altman, the San Francisco-based start-up acquired the company of the former Apple design chief, io, for $6.5 billion in May. However, the duo has been reserved about the specifics of their upcoming projects.

Their goal is to develop a compact device, devoid of a screen, capable of interpreting audio and visual prompts from the environment and fulfilling user commands.

People familiar with their plans said OpenAI and Ive had yet to solve critical problems that could delay the device’s release.

Even with hardware crafted by Ive and his team — celebrated for creating the iconic iMac, iPod, and iPhone designs that transformed Apple into one of the world’s most successful corporations — challenges still exist with the device’s software and the necessary infrastructure to support it.

Key concerns include determining the assistant’s “personality,” addressing privacy concerns, and budgeting for the substantial computing power needed to operate OpenAI’s models on a large customer scale.

“Compute is another significant delay factor,” according to a source close to Ive. “While Amazon has sufficient compute for Alexa and Google for its Home device, OpenAI struggles with securing enough compute for ChatGPT, let alone an AI device — resolving this is essential first.”

A person close to OpenAI said the teething troubles were simply normal parts of the product development process.

Multiple people familiar with the plans said OpenAI and Ive were working on a device roughly the size of a smartphone that users would communicate with through a camera, microphone and speaker. One person suggested it might have multiple cameras.

The gadget is designed to sit on a desk or table but can also be carried around by the user. The Wall Street Journal previously reported some of the specifications around the device.

One person said the device would be “always on” rather than triggered by a word or prompt. The device’s sensors would gather data throughout the day that would help to build its virtual assistant’s “memory”.

OpenAI overtook Elon Musk’s SpaceX to become the world’s most valuable private company this week, after a deal that valued it at $500bn. One of the ways the ChatGPT maker is seeking to justify the price tag is a push into hardware.

The goal is to improve the “smart speakers” of the past decade, such as Amazon’s Echo speaker and its Alexa digital assistant, which are generally used for a limited set of functions such as listening to music and setting kitchen timers.

OpenAI and Ive are seeking to build a more powerful and useful machine. But two people familiar with the project said that settling on the device’s “voice” and its mannerisms were a challenge.

One issue is ensuring the device only chimes in when useful, preventing it from talking too much or not knowing when to finish the conversation — an ongoing issue with ChatGPT.

“The concept is that you should have a friend who’s a computer who isn’t your weird AI girlfriend . . . like [Apple’s digital voice assistant] Siri but better,” said one person who was briefed on the plans. OpenAI was looking for “ways for it to be accessible but not intrusive”. 

“Model personality is a hard thing to balance,” said another person close to the project. “It can’t be too sycophantic, not too direct, helpful, but doesn’t keep talking in a feedback loop.”

OpenAI’s device will be entering a difficult market. Friend, an AI companion worn as a pendant around your neck, has been criticised for being “creepy” and having a “snarky” personality. An AI pin made by Humane, a company that Altman personally invested in, has been scrapped.

Still, OpenAI has been on a hiring spree to build its hardware business. Its acquisition of io brought in more than 20 former Apple hardware employees poached by Ive from his alma mater. It has also recruited at least a dozen other Apple device experts this year, according to LinkedIn accounts.

It has similarly poached members of Meta’s staff working on the Big Tech group’s Quest headset and smart glasses.

OpenAI is also working with Chinese contract manufacturers, including Luxshare, to create its first device, according to two people familiar with the development that was first reported by The Information. The people added that the device might be assembled outside of China.

OpenAI and LoveFrom, Ive’s design group, declined to comment.