Apple wants AI to run directly on its devices rather than in the cloud

Apple’s latest research into running large language models on smartphones provides the clearest sign yet that the iPhone maker plans to catch up with its Silicon Valley rivals in generative artificial intelligence.

The paper is titled “LLM in a flashoffers “a solution to the current computational bottleneck,” its researchers wrote.

Their approach “paves the way for efficient inference on LLMs on memory-constrained devices,” they said. Heuristics refer to how large language models, the large data repositories that support applications like ChatGPT, respond to user queries. Chatbots and LLMs typically run in massive data centers that have much more computing power than an iPhone.

The study was published on December 12, but gained wider attention after Hugging Face, a popular site for AI researchers to showcase their work. Highlight it Late Wednesday. This is Apple’s second paper on generative AI this month, and follows previous moves to enable image generation models like Stable Diffusion to run on its custom chips.

Device manufacturers and chipmakers hope new AI features will help revive the smartphone market, which had its worst year in a decade, with shipments falling an estimated 5 percent, according to Counterpoint Research.

Despite launching one of the first virtual assistants, Siri, in 2011, Apple was largely left out of the wave of excitement around generative AI that swept Silicon Valley in the year following OpenAI’s launch of its chatbot ChatGPT. Many in the AI ​​community view Apple as lagging behind its Big Tech rivals, despite the hiring of John Giannandrea, Google’s chief AI executive, in 2018.

While Microsoft and Google have largely focused on delivering chatbots and other generative AI services over the internet from their vast cloud computing platforms, Apple’s research suggests they will instead focus on AI that can run directly on an iPhone.

Apple’s rivals, such as Samsung, are preparing to launch a new type of “AI smartphone” next year. Counterpoint estimates that more than 100 million AI-focused smartphones will be shipped in 2024, with 40 percent of new devices having such capabilities by 2027.

The head of the world’s largest mobile chip maker, Qualcomm CEO Cristiano Amon, predicted that bringing artificial intelligence to smartphones would create a completely new experience for consumers and reverse declining mobile phone sales.

“You will see hardware launching in early 2024 with a number of generative use cases for AI,” he told the Financial Times in a recent interview. “As these things scale, they begin to make a tangible difference in the user experience and enable new innovation that has the potential to create a new upgrade cycle in smartphones.”

He added that more sophisticated virtual assistants will be able to anticipate users’ actions such as sending text messages or scheduling a meeting, while the devices will also be capable of new types of photo editing techniques.

This month, Google unveiled a version of its new Gemini LLM software that will run “natively” on Pixel smartphones.

Running the massive AI model that powers Google’s ChatGPT or Bard on a personal device brings enormous technical challenges, because smartphones lack the massive computing resources and power available in a data center. Solving this problem could mean that AI assistants respond more quickly than they would from the cloud and even work offline.

Ensuring that inquiries are answered on an individual’s device without sending data to the cloud is also likely to bring privacy benefits, something that has been Apple’s signature in recent years.

“Our experiment is designed to improve the efficiency of inference on personal devices,” its researchers said. Apple has tested its approach on models including the Falcon 7B, a smaller version of the open source LLM software originally developed by the Technology Innovation Institute in Abu Dhabi.

Optimizing MBA to work on battery-powered devices has been a growing focus for AI researchers. The academic papers are not a direct indication of how Apple intends to add new features to its products, but they do provide a rare glimpse into its secret research laboratories and the company’s latest technical achievements.

“Our work not only provides a solution to the current computational bottleneck, but also sets a precedent for future research,” the Apple researchers wrote in the conclusion of their paper. “We believe that as LLM students continue to grow in size and complexity, approaches like this work will be essential to harness their full potential in a wide range of devices and applications.”

Apple did not immediately respond to a request for comment.