Chip designer Arm has entered the artificial intelligence (AI) hardware arena with its first in-house processor designed to power AI agents. Unlike conventional chatbots, these are much smarter systems that can take proactive actions to achieve their goals without as much human input or supervision.
By focusing specifically on powering AI agents, Arm’s chip could help accelerate the adoption and widespread use of agentic AIs, be that in businesses or in one’s personal life, bringing AI much closer to what people would expect from virtual assistants.
Think of a CPU as the conductor of an orchestra of GPUs and other AI accelerators — hardware that’s specifically designed to run LLMs — in this case.
As such, Arm representatives announced in a statement that its new AGI CPU has a custom design — including 3-nanometer process nodes, up to 136 Neoverse V3 cores that can hit 3.7 GHz clock speeds, and a memory bandwidth of 6 gigabytes per second per core — for use in data centers that are powering active AI agents.
All of these capabilities aim to meet the goal of providing better performance and efficiency than classical CPUs that use the x86 architecture, the dominant computing architecture that was developed by Intel in 1978 and is still used in processors today.
Custom chip future
With the inexorable growth of AI and the deployment of smart agents, there’s a need for more data-center-based hardware to power these systems. However, the general-purpose nature of CPUs means they aren’t intrinsically designed to run the specific orchestration needed for agentic AIs.
Arm’s AGI CPU uses the Armv9.2-A architecture at its core. This architecture has been designed with the specialized needs of running AI in action — known as inference. With this specialty, there’s no need for an AGI CPU to hold legacy support for other processes and applications, as seen in x86 chips — conventional processors used in regular computers.
This should make for faster and more efficient performance targeted at AIs. Arm representatives said that its AGI CPU delivers more than twice the performance per server rack versus x86 CPUs.
The AGI CPU has been designed to pack two chips with dedicated memory and in-out (I/O) functionality into a single server blade with a total of 272 cores per blade. The blades can then be stacked into server racks of 30, delivering a total of 8,160 cores with sustained performance for agentic AI workloads at a “massive scale,” thanks to thousands of cores working in parallel.
Arm’s speciality in chip design centers on offering strong performance for relatively lower power consumption. That’s one of the reasons all smartphone chips use Arm-based processors or instruction sets. For example, Qualcomm uses Arm technology in Snapdragon chips and Apple uses it in its iPhone and MacBook chips.
As AI continues to transition from training LLMs to actively deploying agentic AIs, there will be an increased need for CPU-based processing power in data centers. This is expected to drive a huge increase in AI energy demand.
The AGI CPU has been designed to pack two chips with dedicated memory and in-out (I/O) functionality into a single server blade with a total of 272 cores per blade. The blades can then be stacked into server racks of 30, delivering a total of 8,160 cores with sustained performance for agentic AI workloads at a “massive scale,” thanks to thousands of cores working in parallel.
Arm’s speciality in chip design centers on offering strong performance for relatively lower power consumption. That’s one of the reasons all smartphone chips use Arm-based processors or instruction sets. For example, Qualcomm uses Arm technology in Snapdragon chips and Apple uses it in its iPhone and MacBook chips.
As AI continues to transition from training LLMs to actively deploying agentic AIs, there will be an increased need for CPU-based processing power in data centers. This is expected to drive a huge increase in AI energy demand.
Roland Moore-Colyer
If Arm can deliver CPUs that offer strong AI inference performance that’s more power-efficient than x86-based CPUs, it could mitigate some of this demand in terms of performance per watt and energy consumed per workload while also disrupting chipmakers Intel and AMD. It could even shake up AI hardware giant Nvidia, which has its own Arm-based Vera CPUs.
Given that Arm architecture is already used in chips for AI data centers, the chip designer is in a strong position to make its own foray into providing “off-the-shelf” CPUs.
While Arm has traditionally licensed its chip designs to other chipmakers, the AGI CPU will be its first move into producing hardware that companies can buy and deploy in their data centers. This points to a future where more hardware is custom-designed to power AI, whether it’s for the efficient running of LLMs, as seen with the application-specific integrated circuit architecture found in Google’s TPU and Amazon’s Trainium chip, or for inference, in the case of Microsoft’s Maia 200 chip.
Essentially, custom chips that can overcome some of the hardware constraints of operating AI at a large scale could disrupt the traditional makeup of general computing hardware in data centers. This, in turn, could accelerate the path to artificial general intelligence (AGI), the point where AI possesses the ability to learn, understand and apply knowledge across a wide variety of tasks at a human-level or beyond.
