Cloud platform Modal said it will add Nvidia's new Arm-based Vera processor to run isolated sandboxes for AI agents. In early tests under heavy load, the company measured up to twice the throughput of a comparable x86 system. It is Modal's first move beyond running x86 machines exclusively. According to Reuters, Intel and AMD processors will stay in a mixed fleet.

What it means: Talk about AI infrastructure usually centres on graphics cards, but agents change the picture. When a model writes and runs code, installs packages or searches files, that work happens on ordinary processors. On platforms with hundreds of concurrent sandboxes, sudden load spikes cause contention, and every second of waiting costs money, especially when expensive GPUs sit idle waiting for results in reinforcement learning. Nvidia sees a new market here and is moving onto Intel and AMD territory. Moving to Arm is not trivial, though: it requires changes to software images, libraries and build tooling. Modal is opting for a mixed fleet, sending each job to the hardware that runs it best. For Greek development teams using coding agents or automated testing, the message is that cost depends not only on the model but also on the execution environment. Multi-architecture container images, already needed for Apple Silicon Macs, are now also a cloud cost issue.

Read the original on Modal

Related service: Custom software

Related stories