Agentic AI explained: How agentic AI changes computing from cloud to edge

Before your next meeting, a personal AI agent recognizes what is coming up and starts preparing without waiting for a prompt. It checks your calendar, pulls together relevant messages and documents, identifies who you are meeting with and surfaces the information you are likely to need before you join.

Behind that experience, the agent may combine context held on the device with enterprise applications, cloud services, and specialist models. It then determines what information matters, which tools to use, where parts of the task should run and how each result affects what happens next. This is what moves agentic AI beyond a model-centric workload.

Models can provide the reasoning, but completing the task depends on the wider system coordinating memory, software, tools, and different types of compute across multiple steps. Keeping that chain connected, from the first instruction to the outcome, is becoming one of the defining engineering challenges of agentic AI.

Agentic AI starts with the outcome

Laurence Moroney, Director, Solutions Architect for AI at Arm, describes the basic agentic loop in four stages: understand intent, plan, act, and reflect. Here, it is the intent that changes the starting point. Instead of moving through a predefined application flow, an agent can begin with the desired outcome and work out which information, models, tools, or software it needs next.

Dr. Yang Zhang, CEO and Co-founder of AutoCore, sees that changing how people interact with software in the first place. Applications have traditionally required people to translate what they want into interfaces the machine can understand, such as selecting the right application, navigating menus, or entering information in a prescribed format. With agentic AI, Zhang sees that interaction becoming more human-centric and increasingly multimodal, with the system beginning from a simpler question: What do you want to do?

The implication is not that every instruction suddenly becomes complete, as the agent still has to turn that intent into a sequence of actions. A coding agent for instance, might inspect a repository, make a change, run tests, and adjust based on the results. A customer service assistant, on the other hand, could pull information from account records, policies and external services before deciding how to resolve a request. In an industrial setting, an agent may interpret sensor data before deciding which software tool to invoke or which physical action should follow.

As those tasks extend across more steps, context and memory become part of the workload. Previous instructions, retrieved information, and intermediate results can all become relevant again later. On a device specifically, that growing context must fit within limited memory already shared with the operating system, applications and other AI workloads.

Cemal Bilgin, Engineering Manager at Meta sees those constraints through his work on ExecuTorch, Meta’s open source runtime for on-device machine learning. Hundreds of models already run across Meta’s family of apps, and agentic workloads add further demand where the system must retain enough context to continue a task without losing information it has already processed. Keeping more of that context in memory consumes capacity, and compressing or retrieving it again can introduce trade-offs in fidelity, latency, and efficiency.

However, some context is hard to capture. People inside an organization often know why a process exists, what led to an earlier decision, or what happened when a similar problem occurred before. As Bilgin puts it, “For an agent, I might need to spell out, one by one, exactly what it is that we’re looking for.”

More natural, human-centric interfaces can make it easier for people to express intent, but useful agents still need sufficient context to interpret that intent correctly. Documentation, policies, and domain expertise therefore becomes part of the information an agent may need as it works toward an outcome.

That becomes particularly important on personal devices, where the input to an agent can extend beyond an explicit request to what the device can sense, remember and understand about the person using it.

Agentic AI depends on efficient orchestration

Eric Xia, Chief Chipset Planning Expert at vivo, takes that idea further. He describes personal agents that “need to remain aware of what is happening around the user, retain relevant information, anticipate what the user may need and complete tasks across applications and devices.” On a smartphone, those capabilities have to coexist with a fixed battery, shared memory, active sensors and applications already competing for compute throughout the day.

Supporting those capabilities requires more than adding another model or using a larger model. Xia explains that the operating system can no longer act as just a housekeeper managing the hardware. It will have to manage context and user intent alongside the security controls needed for autonomous actions. Hardware will also need to evolve to support specialized models efficiently on-device, particularly within tight power and thermal limits. That means, agentic AI will require the whole system – software and hardware together – to be rethought from the ground up.

Physical AI extends that challenge into systems where software decisions can have real-world consequences. A humanoid robot, for example, may be processing camera feeds, interpreting sensor data and controlling movement while an agent requests additional information or decides what action to follow.

Zhang sees the agentic runtime as an important layer that helps manage that complexity. It can schedule work, control access to resources, manage permissions and coordinate execution across local and cloud environments.

In physical systems, those controls help define the limits of autonomy. Engineers need to know what an agent Is allowed to do, which resources it is using and how its actions interact with the rest of the machine. As agents move closer to real-world decision-making, those boundaries are fundamental to making agentic behavior dependable at scale.

Scaling agentic AI puts pressure on the infrastructure

Google offers an early view of what happens when agentic workflows operate at software scale. Their monorepo contains billions of lines of code, and a large migration can require coordinated changes across builds, tests and applications.

Wolff Dobson, Developer Relations Engineer at Google, who has worked with multi-agent systems designed to tackle parts of that process, describes the arrangement as “several agents inside a trench coat.” An orchestrating agent can divide the migration into smaller tasks, while other agents investigate build failures, modify code or work through tests.

The agents can generate changes quickly, but the engineering systems around them still have to keep pace. Dobson points to Piper, the custom source-control system Google built after conventional systems struggled with the scale of its monorepo. Agentic workflows increase that pressure further as each change may need to be stored, built, tested and evaluated before another action can begin.

Agentic AI can also accelerate the engineering work itself. Bilgin points to the lifecycle of deploying on-device AI at Meta, where model and runtime development, testing, performance, and accuracy optimization, A/B testing and gradual rollout can take several months or longer. Agentic workflows can shorten parts of that process by helping benchmark models, identify crashes or accuracy problems and move more quickly towards another deployment.

That acceleration makes the underlying engineering discipline more important, not less. Moroney draws a distinction between running one agent and operating many at once: one agent calling an API is straightforward; a thousand doing so concurrently creates a familiar systems-scale problem. The same principles still apply – designing for scale, identifying bottlenecks, accounting for constrained resources, and defining clear boundaries around what an agent is allowed to do.

Agentic AI may increase the speed and volume of work, but scaling it reliably still depends on getting those fundamentals right.

Performance will be measured in outcome

Agentic AI also changes what performance looks like. Model latency and throughput remain useful measurements, but they capture only one part of a task that may involve retrieving information, calling tools, waiting for services or moving context between steps.

For the person using an agent, the meaningful measure is the outcome: did the task get done, how long did it take, and how reliably and efficiently did the system complete it? Achieving that end-to-end performance depends on how efficiently the system coordinates the work around the model. A common compute and software platform can reduce the complexity of building and optimizing across cloud, edge, and physical systems by giving developers a consistent foundation as workloads move between them.

But that foundation is only one part of the equation. Agentic systems will combine models, runtime tools, hardware and services from across the industry, making open interfaces and collaboration essential to how efficiently those resources work together. As Xia suggests, “openness needs to be designed in from the outset, with the ecosystem working together rather than creating isolated stacks.”

The combination of efficient system design, a common compute platform and an open, multi-vendor ecosystem is what enables agentic AI to scale. The next phase of AI will therefore depend not simply on how capable individual models become, but on how effectively the wider system orchestrates intelligence into useful action.

https://newsroom.arm.com/blog/scaling-agentic-ai-orchestration-across-cloud-edge-and-devices