Agentic AI security: Designing in control

Agentic AI promises to automate complex workflows, connect fragmented processes, and give people more time to focus on valuable work. It offers real enterprise advantage, but with these benefits comes risks.

As AI agents become increasingly capable and autonomous, agentic AI security is already high on the agenda for governments, regulators, and organisations worldwide – highlighting why it’s more important than ever to design cyber security that can stand up to these increasingly intelligent systems, while also designing agents to act safely, securely, and responsibly.

I recently hosted an event with my colleagues across PA about how we can ensure agentic AI security. One of the key issues we explored was as AI moves from generating answers to taking action, valid credentials and approved permissions are no longer enough to establish trust. As we look ahead, cyber security must understand not only whether an agent can act, but whether its action makes sense and is authorised to do so in ever evolving and nuanced contexts. Reflecting on the discussions in the session, here are my practical recommendations to help ensure AI agents stand up to cyber security requirements.

Why permission is no longer enough for cyber security

Agentic AI poses many security risks and threats that we must confront, before harm occurs. During the session, what started as a conversation about prompt manipulation, data leakage, and excessive permissions, quickly moved into more complex territory.

One of my astute colleagues asked: What if an AI agent is authorised to perform an action within its approved access but inadvertently causes harm or acts inappropriately? Who is accountable, and how can this be prevented?

It’s an important question because this is where the value added by an agent can slip into causing risk. An AI agent is more than a system that generates a response – it's designed to have autonomy. Which means it can interpret an objective, develop a plan, use tools, interact with other systems, and adjust its approach along the way.

Put simply, it can act.

An AI chatbot might provide an answer. But an AI agent could use that answer to update a record, send a message, move data, or start a business process without human interference. When done right, this can be a powerful tool for operational efficiency. But the risk of AI inaccuracy or hallucinations becomes much greater when AI agents have this autonomy to act on their outputs.

This is where agentic AI challenges our existing security assumptions. Cyber security has traditionally focused on establishing identity, controlling access, and detecting unauthorised activity. These controls remain important, but to remain secure while using an AI agent, we need to do more.

An AI agent’s identity may be valid, its credentials may be secure, its action may be permitted. But without the appropriate understanding of the nuance of a given situation, an action may not make sense in the context. So, how can organisations embrace agentic AI as part of their intelligent enterprise, while managing security threats?

Understand behavioural risk

Imagine you have an AI agent designed to process customer refunds. It receives a valid request, accesses the correct system, and uses its assigned permissions to approve a payment. Viewed separately, each step appears legitimate. Yet the overall action taken might be wrong. For example, the request might contain manipulated instructions, the refund might be unusually large, or the agent might have misunderstood the customer’s circumstances.

A conventional control could see an authorised identity performing an authorised action. Whereas a context-aware control would also ask whether the amount, timing, recipient, and sequence of decisions are consistent with the agent’s purpose.

Rather than your authorisation model only asking “Can this agent perform the action?” you must ask “Should this agent perform this action now?”. If your security control mechanism asks the latter, then you’re encompassing behaviour, purpose, and consequence into security decision-making.

Look beyond the model

AI security conversations often focus on the model. The model matters, but it’s only one part of an agentic system. Knowing when to put a human-in-the-loop is essential.

An agent can include prompts, memory, data sources, identities, plug-ins, application programming interfaces, and supporting infrastructure. Each component influences what the agent knows, how it interprets its objective, and what it can do.

Therefore, your threat modelling must follow the complete chain from instruction to outcome. When considering an agentic use case, start with these five questions:

1. What information can the agent access?
2. Which tools and systems can it use?
3. What decisions can it make independently?
4. How could its instructions or context be influenced?
5. How would we recognise behaviour that no longer matches its purpose?

Organisations often log whether access was granted or denied. But agentic systems also require an additional layer of visibility into the instructions an agent received, the reasoning behind its chosen action, the tools it selected, and the outcome it produced. With this additional context, an incident investigation can reveal what happened while also explaining why.

Design meaningful boundaries

There’s no single control that will make an AI agent secure. Safer adoption comes from designing several layers of protection around the decisions and actions that matter most. Plus, clearly establishing when an agent must reach out to a human as a final check and balance. To set out these clear boundaries:

• Give agents a defined purpose. Broad objectives such as “manage customer queries” can leave too much room for interpretation for the agent. A narrower, clearly defined task makes expected behaviour easier to describe, test, and monitor
• Apply least privilege. Each agent should only access the information, tools, and actions required for its specific purpose. Permissions should also distinguish between reading information, recommending an action, and executing it
• Make oversight proportionate. Human approval is particularly valuable when an action is unusual, high-impact, or difficult to reverse. Routine and reversible tasks may justify more autonomy for the agent
• Monitor patterns of behaviour. A single action may appear harmless while a sequence reveals something unexpected. Monitoring activity should consider volume, timing, destinations, tool use, and deviations from the agent’s normal activity
• Make intervention practical and timely. Agents should know when to reach out to a human e.g. at times of uncertainty, or as a random sample check. Teams should know who owns the agent, who can suspend it, how to revoke its access, and how to recover affected services.

These boundaries are not intended to prevent useful autonomy. They provide the confidence to increase autonomy where the evidence supports it.

Follow the chain of trust

In an AI supply chain, an agent might use a model from one provider, retrieve information through another platform, connect to third-party tools, and trigger actions across internal systems, with its behaviour affected by every part of the chain.

A new model version, a changed plug-in, or an unreliable data source could alter the agent’s decisions without anyone changing the agent itself. Security teams need visibility into these dependencies and a way to assess changes throughout the agent’s lifecycle. They also need to understand the boundaries the agent thinks it has for uncertainty and when it would and should bring a human into the loop.

Trusting each component separately does not automatically make the combined behaviour trustworthy. That's why context matters.

Learn from across disciplines

During the event, one of the most valuable things I took away from it was seeing how colleagues challenged the topic from different angles. Cyber security specialists focused on the technical threats. Architects explored dependencies and system boundaries. Risk and governance colleagues considered accountability, oversight, and evidence. Industry experts brought the operational consequences to life.

No single discipline had the complete answer, but together the discussion helped turn agentic AI security from an abstract technology issue into a practical design challenge.

The session also reflected one of the things I love most about working at PA: being in a room with people excited to test emerging ideas, challenge assumptions, share experience, and improve their thinking together.

Secure the action

Realising agentic AI’s potential depends on organisations establishing enough trust in agents to act safely and predictably, strengthening their organisational resilience.

The central lesson is clear: authorisation is necessary, but it’s no longer sufficient alone.

As AI moves from producing content to taking action, security teams must consider purpose, behaviour, context, identity, and access. Organisations that make this shift early will be better placed to increase AI agent autonomy with confidence, respond when behaviour changes, and build agentic systems that remain useful, resilient, and trustworthy.