As Companies Adopt AI Agents, Where Should They Allocate Their Security Budgets?

The company providesAI IntelligenceWhen crunching the numbers, people often prioritize model invocation fees, deployment costs, and savings in man-hours. But once the system is actually integrated into business operations, another set of considerations quickly comes to light: Who authorizes its operation? Who can detect problems if something goes wrong? And who has the authority to halt it in a timely manner? If the security budget remains limited to a single pre-launch audit, it will be difficult to adequately cover the ongoing operation of the AI agent.

The risk mitigation measures announced by the British government on October 12, as well asAnthropicThe cybersecurity verification program, which was revised a week ago, provides two observation windows. This article concludes that enterprise procurement now requires a more comprehensive cost framework: one that factors in licensing, process oversight, independent audits, and incident response alongside model capabilities when determining pricing.

The U.K. Proposes a New Agenda; Businesses Must First Clarify the Scope of Its Application

据British Government Announcement, October 12... The government will expand pilot programs that utilize cutting-edge AI to strengthen cybersecurity defenses and study how to respond to serious incidents involving autonomous intelligent agents. The announcement also proposes exploring ways to strengthen the Cybersecurity and Resilience Act and related systems, as well as studying the identification and reporting of serious incidents and “near misses.”

The key words here are “exploration” and “research.” The details of the measures still need to be discussed with lawmakers, technical experts, and relevant operators; therefore, it cannot be claimed on this basis that the United Kingdom has already required all companies to comply with a new set of incident reporting obligations for artificial intelligence systems.

On the same day, the UK AI Safety InstituteAISI's Technical AgendaFocus on cutting-edge AI development organizations: Risks may arise during the training, evaluation, and deployment phases, and oversight mechanisms and evaluation methods need to keep pace with evolving capabilities. AISI has made it clear that this initiative aims to promote voluntary technical practices, which may evolve into formal standards once mature—it is not intended to impose mandatory requirements on developers.

For ordinary companies, these documents are better suited as a basis for procurement inquiries. It is often more useful for a supplier to explain how the model is monitored before and after deployment and how incidents are detected than to provide a vague promise of “safety and reliability.”

The first funds should be spent on determining who can do what.

Anthropic announced on October 6 thatExpanded Cyber Verification Program (CVP)...is designed for verified cybersecurity professionals and consists of three tiers: Defense, Red Team, and Specialized. The Defense tier covers security operations, incident response, and related tasks; the Red Team tier adds authorized penetration testing; and the Specialized tier is intended for a select few organizations authorized to test systems that could impact human safety or market operations, and to conduct in-depth reviews in collaboration with the U.S. government.

This isn’t a situation where a typical company can simply purchase a more expensive plan to completely lift all restrictions. The Red Team tier still restricts activities that could cause physical harm or widespread destruction, and individual researchers are currently ineligible to apply for the Red Team tier. Access rights are determined by the scope of work and verification results, not solely by a willingness to pay.

Following this approach, companies can start by creating a checklist of actions for their AI agents: accessing customer data, generating modification recommendations, updating business systems, and implementing production changes—each of which requires different authorizations. The budget should cover independent identities, short-term credentials, the principle of least privilege, and approval processes for high-risk operations. Allowing a single generic account to simultaneously access email, databases, and operations platforms may save on initial integration costs, but this could later result in difficulties in assigning accountability.

Process monitoring should focus on actions; it shouldn't rely solely on chat logs.

When an agent completes a task, it may call multiple tools in succession. The fact that the final response appears correct does not prove that every intermediate step was within the scope of authorization. Therefore, enterprises should maintain a traceable record of tool calls, permission changes, external connections, and data writes, and route alerts to the individuals who actually have the authority to handle them.

CVP'sOfficial Safety RequirementsSpecific guidelines are provided: Authorized users and workspaces must have verifiable usage records; in environments where red team activities involve agents or offensive operations, outbound connections must be restricted and logged; and procedures for credential revocation and incident response must be in place. These are the requirements for access to and continued use of the program and should not be interpreted as industry-wide regulations.

For enterprises, the key takeaway is to enforce controls at the system boundaries. For example, which targets a test environment can access should be constrained by network rules that agents cannot modify on their own. Monitoring costs encompass more than just log storage; they also include alert tuning, on-call support, and post-incident analysis. Otherwise, logs will continue to pile up, and business teams may still only become aware of problems after receiving customer complaints.

Independent evaluations and emergency shutdowns must be accepted together.

AISI has identified the capabilities, scope of access, and resources of independent evaluators as technical issues requiring further attention. The implication for buyers is that security assessments require sufficient, verifiable evidence and cannot merely rest on the supplier’s demonstrations. Companies also do not need to outsource all testing to expensive external organizations; instead, they can determine the depth of testing based on business consequences.

For applications that only generate internal drafts, the focus can be on data isolation and manual review; for agents capable of modifying orders, issuing payment instructions, or interacting with production systems, testing should be conducted independently of the development team. Testing should cover actual tool permissions, exceptional inputs, and failure recovery, and should be re-evaluated following any significant changes to the model or toolchain.

Emergency stops, in particular, require on-site verification. The purchasing party may require the supplier to demonstrate in a controlled environment whether the task queue stops after an administrator issues a stop command, whether permissions for external tools can be revoked, and how changes that have already been made can be identified and rolled back. The mere presence of a button on the interface does not mean that the business operation has been terminated.

Accordingly, acceptance criteria could include the time required to detect anomalies, the time required to revoke permissions, and the results of recovery. These are procurement recommendations proposed in this paper and do not constitute the standardized technical criteria already published in the aforementioned U.K. document.

Incident response also requires separate staffing and funding for drills. It should be clearly defined in advance which anomalies must be escalated, who has the authority to revoke access during nighttime hours, and who is responsible for preserving evidence and notifying affected business units. Even if an unauthorized access attempt is promptly blocked, it is worth reviewing how it occurred. Treating “no loss incurred” as a reason not to address the issue will result in missing opportunities to patch control gaps.

The opportunity for domestic companies lies in turning security into a deliverable capability.

When domestic companies evaluate overseas products, they must first confirm the scope of service availability, data flow, and contract terms. CVP should not be assumed to be a capability directly available to all domestic customers. ItsHelp CenterIt also states that authorized access does not automatically extend to customer-facing products or services; separate approval must be sought for related productization uses. There are also specific requirements and exceptions regarding data retention. Security service providers may not treat internal use authorization as authorization for resale to third parties.

It is even more important to include these issues in the procurement scope upfront: who is responsible for integrating the identity system, who maintains the monitoring rules, who receives reports of anomalies, and who handles downtime and recovery. Domestic models and on-premises deployment solutions should also be evaluated based on these criteria. The deployment location alone cannot replace access control design and incident drills.

When budgets are limited, prioritize ensuring permission isolation, revocation, and restoration for high-impact actions, and then gradually expand assessment and monitoring coverage. Every investment should yield a demonstrable, verifiable result with a designated person in charge, to avoid purchasing a large number of features only to find that no one is handling the alerts.

This also creates business opportunities for local security vendors, system integrators, and evaluation agencies: developing sustainable delivery capabilities in agent identity management, tool invocation auditing, independent testing, and emergency response services. Their value will ultimately be reflected in reducing losses from operational errors, shortening downtime, and helping customers confidently expand the scope of their applications. Only when enterprises are confident enough to entrust critical tasks to intelligent agents will security investments truly contribute to productivity gains.

Information as of October 13, 2026, Beijing Time. Source:UK Government Announcement,AISI Technical Agenda,Anthropic CVP AnnouncementandProgram Description,Safety Requirements.