Artificial intelligence has leapt from a fringe technology to a core business enabler, with corporate executives feeling increasingly confident about its ability to deliver value. But according to Amazon Web Services executive Swami Sivasubramanian, the focus of CEOs' concerns is changing.

Sivasubramanian said business leaders are no longer asking if artificial intelligence can benefit their companies, but are instead concerned with a more involved question: Can businesses safely trust AI with important decisions?

CEO Are Becoming More Confident About AI's Business Value

In recent years businesses have poured billions into generative AI, automation and AI-powered software. For many executives these were initially uncertain investments, with doubts about whether the technology investments would deliver tangible returns.

That dynamic is set to change, as companies begin looking at deploying AI projects in the real-world, seeking tangible productivity gains, cost reductions and new sources of revenue.

AWS CEO Matt Garman recently said enterprise customers were seeing positive returns from their AI investments, showing how rapidly corporate attitudes to the technology was shifting.

The New AI Fear: Can Companies Trust It?

• Speaking at the Madrona IA40 Summit, Sivasubramanian pointed to the next major challenge ahead for enterprises: trust in AI systems.
• As AI becomes more capable, enterprises are beginning to consider giving AI systems greater responsibility for making decisions and executing tasks with limited human intervention.
• This brings a different risk profile than businesses face when simply using AI to summarise a document, analyse information or complete routine tasks.
• A business may be happy for an AI system to perform these tasks and even to make judgements about the most suitable approach, but to allow an AI system to make decisions which could have an outsized impact on customers, employees or operations is a much greater step.
• The concern is therefore shifting from 'Will AI create value?' to 'Can we safely give AI more control?'

AI Safety Is Becoming A Business Priority

• AI safety is becoming an increasing concern as businesses deploy more capable AI systems, seeking to understand if models can behave reliably, if their judgements can be audited and whether humans can intervene if they wish.
• Zico Kolter, chair of OpenAI's safety and security committee and a professor at Carnegie Mellon University has also highlighted the challenge of putting powerful AI systems in charge of critical decision-making without appropriate safeguards.
• This is especially true as businesses begin to move towards 'agentic' AI systems, capable of performing a variety of tasks and taking actions, rather than simply responding to individual prompts.
• AWS has already established a dedicated team focused on agentic AI, with Sivasubramanian driving the initiative. The company has described AI agents as an important opportunity for the next phase of enterprise AI development.

Why AI Agents Could Change The Equation

Traditional generative AI applications essentially respond to a user's instructions, but agentic AI is built to be able to plan and execute a wider series of actions in pursuit of a specific objective.

This could have major productivity gains, but has also increased the importance of governance, as an AI agent that has access to a number of the company's systems and tools could take a series of actions which may not be fully understood by a human authorising access.

Organizations need monitoring, security and governance practices to ensure appropriate controls and guardrails are in place.

AWS's own research into enterprise adoption of AI highlights the importance of governance as they begin to roll out thousands of AI agents and connect them to increasingly complex business processes.

The AI Race Is Now Also A Trust Race

• The next phase of artificial intelligence adoption may therefore not be defined by which company possesses the most capable AI model.
• Businesses also need to determine if they have the right infrastructure in place to make safe use of the technology.
• Companies which can balance the capabilities of AI with appropriate governance, security, human oversight and reliable data may have an advantage over those which adopt AI technologies without such controls.
• As AI becomes more deeply embedded in business operations, the question of trust may become just as important as the question of performance.
• For CEOs the discussion around AI is no longer simply about proving that the technology can create value, but what level of responsibility can safely be delegated to artificial intelligence.