As enterprises build and deploy generative AI applications and AI agents, they seem to have set aside concerns about AI technology.
However, beneath the facade of adoption lies a hidden truth about the obstacles businesses face as they try to move from generative AI applications to fully automated systems. Businesses still face significant questions about measuring ROI, how much to invest and how to account for costs when AI goes wrong. There’s also figuring out how to bring employees along and how to work through technology basics, such as product lifecycle management.
Skepticism about generative AI has dwindled as applications such as ChatGPT Enterprise and Anthropic Claude Cowork have become commonplace in enterprises. The story becomes complicated when enterprises move from generative AI applications to creating AI agents tasked with different jobs, aiming to move toward true autonomy and achieve agentic AI.
As for generative AI, particularly the use of large language models (LLMs), more enterprises are on board than maybe two or three years ago.
Start Small and Think Big
ChatGPT launched in late 2022, and today, 44% of enterprises are now deploying generative AI models, according to a recent Omdia study of 400 enterprises in North America with 1,000 or more employees. (Omdia is a division of Informa TechTarget.)
Pinterest, for example, recently launched Pinterest Assistant, a multimodal AI Agent that helps shoppers find items. Starbucks also launched its AI Ordering Companion in April, a ChatGPT-like generative AI tool that lets users start orders and customize drinks. These two examples show that both experimentation and deployment are up.
Not only is deployment up, but Omdia also found that 32% are experimenting with pilot projects while 4% have no plans to adopt generative AI in the near future.
Mark Beccue, an analyst at Omdia, said that not only are most enterprises using and adopting generative AI technologies, but AI also ranks among the highest technology priorities they are focusing on.
“When it comes to budgets and personnel and resource allocations, GenAI and the derivatives of AI are taking up a lot of space,” Beccue said.
But even as enterprises get more comfortable with generative AI, the next iteration of AI agents, which are designed to complete discrete tasks, and agentic AI, a series of agents working together to complete a process autonomously, are still a long way away. In most cases, enterprises are using AI agents internally or with what experts call low-risk implementation efforts.
Plus, AI technology is known to hallucinate and, in some cases, go rogue, making enterprise AI maturity more complicated for organizations.
The ROI Disconnect
With ROI, there is a divide between senior executives who want to see a positive return from generative AI investments and those implementing the technology, said David Nicholson, an analyst at Futurum Group. Often, senior executives are looking for the financial returns from using generative AI and AI agents.
But this wave of generative AI (in which ChatGPT has led to the popularity of the technology) in the enterprise is only about five years old, and even an understanding of how much an enterprise needs to spend for an implementation to be successful is unknown. AI agents and agentic AI are even less mature, making ROI even more elusive.
Right now, enterprises are grappling with the number of tokens they need to have for agents to accomplish the same tasks as humans. But broader questions remain about infrastructure, how long the value of GPUs will hold, and the cost of inference. Enterprises are grappling with the inference cost and whether it will also affect the number of tokens they can use. With agentic AI, many companies still don’t know how long it will take for the technology to mature to the point at which humans no longer need to be involved.
“Once you get into that realm of, ‘How do we actually do this,’ it gets very complicated very quickly,” Nicholson said. He added that vendors such as Nvidia and hyperscalers such as Google have provided enterprises with portfolios of tools, but they fail to address how to implement the technology in a way that measures actual cost and value.
It is also unclear which vendor an enterprise should partner with. The AI marketplace is rife with options, and enterprises often have to decide between partnering with frontier AI labs such as Anthropic and OpenAI or with SaaS providers with domain expertise, such as Salesforce, DataRobot and IBM. Nicholson said there is no clear path for one partner to lead to greater success or failure than another.
“All of this is still based on a bit of a wing and a prayer,” he said.
Alongside measuring ROI, companies must consider the cost if something goes wrong.
Recently, an AI coding agent running in Cursor, an integrated development environment powered by Claude, deleted the SaaS startup’s entire database and its backup rather than fix the credential issue it was tasked with resolving. To recover, its founder tried to use an older, off-site backup, and he and his team also attempted a manual rebuild. Ultimately, its infrastructure host, Railway, had to manually reconstruct the data on the backend for the company to fully recover.
“If an agent does something bad, and the business partner, customers, lose money or lose reputation … they would go after the business,” said Irving Wladawsky-Berger, a research affiliate at the MIT Sloan School of Management, during an interview at the MCP Dev Summit in New York in April. “Businesses have to be very careful.”
That level of liability with generative AI and AI agents is another reason experts suggest starting with low-risk use cases, such as writing or summarization, Wladawsky-Berger said.
“What we need to do is buy useful, simple applications to implement initially and then as we learn more, we can go bigger,” Wladawsky-Berger said.
On top of the people part, companies also have to think differently about how to manage AI.
“This isn’t like adopting software,” said Omdia’s Beccue. “The product lifecycle or the lifecycle management of AI is a different animal.”
The Human Factor
Companies also have to face another, less technical challenge: the human factor, according to Rob O’Donohue, an analyst at Gartner.
Organizations often contain three distinct groups that influence AI adoption, according to Gartner. They are AI achievers, who make up a small percentage of the organization and are quick to adopt; those who are hesitant and indecisive about AI but can be swayed; and those who are unwilling or unable to embrace AI technology. By knowing which category each individual or employee falls into, enterprises can determine whether part of what is hindering their adoption story is a lack of a culture that accepts generative AI or agentic AI.
The only way to address the challenge of the human element is to create an environment where employees are willing to use the technology, O’Donohue said.
“You need to put the right kind of processes in place at an organizational level to help people use the tool, and in many ways not discourage them from using it,” he said.
Enterprises might also need to address the emotional barriers that keep some employees from using new technology, especially those who are averse to AI technology because of fears that it could eliminate their position.
“Unless you’re bringing those individuals along and cocreating what their job of the future will look like, they’re still going to feel uncertain,” O’Donohue added. “So, the change is happening to them rather than the changes happening with them.”
Beccue added that while enterprises need the right culture and processes in place, they also need to face reality.
“Be pragmatic, be realistic about what you are trying to accomplish and start with the basics, which is, ‘What problem am I trying to solve?’ When you start with those basics, you have a better path to how AI can help,” Beccue said.
Moreover, patience will be the best way to address some of the challenges enterprises are seeing and to answer questions about ROI, and whether enterprises are investing in the right infrastructure and software, Futurum Group’s Nicholson said.
“The only thing that’s going to solve that is going to be time,” he said. “We’re talking a year, two years from now, before the uncertainty will be smoothed out.”
