AI Agents and Copilots: SAP Introduces Deeper Integrations
SAP has launched collaborative AI agents in Joule, its generative AI copilot, as a further effort to provide deeper integrations across its software ecosystem.
“You’ve likely all heard others in the industry talk about AI agents a lot over the past few months, but one thing you’re not hearing about is the ability for those agents to interact with other agents, with each other, to help solve more complex business challenges,” said Muhammed Alam, executive board member of SAP and its head of product engineering, at an online press conference Thursday.
“We are infusing Joule with multiple autonomous AI agents that will combine their expertise across the business functions to collaboratively accomplish complex workflows. These AI agents will help organizations unlock massive productivity gains by breaking down silos. This will free workers to collaborate in areas where human ingenuity is best suited.”
With these collaborative agents, SAP reflects a more significant movement in the enterprise software market to use generative AI technologies that leverage large language models (LLMs) such as Llama and Mistral and then training with SAP’s data.
Partnerships With Anthropic, Meta, Mistral AI
At the press event, Alam previewed partnership announcements to be made at this week’s SAP TechEd conference.
“With Anthropic, we’re introducing a capability within SAP AI core that makes it easy to build generative AI use cases for SAP applications with Anthropic models,” he said.
With a Meta partnership, he added, SAP will “continue to expand to execute on our multi-vendors vendor strategy for large language models, to deliver performance, cost efficiency and choice for our customers and internal developers.
“And finally, in our partnership with Mistral AI, we’re expanding to enhance SAP ABAP large language model code generation capabilities and development tools.”
SAP executives emphasized the importance of the cloud to its customers. SAP has invested in cloud native approaches, building its own Kubernetes infrastructure.
At SAP TechEd two years ago, the company detailed to The New Stack how Kubernetes served as an underpinning for SAP Build, a low-code service built on SAP Business Technology Platform (BTP) to scale out its multicloud service. SAP Build uses a metadata-based approach with cloud functions to offer a granular service for business people to build applications and websites, and automate processes.
The large application stack providers see ways with generative AI to better integrate with their software offerings, James Governor, co-founder of the analyst firm RedMonk, told The New Stack.
It’s a never-ending requirement. The need continues to extend with APIs, new interfaces, and better ways to query data stores.
Enterprise customers also want a level of security that they can get from something other than a consumer-oriented LLM. “Enablement, security, data residency, all that stuff is stuff that the enterprises expect their vendors to provide for them,” Governor said. “And that’s a lot of the blocking and tackling that SAP has to do.”
SAP and Kubernetes
Today, we know that Kubernetes has played an important role in generative AI deployments across the enterprise software market. Combined with Nvidia’s efforts to bring more efficiencies in its GPUs, deeper infrastructure utilization also comes. That allows customers to go from larger LLMs to smaller ones. There comes an efficiency that has benefits but also drives performance with AI agents.
SAP supports about 30 large language models, said Walter Sun, senior vice president and global head of AI, in last Thursday’s press conference.
“And so, since we build the AI agents, we can pick a language model by which each agent is powered,” Sun said. He added, “Some agents might be easier to answer questions. Some might be harder, so for the ones that are easier or more lightweight, we provide a smaller or cheaper model. And so that way, we can help control the [return on investment] and make it as cheap as possible.”
Lower costs, he noted, “correlate exactly to energy consumption. So basically, we’re both considering the cost for our customers to make it as low as possible and, at the same time, making sure our carbon footprint, our usage or compute on the environment is as small as possible.”
More Capabilities for SAP Build Code
Two years ago at TechEd, SAP started its journey in low-code development with SAP Build. Last year, SAP added more capabilities for Java and JavaScript developers with generative AI capabilities using Joule, the company’s proprietary co-pilot platform.
This year, SAP is building new generative capabilities within SAP Build Code and adding support for ABAP development. SAP ABAP is a high-level programming language used in the SAP software ecosystem. Its origins date back to the 1980s. SAP Build Code allows developers to build applications that bridge modern development practices and cloud native approaches to ABAP to enhance and better optimize application environments, particularly SAP’s Enterprise Resource Planning (ERP) technologies.
SAP’s newest work involves its generative AI capabilities across its ecosystem. Joule, their copilot, provides code explanations, documentation, and collaborative AI agents. The company sees generative AI as applicable for better understanding legacy code.
Among the changes ahead:
- In early 2025, ABAP developers will get access to Joule to generate code and perform unit tests, further allowing developers to understand ABAP legacy code bases better.
- Joule will also be added to the SAP HANA Cloud by the end of the year. The goal is to make the in-memory database easier to use across various services and again using natural language processing. Through this new capability, people with less sophisticated tech skills will be able to ask questions and receive suggestions, which previously only people with extensive knowledge in database administration would understand how to access.
In addition, SAP has launched a knowledge graph engine that will integrate with Joule to help developers use data more effectively. SAP Knowledge Graph connects the business context captured in SAP applications. The solution anchors SAP’s Joule co-pilot with business data, so it can deliver more reliable, context-aware business insights with far fewer inaccuracies.
SAP Knowledge Graph grounds AI in specific SAP Business semantics and their interrelationships. SAP executives say this reduces the risk of inaccurate and irrelevant results and makes it easier for organizations to build intelligent applications and leverage generative AI more effectively by offering ready-to-use relationships between business entities like purchase orders, invoices, and customers.
The service reduces the complexity of manual data modeling. SAP also introduced new, embedded data lake capabilities for SAP DataSphere. Businesses can store raw data in its original context, allowing them to improve decision-making, operational efficiency, and AI-driven capabilities to provide context-aware insights for AI use cases.
Other enhancements include:
- Generative AI will allow customers to connect data from different systems to use on the sites using SAP Build Work Zone, a low-code tool for building personalized business sites. Joule’s generative AI capabilities will offer users intelligent support ways to connect data.
- Joule Studio provides SAP Build with natural language processing for planning and defining workflows.
4 Challenges for Developers
Developers in the SAP ecosystem face four primary challenges, Sid Misra, SAP Build’s vice president for product marketing, told The New Stack.
- There’s a developer scarcity, and it’s hard to find people with the right expertise in the organization.
- Often, the problems get tackled, starting from scratch, which means development delays.
- AI today is pretty generic. It needs to understand your data, business or company processes.
- Applications and platforms represent two different worlds, layered together. So, anyone building solutions today is bolting applications on top of existing critical platforms.
Low code, combined with generative AI, gives business users tools for connecting data across the SAP ecosystem. With Joule, a user may provide a prompt asking for the correct SQL query. Joule may ask for APIs that a user may leverage. But really, the assistance applies across the board. Joule can help make suggestions for data models and help create APIs to access SAP HANA, the company’s in-memory database.
Professional developers play a more significant role in SAP’s strategy for 2025, especially as the demand continues to integrate services and applications from outside the SAP ecosystem.
For example, SAP Fiori, a user experience layer, fits with SAPUI5, a Javascript framework that integrates with Joule. In a recent post on the SAP site, Paola Laufer explained how to create a full-stack application with SAP Fiori UI using Joule in SAP Build Code. It shows how to use Joule to create data entities, enhance application data and make application logic.
To help developers start from scratch, SAP has built hundreds of pre-build solutions for developers.
“So that adds the faster time to market and more developer efficiency for them,” Misra said.
Code Explanation and Documentation Capabilities
In 2023, SAP added code generation capabilities to generate Java and JavaScript code for business logic, data models and testing. This year, SAP also added code explanation capabilities and documentation.
“So this is very helpful from a developer productivity perspective, especially if we knew developers joining the team,” Misra said. “You want to know, hey, what is this code? What does it do? So we have that generative AI capabilities around code explanation and also creating documentation.”
ABAP developers will also get new code explanation capabilities.
“We have 5 million ABAP developers,” Misra said. “We want to make them even more productive. So what we’re doing with ABAP developers is, we’re adding new code generation and also that code explanation capabilities within ABAP. So what’s very special about Joule co-pilot is the variety of LLMs we have and the quality of the data; we have SAP trained on that as well. And, just the context of best practices around software development and also SAP development. We’ve trained Joule for that as well.”
AI agents have made significant progress in the past year. The market has moved from single agents to more coordinated, orchestrated capabilities.
“What is really important, when it comes to agents, is both a higher level of autonomy and a higher level of collaboration,” said Philip Herzig, SAP’s chief AI officer. “Autonomy is achieved through things like planning, reasoning, reflection and taking not only some of the latest models — like, for example, one from OpenAI, which is much better in terms of reasoning and reflection capabilities, a chain of thought prompting, or other techniques that you may use for this.
“But of course, also to self-correct or to break a complex task down into multiple steps, which is maybe not answered only by a single, large language model, but maybe multiple, specialized models that underlie these agents.”
For instance, he said, “Take, for example, an agent that is more capable or more specialized — for example, in the financial domain. And then you have another agent, for example, where you have a large language model or some other model underneath that is more specialized in the [human resources] domain, and they really work together to come up with individual solutions that neither of those can solve alone.”