Microsoft’s new AI agents support 1,800 models (and counting)

AI agents are the talk of the enterprise right now. But, business leaders want to hear about tangible results and relevant use cases — as opposed to futuristic, not-quite-there-yet scenarios — and demand tools that are easy to deploy and use and, further, that support their preferred model(s). 

Microsoft claims to have all these concerns covered with new no-code and low-code capabilities in Microsoft 365 Copilot. Today at Microsoft Ignite, the tech giant announced that users can now build their own custom autonomous agents or deploy out-of-the-box, purpose-built agents. And, they can do this via a bring-your-own setup that provides them access to the 1,800-plus models in the Azure AI catalog. (See our separate story today about how Microsoft has quietly assembled the largest AI agent ecosystem — and no one else is close).

“Companies have done a lot of AI exploration and really want to be able to measure and understand how agents can help them be more efficient, improve performance and decrease cost and risk,” Lili Cheng, corporate VP of the Microsoft AI and research division, told VentureBeat. “They’re really leaning into scaling out their copilots.”

Supporting bring-your-own-knowledge, bring-your-own-model

According to IDC, in the next 24 months, more and more companies will build custom, tailored AI tools. Indeed, vendors — from tech giants such Salesforce and Snowflake to smaller players like CrewAI and Sema4.ai — are increasingly pushing platforms to market that promise to revolutionize enterprise operations. 

Microsoft introduced Copilot in February 2023, and has now infused it with a suite of new capabilities to support agentic AI. Autonomous capabilities now in public preview allow users to build agents that act on their behalf without additional prompting. This means agents can work and act in the background without human oversight. 

Users can use templates for common scenarios (such as sales order and deal accelerator agents) in Copilot Studio. Or, more advanced developers can take advantage of a new Agent SDK (now available in preview) to build full-stack, multichannel agents that integrate with various Microsoft services and can be deployed across Microsoft, third-party and web channels. 

New integrations with Azure AI Foundry will support bring-your-own-knowledge (custom search indices can be added as a knowledge source) (now in preview) and bring-your-own- model (now in private preview). This will allow users to pull from the 1,800-some-odd models (and counting) in Azure’s catalog. 

This element is critical, as users are demanding the ability to securely use proprietary data and combine and test different models without getting locked in to one or the other. “People want a variety of models, they want to be able to fine-tune models,” said Cheng. 

Ready-made agents for HR, translation, project management

But not all tasks require a custom solution; already-built models can be useful across enterprises. Microsoft is releasing several ready-made agents in Copilot that can handle simple, repetitive tasks or more complex multi-step processes. These include: 

• Agents in SharePoint, which allows users to create their own tailored agents that they can give names and personalize. Users can ask questions and receive real-time answers and share agents across emails, meetings and chats. Microsoft emphasizes that agents follow existing SharePoint user permissions and sensitivity labels to help ensure that sensitive information isn’t overshared.
• Employee self-service agent, which answers common workplace policy-related questions and takes action on HR and IT-related tasks. For instance, employees can retrieve benefits and payroll information, request a new device or start a leave of absence form. 
• Facilitator agent, which takes real-time notes in Teams and chats and provides a summary of important information as the conversation is unfolding. 
• Interpreter agent, which provides real-time translation in Teams meetings in up to nine languages. Participants can also have Interpreter simulate their voice.
• Project Manager agent, which automates processes in Planner, handling projects from creation to execution. The agent can automatically create new plans from scratch or use templates; it then assigns tasks, tracks progress, sends notifications and provides status reports. 

Further, a new Azure AI Foundry SDK offers a simplified coding experience and toolchain for developers to customize, test, deploy and manage agents. Users can choose from 25 pre-built templates, integrate Azure AI into their apps and access common tools including GitHub or Copilot Studio. 

Cheng pointed to the importance of low-code and no-code tools, as enterprises want to accommodate teams with a range of skills. “Most companies don’t have big AI teams or even development teams,” she said. “They want more people to be able to author their copilots.”

The goal is to greatly simplify the agent-building process so that enterprises “build something once and use it wherever their customers are,” she said. Tooling should be simple and easy to use so that app creators don’t even know if things are getting ever more complicated on the back end. Cheng posited: “Something might be more difficult, but you don’t know it’s more difficult, you just want to get your job done.”

McKinsey, Thomson Reuters use cases

Initial use cases have revolved around support, such as managing IT help desks, as well as HR scenarios including onboarding, said Cheng. 

McKinsey & Company, for its part, is working with Microsoft on an agent that will speed up client onboarding. A pilot showed that lead time could be reduced by 90% and administrative work by 30%. The agent can identify expert capabilities and staffing teams and serves as a platform for colleagues to ask questions and request follow-ups. 

Meanwhile, Thomson Reuters built an agent to help make the legal due diligence process — which requires significant expertise and specialized content — more efficient. The platform combines knowledge, skills and advanced reasoning from the firm’s gen AI tool CoCounsel to help lawyers close deals more quickly and efficiently. Early tests indicate that several tasks in these workflows could be cut by at least 50%. 

“We really see people combining more traditional copilots — where you have AI augmenting people skills and providing personal assistance — together with autonomous systems,” said Cheng. Agents are increasingly authoring processes and workflows and working across groups of people and in multi-agent systems, she noted. 

AI agents aren’t new (but using them on top of LLMs is)

While they may be all the talk now, agents are not new,  Microsoft Source writer Susanna Ray emphasizes in a blog post out today. “They’re getting more attention now because recent advances in large language models (LLMs) help  anyone — even outside the developer community — communicate with AI,”  she writes. 

Agents serve as a layer on top of LLMs, observing and collecting information and providing input so that together they can generate recommendations for humans or, if permitted, act on their own. “That agent-LLM duo makes AI tools more tangibly useful,” Ray notes, adding that agents will become even more useful and autonomous with ongoing innovations with memory, entitlements and tools.

Cheng pointed out that Microsoft began talking about conversational AI about eight years ago. Before AI agents, conversation data “was always kind of lost and siloed.” Now, agentic AI can bring intelligence to users and provide context in real time. 

“People just want that tooling to be more natural,” she said. “It’s phenomenal that we can do a lot of these things that we dreamed about. Being able to combine all these sources effortlessly is really groundbreaking.”