Microsoft is aiming to expand its ecosystem of AI-powered apps and services called “copilots” with plugins from third-party developers.
Today at its annual Build conference, Microsoft announced that it will use the same plug-in standard that its closest collaborator, OpenAI, introduced for ChatGPT, its AI-powered chatbot. Microsoft Edge sidebar), Dynamics 365 Copilot, Microsoft 365 Copilot, and the recently launched Windows Copilot.
“I think this will become an expectation for how all software works for years to come,” Kevin Scott, Microsoft’s CTO, said in a blog post shared with AapkaDost last week.
Bold statements aside, the new plugin framework lets Microsoft’s family of “copilots” — apps that use AI to help users with various tasks, such as writing an email or generating images — interact with a range of various software and services. Using IDEs such as Visual Studio, Codespaces, and Visual Studio Code, developers can build plug-ins that retrieve real-time information, ingest business or other corporate data, and take action on behalf of a user.
For example, with a plug-in, the Microsoft 365 Copilot can arrange a trip in accordance with a company’s travel policy, query a site like WolframAlpha to solve a comparison, or answer questions about how certain legal issues at a company have worked in the past. were handled.
Customers in the Microsoft 365 Copilot Early Access Program (plus ChatGPT Plus subscribers) will have access to new plugins from partners including Atlassian, Adobe, ServiceNow, Thomson Reuters, Moveworks, and Mural in the coming weeks. Bing Chat, meanwhile, will see new plugins added to the existing collection of Instacart, Kayak, Klarna, Redfin, and Zillow, and those same Bing Chat plugins will come to Windows within Windows Copilot.
For example, the OpenTable plugin allows Bing Chat to search restaurants for available bookings, while the Instacart plugin lets the chatbot create a dinner menu, turn it into a shopping list, and place an order for the ingredients to be delivered. Meanwhile, the new Bing plugin brings web and search data from Bing to ChatGPT, complete with quotes.
A new framework
Scott describes plugins as a bridge between an AI system, such as ChatGPT, and data that a third party wants to keep private or proprietary. A plug-in gives an AI system access to those private files, allowing it to answer a question about company-specific data, for example.
There is certainly a growing demand for such a bridge as privacy becomes a major issue with generative AI, which tends to leak sensitive data, such as phone numbers and email addresses, from the datasets it has been trained on. To minimize the risks, companies including Apple and Samsung have banned employees from using ChatGPT and similar AI tools amid concerns that employees may mishandle and leak confidential data to the system.
“What a plugin does is it says, ‘Hey, we want to make that pattern reusable and put some limits on how it’s used,'” John Montgomery, CVP of AI platform at Microsoft, said in a canned statement. .
There are three types of plugins within Microsoft’s new framework: ChatGPT plugins, Microsoft Teams message extensions, and Power Platform connectors.
Image Credits: Microsoft
Teams message extensions, which let users interact with a web service through buttons and forms in Teams, aren’t new. Neither are Power Platform connectors, which act as a wrapper around an API that allows the underlying service to “talk” to apps in Microsoft’s Power Platform portfolio (e.g., Power Automate). But Microsoft is expanding their reach, allowing developers to tap into new and existing message extensions and connectors to extend Microsoft 365 Copilot, the company’s assistant feature for Microsoft 365 apps and services like Word, Excel, and PowerPoint.
For example, Power Platform connectors can be used to import structured data into the “Dataverse”, Microsoft’s service that stores and manages data used by internal business apps, which Microsoft 365 Copilot can then access. In a demo during Build, Microsoft showed how Dentsu, a PR firm, tapped Microsoft 365 Copilot along with a plug-in for Jira and data from Atlassian’s Confluence without having to write new code.
Microsoft says developers can create and debug their own plugins in a number of ways, including through the Azure AI family of apps, which adds capabilities to run and test plugins against private company data. Azure OpenAI Service, Microsoft’s managed enterprise-focused product designed to give businesses access to OpenAI technologies with added governance features, also supports plugins. And Teams Toolkit for Visual Studio is getting plugin testing features.
Switching to a platform
As for distribution, Microsoft says developers can configure, publish, and manage plugins through the Developer Portal for Teams, among other things. They will also be able to monetize it, although the company was not clear on exactly how the prizes will work.
Anyway, with plugins, Microsoft is playing to stay in the highly competitive generative AI race. Plugins are essentially transforming the company’s “copilots” into aggregators – putting them on their way to becoming one-stop shops for both business and consumer customers.
Microsoft no doubt views the lock-in capability as increasingly important as the company faces competitive pressure from both startups and tech giants building generative AI, including Google and Anthropic. You might imagine plugins becoming a lucrative new revenue stream as apps and services increasingly rely on generative AI. And it could allay the fears of companies claiming that generative AI trained on their data violates their rights; Getty Images and Reddit, among others, have taken steps to prevent companies from training generative AI on their data without any form of compensation.
I would expect rivals to answer Microsoft and OpenAI’s plugin framework with their own plugin frameworks. But Microsoft has a first-mover advantage, as OpenAI had with ChatGPT. And that is not to be underestimated.