Skip to main content

Nvidia and Microsoft are solving a big problem with Copilot+

The Surface Laptop running local AI models.
Luke Larsen / Digital Trends
Computex 2024 logo.
This story is part of our coverage of Computex, the world's biggest computing conference.

When Microsoft announced Copilot+ PCs a few weeks back, one question reigned supreme: Why can’t I just run these AI applications on my GPU? At Computex 2024, Nvidia finally provided an answer.

Nvidia and Microsoft are working together on an Application Programming Interface (API) that will allow developers to run their AI-accelerated apps on RTX graphics cards. This includes the various Small Language Models (SLMs) that are part of the Copilot runtime, which are used as the basis for features like Recall and Live Captions.

Recommended Videos

With the toolkit, developers can allow apps to run locally on your GPU instead of the NPU. This opens up the door to not only more powerful AI applications, as the AI capabilities of GPUs are generally higher than NPUs, but also the ability to run on PCs that don’t currently fall under the Copilot+ umbrella.

Get your weekly teardown of the tech behind PC gaming
Check your inbox!

It’s a great move. Copilot+ PCs currently require a Neural Processing Unit (NPU) that’s capable of at least 40 Tera Operations Per Second (TOPS). At the moment, only the Snapdragon X Elite satisfies that criteria. Despite that, GPUs have much higher AI processing capabilities, with even low-end models reaching to 100 TOPS, and higher-end options scaling even higher.

In addition to running on the GPU, the new API adds retrieval-augmented generation (RAG) capabilities to the Copilot runtime. RAG gives the AI model access to specific information locally, allowing it to provide more helpful solutions. We saw RAG on full display with Nvidia’s Chat with RTX earlier this year.

Performance comparison with the RTX AI toolkit.
Nvidia

Outside of the API, Nvidia announced the RTX AI Toolkit at Computex. This developer suite, arriving in June, combines various tools and SDKs that allow developers to tune AI models for specific applications. Nvidia says that by using the RTX AI Toolkit, developers can make models four times faster and three times smaller compared to using open-source solutions.

We’re seeing a wave of tools that enable developers to build specific AI applications for end users. Some of that is already showing up in Copilot+ PCs, but I suspect we’ll see far more AI applications at this point next year. We have the hardware to run these apps, after all; now we just need the software.

Jacob Roach
Lead Reporter, PC Hardware
Jacob Roach is the lead reporter for PC hardware at Digital Trends. In addition to covering the latest PC components, from…
Microsoft outlines Recall security: ‘The user is always in control’
Recall promotional image.

Microsoft just released an update regarding the security and privacy protection in Recall. The blog post outlines the measures Microsoft is taking to prevent a data privacy disaster, including security architecture and technical controls. A lot of the features highlight that Recall is optional, and that's despite the fact that Microsoft recently confirmed that it cannot be uninstalled.

Microsoft's post is lengthy and covers just about every aspect of the security challenges that its new AI assistant has to face. One of the key design principles is that "the user is always in control." Users will be given the choice of whether they want to opt in and use Recall when setting up their new Copilot+ PC.

Read more
Microsoft is giving up control of the Copilot key
Windows 11 logo on a laptop.

In a Windows Insider Blog post, Microsoft recently announced that it is rolling out the Windows 11 Insider Preview Build 22635.4225 (KB5043186) update. It's a relatively small update, but it finally gives users control of the dedicated Copilot key that's showing up on an increasing number of laptops.

In the blog post, Microsoft detailed how it is giving users more customization freedom by adding the option to configure the Copilot key, which can open an app that's MSIX packaged and signed. This is good news since the app meets security and privacy requirements to keep your PC safe. When the option is available more broadly, you should find it by going to Settings > Personalization> Text Input.

Read more
Copilot Wave 2: Here are all the new AI features to try out
Copilot Pages open in a graphic.

Microsoft has announced an update to Copilot, the company's all-in-one AI assistant. "Wave 2," as Microsoft calls it, is a series of updates that gives Copilot more capabilities within popular Office applications, Copilot agents for businesses, and even a new feature called Copilot Pages.

Let's start with Pages first. Microsoft calls it a "dynamic, persistent canvas" that's designed for "multiplayer" collaboration, built right into Copilot. Microsoft has been busy integrating Copilot into most every application imaginable, but think of Pages as a way of allowing you to get more done without having to ever leave Copilot itself.

Read more