Microsoft Will Ship an On-Device AI Model With All Copilot+ PCs

Microsoft is shipping an on-device large language model (LLM) with all Copilot+ PCs. The company announced its latest variant of the Phi-3 AI model dubbed Phi-Silica at the Microsoft Build 2024 event. This has now become the fifth variant of Phi-3 and officially the smallest LLM (or SLM) at just 3.3 billion parameters. This new AI model will now be available locally within every PC part of the Copilot+ AI PC class. Notably, Microsoft also announced several new AI features for Copilot+ PCs during its Surface event.

The announcement was made by Microsoft CEO Satya Nadella during the keynote session of its Build event where the new Phi-Silica AI model was introduced as an on-device LLM for Copilot+ PCs. These will run on the local Neural Processing Unit (NPU) hardware equipped on these devices. With just 3.3 billion parameters, the model is relatively smaller than most of the SLMs. Such smaller models are generally better in terms of particular tasks but do not have the general conversational capabilities of LLMs such as OpenAI’s GPT-4, Google Gemini 1.5 Pro, or Claude 3 by Anthropic.

The development is still considered a major milestone as this is the first time AI models are being added to PCs out of the box. While open-source models are available for enthusiasts to download and run locally, such as the LLMs released by Mixtral, they were not offered from the OEM or the operating system’s side. With Phi-Silica being integrated into the Windows workflow, it is believed that its functionality will also be optimised.

Speaking with a Microsoft spokesperson, a VentureBeat report noted that Phi-Silica’s differentiation is its distinction as Windows’ first locally deployed language model. The spokesperson also confirmed that the model is optimised to run on Copilot+ PCs NPU and will perform relatively lag-free. However, its specific functionalities are not known. Further, these PCs will also come integrated with the Windows Copilot Library which will run on servers and offer all the existing and future Copilot features.

Microsoft also announced the general availability of the previously unveiled Phi-3 models as well as previewed a new Phi-3-vision model, which comes with multimodal and computer vision processing capabilities.