As Microsoft says goodbye to Windows 10 and prepares to mark the 40-year milestone of its operating system, it’s looking forward to what’s next for Windows. Microsoft may not be ready to announce Windows 12 just yet, but it clearly wants to turn every Windows 11 PC into an AI PC that Copilot controls and that users talk to.
“We think we’re at the cusp of the next evolution, where AI doesn’t just happen in that chatbot and becomes naturally integrated into the billions of experiences that people use every day,” Youssef Mehdi, Microsoft’s executive vice president and consumer chief marketing officer, said in a briefing. The Verge“The vision we have is this: Let’s rewrite the entire operating system around AI, and essentially build something that truly becomes an AI PC.”
Microsoft is launching a set of capabilities in Windows today that will let consumers start adding AI features to regular Windows 11 PCs instead of purchasing a special CoPilot Plus PC. The biggest change is that Microsoft thinks people will want to talk to their computers and ask Copilot to take actions on their behalf.
“You need to be able to talk to your PC, have it understand you, and then be able to do magic with it,” says Mehdi. “With your permission, we’d like people to be able to share with our AI on Windows what they are doing and seeing. The PC should be able to take actions on your behalf.”
Microsoft is relying on CoPilot’s voice and vision capabilities to make this a reality, with a new “Hey, CoPilot!” Thanks for. Wake Word which is now running on Windows 11 PC. “In our minds, voice will now become the third input mechanism you use with your PC,” says Mehdi. “It doesn’t necessarily replace the keyboard and mouse, but it’s an additional thing and it will be much deeper and a new way of doing it.”
We have been here many times before. Microsoft tried to convince people to use Cortana on Windows 10 PCs a decade ago, and has added various voice features to Windows for accessibility over the past 40 years. Microsoft is now confident that AI will somehow bring about a change in behavior and make people understand that talking to a PC is not strange.
“All the data we see is when people use voice, they love it,” says Mehdi. Some of that data is the billions of minutes people spend talking in Microsoft Teams meetings. “They’re talking through their computers today, and I think this change of ‘get together and talk out’ will come to reality and we’ll see this thing really take off,” says Mehdi.
I’m not sure most people want to talk to their computer, even though it’s great for scenarios like accessibility and getting help with apps. “Doctors are taking transcriptions while doing exams, people are using it for searches, and our work with the accessibility community has taught us a lot about what makes voice access and voice typing really valuable,” says Mehdi.
To control a PC and take actions on the user’s behalf, AI must first be allowed to see what is on the screen. Microsoft has been testing Copilot Vision in recent months, a feature that can scan everything on your screen and coach you on using apps or answer questions about photos and documents.
CoPilot Vision is now available in all markets around the world where CoPilot is available, and will help you use apps, troubleshoot PC problems, learn new functions, and even get step-by-step guidance in games. Unlike the Recall feature that automatically takes a snapshot of your PC, Copilot Vision is an opt-in feature where you essentially stream what you’re seeing on your screen like you do in a Teams call.
The next step beyond Copilot Vision is Copilot Action, which allows Microsoft’s AI assistant to perform actions on a local PC, like making edits to a folder full of photos. Microsoft is starting to test these actions on Windows PCs through a preview program, limited to a narrow set of use cases while Microsoft optimizes the AI model.
“Initially you may notice that the agent makes some mistakes, or encounters some challenges when trying to use some complex applications,” explains Navjot Virk, corporate vice president of Windows Experience. Having an AI agent make mistakes using a computer doesn’t fill me with confidence, which is probably why Microsoft is limiting it to Copilot Labs for now. “We’re fully committed to learning how people use it, and we want to continue to improve the experience to make it more efficient and streamlined over time,” says Virk.
CoPilot Actions launches in a separate Windows desktop “secure and contained environment” and uses an AI agent to complete the task you’ve asked it to do. You can keep it running in the background while you go on and do something else, and it will list all the steps it takes and you can sit back and watch it complete it.
Microsoft is also integrating copilot in windows taskbarWith one-click access to these new CoPilot Vision and Voice features. There’s also a new integrated search experience that makes it faster to find local files, apps, and settings.
After the recall failure last year, I think Microsoft will have a hard time convincing people to trust its CoPilot Vision and CoPilot Action features, and will have an equally challenging time getting people to talk to their PCs. However, that’s not stopping Microsoft from trying. The company plans to run television commercials that will highlight these new AI features in Windows 11, with the tagline “Meet the computer you can talk to.”
The ads coincide with the end of support phase for Windows 10 earlier this week, and Microsoft is once again promoting Windows 11 PCs that consumers can upgrade to. “We want everyone to experience what it means to have a PC that is not just a device, but a true companion,” says Mehdi.
