Microsoft has recently initiated testing for a groundbreaking update to its Copilot app on Windows, introducing a feature that allows users to share their screen or applications directly with the AI assistant. Known as Copilot Vision, this cutting-edge functionality was originally confined to the Microsoft Edge browser but is now expanding its capabilities to encompass any application on your PC.
With Copilot Vision, users can receive real-time assistance in a variety of tasks. For instance, it can coach you through the features of Adobe Photoshop, providing step-by-step guidance to enhance your editing skills. Additionally, it has the capability to analyze images and webpages you are currently viewing, making it a versatile tool for productivity.
At Microsoft’s 50th anniversary celebration, I had the opportunity to preview an early version of Copilot Vision on Windows. During this event, the AI assistant guided me through a game of Minecraft, helping me optimize settings in Microsoft’s Clipchamp video editor. This hands-on experience demonstrated the practical applications of the AI in real-time scenarios.
Currently, Copilot Vision is being tested exclusively with Windows Insider participants, primarily limited to users in the United States. As a result, I have not been able to conduct a full test of the feature. One of the anticipated enhancements of this initial beta version is the ability for Copilot to highlight specific areas on your screen to assist you in navigating various applications. However, this feature is not yet activated in the current testing phase.
While Copilot Vision may seem akin to Microsoft’s Recall feature, which automatically captures snapshots with user consent, its functionality is distinct. It operates more like a screen-sharing tool, allowing users to present an application or their entire desktop, similar to sharing screens during a Microsoft Teams call.
In addition to the screen-sharing capabilities, Microsoft has also begun testing a file search function within the Copilot app on Windows. This feature enables users to inquire about the contents of files stored on their PC, supporting various formats including .docx, .xlsx, .pptx, .txt, .pdf, and .json files. With Copilot, you can effortlessly locate documents you've previously worked on, enhancing your workflow and productivity.
Both the Copilot Vision and file search features require only the Copilot app on Windows, eliminating the need for a full Copilot Plus PC. Additionally, users can enjoy the benefits of Copilot Vision on both iOS and Android devices, ensuring that assistance is available across multiple platforms.
Microsoft is actively testing these innovative Copilot features with Windows Insiders, paving the way for a broader rollout to all Windows 11 users in the coming weeks or months. As these features become available, they promise to enhance user experience and productivity significantly.
