This morning, I had the opportunity to explore Gemini in Chrome, an innovative integration that seamlessly incorporates the AI-powered assistant directly into your web browser. Gone are the days of navigating to a separate chatbot web app; with the new Gemini button conveniently located in the top-right corner of Chrome, initiating a conversation has never been easier. What sets this integration apart is Gemini's ability to “see” the content on your screen as you browse the web, enhancing the overall user experience.
In my experience, the integration of Gemini into Chrome appears to be just the beginning of Google’s vision to create a more agentic AI. I found myself wishing for its capabilities to extend beyond its current functionality. As of now, only users subscribed to the AI Pro or AI Ultra plans can access the early version of Gemini in Chrome, and it requires the Beta, Dev, or Canary versions of the browser.
I began my journey by utilizing Gemini to summarize articles from The Verge and to discover gaming news. Notably, it highlighted the new Game Boy games that Nintendo has added to its Switch Online service, the anticipated Elden Ring film adaptation, and a significant update for Valve’s Steam Deck. However, Gemini’s ability to summarize content is limited to what is visible on the screen. For instance, if you want it to summarize comments from The Verge, you’ll need to make that section visible first.
A standout feature of Gemini in Chrome is its “Live” mode, which allows users to pose questions verbally. By clicking the button in the bottom-right corner of the dialogue box, you can ask Gemini anything aloud. I found this particularly beneficial while watching YouTube videos. For instance, during a bathroom remodeling video, I asked, “What tool is he using?” and Gemini promptly responded, “It looks like he’s using a nail gun to fasten some wood pieces together.” This capability also extends to identifying components in videos, such as recognizing a capacitor on a motherboard.
While Gemini can summarize videos and provide information about specific segments, its accuracy can falter, especially if videos lack labeled chapters. One of my favorite applications of this integration was having Gemini extract recipes from YouTube videos, eliminating the need for me to manually jot down ingredients. Additionally, it proved useful for identifying waterproof bags during an Amazon search. However, consistency was sometimes lacking; when I inquired about MrBeast’s location in a video exploring ancient Mayan cities, Gemini initially stated it could not provide real-time information but later identified the location as Mexico.
Occasionally, I found Gemini's responses to be overly lengthy for the small pop-up window in Chrome. While it is possible to extend the dialogue box, space remains limited on my MacBook Air’s 13-inch display. One of the primary advantages of AI is its potential to save time by offering quick, concise answers. Unfortunately, Gemini didn’t always deliver on this promise unless I explicitly requested brevity. Additionally, its follow-up questions tended to become repetitive, which could detract from the user experience.
Despite these limitations, I can envision a future where Google expands the capabilities of Gemini in Chrome beyond simple Q&A. Google’s goal is to create an AI that can perform tasks on behalf of users, and Gemini appears to be on a trajectory toward achieving this. For example, after asking it to summarize a restaurant’s menu, I contemplated whether it could eventually place a pickup order — a task it currently cannot perform.
Looking ahead, I anticipate that Gemini could assist with tasks like bookmarking travel-related pages or even curating YouTube recipes to my Watch Later playlist. With Project Mariner's upcoming “Agent Mode” for the Gemini app, which will allow it to manage multiple tasks simultaneously and search the web on your behalf, the integration of these capabilities into Gemini in Chrome seems like a promising possibility.
