Much of the time, when we need help from an AI assistant, it has something to do with content on our screen. Over the course of several months, Gemini has been slowly gaining the ability to see what’s on your screen. Well, Google just added a new Gemini Screenshare feature that completes the puzzle.
There are a few features that allow Gemini to read what’s on your screen. Firstly, when you summon Gemini, you’ll see the Ask about this screen button. When you press it, you’ll be able to ask Gemini questions about what’s on your screen. The Screenshots feature is similar in that it can read what’s in your screenshots. Also, while this isn’t technically a Gemini feature, Circle To Search lets you quickly do a Lens search of what’s on your screen.
However, people have been waiting for an all-in-one solution to ascertain what’s happening on their phones or tablets. It’s something that all of the major AI companies are working on.
Gemini now has a Screenshare feature
This is something that Google unveiled during Google I/O last year, so people have had quite the wait. While Gemini can already read your screen, Google has added a much more seamless method of doing so. As it stands, you can only ask questions about what’s on one screen. If you want to change screens or scroll, then you’d need to initiate a new conversation.
However, on Monday, Google announced a new Gemini feature called Screenshare. This feature will share your screen to Gemini in real-time. In the video below, Google gave an example of a person looking at a pair of jeans on their screen. Then, while sharing their screen, the person was able to scroll down to a lower part of the screen and ask Gemini another question.
So, the person is able to have a live conversation about the pants as though they were talking to an actual person. Not only can you scroll while Gemini is talking to you, you won’t need to summon Gemini when you want to ask another question. Just like with Gemini Live, you’ll be able to have a natural conversation.
Gemini Live with video
What’s perhaps the most anticipated feature is Gemini Live with video. You may remember this from the Project Astra showcase. It showed people asking Gemini questions about a live video feed. In the video below, we see the person asking about which color glaze they should add to a vase. Well, Gemini was able to give them suggestions in real-time. Just like with the Screenshare example, the person was able to have a live conversation with Gemini.
Since this is a pretty new and exciting Gemini feature, it’ll be exclusive to Gemini Advanced users. Google will roll the Gemini Screenshare feature out to paid users later this month.
