Google’s Project Jarvis AI is coming to Chrome to assist users with various tasks

Google is reportedly working on a new AI that will perform day-to-day tasks in Chrome. The technology is called Project Jarvis, and it could be unveiled before the end of the year.

Did you know? Google Chrome may send AI search history to Google.

Google’s Project Jarvis AI can perform various tasks for you

The news comes via The Information (paywalled). An article by 9to5Google explains more about the upcoming AI, which is powered by Gemini. Project Jarvis’ name seems to have been inspired by Marvel’s Iron Man’s AI-assistant J.A.R.V.I.S (Just A Rather Very Intelligent System). The AI is designed to assist users perform various tasks such as conducting research, purchasing products or booking flight tickets.

Google first unveiled its plans for AI Agents in April 2024, at the Google Cloud Next 24 event. The Mountain View demoed a couple of prototype AI agents at the Google I/O 2024 Developer Conference in May, these were designed. You can watch the video here (it’s around the 18-minute mark). Let’s discuss this briefly.

Google's AI Agents

Sundar Pichai, the CEO of Google, described Agents as intelligent systems that can reason, plan, and use memory to think. In fact, the Agents can plan multiple steps ahead to assist the user. The AI is designed to work across software systems and systems, it likely refers to operating systems and devices.

Google's AI Agents example

One of those examples highlighted a shopping experience, where a user could take a photo of a pair of shoes they had ordered, and ask Google Gemini to assist them to return the shoes. Gemini scanned through mails in Gmail, found the receipt, filled out a return form, and added a reminder in Google Calendar to notify the user about the order’s pick-up.

The other example showcased how Google could assist someone who had just moved to a new city, and offered suggestions to explore the city, listed suggestions for shops and services. The AI could even help them update their address across different websites, and prompted the user with questions as and when required.

Google's AI Agents example 2

It is worth noting that the scenarios were not running via an app/UI, they were likely demoed using Gemini. This is what Google plans to unveil in the coming months, integrating the AI features in Chrome. While it may sound like the AI is taking over a computer, it is pretty clear that the user has to trigger the action manually, and the app does the rest.

That said, there are some concerns about Project Jarvis.

How will Project Jarvis work?

Jarvis will capture screenshots of the user’s screen frequently, and analyze the data in the images. It will interpret actions such as clicking on a button or typing something in a text field. This could be potentially be worse than Microsoft Recall. The Redmond company has confirmed that Recall stores the database on the user’s computer, the data is encrypted, and that user data is not shared with Microsoft or third-parties. Google is an advertising company, so its motivations will come under scrutiny from regulatory agencies. It remains to be seen how it will protect the user’s privacy.

Project Jarvis could be powered by Gemini 2.0, and could be released in a preview form in December 2024. On a sidenote, Anthropic announced Claude 3.5 Sonnet, which has a new feature called “computer use”, that is capable of moving the mouse cursor, click on buttons, and typing text. AI and Automation, what could possibly go wrong?

While it sounds impressive on paper, it is too early to say how Project Jarvis will actually perform the tasks.

Would you use such AI tools, or rather protect your privacy?

Summary