Google I/O 2024

You’ve long been able to search Google using still images, but now the company bringing “ask with video” search to Google Lens.

In an example during the I/O keynote, Google’s Rose Yao asked Lens why her turntable’s tonearm wouldn’t stay still, recording a brief clip to demonstrate the issue. Not exactly mind-blowing, but certainly helpful!

Onstage at Google I/O, the company is showing off its AI enhanced Google Search that can take a complex and long entry and generate an AI Overview. In an example, Google searched “Find the best yoga or pilates studios in Boston and show details on their intro offers and walking time from Beacon Hill,” and it provided all of that for the user.

CEO Sundar Pichai just announced new Trillium chips, coming later this year, that are 4.7 times faster than their predecessors, as Google competes with everyone else building new AI chips. Pichai also highlighted Axion, Google’s first ARM-based CPU, which the company announced last month.

Google will also be “one of the first” cloud companies to offer Nvidia’s Blackwell GPU starting in 2025.

Correction: Axion was announced last month, not last year. Also, corrected the spelling of Axion.

Google has announced Gemini’s 1.50 Pro model is coming to its AI-powered note-taking app. Soon, users will be able to create notebook guides out of student notes with summaries, quizzes, and FAQs.

But the real star is the Audio Overviews feature, which can turn the material into an interactive discussion and answer students' questions.

The faster version of its next-gen large language model offers similar multimodal reasoning and long context capabilities to Gemini 1.5 Pro (also announced today) but optimized for low-latency responses and overall efficiency.

Google announced Gemini 1.5 Flash is available for developers to try in Google AI Studio and Vertex AI today, with 1 million tokens to start and 2 million available upon request.

Google CEO Sundar Pichai just announced that the AI-generated summaries, now known as “AI Overviews,” will be launching to everyone in the US “this week,” with more countries coming soon.

As usual, Google has chosen to reveal the date of its I/O developer conference through a collaborative puzzle. For I/O ’24, it’s a Pipe Dream-style game, guiding a marble from point A to point B.

Completing your levels helps the overall progress (and probably trains generative AI models somehow), which you can monitor on the main event page. Or just wait for others to finish it while you dig through the I/O 23 announcements to see what has or has not shipped yet.

Update (5:03PM ET): The puzzle’s done, and we have a date (May 14th).

Google announced at IO that Bard would include images where relevant in its responses, and that change is live. Below, a tweet shows what that looks like. The images come from sites like Pinterest. I wish they were AI-generated for extra fun, but alas.

Still, I tried asking it for the best food in Austin, TX, and it failed to highlight Casino El Camino.

From yesterday’s Command Line newsletter:

Throughout the series of closed-door policy panels yesterday, I’m told that global affairs president Kent Walker and other Google execs fielded pointed questions about the company’s use of personal data for everything from large language models to ad targeting. Applying copyright law to generative AI was discussed but seemingly not a top concern for the room. Instead, most of the questions were about the use of AI as it relates to user privacy and safety. Regulators wanted to know how Google thinks about using sensitive data from both individual users and enterprise customers for its training sets. And during a talk dedicated specifically to AI and search, concerns were raised about showing harmful, AI-generated results...

You can sign up to read the full thing at the link below.

Like Face Landmarker, which can apparently let you make something like what’s shown in the below GIF. (Which admittedly looks a lot like an Apple Memoji.) Very interested to see what developers make with these tools.

We have a longer recap video too, of course, plus every update from the event and a list of the key highlights, but this is how it sounded for the most part.