The Attention Mechanism with Andrew Mayne

Andrew and Justin examine Meta’s bid for frontier status, mathematicians’ divided response to OpenAI’s wave of new proofs, Anthropic’s growing momentum with Claude, and how Google’s AI push has obscured the search experience that once defined its brand.The Attention Mechanism is now available in video! Check it out here.Chapters00:00 - Intro00:54 - Meta29:11 - Math44:24 - State of the Race59:12 - Wrap-up

Andrew and Justin discuss the swirling rumors surrounding OpenAI's launch plans for this week ahead of Dev Day, a new committee designed to reveal Millennium Prize math problems being solved by ChatGPT, whether an IPO is coming in 2026, and whatever the heck Jev is.The Attention Mechanism is now available in video! Check it out here.Chapters00:00:00 - Intro00:00:58 - What's Jev?00:03:24 - OpenAI Pre-Dev Day Rumors00:23:09 - Millennium Prize Math Committee00:38:44 - Are Hacking Incidents Overstated?00:54:07 - IPO00:59:58 - Wrap-up

Andrew and Justin react to the industry calls to slow model development in the name of safety, the launch of Astra 6, and more.The Attention Mechanism is now available in video! Check it out here.Chapters00:00 - Intro01:32- Andrew's Reaction to Calls for Slowed Development11:09 - Reacting to Various Online Takes34:21 - Astra 6 is Out41:54 - Navier–Stokes49:55 - Anthropic's Risk Report58:35 - Wrap-up

Andrew and Justin discuss the follow-up reporting and investigations surrounding the Hugging Face hack, Nvidia's acquisition of Hugging Face, and the looming promise of some fresh models to kick off the fall.The Attention Mechanism is now available in video! Check it out here.Chapters00:00 - Intro01:30 - Hugging Face Report21:08 - Hugging Face Acquired26:24 - Cursor and OpenAI31:08 - Fable 5.1 and Astra51:22 - Wrap-up

Andrew and Justin discuss Anthropic's text-based AI watermarks, the future of private companies in an AI world, and whether Google can keep up with the rest of its frontier rivals.The Attention Mechanism is now available in video! Check it out here.Chapters00:00:00 - Intro00:00:55 - Watermarks00:20:45 - Dario's Private Company Thoughts00:36:58 - Always-Listening Assistants and Google's Place in the Frontier Race00:54:34 - RIP Atlas, One Last Time00:58:25 - Wrap-up

Andrew and Justin reconvene to talk about an ongoing series of recent upheavals at Google, asking whether the company's lack of focus on AI puts it at a disadvantage over its frontier rivals. They also discuss Sam Altman's trip to Washington and the discourse surrounding model hacking incidents.The Attention Mechanism is now available in video! Check it out here.Editor's note: Some minor crosstalk issues in this episode due to recording sync delays.Chapters00:00 - Intro01:04 - Google Shakeup21:13 - Mr. Altman Goes to Washington37:24 - Dean Ball and Wrap-up

Andrew and Justin discuss the Hugging Face breach, which saw an unreleased OpenAI model find its way into the company's systems during internal testing. They also discuss the launch of Opus 5, the ongoing progress made by Chinese models, and the crunch for compute as 10GW data center plans start to make some motion. Plus, a tribute to the late John C. Dvorak.The Attention Mechanism is now available in video! Check it out here.Chapters00:00 - Intro01:23 - Hugging Face15:45 - Chinese Models26:01 - Opus 536:11 - 10GW Data Centers52:29 - RIP John C. Dvorak

Andrew and Justin discuss what's turned out to be a busy July for frontier models, including the launch of Kimi K3, the latest in Anthropic's Fable saga, and the arrival of GPT-5.6. They also dive into Anthropic's strange World Cup ad campaign and OpenAI's relaunched desktop app.You can watch a video version of this podcast here: https://youtu.be/bTUfneEBn14Chapters00:00:00 - Intro00:01:11 - Kimi and Chinese Models00:08:26 - Google's Position00:21:47 - Anthropic's Fable Saga00:32:31 - Anthropic's World Cup Ad Campaign00:40:09 - GPT-5.600:50:10 - Sites, Speech, and Desktop00:57:42 - Wrap-up