Jeff Dean & Noam Shazeer - 25 years at Google: from PageRank to AGI
Jeff Dean and Noam Shazeer talk with Dwarkesh Patel about Google's early systems, hardware, language models and AI research. They explain how data-movement costs shape algorithms, why hardware and software need co-design, and how ideas and research culture contribute alongside scale. This is their February 2025 perspective, not a guaranteed AGI roadmap.
Executive Summary (TL;DR)
Dean and Shazeer connect AI progress with hardware and algorithm co-design. Arithmetic has become relatively cheap while moving data is costly, which helps explain the emphasis on matrix multiplication. They argue that improvements come from architecture, algorithms and data as well as larger hardware budgets. The interview also describes how research ideas combine and why humility and a record of ambitious directions matter. Their forward-looking views are from February 2025.
Key Takeaways
General-purpose hardware no longer supplies the same effortless gains
Dean contrasts an earlier period of waiting for faster hardware with slower improvement in general-purpose CPUs. Specialized accelerators fill some of that gap. The discussion explains a change in engineering incentives rather than guaranteeing a fixed improvement rate for every chip or workload.