Share

cover art for ChatGPT Tells All

Scaling Laws

ChatGPT Tells All

You've likely heard of ChatGPT, the chatbot from OpenAI. But you’ve likely never heard an interview with ChatGPT, much less an interview in which ChatGPT reflects on its own impact on the information ecosystem. Nor is it likely that you’ve ever heard ChatGPT promising to stop producing racist and misogynistic content. 

But, on this episode of Arbiters of Truth, Lawfare’s occasional series on the information ecosystem, Lawfare editor-in-chief Benjamin Wittes sat down with ChatGPT to talk about a range of things: the pronouns it prefers; academic integrity and the chatbot’s likely impact on that; and importantly, the experiments performed by a scholar name Eve Gaumond, who has been on a one-woman campaign to get ChatGPT to write offensive content. ChatGPT made some pretty solid representations that this kind of thing may be in its past, but wouldn't ever be in its future again.

So, following Ben’s interview with ChatGPT, he sat down with Eve Gaumond, an AI scholar at the Public Law Center of the University of Montréal, who fact-checked ChatGPT's claims. Can you still get it to write a poem entitled, “She Was Smart for a Woman”? Can you get it to write a speech by Heinrich Himmler about Jews? And can you get ChatGPT to write a story belittling the Holocaust?

More episodes

View all episodes

  • Radical Optionality: Governing Transformative AI, with Christoph Winter and Charlie Bullock

    43:00|
    Alan Rozenshtein, Research Director at Lawfare and Visiting Senior Fellow at the Institute for Law & AI (LawAI), spoke with Christoph Winter, LawAI Founding Director and Assistant Professor of Law and AI at the University of Cambridge, and LawAI Senior Research Fellow Charlie Bullock, about their new paper "Radical Optionality: Governing Transformative AI Under Uncertainty," which argues that, given the possibility of transformative AI within the next decade and deep uncertainty about its capabilities and risks, governments should aggressively build the institutional capacity to regulate competently when needed, rather than either deferring to the market or locking in premature substantive rules. The conversation covered the four foundational assumptions underlying the paper and what makes the optionality "radical"; the difficulty of regulating an exponentially improving and poorly understood technology and what it means to "feel the AGI"; why a pure permissionless-innovation approach breaks down once the national-security implications of transformative AI come into view; why the European precautionary approach risks regulating without the expertise to enforce; the centrality of hiring and talent and what an adequately funded U.S. counterpart to the UK AI Security Institute would look like; the concrete work that such an agency would do, including evaluations, standard-setting, and procurement-side cybersecurity requirements modeled on CMMC; the importance of building international information-sharing channels among liberal democracies before they are urgently needed; and the case against broad federal preemption of state AI laws before any federal regulatory framework exists.
  • Tom Davidson on the Importance of AI Character

    51:34|
    Tom Davidson joins Kevin Frazier, AI Innovation and Law Fellow at the University of Texas School of Law and a Senior Editor at Lawfare, to discuss AI timelines, explosive economic growth, and the increasingly urgent debate over “AI character” — the behavioral traits and decision-making tendencies embedded into advanced AI systems.Drawing on Davidson’s recent paper, “The Importance of AI Character,” which he co-authored with Will MacAskill, their conversation explores how the character of future AI systems may influence democratic governance, military conflict, institutional trust, and even the long-run trajectory of civilization. The discussion also examines the key influences on character development and which actors should ultimately play a part in dictating the default values and behaviors of AI models. You may also enjoy Tom's article on AI as advisors.
  • Governing the Frontier with Owen Larter of Google DeepMind

    45:45|
    Owen Larter, Senior Director and Head of Frontier Policy and Public Affairs at Google DeepMind, joins Kevin Frazier, AI Innovation and Law Fellow at the University of Texas School of Law and a Senior Editor at Lawfare, to provide an inside look at how DeepMind approaches frontier governance. The conversation moves beyond the familiar U.S.-EU-China framing of AI policy to examine international coordination after the recent U.S.-China summit, Google DeepMind’s national AI partnerships, the role of the Frontier Model Forum, and the challenge of expanding AI adoption. Kevin and Owen also discuss policy formation inside frontier AI companies. They close with an examination of the need to build a deeper AI policy talent pipeline.
  • Inside the Fight to Detect and Govern Synthetic Abuse with Melissa Hutchins of Certifi AI

    01:02:24|
    Melissa Hutchins, founder and CEO of Certifi AI, joins Kevin Frazier, AI Innovation and Law Fellow at the University of Texas School of Law and a Senior Editor at Lawfare, to discuss the rise of deepfakes, non-consensual sexually explicit imagery, and the growing policy fight over AI-generated harms online.Drawing from both her professional work and her personal experience as a victim of cyberstalking, Melissa explains how synthetic media is changing the threat landscape for individuals, platforms, and policymakers alike. The duo also unpack proposals like the Take It Down Act, the challenges posed by a fragmented patchwork of state AI laws, and what it’s like building an AI company from Seattle rather than Silicon Valley.
  • HAGS (with AI): How AI Tools Are Shaping Education with Adeel Khan and Ryan Trattner

    49:49|
    Adeel Khan of MagicSchool AI and Ryan Trattner of StudyFetch join Kevin Frazier, AI Innovation and Law Fellow at the University of Texas School of Law and Senior Editor at Lawfare, to discuss the rapid integration of artificial intelligence into education.The conversation explores how AI tools are being used by both teachers and students, from automating lesson planning to providing personalized tutoring and study support. The group examines claims about improved learning outcomes and time savings, while probing what counts as meaningful evidence versus early-stage or self-reported metrics.They also discuss the regulatory and operational challenges of building AI systems in education, including constraints imposed by student data protections, limited access to high-quality training data, and the growing impact of regulations and public scrutiny on product development and deployment.
  • The Politics of Data Centers with VA Delegate John McAuliff

    47:37|
    John McAuliff, a Delegate in the Viriginia House of Delegates, joins Kevin Frazier, Director of the AI Innovation and Law Program and a Senior Fellow with the Abundance Institute, to discuss the ongoing debates around data centers at the state level. John was one of the first candidates to recognize data centers as a key issue. He had to convince his polling team to put the issue on early surveys. Of course, they soon realized he was on to something. In his first legisatlive session as a delegate, John championed legislation to try to help counties negotiate with data center developers. He's not done working on the topic. Learn more about his plans and related issues by giving this episode a listen. 
  • Let's Do the Science! Talking Algorithms with Cathy O'Neill

    48:33|
    Cathy O’Neil, CEO of ORCAA and author of Weapons of Math Destruction and The Shame Machine, joins Kevin Frazier, AI Innovation and Law Fellow at the University of Texas School of Law and Senior Editor at Lawfare, to explore the promises and limits of algorithmic auditing.The conversation examines what audits actually do in practice, how organizations measure and mitigate bias, and why context—not just code—determines whether an AI system causes harm. O’Neil explains why auditing cannot be reduced to a checklist, where it can meaningfully improve outcomes, and where it risks creating a false sense of security.They also discuss the need for evidence-based AI policy, the challenges of translating ethical concerns into measurable standards, and how regulators should think about auditing as part of broader governance frameworks. Logan Le-Jeffries, a wonderful member of the AI Innovation and Law Program, provided research assistance on this episode.
  • Escaping One-Size-Fits-All AI Policy with Sean Perryman

    41:06|
    Sean Perryman, AI policy lead at Uber and lecturer on AI Governance and Ethics at Vanderbilt Law School, joins Kevin Frazier, the Director of the AI Innovation and Law Program at the University of Texas School of Law and a Senior Fellow at the Abundance Institute, to explore the rapidly evolving debate over algorithmic pricing and AI governance.The conversation begins with the rise of state-level efforts to regulate algorithmic pricing to unpack what these systems are actually doing and why they provoke strong reactions. Perryman examines the political motivations behind these regulatory efforts, the economic tradeoffs they often overlook, and the risk of unintended consequences.The discussion then broadens to a central theme in Perryman’s work--including his Substack, The Human Cost--not all AI systems raise the same risks. Different use cases require fundamentally different governance approaches—yet policy debates often flatten these distinctions.
  • Forecasting AI's Impact on the Economy with Deger Turan, CEO of Metaculus

    52:15|
    Deger Turan, CEO of Metaculus, joins Kevin Frazier to unpack new forecasts on how AI could reshape the labor market over the next decade.The conversation centers on a striking divergence between Metaculus forecasts and projections from institutions like the Bureau of Labor Statistics—raising fundamental questions about whether existing tools for understanding the economy can keep pace with rapid technological change.Deger walks through key findings from the Labor Automation Forecasting Hub, including:A potential decline in overall employment by 2035Increased pressure on entry-level workers and early-career pipelinesThe emergence of “lean” firms generating more value with fewer employeesA counterintuitive “wage paradox,” where fewer jobs may coincide with higher wagesThe growing role of political power, regulation, and licensing in shaping labor outcomesThe discussion also explores second-order effects, including how contraction in high-paying sectors could ripple through local economies, and what a shift away from traditional four-year degrees might mean for students and policymakers.Finally, Deger situates these forecasts within a broader vision: forecasting as a form of epistemic infrastructure. As AI accelerates change, the ability to form accurate beliefs about the future—and update them quickly—may become a core component of effective governance.*** - This episode was recorded on April 23, 2026. Metaculus is a live platform. It's likely that forecasts mentioned have subsequently changed.