Share

cover art for Mark Miller | Paperclips and Pyramids: Misdiagnosing AI Risks

The Foresight Institute Podcast

Mark Miller | Paperclips and Pyramids: Misdiagnosing AI Risks

Season 1

Mark S. Miller is the Chief Scientist at Agoric. He is a pioneer of agoric (market-based secure distributed) computing and smart contracts, the main designer of the E and Dr. SES distributed persistent object-capability programming languages, inventor of Miller Columns, an architect of the Xanadu hypertext publishing system, a representative of the ECMAScript committee, a former Google research scientist, and member of the WebAssembly (Wasm) group.


Key Highlights

  • This podcast addresses common misconceptions about the risks of advanced AI – an example being the paper clip scenario
  • How more pressing concerns regard one entity gaining unchecked power and that solutions lie not in just dispersing power but in structuring institutions effectively – similar to the separation of powers principle.
  • The blend of human and AI intelligence is the cornerstone of future civilization, demanding innovative governance to ensure harmony.


Dive deeper into the session: Full Summary


The Foresight Institute

A research organization and non-profit that supports the beneficial development of high-impact technologies. Since our founding in 1987 with a vision of guiding powerful technologies, we have continued to evolve into a many-armed organization that focuses on several fields of science and technology that are too ambitious for legacy institutions to support.


Allison Duettmann

The President and CEO of Foresight Institute, Allison Duettmann directs the Intelligent Cooperation, Molecular Machines, Biotech & Health Extension, Neurotech, and Space Programs, alongside Fellowships, Prizes, and Tech Trees. She has also been pivotal in co-initiating the Longevity Prize, pioneering initiatives like Existentialhope.com, and contributing to notable works like "Superintelligence: Coordination & Strategy" and "Gaming the Future".


Get Involved with Foresight


Follow Us: Twitter | Facebook | LinkedIn


Note: Explore every word spoken on this podcast through Fathom.fm, an innovative podcast search engine.

More episodes

View all episodes

  • Vision Weekend France: Energy and Space Panel

    39:23
    Panellists: Andre Losekrug, Michael Gibson, Anders Sandberg, and Robin Hanson.Recorded at Vision Weekend France 2023.Energy Key Highlights:Excitement about energy advancements, specifically in the field of fusion. This could lead to an increased focus on policy changes to address climate change and the building of reactors.The growing entrepreneurial revolution in fusion, with various start-ups and innovative reactor designs emerging. China's recent approval of a molten salt thorium breeder reactor was highlighted as a further acceptance of nuclear power.Space Key Highlights:Advancements in satellite technology and constellations are anticipated, along with the need to explore intermediate solutions between current space exploration and future ambitious projects. It is noted that space exploration offers opportunities for freedom and innovation, with new approaches to manufacturing and remote sensing potentially leading to breakthroughs in various industries.Other Topics:Offshore water restoration, international disputes, prioritizing space development, launch services, space tourism, addressing space debris, and the prediction of detecting extraterrestrial intelligent life within the next century.About Foresight InstituteForesight Institute is a research organization and non-profit that supports the beneficial development of high-impact technologies. Since our founding in 1987 on a vision of guiding powerful technologies, we have continued to evolve into a many-armed organization that focuses on several fields of science and technology that are too ambitious for legacy institutions to support.Allison DuettmannThe President and CEO of Foresight Institute, Allison Duettmann directs the Intelligent Cooperation, Molecular Machines, Biotech & Health Extension, Neurotech, and Space Programs, alongside Fellowships, Prizes, and Tech Trees. She has also been pivotal in co-initiating the Longevity Prize, pioneering initiatives like Existentialhope.com, and contributing to notable works like "Superintelligence: Coordination & Strategy" and "Gaming the Future".Get Involved with Foresight:Apply: Virtual Salons & in-person WorkshopsDonate: Support Our Work – If you enjoy what we do, please consider this, as we are entirely funded by your donations!Follow Us: Twitter | Facebook | LinkedInNote: Explore every word spoken on this podcast through Fathom.fm, an innovative podcast search engine.
  • 23. Existential Hope: Danielle Strachman | Education, Agency, and the Ability to Experiment

    49:55
    Danielle Strachman is a cofounder and general partner of 1517. She has worked with young entrepreneurs for about a decade. In 2010, during the founding of the Thiel Fellowship, Danielle joined to lead the design and operations. She's worked with some of the most prestigious founders, such as Vitalik Buterin and Ritesh Agarwal. Previous to her work with Peter Thiel, Danielle founded and directed Innovations Academy in San Diego, a K-8 charter school serving 400 students, with a focus on student-led project-based learning and other alternative programs.Session SummaryDanielle envisions a world where fostering personal agency and curiosity is the key to progress. She imagines a future where technology, particularly AI, catalyzes human potential and societal transformation, emphasizing the importance of environments that champion experimentation and a fail-forward mentality.Specifically, she advocates for special economic zones as laboratories for innovation, suggesting governance that adapts to technological advancements. She dreams of a solarpunk future, where sustainability is achieved through technological and natural harmony. Acknowledging the complexities ahead, she views the journey to this future as requiring a balance between innovation and ethics. Her vision is grounded in the belief in human capacity to navigate the unknown and transform existential risks into opportunities for growth.Full transcript, list of resources, and art piece: https://www.existentialhope.com/podcastsExistential Hope was created to collect positive and possible scenarios for the future so that we can have more people commit to creating a brighter future, and to begin mapping out the main developments and challenges that need to be navigated to reach it. Existential Hope is a Foresight Institute project.Hosted by Allison Duettmann and Beatrice ErkersFollow Us: Twitter | Facebook | LinkedIn | Existential Hope InstagramExplore every word spoken on this podcast through Fathom.fm.
  • Vision Weekend France: Intelligent Cooperation Panel

    39:52
    Panellists: Divya Siddarth, Jonathan Prasserat-Palmbach, Lewis Hammond, Primavera De Filippi, Stuart Armstrong, and Trent Mcconaghy. Recorded at Vision Weekend France 2023. Key HighlightsPanellists discussed the potential development of intelligent cooperation between humans and AI, focusing on secure collaboration and cryptography methods. Concerns about AI's impact on society and the economy also arose, and an acknowledgement of the importance of designing AI systems for collaboration with humans and nature was made.Open source versus closed AI was debated – the risks of opening powerful technologies to a large number of developers vs. the potential for public collaboration and counteracting evil AI models. Other topics included concerns about governmental control of AI and data, potential risks of AI accidents, abuse, and misalignment, and the intersection of blockchain and AI to create decentralized and sovereign AI systems.Dive deeper into the session: Full SummaryAbout Foresight InstituteForesight Institute is a research organization and non-profit that supports the beneficial development of high-impact technologies. Since our founding in 1987 on a vision of guiding powerful technologies, we have continued to evolve into a many-armed organization that focuses on several fields of science and technology that are too ambitious for legacy institutions to support.Allison DuettmannThe President and CEO of Foresight Institute, Allison Duettmann directs the Intelligent Cooperation, Molecular Machines, Biotech & Health Extension, Neurotech, and Space Programs, alongside Fellowships, Prizes, and Tech Trees. She has also been pivotal in co-initiating the Longevity Prize, pioneering initiatives like Existentialhope.com, and contributing to notable works like "Superintelligence: Coordination & Strategy" and "Gaming the Future".Get Involved with Foresight:Apply: Virtual Salons & in-person WorkshopsDonate: Support Our Work – If you enjoy what we do, please consider this, as we are entirely funded by your donations!Follow Us: Twitter | Facebook | LinkedInNote: Explore every word spoken on this podcast through Fathom.fm, an innovative podcast search engine.
  • Vision Weekend France: Neuotech Panel

    40:55
    Panellists: Anders Sandberg, Bradley Love, Robin Hanson, Brett KaganRecorded at Vision Weekend France 2023. Key HighlightsThe panel explored how brain emulation is increasingly becoming a viable possibility with technological advancements.They discussed the availability of tools that help understand the neurochemistry and biology of cells, which is crucial for brain emulation. For example, the use of expansion microscopy for imaging neurons on a larger scale was highlighted, showcasing its importance in brain emulation research.However, they also note that the mapping efforts required for brain emulation are proving to be more complex than initially anticipated.It was emphasized that there is a critical need for collaboration between biologists and computer scientists in advancing brain emulation research.They explored the potential of using neurons as an engineering resource to create intelligent devices.The panel concluded that achieving smaller milestones is crucial before envisioning the broader potential of brain emulation technology.About Foresight InstituteForesight Institute is a research organization and non-profit that supports the beneficial development of high-impact technologies. Since our founding in 1987 on a vision of guiding powerful technologies, we have continued to evolve into a many-armed organization that focuses on several fields of science and technology that are too ambitious for legacy institutions to support.Allison DuettmannThe President and CEO of Foresight Institute, Allison Duettmann directs the Intelligent Cooperation, Molecular Machines, Biotech & Health Extension, Neurotech, and Space Programs, alongside Fellowships, Prizes, and Tech Trees. She has also been pivotal in co-initiating the Longevity Prize, pioneering initiatives like Existentialhope.com, and contributing to notable works like "Superintelligence: Coordination & Strategy" and "Gaming the Future".Get Involved with Foresight:Apply: Virtual Salons & in-person WorkshopsDonate: Support Our Work – If you enjoy what we do, please consider this, as we are entirely funded by your donations!Follow Us: Twitter | Facebook | LinkedInNote: Explore every word spoken on this podcast through Fathom.fm, an innovative podcast search engine.
  • Vision Weekend France: Molecular Machines Panel

    35:29
    Panellists: Alexis Courbet, Erik Benson, Stephane Redon, Dean Thomas. Recorded at Vision Weekend France. Key HighlightsThe panel discusses molecular nanotechnology and the potential for compact machines to replace industrial facilities.They discuss advances in the field, which include protein design, protein folding, and dynamic structures that respond to stimuli.It is noted how molecular machines can automate the synthesis of other machines, similar to biology – computational protein design allows for the creation of new machines from scratch.The panel concluded by highlighting the potential for molecular nanotechnology to bridge the gap between organic chemistry and automation.About Foresight InstituteForesight Institute is a research organization and non-profit that supports the beneficial development of high-impact technologies. Since our founding in 1987 on a vision of guiding powerful technologies, we have continued to evolve into a many-armed organization that focuses on several fields of science and technology that are too ambitious for legacy institutions to support.Allison DuettmannThe President and CEO of Foresight Institute, Allison Duettmann directs the Intelligent Cooperation, Molecular Machines, Biotech & Health Extension, Neurotech, and Space Programs, alongside Fellowships, Prizes, and Tech Trees. She has also been pivotal in co-initiating the Longevity Prize, pioneering initiatives like Existentialhope.com, and contributing to notable works like "Superintelligence: Coordination & Strategy" and "Gaming the Future".Get Involved with Foresight:Apply: Virtual Salons & in-person WorkshopsDonate: Support Our Work – If you enjoy what we do, please consider this, as we are entirely funded by your donations!Follow Us: Twitter | Facebook | LinkedInNote: Explore every word spoken on this podcast through Fathom.fm, an innovative podcast search engine.
  • 22. Existential Hope: Gus Docker | Beyond Survival

    47:11
    Gus Docker hosts The Future of Life Institute podcast, which features conversations with prominent researchers, policy experts, philosophers, and influential thinkers of all kinds. Gus studied philosophy and computer science at the University of Copenhagen and is active in Effective Altruism Denmark.Despite the ongoing threat of existential risks from AI, Gus discusses his excitement about the future's potential. He acknowledges the difficulty in envisioning the far future's technological changes, comparing the gap to that between the Stone Age and today – however, he remains excited about technologies enabling diverse experiences and consciousness exploration. Optimistic, Gus foresees a major positive shift in mental health, where individuals easily manage and enhance their mental well-being, harmonizing happiness and productivity, thus contributing to societal welfare.Full transcript, list of resources, and art piece: https://www.existentialhope.com/podcastsExistential Hope was created to collect positive and possible scenarios for the future so that we can have more people commit to creating a brighter future, and to begin mapping out the main developments and challenges that need to be navigated to reach it. Existential Hope is a Foresight Institute project.Hosted by Allison Duettmann and Beatrice ErkersFollow Us: Twitter | Facebook | LinkedIn | Existential Hope InstagramExplore every word spoken on this podcast through Fathom.fm.
  • Dr. Scott Hubbard | Exploring Mars: Following the Water

    01:00:18
    Dr. Scott Hubbard has been a leader in space exploration for almost 50 years. His career includes a National Lab, a start-up venture, NASA, and Stanford. At NASA, Dr. Hubbard is best known as Director of NASA’s Ames Research Center, the first Mars Program Director (aka the "Mars Czar"), and the founder of NASA's Astrobiology Institute.Key HighlightsHubbard shared his deep passion for Mars and recounted his role in restructuring the Mars program.He described Mars as the most Earthlike planet, highlighting its potential for past or present life evidence.The exploration of Mars, as explained by Hubbard, focuses on life, climate, solid rock geology, and human exploration.Emphasizing the mantra "follow the water," Hubbard underscored water's essential role in supporting all known life forms.He pointed out the challenges posed by Mars' launch window, occurring every 26 months, which significantly impacts the mission's rocket science and budget.Dive deeper into the session: Full SummaryAbout Foresight InstituteForesight Institute is a research organization and non-profit that supports the beneficial development of high-impact technologies. Since our founding in 1987 on a vision of guiding powerful technologies, we have continued to evolve into a many-armed organization that focuses on several fields of science and technology that are too ambitious for legacy institutions to support.Allison DuettmannThe President and CEO of Foresight Institute, Allison Duettmann directs the Intelligent Cooperation, Molecular Machines, Biotech & Health Extension, Neurotech, and Space Programs, alongside Fellowships, Prizes, and Tech Trees. She has also been pivotal in co-initiating the Longevity Prize, pioneering initiatives like Existentialhope.com, and contributing to notable works like "Superintelligence: Coordination & Strategy" and "Gaming the Future".Get Involved with Foresight:Apply: Virtual Salons & in-person WorkshopsDonate: Support Our Work – If you enjoy what we do, please consider this, as we are entirely funded by your donations!Follow Us: Twitter | Facebook | LinkedInNote: Explore every word spoken on this podcast through Fathom.fm, an innovative podcast search engine.
  • Mark Hamalainen | A Coherent, Executable Plan to Achieve Unlimited Healthy Lifespan

    51:02
    Mark Hamalainen is cofounder of the Longevity Biotech Fellowship. His career has progressed from manual bench work in academia, to lab automation at Synthego, to longevity movement building - always seeking better methods and higher leverage ways to accelerate progress.He shares an actionable roadmap for an unlimited healthy lifespan, divided into four promising areas for progress. He shares updates and information regarding The Longevity Biotech Fellowship, whose mission is to enable a future where everyone can access a healthy, unlimited lifespan. Dive deeper into the session: Full SummaryAbout Foresight InstituteForesight Institute is a research organization and non-profit that supports the beneficial development of high-impact technologies. Since our founding in 1987 on a vision of guiding powerful technologies, we have continued to evolve into a many-armed organization that focuses on several fields of science and technology that are too ambitious for legacy institutions to support.Allison DuettmannThe President and CEO of Foresight Institute, Allison Duettmann directs the Intelligent Cooperation, Molecular Machines, Biotech & Health Extension, Neurotech, and Space Programs, alongside Fellowships, Prizes, and Tech Trees. She has also been pivotal in co-initiating the Longevity Prize, pioneering initiatives like Existentialhope.com, and contributing to notable works like "Superintelligence: Coordination & Strategy" and "Gaming the Future".Get Involved with Foresight:Apply: Virtual Salons & in-person WorkshopsDonate: Support Our Work – If you enjoy what we do, please consider this, as we are entirely funded by your donations!Follow Us: Twitter | Facebook | LinkedInNote: Explore every word spoken on this podcast through Fathom.fm, an innovative podcast search engine.
  • 21. Existential Hope Podcast: Jason Crawford | Progress: An Ever-Evolving Journey

    49:51
    Jason Crawford is the founder of The Roots of Progress, where he writes and speaks about the history of technology and the philosophy of progress. Previously, he spent 18 years as a software engineer, engineering manager, and startup founder.Session Summary: Jason envisages a future marked by dynamic, continuous progress, encapsulated in the concept of protopia. This vision diverges from a traditional notion of a utopia, and instead embraces a reality of constant, incremental improvement. In Jason's view, progress is an ever-evolving journey, not a destination. It's a series of small, significant steps that, over time, lead to profound transformations in our world.Central to Jason's perspective is the transformative potential of AI, paralleling historical technological leaps like the steam engine and personal computing. He views AI as a catalyst for a new era in human history, one that could redefine societal structures by making high-quality services accessible to a broader demographic. This democratization of resources, akin to services becoming as affordable as a Netflix subscription, could bridge societal gaps. However, Jason emphasizes that this protopian future requires collective agency, responsibility, and a balanced understanding of our role in shaping it. He believes that progress accelerates over time, with each innovation building upon the last, thus speeding up future advancements.Full transcript, list of resources, and art piece: https://www.existentialhope.com/podcastsExistential Hope was created to collect positive and possible scenarios for the future so that we can have more people commit to creating a brighter future, and to begin mapping out the main developments and challenges that need to be navigated to reach it. Existential Hope is a Foresight Institute project.Hosted by Allison Duettmann and Beatrice ErkersFollow Us: Twitter | Facebook | LinkedIn | Existential Hope InstagramExplore every word spoken on this podcast through Fathom.fm.