We talk a lot about how executives think AI is being used in their organizations. This episode is about how it is actually being used — at scale, in the real world, across a quarter million real conversations.
Stanford University's SALT Lab analyzed 249,834 real Claude.ai conversations to answer three questions: what work people bring to AI, what control they retain over the process, and where the collaboration breaks down. The findings challenge some of the most common assumptions executives hold about AI adoption.
The most striking finding: 56% of conversations involving actionable tasks were consequential or hard-to-reverse work. Legal guidance. Financial decisions. Strategic choices. People are not using AI for trivial tasks — they are bringing their most difficult problems to it, often without any formal organizational sanction or governance framework in place. Legal and financial guidance had the highest Task Criticality Index at 2.14.
And yet — the human remains in control. 72% of conversations were human-led, with AI assisting rather than directing. Users typically adapted AI outputs rather than using them verbatim. Higher-stakes tasks nearly doubled conversation length and engagement. And friction — which appeared in roughly half of all conversations — was often productive: users who encountered resistance from the AI emerged more engaged and more capable.
The central tension the study surfaces: does working with AI build human capability, or substitute for human effort? Active teaching — where users explain context, correct outputs, and shape results — occurred in 67% of conversations. But the benefits were unevenly realized. The executives who understand this dynamic will design better governance. The ones who don't will discover the gap the hard way.
Key Takeaways
Source: Shao, Y., Zhao, D., Padmakumar, V., Wang, J., & Yang, D. (2026). Human–AI Collaboration at Scale: Task Criticality, Agency, and Friction Across 250,000 Conversations. Stanford University SALT Lab. https://arxiv.org/abs/2608.human-ai-collaboration-at-scale
","author_name":"Kenza Ait Si Abbou"}