Skip to main content

The Stochastic Mindset

AI tools will change the way we work by changing the way we think.

Most descriptions of AI’s potential to change how we work are by now familiar: AI will automate repetitive tasks, increase worker impact, improve job satisfaction and disrupt entire industries. But how will using AI tools change the actual experience of work? Will AI change the way we think?

AI is the next era of computing, and one of its hallmarks is moving from deterministic to probabilistic or “stochastic” outputs. As AI spreads, workers are engaging with a new software paradigm that calls for a new mindset. In a recent episode of Sequoia’s Training Data podcast, Dust co-founder Gabriel Hubert introduced the concept of the “stochastic mindset” as “the biggest shift in the use of the tools that we have since the advent of the computer.”

Randomness and uncertainty are a part of life, but in the context of modern knowledge work we have been trained to optimize for certainty and predictability. The stochastic mindset invites us to move from rote workflows to iterative development of tools, content, strategy and more.

The stochastic mindset moves us from having minimal leverage on a task and 100% certainty of its outcome to 100% leverage on a task and far less certainty on the exact manifestation of its outcome. This transition is the difference between doing something yourself and delegating a task to someone else. 

AI as exoskeleton for work

The shift to the stochastic mindset shows how the impactful productivity gains from AI will actually be achieved. By making it easier to access and synthesize information, workers will consume and produce  much more information. OpenEvidence, Harvey and Dust are examples of products that take the friction out of accessing and making use of relevant information in the context of doctors, lawyers and knowledge workers more generally.

AI expands information through generalization. But it also reduces by summarizing information. In the AI age, there is uncertainty in exactly the message being delivered in exchange for speed and leverage. The quantity of information itself requires probabilistic approaches to manage and infer simplifications.

Age of AI (Circa 2023)

AI tools provide drafts or suggestions, not definitive answers. They will improve—but always to the point of some probability. Those reading—especially the quantum and stats enthusiasts—will be quick to assert that everything is probabilistic. Fair. But the history of computing has been one darn near determinism. AI is reaching a threshold of scale where being probabilistic is more efficient than being deterministic. 

What this means for us as workers will be seismic. Our tools imitate us and then we imitate them. If our predominant model of a machine becomes stochastic—embracing randomness—our minds will follow suit. We will adopt a more questioning stance toward the data at hand, realizing it is infused with randomness. Perhaps most importantly, the stochastic mindset can accommodate change and is adapted specifically for change. 

Whether you’re a founder or a knowledge worker or a student, your future prospects will have a lot to do with how well you adapt to change. Engaging with more uncertain outputs from AI systems, workers will need to strengthen their critical thinking skills. This is not just a short term adaptation resulting from the tendency of LLMs to hallucinate. Our demands on these systems will continue to outstrip their abilities, and for some use cases may become more variable over time, not less. As Ilya Sutskever said about AI reasoning systems in his recent talk at NeurIps 2024, “the more it reasons the more unpredictable it becomes.”

Dust is an example of a platform making custom agents accessible to non-technical workers. They aim to build “horizontal sandboxes” where workers can create their own agents, assistants and tools. Over time, teams will manage AI agents. As Jensen Huang put it at his CES Keynote this month, “IT teams will become HR departments of AI.” These systems are the record of management and action. Companies like Factory will enable management of coding agents, XBOW will be the management of cybersecurity agents and Rox will provide management of selling agents. 

Instead of thinking about AI purely in terms of how much opex it will save companies, the stochastic mindset focuses our attention on AI as a power assist—an exoskeleton for work.

Adapting to 21st century realities

We can see the stochastic mindset as an evolutionary and adaptive response to the reality of the world in the 21st century. Humanity is faced with increasing levels of social, political, economic and environmental uncertainty. The accelerating speed of change (models/robot/rockets keep getting better/faster/cheaper) also means accelerating volatility.

Fast iteration and execution favors small teams with AI superpowers. The stochastic mindset is also the builder mindset: iterative, experimental, skeptical and data-driven. An intriguing detail that Gabriel reports is that the patterns of usage of Dust within organizations becomes a heat map for identifying builders. 

The stochastic mindset is also the scientific mindset: forming hypotheses and seeking to prove or disprove them. Promoters of AGI and superintelligence (including Vlad Tenev at Harmonic) have science firmly in their sights. But the bigger consequence of the diffusion of stochastic AI systems may turn out to be the increasing prevalence of scientific thinking among humans.

Founders need the stochastic mindset

When people say that product-market fit is as much art as science they are perhaps thinking of science in deterministic terms. Founders have a stochastic learning problem that they confront every day: the changing needs of their customers in a changing world.

Among other things, contemporary founders need the stochastic mindset to take advantage of assumed but unpredictable improvements in AI models. Their teams need to plan product roadmaps in anticipation of what may be possible soon. Founders also now need to engage with research in an open-ended way—uncertainty can also be opportunity.

Part of the stochastic mindset is the ability to understand constraints. Any process has to run in a limited time. Compute resources are limited and so is communication within a given system. All of these factors lead to non-determinism, but the constraints on human cognition are notoriously more stringent than those of machines. 

Most importantly, by developing their own stochastic approach, founders can solve their customers’ problems too. By infusing the stochastic mindset into their products, founders can help users take advantage of these new capabilities:

  • Help workers become builders
  • Help students become researchers
  • Help consumers make good choices among complex offerings
  • Help everyone learn to consume AI services productively

Giving humans more time to think

A central problem in making AI agents reliable is endowing them with the right guardrails, but their unpredictable nature is also a feature, not just a bug.

Indeed, one of the big takeaways from the story of AlphaGo turned out to be its effect on human Go players. A 2023 paper on the effect of superhuman artificial intelligence on human decision-making concluded that the novelty in moves by human Go players increased significantly after AlphaGo beat Lee Sedol. As Garry Kaspirov put it, “I was the first knowledge worker whose job was threatened by a machine.” That was back in 1996. Yet the game of chess is far from a relic of the past, it has wider adoption than ever and new techniques. AI is used both as a sparring partner and a coach.

The stochastic mindset is not actually a new way of thinking—though the adoption of AI tools is about to make it much more prevalent. Less line workers and more creative artists and strategic managers. Less programming and more teaching. The stochastic mindset will help make us all more comfortable at a higher level of abstraction. From there we can see AI as both an executor of our intentions and as a teacher. In a world where nothing is certain, adaptability becomes even more valuable, allowing us to navigate through the unpredictable and embrace new possibilities with self-determination and creativity.

Related Topics