Five-plus years ago, I sat next to my friend Andrej Karpathy as he published the blog post “Software 2.0.” It was a different time in machine learning—we were using Nvidia V100s, and the open source and foundation models we see today were nascent. The level of tooling we had long been used to in traditional software was also absent from the world of machine learning: where were the GitHub, Vercel, Docker equivalents? As Andrej described in his post:

“In particular, we’ve built up a vast amount of tooling that assists humans in writing 1.0 code, such as powerful IDEs with features like syntax highlighting, debuggers, profilers, go to def, git integration, etc. In the 2.0 stack, the programming is done by accumulating, massaging and cleaning datasets. … Who is going to develop the first Software 2.0 IDEs, which help with all of the workflows in accumulating, visualizing, cleaning, labeling, and sourcing datasets? … Github is a very successful home for Software 1.0 code. Is there space for a Software 2.0 Github? In this case repositories are datasets and commits are made up of additions and edits of the labels. Traditional package managers and related serving infrastructure like pip, conda, docker, etc. help us more easily deploy and compose binaries. How do we effectively deploy, share, import and work with Software 2.0 binaries?”

I met Replicate co-founders Ben Firshman and Andreas Jansson a few years later. They were the creators of arXiv Vanity, which turned scientific papers into responsive web pages—a nifty product I was grateful for. 

We talked about Andreas’ frustration with the problem of deploying machine learning models to production while at Spotify, which, as Ben describes, required extensive collaboration between researchers and engineering—including deciding on an API, getting a server written, packaging up the dependencies, getting it hooked up to a GPU and running efficiently. To solve this pain point, Andreas built a set of tools and templates that let researchers package up and deploy models themselves.

Ben came from the world of developer tools. Most notably, he’d built the product that eventually became Docker Compose. He understood the quality of developer tools we were used to in the world of software development, and the opportunity to bring that to the world of machine learning. 

It was a thesis we at Sequoia loved and believed in deeply, and we quickly started working with Ben and Andreas, leading Replicate’s seed round and building on our shared dream of what we believed the world could look like. 

Today, Replicate provides an easy, fast way for anyone to run generative AI models in the cloud. Instead of battling with GPUs and keeping expensive AWS instances switched on, developers can deploy with just one line of code—in the cloud, billed by the second. And in addition to curating the best models from the open-source ecosystem, the platform allows companies to run their own custom models, which Replicate automatically packages, optimizes and deploys—shrinking the process from weeks to minutes and reducing the need for an infrastructure team’s help. 

It’s been so fun to see the variety of ways developers use Replicate. Character.AI leverages DreamBooth to enhance the conversations people have with characters on their platform, while Vercel uses Replicate for its AI Templates. As the use cases shared on the team’s Twitter feed shows, the ecosystem is burgeoning with creativity.

We are delighted to deepen our partnership with Ben and Andreas through this Series A round and to support the team’s continued growth. AI is the most important theme of the next decade—driven by the scale of data, new technological breakthroughs in generative models and advancements in compute—and as the infrastructure stack of this Software 2.0 world is written, we believe Replicate will play a fundamental role. We could not be more excited to build this future together!