Chai Discovery’s Bitter Lesson: Drug Design Is Another Scaling Problem

Intro

Matthew McPartlon: One of our big guiding principles is just simplicity. So when you look at a model like let’s say Chai-1, I think there are 23 distinct submodules in Chai-1. And when you’re trying to iterate on something like that, it gets really hard because you’re like, I need to understand each of these submodules independently. I need to understand all of their behaviors, their dynamics. And that doesn’t actually scale that well. So you can start to think, okay, how do I simplify this? How do I identify what’s really important? And once you have that, the whole research process and identifying these types of scaling directions becomes a lot simpler.

Main conversation

Pat Grady: We’re here with Josh and Matt, two of the co-founders of Chai Discovery. Chai is engineering molecules with AI, so it’s something of a foundation model lab for biology. Sounds like a big idea, but let me just start there. What is the big idea?

Joshua Meier: Thanks for having us on the show. One of the exciting things we’re trying to do at Chai is to make the drug discovery process look a little bit more like engineering. And we’ve seen all the work happening with LLMs for code generation, for instance, right? And that works really well because code’s a very simple abstraction to get across what you want to do.

Biology doesn’t look like that today. It’s a lot of trial and error. I think if we go back to the early days of just modern biotech, a lot of the medicines that we have today were discovered quite randomly. It was a bit serendipitous. And what we’re trying to do is allow us to industrialize that process a little bit more and try to come up with the tools that we’d need in order to have abstraction layers, the same way we have that in code and in modern engineering, that allows us to iterate really quickly, and try to bring that into biology and into drug discovery.

Pat Grady: So let me ask you a question on that then, and tell me if this is a reasonable way to frame it. There’s almost this boundary between that which can be engineered and that which needs to be tested in the real world. And it feels like that boundary has been moving over time, like, the proportion of the drug development process that can be engineered as opposed to trial and errored seems to be increasing. Is that a reasonable way to think about it? If so, can you talk about what specific developments have pushed that boundary over time?

Joshua Meier: I think that’s one way to think about it. The way that we often look at this is that the lab is an important part to verify that what you’re doing is correct. And actually, verification is a very big theme in AI as well. If you can evaluate that your model works, you can verify it, then you can start to hill climb that and you can make progress on it. So I think the lab is a very important part of this.

I think the question is: How do we take drug discovery and make it look a lot more like drug design? So one of the reasons why we call this field “drug discovery” is we’re often looking for a needle in a haystack. We’ll screen millions, billions of molecules, try to find one that works. And if we can instead put in what is the dream state of the molecule you want, and then the model can materialize that, that’s going to be really powerful. So it’s not even a question of reducing lab testing. I mean, maybe that happens as a result. I actually might even take the opposite side of the coin, and we can talk about that, where maybe we’d actually do even more lab testing because the ROI will increase the same way there’s more software engineers, or there’s more demand for software engineers now that they become more productive. There might be more demand for the lab. But I think the key part is how do we just change the paradigm here and how do we make it more design-oriented?

Pat Grady: Can we talk about what is state-of-the-art today? And then maybe let’s take a little trip down memory lane. Five years ago, three years ago, one year ago, what have been some of the major breakthroughs and how has state-of-the-art changed over the recent years?

Matthew McPartlon: Yeah, I think the field has really evolved, especially over the last decade. So it really wasn’t until—you can actually look back, there’s this biannual protein folding competition. So every two years, usually a bunch of academic groups would compete in this protein folding competition. What would happen is you hold out these protein targets that no one’s ever seen before, they never get deposited publicly. And then all these groups compete so you can predict the proteins the best. And really it wasn’t until 2018 where you started to see this big step change in performance. And then finally again in 2020, with AlphaFold 2. But really, it was kind of the advent of deep learning that really kicked off this whole field.

So at first, what you do—these are bringing me back to my original days in my advisor’s lab. So we worked on one of the first systems to do this with deep learning. What we would try to do is predict the distances between amino acids in a protein. And then we’d have some renderer, which then took in these kind of noisy, incomplete-looking distances and then emitted some protein from that. And what’s pretty remarkable is that with relatively minimal information, you could build a machine learning system that could actually produce something that really looks like a protein and for the most part was correct. There were still pretty big gaps. It didn’t quite have the resolution of what we have today, almost like the early image generation models where it was a little bit grainy, a little bit blurry. And now it’s like, oh my gosh, this is crazy high-definition images in seconds. So we saw that same evolution happen over time.

So it started with protein folding. And then I guess once that became more realizable, there were these other subproblems that people wanted to solve. So now, given a protein structure, can I design, say, a sequence which might fold to that? And that’s getting closer and closer to, like, the drug design problem. So you’re now thinking, okay, how do I start engineering proteins that match a certain shape or might perform a specific function?

And so people studied that problem independently, and that was 2021, 2022 time. And then these ideas began to merge together. It was really the advent of diffusion models where we started to be able to be like, okay, I can now generate a protein structure and a sequence simultaneously, and I can start making these what we’d call prompts more and more realistic. So I can now prompt these models on a target that might have a certain shape, and I can say, hey, I want to bind over here. Add more real-world constraints on the problem.

Pat Grady: What did you guys see in 2024 that made you think that was the right time to start the company?

Joshua Meier: Yeah, there were a lot of discussions that went into it. I remember one of these early discussions actually back in Matt’s house when Matt was showing me some of these results on antibody antigen structure prediction. Matt was just talking about protein folding. But for a long time, people thought that protein folding for antibodies was just too hard of a problem. People were like, there was not enough data for antibodies, in the Protein Data Bank, for instance, to solve this problem. And people, as a result, thought that antibody design was going to be out of reach.

A lot of the work that people even did on protein design in the early days with deep learning, it wasn’t antibodies. It was this different class of proteins called mini-proteins, which are actually really interesting in their own right, but they’re not what most of the drug industry is looking at. So the holy grail was whether we could design antibody proteins on the computer, especially ones that had all that therapeutic function. And our thinking was that if you couldn’t predict what an antibody looks like, how are you ever going to design one?

Matthew McPartlon: Traditionally, people have thought biology is scary. There’s so much to know.

Pat Grady: I’m terrified of biology.

Matthew McPartlon: I am as well, honestly.

Pat Grady: [laughs]

Matthew McPartlon: Like, my background was never biology. I studied pure math and started my PhD in theoretical computer science. And it was only after my third year that I ended up switching into deep learning, protein structure prediction, all of this stuff. So it was a totally new field to me, seemed insane. But at the end of the day, it’s much simpler, and the problems are much more interconnected than one might think. So I think people, when they start going into the field, they’re like, oh man, what’s an antibody? What’s a mini protein? These are all just sequences of amino acids. At the end of the day, these are just different types of prompts for the model. But in the same way where you might have a math problem that goes into ChatGPT, ChatGPT can both answer your math problem and help you with your English homework. So really, we have the same type of thing going on with our models. We just have some way of representing these sequences of amino acids, then we have a way of designing and predicting those as well. And I think in that lens, things become a lot more clear.

Pat Grady: And Matt, you mentioned your background a little bit. Josh, talk a bit about your background, and then more broadly, in order to pull this off, you have a bunch of different disciplines that kind of come together. So can you just talk a bit about your background and that of some of the other core members of the team and how these things all fit together?

Joshua Meier: Yeah, I’ve been excited about AI and biology since I was a kid. So I guess I’m mad I didn’t start with theoretical CS and get into it that way. But I went to a high school with a stem cell lab, so I was just always excited about biology as a kid, and I grew up as a programmer. I really started my career at OpenAI, so it was on the early team there. It was a nonprofit back then, so it was a pretty good time to be there. We did GPT-1, GPT-2, scaling laws. And the question was: If the models can learn to speak English, German, French, why can’t they learn to speak DNA and protein? And that was my research agenda since then. I think that intersects with around the time Matt got into the field as well. And I think it was a pretty important time as well, right? Because if you look at the kind of methods that we were bringing in, there’s been a lot of these changes on the edges, if you will, right?

And Matt talked a bit about the history of what’s happened in the field here. But it’s all about how do we find the right deep learning architectures with the right compute configuration and the right model architectures to make this happen? What are the right tasks to apply it to? We were talking about how we even knew that 2024 is the right time to start the company. As Matt was saying, these are all different kinds of amino acid sequences, and people thought that antibody class of problem was going to be too hard. And we started to see the first signs of life that actually this was starting to work. I think a lot of it was fueled by some of the new architectures we were bringing to the problems. I think it was diffusion models back then.

Matthew McPartlon: The first time anyone was able to generate reasonable-looking proteins was the advent of diffusion models. So it was pretty crazy. There were a bunch of generative modeling approaches that would kind of work if you had a bunch of data. So people got these working for images. There are a bunch of tricks to make this better and better along the way. We’ve definitely borrowed a lot of those ideas in our domain as well. But it was really once diffusion models came around …

Pat Grady: And is there an intuition for why diffusion models work?

Matthew McPartlon: Yeah. So diffusion is not a one-step process. So I think up until this point, the main generative design paradigm was called variational autoencoders. And in that case, you’re saying I want to just compress my input distribution. So you have some proteins, you want to make these look like fuzzy Gaussian vectors. That task is just really hard. And maybe today if we tried super hard, I think we’d crack it. But at the time, I think it wasn’t really developed enough to work for our problems. What turned out working really well was just giving the model more time to think and showing it more examples, like here’s a slightly broken looking protein, how do you make it better? And you can break that protein more and more and more, and you can make it look more and more noisy, more and more broken, and teach the model just quick little shortcuts. All right, here’s how I make it slightly better. And you can just keep asking over and over again, make it slightly better, make it slightly better. And breaking the problem down to that scale worked really, really well for biology.

Pat Grady: Yeah, the make it slightly better reminds me of a game that I like to play with my daughters where we have ChatGPT give us a unicorn and then we make it stronger. And we just keep telling it to make it stronger. And by the time we’re done, we have the strongest unicorn in the world. So about the same, right?

Matthew McPartlon: Yeah, that’s—it’s that easy.

Pat Grady: Okay, maybe not the same.

Sonali Singh: Yeah. So I feel like in this domain, you need to assemble a kind of quadrilingual group of people, like an Avengers squad of chemistry people, biologists, AI folks. And so that’s a challenge. How have you guys gone about finding people, convincing people to join the team, and who are your superstars?

Joshua Meier: We’ve been really pragmatic about this at Chai. If you look at the founding team, it was mostly AI researchers, so people who had worked on either scaling models or getting them to work in this domain. But really, with each generation of model, the kind of people we’ve needed for the next milestone has changed, or I’d say probably has expanded, right?

So if you look at Chai-2, that’s the point when we started to bring in some of the most incredible antibody engineers and scientists in the world. One of the scientists named Andy, Andy Yeung, actually, when we hired him, people asked us if we had pivoted into building a full-stack drug pipeline because they’re like, you’d be crazy not to do that if Andy joined your team. But I think Andy has enjoyed running more antibody campaigns in the past couple of months than he’s probably run in his whole career, which is very cool to see.

You have folks like Nathan Rollins on the team. Nathan was actually homeschooled and then started college very early on. So he joined David Baker’s lab, who won the Nobel Prize for protein design, when he was 14, started his PhD when he was 18, and has so many creative ideas. As the models started to get better, we needed to build up a product team, because while the researchers might get the models to a point that they’re very powerful, you need to build the right product interfaces so that the models are actually useful. And that’s where we started to bring on people who have built some of the most exciting products that we know about today. Like our co-founder Jack worked at Stripe, Munaz, who was one of the top 10 code contributors at Stripe, Neil, who ran his own cybersecurity company before security started to become very important as we deployed this to our big partners. As we started to scale up, we’ve brought in people who’ve really done a lot of the GPU hacking, if you will, in order to scale up our systems so that they don’t break when we’re running them at scale. We had an email from—or a Slack message from one of our hyperscalers the other day where we had a cluster, I think that had some issues, too. And they’re like, “Oh, the GPUs got too hot. I think you guys are running too many—” and we’re like, isn’t that the point? Right? I was like, “Good, we’re doing our job, at least.”

Yeah, how do you convince those people to join? I think, again, a lot of it comes back to the results and a clear need. We didn’t hire antibody engineers before an antibody design model. What are those folks going to do? We were even, I think, worried when we started that trend because for some of the next-generation formats, the complicated antibodies like multispecifics, they didn’t even work with Chai-2. So it actually took a couple of weeks when some of those people showed up before the models could work at a point that they could work on some of these interesting case studies. But fortunately, the progress was fast enough to bring that online. So I think we’re always evolving that team and going for that next milestone. We’ve got the team very small as a result, too. So this way, everyone is a little bit slightly over capacity, I think, which means we have to prioritize. It forces us to work on the things that really matter most.

Matthew McPartlon: I think on the research side as well, one of the founding engineers, Kevin Wu, he had the first—I think it was the first protein diffusion model ever. And that speaks to Kevin’s speed of execution. He is a heck of an engineer. And I think engineering has just always been important since day one. So, like, really, even our researchers, they’re all excellent engineers. And we really care about building a high-quality codebase. At the end of the day, we are technically a software company. We’re AI researchers, we’re protein designers, we’re a lot of things, but our deliverable is some piece of software. So we’ve kept that bar really high while also trying to level that with great research talent and people that can actually push the frontiers of what’s possible.

Sonali Singh: And you’ve had a number of, it seems like, aha moments in the field. Like, AlphaFold was an “Aha, we can figure out how a protein folds.” And then the diffusion models, “Aha, we can generate proteins.” And it seems like your latest models are kind of another aha moment where you’re not only generating molecules that look like proteins, but they also have therapeutic properties, like they can bind really tightly, they have really high hit rates. Can you talk about the quality of the molecules that your models are producing now, and everything that went into those models to make them able to do that?

Joshua Meier: Yeah, if we look at the quality of the molecules that’s coming out, if it goes back to one of the theses when we started the company that we really wanted to focus in on this de novo generation of the molecules. If you look at what a lot of the drug discovery AI work was at the time, it was about how do I take a molecule and just make it a little bit better, right? Which we just talked about in a sense, but it was doing it with a lab-in-the-loop style, where I take some data, try to make it better that way. And the question we started with is: Well, can we actually just do all that on the computer? Is there a way that maybe there would be enough data, or we could collect enough data, so that we could just zero-shot a molecule that has a lot of these properties?

So the first thing we needed to do to get there was to design molecules with really high success rates. When we started the company, the state of the art for antibody design was about a 0.1 percent binding rate so 1 in 1,000 of the molecules you design would actually bind in the lab. So first of all, that means you have to screen a lot of molecules to find some good ones. It also means that the gradient you get on your process is actually quite weak as well. So for many targets, you won’t get any hits. For the ones that you do get some hits, you won’t have enough to actually see whether you’re having the drug-like properties. So we really focused in on how do we just make this process more accurate? We got to with our Chai-2 model about a 15 percent success rate. So now if you screen 1,000 molecules, you’re getting 150 back. Now you can start to get some interesting statistics on the properties of the molecules. And that allowed us to iterate on that, build the right evaluations around that in the lab, and really try to hill climb that as well. And we’re getting to a point now where we can actually bake in a lot of these different properties from the start into this engine. And then maybe if you think about how does this happen or how are we approaching this as well and why do we think it’s going to continue improving?

Matthew McPartlon: Yeah. So honestly, I think Josh and I were both surprised at how quickly this worked. When we were originally budgeting this, we’re like, maybe 20 percent hit rate in three or four years. We were really shooting for a one percent hit rate. We thought 1 in 100 would be amazing. We’re like, this is going to be a groundbreaking thing. We had a philosophy or an approach that we wanted to take, and it just ended up working really well. And that approach is very similar to what’s worked in the rest of machine learning. So people kind of treat biology as this bespoke problem or bespoke field, but really it’s the same principles as, like, self-driving, LLMs.

Pat Grady: Well, same principles, but one of the things we’ve talked about before is how you manage to find scaling laws. And it’s one thing to tokenize a string of text. It’s another thing to tokenize biology. Can you say a couple words about—without giving away any of the magic, can you just say a couple words about that challenge and how you guys solve that?

Matthew McPartlon: Yeah. So one thing that I really like about Chai is we’re a very bitter lesson pill company. So we really believe in scaling data, scaling models, scaling compute. In order to do that, obviously you need to identify scaling laws. Otherwise, you’re just wasting time and resources. And I think without giving away too much, one of our big guiding principles is just simplicity. So when you look at a model, let’s say Chai-1, I think there are 23 distinct submodules in Chai-1. And when you’re trying to iterate on something like that, it gets really hard, because you’re like, I need to understand each of these submodules independently. I need to understand all of their behaviors, their dynamics. And that doesn’t actually scale that well. So you can start to think, okay, how do I simplify this? How do I identify what’s really important? And once you have that, the whole research process and identifying these types of scaling directions becomes a lot simpler.

Joshua Meier: One of the other things, too, I think is interesting about this is we take a lot of these lessons from what’s worked in the rest of the deep learning space, but as you’re pointing out, the data itself is different. The models at Chai are completely built from scratch. We’re not fine-tuning GLM or something like that on some protein data. We build everything from the ground up. I think a lot of the company-building process, though, is taking a philosophy and actually sticking with it and iterating on that and just having some guiding principles. When you’re building a blue-sky research company, if you will, Chai is almost like one of these neolabs, right? We have this big AI problem we’re going after where as we make progress on it, that opens up opportunity for our customers. But if you’re going to work on something so open-ended that way, you need some principles to guide you. And I think we’ve done a very good job on tracking those principles in the company, working on it. So a lot of the things that Matt said …

Pat Grady: Wait, can you share them? What are the guiding principles?

Joshua Meier: So I think simplicity was one of the ones that Matt mentioned. It’s this bitter lesson pillness of, like, scaling compute and data and models. It’s being really rigorous. That’s something that’s so important in this space. You can fool yourself so easily in biology. The error bars in the wet lab are actually quite large as well. So it’s actually a little bit different than if you look at code generation, for instance. If you look at SWE-Bench, people were like, oh, I—this is maybe a year ago, people were like, oh, I got 16 percent, then 17 percent, then 18 percent. I mean, in biology, if you’re plus or minus five percent in your lab, that might all be the same. So it actually just means the bar is really high in terms of the step changes that you want to see with the models. But you also need to be really honest with yourself about whether you’re making progress or not. So you could come up with some fancy model that looks like it works well on one or two new tasks, but it’s very important to show that that works more generally if you’re actually trying to build a product that can bring the field forward.

Sonali Singh: Yeah, and that’s, I think, pretty interesting because biology is one of those inherently not so verifiable domains. And you guys have been really good at showing your progress to customers and to people like us who know very little about biology. And so could we talk a little bit about the evals and the verifiable part of the model progress? Like, how do you guys know that your models are getting better?

Joshua Meier: Well, I would say first of all that I actually think this is one of the more verifiable domains. It’s actually a very objective readout. If you look at something even like codegen, right? Maybe as a verifiable task, did my code compile? Did it solve these unit tests? But how do you think about the taste, right? Did I write some really sloppy code that can’t be maintained? What does that look like?

When we think about designing a molecule in the lab, we can actually be quite specific about many of these properties, right? So maybe we get a molecule that binds the target, but can I manufacture it, right? That might be your version of some tech debt. But you can measure that. And I think those evals, again, actually make this domain more verifiable. Maybe it takes a little bit longer to validate it. It’s not like five seconds to get a readout and run a unit test. You might have to spend a couple days, a couple weeks in the lab to get that readout, but at least you can be honest with yourself.

Sonali Singh: Yeah. When we talk about progress and how good the models are, there’s a domain of targets in biology that you can, as you mentioned, sort of address with traditional screening methods. They take a long time, they’re very slow and rudimentary. And then there are targets that just aren’t addressable with existing methods. They’re not druggable for whatever reason. And so where are we in terms of model progress, in terms of working on existing targets and generating molecules faster? That’s one end of the spectrum. And then the other end of the spectrum is unlocking novel biology and new targets and things that we couldn’t drug before.

Matthew McPartlon: This kind of goes back to our core modeling philosophy. So there have actually been plenty of times where, like, man, if we add a 24th module, we can actually unlock that new target. And we’re like, is that really something that we want to maintain long-term? Is this incremental, or is this actually a compounding improvement? Will this actually help us generalize to the broader class of this whole class of targets that we really can’t hit? And so our philosophy has been, like, all right, let’s just continue to focus, identify your scaling laws. There comes a point where if the model is able to push loss down even further, it has to understand something very intrinsic about the target that it’s operating on.

One example we were talking about the other day, Paul and I, maybe the way that we’re looking at certain glycosylation sites on proteins, we’re like, oh, we might want to represent them differently or something like this. And we’re like, well, even if we didn’t represent them, there are certain sequence motifs that will tell the model there should be a glycosylation site here. In order to drive loss down further, the model should just have to learn that. So there are all these hidden features of targets where, like, if you really believe in scaling laws and you believe the models will get there, these types of targets should just unlock with better models. And of course, you still have to take this very seriously and you still need all the proper validation. You need to really challenge yourself and make sure that this is truly working. But I think our approach has always been with better models, we should be able to unlock a lot of these targets.

Joshua Meier: One of the other interesting things is if we look at—we talked about the interdisciplinary nature of this. If we look at the different teams at Chai, what people will call a hard target is actually different in literally every team. So on the science team, it might be like an undruggable GPCR target. No one’s gotten something that has modulated that in a functional way. On the ML research team, it’ll be something like, oh, the model just can’t fold this thing up. It doesn’t know what it looks like.

Matthew McPartlon:Yeah.

Joshua Meier: And then on the product team, it might look like, oh, I’ve got all these modifications like my glycosylations, and it’s a membrane protein, how do I represent that to the user? And actually, the fact that it’s different for each of these groups, I think, is a feature rather than a bug. And it means that if we want to make broad progress over here, everyone is kind of pushing in parallel on these different ways. And that means that there’s very smooth progress that we can make all the time.

And there’s usually not one bottleneck at Chai. It’s not like, oh, if we only had that one extra module, things would work. Or if we only try to push this into the product in some way, we can unlock it. We’re trying to build this unified solution, because at the end of the day, the goal of the company is to build a computer-aided design suite for molecules. It’s not to make one or two molecules. It’s not to get a pipeline of five interesting therapies that we bring to market. It’s to change the way that medicines are discovered. And if we’re going to do that, we need to work on all the hard targets regardless of how you define hard.

Matthew McPartlon:Yeah.

Pat Grady: Say more about this idea of a computer-aided design suite for molecules. What does that mean?

Joshua Meier: So at its core, it goes back to this point about making biology look more like an engineering discipline. So we’re not going and fishing something out of a large library or doing a ton of trial and error. You want to be able to specify upfront the principles that go into designing your molecule, and then have an engine that can actually realize that into some molecule that we’re going to go and create in the lab. And look, you still might do some iteration on the lab and on your model, because maybe your hypothesis was wrong. But what we want to speed up is actually again, have that computer-aided design suite so that you can go from idea to testable hypothesis very quickly. And if that loop right now takes something like nine months, I don’t know, to go and discover your molecule versus if it takes nine weeks or it takes nine days, you know, each order of magnitude just scales in a very big way the number of ideas you can really sort through. And I think that’s ultimately how the field is going to converge on better medicines.

It really comes back to, like, people sometimes talk about, do we care—and you kind of noted on it, do we care about speed or do we care about the difficulty of the targets? At some point, they converge in this way as well, because a hard target, if we can iterate through hypotheses a lot faster, then maybe it’ll be easier to crack it.

Pat Grady: So if we can maybe detach ourselves from the present reality and go far enough into the future that we’re not thinking present forward, we’re actually thinking future back—2035, 2040, 2100, whatever you want—what does the industry look like? Like, let’s imagine that computer-aided design suite for molecules has become a standard. Let’s imagine a lot of innovation has flowed downstream into some of the wet lab parts of the process. Can you paint the picture of what the industry might look like 10 years from now?

Joshua Meier: Yeah, I think it’s going to be a really exciting time. And we can look at this on a couple of angles. So first of all, the quality of the medicines that we develop will hopefully go up. There’s a lot of molecules we put into the clinic today that there is—it’s really hard to discover a molecule. You find something that’s like 80 percent of the way there, maybe I’m just going to advance it anyways to hit my timelines. It’s probably going to benefit some patients, but then you get beat a year later by someone else. And it’s really just not the most efficient spend of resources in the industry. You’ve got the kind of diseases that are just too hard to go after today. People have been trying to drug Alzheimer’s forever. And unfortunately, haven’t made as much progress as we’d like. You have things that just aren’t economical to go after today. Think about personalized medicines, rare diseases, things where maybe the patient population is going to be smaller. But again, if we can iterate through those ideas faster, if we can launch something faster, if we can do it in a less expensive way, then those probably come into reach as well. So I think there are just so many different axes that we’re able to push on. And I think that means that the future is really bright.

Matthew McPartlon: It used to be like you either want to be first in class or best in class.

Pat Grady: Yeah.

Matthew McPartlon: And now it’s like you want to be last in class, because you actually just want to be the final answer. So I think a lot of what you’ll see is just way more intentionality in the types of drugs that you’re designing. Like, this drug will be super specific to the disease of interest. It won’t have certain interactions, like certain negative interactions that a lot of drugs today do. A lot of this stuff is actually lie, you’re able to model most of this computationally, maybe not today, but there’s definitely a path towards getting there. And I think that’s one of the most exciting things for me.

Pat Grady: Very cool. Can we talk about a business model decision that you guys made? Because I think a lot of times folks think about Chai and Isomorphic in the same neighborhood. Isomorphic is developing drugs. You guys are enabling the existing industry to develop drugs more efficiently, better, faster, cheaper than they have before. Why did you decide to go down that path versus the Isomorphic path?

Joshua Meier: Yeah. first of all, I think both of these paths are great and they can create tremendous value. We’ve always been really excited about building infrastructure for the industry. Our bet when we started to go back to those results in Matt’s house a few years ago was that this is how most future drugs are going to be discovered. And if that’s the case, somebody needs to go and build that infrastructure and make it happen.

I think part of this, too, is a lot of our founding team, including Matt and I, had worked in companies before where we had built these full-stack drug pipelines, right? And we were building AI models, we were putting those drugs into the clinic. Again, we’re pretty bitter lesson pilled as well. And our thinking was as the models get better, we want to be spending more of the money making better models as opposed to diverting those resources into clinical trials and things like that.

And one of the interesting things about our business models is that as the models get better, it actually wins us the right to continue investing more in them, right? And you have partnerships with these pharma companies that are paying off today and allow us to continue to invest in it. So it’s a much more scalable business in that way. And I just think about the ultimate impact that we can create for the world is a lot larger.

We’ve always wanted to just partner broadly with the ecosystem. It goes back to this point about fooling yourself in biology. If you work on a small number of drug targets, you might come up with the most exquisite molecules, creating a ton of value for the world by doing that, but you might miss the forest for the trees because maybe your model doesn’t generalize to the other 500 molecules that people are going to make that year. And if you go and partner with people, you just can’t fool yourself.

Like, if you look at our partners, Eli Lilly, Novartis, argenx, Pfizer, like, these are not companies that take this stuff for granted. You have to really deliver on these partnerships for them to take you seriously. And that means it can’t just work some of the time. When we ship models at Chai, they really have to work. They have to deliver value to our partners. And it’s not like, oh, we made some molecule, doesn’t fully work. We’re going to have our chemists clean it up a little bit. So it’s almost a harder business to pull off. I think that’s one of the reasons why you haven’t seen it pursued many times. If you work on a drug pipeline again, there might be ways to fix things up later. There will be the proof of what happens in the clinic, of course, but when you have this partnering-based model, you have to be really rigorous about your models. Your models have to work really well because otherwise those partners are not going to come easily. So it’s made our life harder, I think, in many ways, but I think it’s also the more rewarding path if we can get it to work.

Pat Grady: Well, what have you learned? I mean, you’re out of the lab, so to speak, and in the real world delivering real value for actual pharma companies. What have you learned as you started to work with these partners in terms of any surprises, in terms of how their needs might have differed from what you expected, or how their level of sophistication around this might be different than what you expected? Have you had any surprises or any learnings from working with these partners?

Joshua Meier: Yeah, so when we went into these partnerships, I think a lot of people told us that, like, pharma doesn’t know how to use AI, these are not—it’s not like a tech-forward industry and things like that. And to be honest, that hasn’t really been our experience. I think that these are—again, they’re very rigorous partners, very rigorous customers, right? And they’re going to test every one of our claims before they start to deploy these things. But when they see the data, they go all in, right? Because pharma is an innovation industry. I think it’s interesting just to think about even the whole economics of the pharma industry. If you build a product in pharma, like a drug, you only have exclusivity on that drug for a certain period of time. And you have to continue to reinvest.

Eli Lilly is a trillion-dollar pharma company right now. If they don’t get more blockbuster drugs, they will not be a trillion-dollar pharma company forever. And I think that forces these companies to really be on their game of adopting new technologies and deploying them and trying to stay ahead. Pharma is a very competitive arena. There are a lot of people trying to bring these drugs to patients, which by the way is great for patients, but it means that you have to be on top of your game here. And I think that means that once you’re through that door and your models are working, we’ve seen an upscale of this adoption very quickly and people thinking about how to use the models in incredibly creative ways.

Matthew McPartlon: I really like the incentive structure that it creates as well, like, taking more of the partnership model. As our models get better, we get better results to our customers and so on. So I think that’s just a nice side effect for us as well. Chai has been incredibly focused. And really, as the models get better, you can iterate on a better model with better data and so on, so a lot of what we see and a lot of what our partners are using the model for, like generating binders, antibodies, and so on, we’re also doing a lot of dogfooding in-house. We have a whole science team that’s using the models and just trying to understand what they can do.

And a lot of that comes back to, like, okay, now that we’ve unlocked use case X, what type of new data can we generate? How can we make the model better that way? And really, that’s been the long-term vision of Chai is we never thought of we’re going to build this one model that’s going to solve all of this. We know that just like in other fields, there are going to be multiple iterations of this model. And as you get better and better models, you get this flywheel effect, I guess.

Pat Grady: And you mentioned what data can we generate? Can we touch on data for a minute? You guys can’t exactly just go scrape the internet and have all the data you need to build your models. Where does the data come from? How does it compound over time? Can you just say a word about data as an input to your models?

Matthew McPartlon: Yeah. So I think the primary source of data or the gold standard source of data is this protein database. So this is actually just legit lab scientists who since 1970 have just been depositing crystal structures of proteins and other molecules over time. And really without that, structure prediction and design wouldn’t have been a thing.

So that’s one source of structural data. When I started in the field, I really took a structure-pilled approach. I was super interested in predicting structure. How do you think about a machine learning model that can output 3D coordinates? That doesn’t look like an LLM. It doesn’t look like an image model. This is really in its own class.

Josh, interestingly, was taking the exact opposite approach. So he was one of the original authors on ESM, and that was really a seminal work in understanding how to apply language models to protein sequences. What’s really cool about that work is if you can train a language model to understand protein sequences, what ends up happening is it ends up representing the 3D structure internally.

And there’s a really interesting reason why this happens. So in order to predict missing amino acids, same way that you’d predict next word in a sentence, I want to predict next amino acid in a protein. In order to do that effectively, you really need to understand, okay, what does that amino acid’s immediate microenvironment look like? Because that kind of tells you, okay, what are the compatible amino acids with everything surrounding it? And in order to do that well, you need to understand the protein’s 3D shape.

So I was taking this really structure-based approach and Josh was even more bitter lesson-pilled than me. He’s like, we’re just going to look at the sequences and this is just going to emerge. So yeah, the two main sources of this data, again, protein database for structures and then these massive, massive, maybe even order of trillions of token sequence databases. And what you can do once you have really good models is just run them on the sequence databases to get new structures out. So again, you have this compounding effect. As your models get better, they get more and more accurate at predicting these structures, and then you have more and more training data for the next series of models.

Joshua Meier: Sometimes people ask us at Chai about—these days at Chai, which paradigm are you actually going after? I think one of the things I love about our team is that it’s actually just neither. We’re very pragmatic. We want to solve the problem. We don’t really care, is it a sequence approach, is it a structure approach? In practice, it’s going to end up being both, of course. I think you’d be surprised, but there might even be more biological sequence tokens on the internet than English language tokens. Now a lot of that data is not that useful—it might not be redundant, it might be very noisy, but there’s a lot of data out there. There’s a lot of art in bringing it together. I think also the exciting thing is as the models have gotten to a point now where we can design things in the lab, write to new targets, for instance, we can actually use the models to generate data as well.

So there’s a lot of exhaust from all the experiments that we’re doing at Chai, which also help to make the models even better. So it’s a similar kind of takeoff that we saw with LLMs. Like, when I was at OpenAI, we worked on reinforcement learning of a GPT-1 architecture. It did not work because the models weren’t good enough, but once the base model got good enough, then you could start to do those kinds of experiments. And I think there’s a similar analogy that’s starting to happen in our world now as well, where the models have reached a point where there’s actually a renewed interest in data and how do we actually bring the models into the loop on making that happen? I think that creates another really interesting cycle on compounding improvement to the models.

Sonali Singh: We were talking a bit about compounding improvement in the models, and you said something about how pharma as an industry is an incredibly competitive landscape. And it’s interesting, because I think there’s been a renewed interest in using AI and using ML to generate proteins and molecules. And so your arena has actually become quite competitive, and you guys have obviously done an amazing job at staying at the frontier. It’s been the year of deployment for you. You’ve locked up a number of pharma partnerships that are making your models better. But how do you guys think about the competitive landscape and staying at the front of the frontier?

Joshua Meier: Well, first of all, I think it goes back to looking at the results and not fooling yourself and being rigorous. So one of the reasons why we do a lot of evaluation of the models, is we mainly do it just to hill climb on the models itself. I think if you look at many of the capabilities that we’ve brought in have kind of been first in the field. You look at our Chai-2 model, getting to success rate of the models that you didn’t have to do large library screening anymore to see results. A couple months later, showing how we could bring in a lot of these developability properties we’ve talked about before, like the manufacturability of the molecules.

So in many cases, we are pushing the model forward and trying to see these capabilities emerge. And then we try to quickly lock those in. How do we make those capabilities even more pronounced that they become production ready and we can ship them to our customers?

I think one of the things I like to tell the team is that it’s not like we are head-to-head with other model providers or something like that, where all of us, I think, are working against nature. Nature’s actually been a pretty good baseline. People have done drug discovery a certain way for a long time. And Matt talked about, like, how maybe you don’t want to add module 24 to the Chai model, but people have added module 240 to the existing wet lab protocols. And they have been tuned quite considerably. One of the scientists on our team, Andy Yeung, was one of the first people working on yeast display at MIT, actually, two decades ago. And he’s got 20 years’ experience in Pfizer and Genentech, really honing in these methods, has a drug approval to his name in antibodies.

And I think you look at someone like that, and Andy knows how to make a good antibody with existing tools, and that is actually the bar that we need to clear. Now of course, I think the ceiling on AI is going to be a lot higher than what we’ve managed to do before. Otherwise, what would be the point of doing this? We didn’t start the company just to make a 10 times faster mouse, right? We started this to make breakthrough medicines that weren’t possible before. But ultimately, that is the bar that we needed to clear in order to get adoption. I think we hit that inflection point a couple of months ago. That’s why you’ve seen a lot of these big pharma announcements. But now we just need to continue to hone in on making these things even better.

Pat Grady: For somebody who’s listening who thinks, wow, this sounds pretty cool, I wonder what it would be like to work at Chai. What is the best thing about working at Chai and what is the worst thing about working at Chai?

Matthew McPartlon: Yeah, I can speak to some of this. I’ll think of this on the fly, the worst thing. So I think the best thing is just how actually mission-driven everybody is. Everyone is so dedicated to what they’re doing. I’ve worked at other companies, and the closest I’ve ever seen to this is maybe some of the guys in my PhD lab. But everyone is just incredibly, incredibly motivated. We all work really hard. There’s an obvious shared goal. And I think that’s really rare to see. And I think this goes back to just the focus that we’ve had since the beginning. And we’ve always had a clear philosophy, a clear plan on how things are going to get there, how things are going to get better. And really, everyone at Chai is very bought into this. It’s pretty amazing just to see the amount of dedication that everyone’s putting in.

Least favorite thing about Chai? Not directly on top of dandelion chocolate, maybe?

Pat Grady: [laughs]

Joshua Meier: Maybe the next office.

Matthew McPartlon: Yeah. Actually, so probably the least favorite part now is just, I guess, it’s getting things to work at scale. And really, I didn’t even know what that meant. Actually, when we started Chai, we had 128 GPUs and I was like, this is the most scale possible for a lab. This is crazy. I came from my PhD group where there were four of us sharing eight GPUs and I just felt GPU rich, which was crazy. So now at Chai, we have a lot more infrastructure to maintain. We have a lot more compute resources. Luckily, GPUs are parallelizable, but that also kind of brings up its own set of problems. So just like how do you keep a cluster healthy over time? How do you get that large training run? How do you keep that running for months on end? And even when it does die, how do you automatically resume these things? How do you keep all of your communication down? How do you optimize the models and make the best use of the resources that you have? So I think these are a lot of problems that continuously pop up. They’re good problems to have, but I think they’re also really difficult to solve. Yeah, and I’m just excited to work on those.

Pat Grady: Yeah, I remember one of the CEOs that we’ve worked with a couple of times, Frank Slootman, had this line about you either have the pain of failure or the pain of growth. You’d much rather have the pain of growth.

Matthew McPartlon: Yeah, that’s exactly right.

Joshua Meier: I think my favorite part is probably the results. And that sounds a bit cliché, but there’s nothing …

Pat Grady: That’s working.

Joshua Meier: It’s working, right? And just knowing that you’re—I think many of us in the company, we’ve been working in AI for a long time, right? There’s all this experience you’ve built up. And to know that you’re applying it to something that really matters, just even, I think, just take Matt and I. We’ve been working on this problem for, like, 10 years, right? And a couple of years ago, I’m looking at something like, yeah, we’re writing some cool papers. Everyone is celebrating this. Are we actually making the world better? Is this actually going to impact some patients? And I think now the answer is actually, yes, we have reached the point where this is going to make a big difference in the world. And every time you get one of these breakthrough results, anytime there’s a new feature on the product that makes our lives of our customers easier, whenever there’s new lab results coming back from the science team, it’s just always so honestly exhilarating to realize this is actually going to change the world in a pretty profound way.

I think that also then maybe comes to the least favorite side, right? We’re running a company and it’s like we have real partners that are relying on this. And things have to work, right? And you ship a new model generation. How do you make sure that there’s no bugs in that? How do you make sure you don’t have regressions, right? This is no longer just, again, the blue-sky research problem of, oh, we got some cool results and we move on. We’ve had to have really high priorities on having production-level codebases as the team grows. How do we make sure that the codebase is in a state that more people can contribute to this?

So something that one of our other co-founders, Jack, likes to say is that if you want to move fast in the long term, you sometimes have to just move a little bit slower in the short term, and make sure that you are building something—again, that goes back to that compounding idea. It goes back to we don’t add module 25 to make the next thing work. So sometimes you’re so excited to get to the next result and you just want to jump into it. But we have real partners, some of the biggest companies in the world that are now relying on us. And it’s important that we realize that, we take that responsibility to heart, and we make sure we’re building systems that continue to work.

Sonali Singh: I have a burning question. Why are you guys called Chai Discovery?

Joshua Meier: Chemistry and AI. But we love chai tea as well.

Pat Grady: Oh!

Joshua Meier: There’s a lot of chai-themed stuff in the office.

Sonali Singh: That’s smart.

Pat Grady: That’s a good question. I didn’t know that either.

Matthew McPartlon: Josh is the visionary. That was all him.

Pat Grady: It is a very user-friendly name.

Joshua Meier: Yeah, we also wanted a name that—biotech companies have such complicated names.

Matthew McPartlon: Yeah.

Joshua Meier: We wanted something that’s going to be much simpler.

Matthew McPartlon: Yes.

Joshua Meier: We’re trying to make this whole thing simpler, right?

Pat Grady: Yeah.

Joshua Meier: So we need a simple name to go along with it.

Pat Grady: Awesome. What are you guys most excited about in the next six to twelve months?

Joshua Meier: I think for me, it’s just the deployments that are happening. So we’ve announced a couple of these partnerships, and I’m really excited just to hear about the results that our partners are bringing online. It goes back to this point of making a real difference, and also why I’m so happy to see how these partnerships are going, even post the agreement and as we’re working with these folks that the models are not just sitting on a shelf somewhere, they’re actually being used on real programs, people trying to approach devastating diseases where if Chai could give them a molecule that works, it could really change the lives of patients. So I’m really excited to see how that goes.

Just the pace of progress here is incredible, but also just the pace of the models. Like, a year ago, you could not zero-shot a molecule and have a good sense that your program was going to work. Now that’s changed. Someone might zero-shot a molecule and be like, I think we’re going to bring this program to the clinic now. And then a year later, they might even have some of those first molecules going into patients. And just the speed of that is just incredible. And sometimes you get some shivers thinking about this that, like, okay, my model is going to—Matt has some patents from our last company about just generating the molecule on the computer, and these things are now in patients. And just to think about the scale, I don’t know, a few years from now, do we have dozens? Do we have hundreds of Chai molecules going into people? It’s a bit mind-blowing to think about what that might look like for patients.

Matthew McPartlon: I think one of the things that again, what makes Chai unique, is the dedication. People are like, man, you work a lot. And aren’t you burnt out or whatever? It’s actually really easy and it’s very motivating when you’re making progress. Seeing the progress that we’re making and just thinking, man, the next model is going to be even better than the last. We identified this new thing and so on. Like, that’s so incredibly motivating. For me, it’s more of just what can we unlock next? And how do we make these things more controllable? And when someone comes to us with a certain target, instead of just hoping we get good affinity or something like that, can we actually control this? Can we say, like, we want exactly a 10 nanomolar binder or things like that? There are a lot of technical things that I think we’re kind of right on the brink of solving. And for me, it’s really motivating just to pin those things down and just get all of this over the line and see kind of where that leads to next.

Pat Grady: Awesome. Matt, Josh, thank you for engineering biology, and thank you for sharing your story with us today.

Matthew McPartlon: Thank you, guys.

Sonali Singh: Thank you.

Joshua Meier: Thanks for having us.