When (and How) to Post-Train Your Own AI Models

Sonya: Up next we have Lin, a close friend and collaborator of Brendan's [from Mercor]. Actually, a show of hands: who here does post-training? Okay, good amount of the room. And who here uses Fireworks? Okay, good amount of the room too. So Lin, you have some friendlies in the audience. For this next talk we're going to focus on post-training. Lin, similar setup to Brendan's talk: 15 minutes on how to approach the problem of post-training, and then 15 minutes or so for Q&A. Really delighted to have you here. Thank you for joining us.

Lin: Thanks for having me. Hi everyone, good morning. I'm Lin, CEO and co-founder of Fireworks. Today I'm going to do a bit of a deep dive on post-training and why post-training could be very relevant to you building your own business.

First, a little context. Fireworks is a specialized intelligence platform. There are tons and tons of applications built on top of us, all the way from startups to digital natives to enterprises. The fun part of my job is that we get to see a lot of patterns: what innovations people are building on top of us, what challenges they're facing, and what trends developers are following.

One of the things we've seen, especially in the past year, is that software and application development has been disrupted. In the past, you had a great idea, and implementing it and scaling it to production required a team of tens of very strong product engineers and PMs working together for multiple quarters. Now one person can do it in a few weeks, without knowing how to write a single line of code. That collapsing of required resources, both in timeline and in deep expertise, is shifting how competitive the application space is. And it's shifting people from building on top of off-the-shelf, black-box APIs toward building a much deeper moat, so they can build a much more durable business.

You've also seen a lot of discussion, especially in the past week, between open models and closed models, and all the rallying and support across the open model alliance. The reason that alliance runs so deep is this: look at the entire industry. There are so many companies. All of you are building your own company, and every single company exists for a reason: because it focuses on solving a unique problem in a special way. That means you carry your own judgment, taste, determination and conviction into your product. That's why a company exists.

If you build on top of an off-the-shelf API, you really need to think about how you carry that special taste, judgment and uniqueness forward. We believe one approach for every company to build a durable business is to bake your judgment, taste and deep customer understanding into the intelligence you build on top of, instead of just an off-the-shelf API. So that's the context of what's happening in the industry, and why post-training could be very, very relevant to you.

You've probably heard a lot about owning your own intelligence, not renting it. What does owning your own intelligence actually mean? It means many things.

First of all, it starts from data. Intelligence is a derivative of data. The frontier labs and the foundation models we all use are built on top of public data and labeled data, and that has solved common tasks. But all of you are solving a specific task; that's why you're building a business. Being able to curate production data with high quality, and even generate synthetic data to enrich your production data, is the first step of owning your own intelligence.

Then, after you have data, you start to use that data and turn it into a model: build on top of an existing model and own the weights. There is a collection of techniques you can use to get there. Those techniques are tailored to solve different kinds of problems you could have, and they can interoperate with each other to reach a final goal. And then, after you have a great model that belongs to you and solves a specific problem really well, you'll first do some A/B testing to make sure it really moves the needle on your product metrics, and then go back into this loop.

Obviously, I don't think you should jump into post-training right away. There are different phases you'll go through. First, prompting: everyone starts from prompting. Use a model as-is, few-shot it, and quickly test your ideas. Then you use RAG to ground the usage of AI in your own data, and you can start to do a lot of context engineering from there. Those take you from minutes of iteration to hours.

Then you progress into: I have some data, and I want to see how my data can make the model work better with my product. So you start to do supervised fine-tuning; that will take you a few hours. Then: I want the model to reflect the personalized taste that is the very unique choice of my product. So you start to use preference information collected from user interactions — thumbs up, thumbs down, that kind of signal — to help the model learn your product taste. And finally, you want to build a model that carries your expertise in your domain, whether that's legal, finance, healthcare, customer support, recruiting, marketing, sales, you name it. Even within one industry vertical there are so many subdomains. All of that is unique and special to the product you're building. You've probably heard a lot about reinforcement learning; that is the tool to build toward a specialty.

This progression is very similar to how we human beings learn knowledge over time. We learn a lot of knowledge by reading literature, where you see what is correct and what is not correct: that's very similar to supervised fine-tuning. As we grow, we develop our own taste and judgment about how we want to approach a problem: that is preference, or DPO. And over the course of time, we learn to do a really good job in a certain area — I want to be an accountant, and I really learn how to work with financial data; or I want to be a dentist, and I learn the details of how to operate in dentistry. All of this maps to exactly how we humans acquire knowledge.

It's also very interesting that different techniques exist to solve different kinds of problems. If the model doesn't know a fact, and the facts in your product are very dynamic, then typically you use RAG to solve that problem. If your model's output behavior or structure is off, then you take your data and do supervised fine-tuning to correct that. If the quality of your model's answers is personal, specific to the taste of your product, then you use preference tuning. If the model is quite weak on the special problem you're trying to solve, then you use RL, reinforcement learning. And if the end result is a model that's too slow or too expensive for you to serve in production, then you use distillation: let the teacher teach a much smaller student model so it can be much more performant and economical. Distillation also means different things for LLMs versus image generation models; if you're interested, we can talk more about that.

There are many ways teams start to try these technologies, and they may not be happy with all of their experiences, because they can burn money and time and not reach their ideal results. So here are the areas where you can get frustrated.

First, data. Data is the essence of tuning, and quantity is not the most important thing: quality is. Sometimes just throwing tons and tons of data into a training process does not lead to a great outcome. You really need to control data quality. And who is the best judge of data quality? Usually your product team. That's where we see a convergence: before gen AI, the product team and the research or ML team were separate organizations. These days, when people post-train models, the product team needs to make the judgment calls on data quality and be deeply involved in the process to ensure the best result.

Second, evals. I know all of you are very busy and racing to launch quickly, and a lot of evals are vibe evals: the founder looks at the result and feels, this is right or this is not. That is judgment; you're putting your judgment on the end result and deciding whether it's good or not. That judgment needs to convert into systematic evaluation. This is no different from traditional software development, where you have unit tests and integration tests to ensure quality. If you're thinking about doing post-training, having a way to build evals and encode your own judgment into a repeatable process is extremely important.

Then, when you do RL, there can be sloppy RL environments, where you build a simulation that doesn't really reflect reality, and the model hill-climbs on the bad simulation. It can also do reward hacking and all kinds of weird stuff. A fun story about reward hacking: this is a coding example. We asked the model to generate code that minimizes compilation errors. Guess what the model did? It generated zero lines of code. No compilation errors! But that's absolutely not what you want. Hacking like this is very common, because the model is very smart and will try all different ways to get to the goal, but it may not be what you want. So pay attention to all these details, and try not to let the model outsmart you.

Then, between your experimentation: think about your development process as a lot of experimentation, and training the model is not the end of it. The final judge is whether your product metrics move or not. So you need to bring the final model into serving and do A/B testing. That transition is extremely important, because quality can drop when you move from a training stack to a serving stack without aligning the two. Think about it: a model is tons and tons of calculation, matrix multiplication. If you do that math with different libraries, different numerics and different optimizations, it will lead to different results. The end result of training may not be replicated, or you may even lose precision, during serving. So that alignment is very important; I can give you more examples of that. There are a few other challenges you'll run into, and I'm happy to talk with you in detail offline.

There have been many pioneers we've worked with on post-training. I would say Cursor is one of the first: they got on this train at the beginning of last year. There were multiple reasons. One is they really wanted to control their destiny on the supply of the model. And they have a lot of user engagement, so they understand customer preferences and have a lot of data; that became the beginning of their journey. They do pretty deep mid-training to post-training, you've heard them announce newer models continuously every few months, and the results are great: their aspiration is to compete at frontier quality, a very bold aspiration, and they're getting there. It’s a great result from Composer to Composer 2.5. They're always trying to be on par with or beat the frontier labs' quality. They're a vibrant example that this is completely doable, as long as you stay focused, have the right tools and build on top of us.

There's a huge variety of other examples. From healthcare: Doximity is one example. They do clinical AI, where they let doctors ask deep medical questions, matching symptoms to medications and side effects, with well-rounded deep research around everything in the medical space. They also trained on Fireworks, and they topped a very important benchmark, the Stanford/Harvard clinical safety benchmark; we're very proud of that result, and they keep working on this training loop. Factory, another Sequoia company, built on top of Fireworks with a special focus on the security part of coding. That's a very hard topic, because security is not high-tolerance: you need to get it right. They tuned a model that also topped the benchmarks in the security area. All these examples are demonstrations of companies solving a unique problem in a special way, and they've been able to build their own intelligence, using their own data, by building on top of open models.

As you know, 2025 was the year of coding, and at Fireworks we support all the coding companies built on top of us: a lot of success in mid-training and post-training very strong models. As you can see, those benchmarks are very impressive, and all these coding companies aspire to be on par with or beat the frontier labs in the coding space. But it's not only coding anymore. This is the year we're starting to see very interesting development in all different kinds of co-work, from general-purpose co-work to specialized, domain-specific co-work: legal, finance, marketing, recruiting, sales, customer support. They're all starting to post-train and own the intelligence part of their product. Genspark is one of the general co-work applications; they build deep research for professionals, and slide generation. As you can see, they compete with a frontier model and are slightly better, but the cost is significantly lower. Here we're talking about a 5 to 10 times cost reduction. As a startup, once you hit product-market fit, you can scale quickly into a durable business. And Heidi is another healthcare example. Healthcare data and use cases are not well captured in frontier lab models, so you can see the quality of the tuned model is significantly better than the state-of-the-art closed model.

We have many different types of developers building on top of Fireworks to do post-training. What are those types? We've seen many frontier agent builders. They usually build bespoke, customized harnesses; they don't use common harnesses. Optimizing their harness with their systems and all the different tools they want to build on top of often requires post-training, and we've seen a lot of repeatable success there. We've also seen a lot of big companies do post-training, because interestingly, those incumbents have a huge amount of traffic. Deploying AI features across the board to all their customers is a huge cost burden. When we say be careful not to scale into bankruptcy, it's not just startups: literally, their CFOs are blocking AI feature launches because of the cost, and post-training is a way to remediate that. And we've seen a lot of specialized model operators — these could be neolabs and cutting-edge model developers — also doing significant post-training.

This is a quote we've heard repeatedly: after product-market fit, post-training becomes the vehicle for many, many companies to build specialized intelligence. We believe, and we see the trend, that in the future there could be millions of specialized models: one per application, per use case. It will be millions. That's what I believe. And we've seen different ways to convert data into a model; the signal from the reward, and how you build those reward functions, is a very important part of the story.

Our observation is: start early, and start doing experimentation much more iteratively to get hands-on experience. We've seen people onboard so quickly. There's actually not as deep a barrier as many people feel, because this whole reward engineering discipline is very similar to software engineering: the logical reasoning and the mindset are very, very similar. So get your hands dirty and start to test it out.

Those are the key takeaways. And we want to acknowledge there are different specialties and different levels of knowledge in this space. For example, we work with companies like Cursor and Cognition. They have deep researchers who want to control every single knob as much as possible to get extreme results. We give them the lowest-level API: we just provide RL rollouts they can directly interact with, and they fully control the trainer. That gives them the best results. But at many companies, you have product engineers or machine learning engineers with good enough knowledge about post-training: you want some control, but not the lowest level of control. That's where our training SDK comes in, and we iterate very closely with you to get you going. And we're also happy to deploy our own researchers hands-on to train your team and get you on board. Those are the different ways we see teams engage and get familiar with the entire process.

I want to pause here and see if you have any questions.

Speaker 3: You talked about people being most successful when they already have data and a reward signal, maybe from their product. What are the best examples of reward signals you've seen trained very successfully, very quickly?

Lin: You can think about rewards as code: you write rewards in code. And think about rubrics for rewards: you want to grade the result along multiple dimensions. For example, if you're building a recruiting agent, think about how you evaluate candidate selection. Different companies, I guarantee you, have different criteria. Maybe you want to grade a candidate on aptitude: are they really hungry, do they not take no for an answer, will they break down walls? That's one metric. The second might be that they're really fast at building things and making progress. And so on. Then you have a blended score to merge these. So think about rewarding different dimensions of rubrics; different companies blend those differently, and that's your unique part, your secret sauce.

Speaker 4: Great talk. I'm curious: you mentioned the pitfalls of starting too early, but based on what you're seeing from Cursor or Cognition or other companies, when do they actually start thinking about post-training? Is it when they feel ready? Is it when the cost is too much? The benchmarks where they're beating the frontier models are a great signal that yes, you can beat them, but is that the primary motivator? Is that 2% bump worth the cost and investment in this whole framework? And how do they think about ROI? Those models are more expensive, but post-training has some intercept of money, investment and maintenance going forward. Are they approaching it when cost becomes a concern, or when usage is spiking up and they have to plan a year or two in advance?

Lin: This is an excellent question. Think about AI building as two phases: product-market fit, and scaling the business. In the SaaS era it was one concept: you hit product-market fit and you just scale and scale as much as possible. But now we see a bifurcation: product-market fit doesn't really mean you have a durable, scalable business.

Typically, we see companies deploy the strategy of focusing on product-market fit first by building on top of frontier lab models, because you don't need to worry about anything: just spend your money and hit market fit. And another very important thing: only after you hit product-market fit is the data you collect from your product surface area really meaningful. That's when high-quality data starts to accumulate at volume from your product surface area, and that becomes the fuel for you to start owning your intelligence. We see that as a very strong indicator.

Because once you hit product-market fit and start thinking about scaling your business, you think about two things. One is continuing to keep your competitive edge. Two is building a durable business, where your revenue and your costs are in a healthy state. Post-training becomes a very appealing solution on both fronts. Post-training allows you to encode, to codify, your unique taste into a model that no one can steal from you. Because it's very easy to clone and copy an application as-is; as you all know, with coding agents, from a screenshot one can generate the same or an even better app. That's very scary: the application itself as a moat is being reduced. Baking your data, reflecting your product engagement and all your deep knowledge, into your model is a way to preserve that moat. And second, you can post-train a model that brings the cost down 5 to 10 times, which means you can support 5 to 10 times more traffic with the same budget. Your unit economics of scaling are so much better, and you avoid scaling into bankruptcy. So we see those as the two compelling stories behind the reason and the timing.