In a while after Hunter Lightman joined OpenAI as a researcher in 2022, he watched his colleagues release ChatGPT, one of the most fastest-growing merchandise ever. In the meantime, Lightman quietly labored on a crew educating OpenAI’s fashions to resolve highschool math competitions.
Nowadays that crew, referred to as MathGen, is regarded as instrumental to OpenAI’s industry-leading effort to create AI reasoning fashions: the core generation in the back of AI brokers that may do duties on a pc like a human would.
“We have been seeking to make the fashions higher at mathematical reasoning, which on the time they weren’t excellent at,” Lightman instructed TechCrunch, describing MathGen’s early paintings.
OpenAI’s fashions are a long way from very best nowadays — the corporate’s newest AI programs nonetheless hallucinate and its brokers battle with advanced duties.
However its state of the art fashions have advanced considerably on mathematical reasoning. One in all OpenAI’s fashions lately received a gold medal on the Global Math Olympiad, a math festival for the sector’s brightest highschool scholars. OpenAI believes those reasoning features will translate to different topics, and in the long run energy general-purpose brokers that the corporate has at all times dreamed of establishing.
ChatGPT used to be a cheerful twist of fate — a lowkey analysis preview became viral client trade — however OpenAI’s brokers are the made from a years-long, planned effort inside the corporate.
“Ultimately, you’ll simply ask the pc for what you wish to have and it’ll do all of those duties for you,” mentioned OpenAI CEO Sam Altman on the corporate’s first developer convention in 2023. “Those features are frequently mentioned within the AI box as brokers. The upsides of this are going to be super.”
Techcrunch match
San Francisco
|
October 27-29, 2025
Whether or not brokers will meet Altman’s imaginative and prescient continues to be noticed, however OpenAI stunned the sector with the discharge of its first AI reasoning fashion, o1, within the fall of 2024. Not up to a yr later, the 21 foundational researchers in the back of that leap forward are probably the most extremely sought-after ability in Silicon Valley.
Mark Zuckerberg recruited 5 of the o1 researchers to paintings on Meta’s new superintelligence-focused unit, providing some repayment programs north of $100 million. One in all them, Shengjia Zhao, used to be lately named leader scientist of Meta Superintelligence Labs.
The reinforcement finding out renaissance
The upward push of OpenAI’s reasoning fashions and brokers are tied to a gadget finding out practicing methodology referred to as reinforcement finding out (RL). RL supplies comments to an AI fashion on whether or not its possible choices have been proper or now not in simulated environments.
RL has been used for many years. For example, in 2016, a few yr after OpenAI used to be based in 2015, an AI gadget created by way of Google DeepMind the usage of RL, AlphaGo, received international consideration after beating an international champion within the board sport, Pass.
Round that point, considered one of OpenAI’s first workers, Andrej Karpathy, started brooding about tips on how to leverage RL to create an AI agent that would use a pc. However it might take years for OpenAI to increase the important fashions and coaching tactics.
Through 2018, OpenAI pioneered its first huge language fashion within the GPT sequence, pretrained on huge quantities of web knowledge and a big clusters of GPUs. GPT fashions excelled at textual content processing, ultimately resulting in ChatGPT, however struggled with simple arithmetic.
It took till 2023 for OpenAI to succeed in a leap forward, first of all dubbed “Q*” after which “Strawberry,” by way of combining LLMs, RL, and a method referred to as test-time computation. The latter gave the fashions additional time and computing energy to devise and paintings via issues, verifying its steps, sooner than offering a solution.
This allowed OpenAI to introduce a brand new way referred to as “chain-of-thought” (CoT), which advanced AI’s efficiency on math questions the fashions hadn’t noticed sooner than.
“I may see the fashion beginning to reason why,” mentioned El Kishky. “It could realize errors and back off, it might get annoyed. It in point of fact felt like studying the ideas of an individual.”
Despite the fact that for my part those tactics weren’t novel, OpenAI uniquely mixed them to create Strawberry, which immediately resulted in the advance of o1. OpenAI briefly recognized that the making plans and truth checking talents of AI reasoning fashions may well be helpful to energy AI brokers.
“We had solved an issue that I were banging my head towards for a few years,” mentioned Lightman. “It used to be one of the crucial thrilling moments of my analysis profession.”
Scaling reasoning
With AI reasoning fashions, OpenAI made up our minds it had two new axes that may permit it to fortify AI fashions: the usage of extra computational energy all over the post-training of AI fashions, and giving AI fashions extra time and processing energy whilst answering a query.
“OpenAI, as an organization, thinks so much about now not simply the best way issues are, however the best way issues are going to scale,” mentioned Lightman.
In a while after the 2023 Strawberry leap forward, OpenAI spun up an “Brokers” crew led by way of OpenAI researcher Daniel Selsam to make additional development in this new paradigm, two assets instructed TechCrunch. Even though the crew used to be referred to as “Brokers,” OpenAI didn’t first of all differentiate between reasoning fashions and brokers as we bring to mind them nowadays. The corporate simply sought after to make AI programs able to finishing advanced duties.
Ultimately, the paintings of Selsam’s Brokers crew changed into a part of a bigger challenge to increase the o1 reasoning fashion, with leaders together with OpenAI co-founder Ilya Sutskever, leader analysis officer Mark Chen, and leader scientist Jakub Pachocki.
OpenAI must divert treasured sources — principally ability and GPUs — to create o1. All the way through OpenAI’s historical past, researchers have needed to negotiate with corporate leaders to acquire sources; demonstrating breakthroughs used to be a surefire approach to protected them.
“One of the most core elements of OpenAI is that the whole thing in analysis is backside up,” mentioned Lightman. “Once we confirmed the proof [for o1], the corporate used to be like, ‘This is smart, let’s push on it.’”
Some former workers say that the startup’s challenge to increase AGI used to be the important thing issue achieve breakthroughs round AI reasoning fashions. Through that specialize in creating the smartest-possible AI fashions, slightly than merchandise, OpenAI used to be ready to prioritize o1 above different efforts. That form of huge funding in concepts wasn’t at all times probable at competing AI labs.
The verdict to take a look at new practicing strategies proved prescient. Through past due 2024, a number of main AI labs began seeing diminishing returns on fashions created via conventional pretraining scaling. Nowadays, a lot of the AI box’s momentum comes from advances in reasoning fashions.
What does it imply for an AI to “reason why?”
In some ways, the purpose of AI analysis is to recreate human intelligence with computer systems. Because the release of o1, ChatGPT’s UX has been full of extra human-sounding options similar to “considering” and “reasoning.”
When requested whether or not OpenAI’s fashions have been in reality reasoning, El Kishky hedged, announcing he thinks about the idea that on the subject of laptop science.
“We’re educating the fashion tips on how to successfully burn up compute to get a solution. So when you outline it that means, sure, it’s reasoning,” mentioned El Kishky.
Lightman takes the way of that specialize in the fashion’s effects and now not as a lot at the approach or their relation to human brains.
“If the fashion is doing onerous issues, then it’s doing no matter important approximation of reasoning it wishes as a way to do this,” mentioned Lightman. “We will name it reasoning, as it seems like those reasoning lines, however it’s all only a proxy for seeking to make AI gear which can be in point of fact robust and helpful to numerous other people.”
OpenAI’s researchers word other people might disagree with their nomenclature or definitions of reasoning — and indisputably, critics have emerged — however they argue it’s much less necessary than the features in their fashions. Different AI researchers have a tendency to agree.
Nathan Lambert, an AI researcher with the non-profit AI2, compares AI reasoning modes to airplanes in a weblog put up. Each, he says, are artifical programs impressed by way of nature — human reasoning and fowl flight, respectively — however they function via solely other mechanisms. That doesn’t cause them to any much less helpful, or any much less able to attaining identical results.
A bunch of AI researchers from OpenAI, Anthropic, and Google DeepMind agreed in a up to date place paper that AI reasoning fashions aren’t effectively understood nowadays, and extra analysis is wanted. It can be too early to expectantly declare what precisely is occurring within them.
The following frontier: AI brokers for subjective duties
The AI brokers in the marketplace nowadays paintings perfect for well-defined, verifiable domain names similar to coding. OpenAI’s Codex agent objectives to lend a hand instrument engineers offload easy coding duties. In the meantime, Anthropic’s fashions have develop into specifically well-liked in AI coding gear like Cursor and Claude Code — those are one of the vital first AI brokers that individuals are prepared to pay up for.
Then again, total aim AI brokers like OpenAI’s ChatGPT Agent and Perplexity’s Comet battle with most of the advanced, subjective duties other people wish to automate. When making an attempt to make use of those gear for on-line buying groceries or discovering a long-term parking spot, I’ve discovered the brokers take longer than I’d like and make foolish errors.
Brokers are, after all, early programs that can indisputably fortify. However researchers will have to first work out tips on how to higher teach the underlying fashions to finish duties which can be extra subjective.
“Like many issues in gadget finding out, it’s a knowledge drawback,” mentioned Lightman, when requested in regards to the boundaries of brokers on subjective duties. “One of the analysis I’m in point of fact desirous about at the moment is determining tips on how to teach on much less verifiable duties. We have now some leads on tips on how to do this stuff.”
Noam Brown, an OpenAI researcher who helped create the IMO fashion and o1, instructed TechCrunch that OpenAI has new general-purpose RL tactics which permit them to show AI fashions talents that aren’t simply verified. This used to be how the corporate constructed the fashion which completed a gold medal at IMO, he mentioned.
OpenAI’s IMO fashion used to be a more recent AI gadget that spawns more than one brokers, which then concurrently discover a number of concepts, after which select the most productive probable resolution. Some of these AI fashions are gaining popularity; Google and xAI have lately launched state of the art fashions the usage of this method.
“I believe those fashions will develop into extra succesful at math, and I believe they’ll get extra succesful in different reasoning spaces as effectively,” mentioned Brown. “The development has been extremely rapid. I don’t see any reason why to assume it’s going to decelerate.”
Those tactics might lend a hand OpenAI’s fashions develop into extra performant, features that would display up within the corporate’s upcoming GPT-5 fashion. OpenAI hopes to claim its dominance over competition with the release of GPT-5, preferably providing the perfect AI fashion to energy brokers for builders and shoppers.
However the corporate additionally desires to make its merchandise more practical to make use of. El Kishky says OpenAI desires to increase AI brokers that intuitively perceive what customers need, with out requiring them to choose explicit settings. He says OpenAI objectives to construct AI programs that perceive when to name up positive gear, and the way lengthy to reason why for.
Those concepts paint an image of an final model of ChatGPT: an agent that may do anything else on the web for you, and know how you need it to be accomplished. That’s a far other product than what ChatGPT is nowadays, however the corporate’s analysis is squarely headed on this path.
Whilst OpenAI indisputably led the AI {industry} a couple of years in the past, the corporate now faces a tranche of worthy combatants. The query is not simply whether or not OpenAI can ship its agentic long run, however can the corporate achieve this sooner than Google, Anthropic, xAI, or Meta beat them to it?
