OpenAI has gone pretty quiet as soon as once more, with GPT-4o’s much-hyped voice chat options rolling out way more slowly than anybody had anticipated.
However there have been murmurings about new tasks within the works, together with SearchGPT, which mixes generative AI and internet looking and the extra mysterious “Undertaking Strawberry.”
Strawberry’s origins lengthen again to November 2023, when a mannequin (extra so a coaching approach) named Q* surfaced in leaks from Reuters.
It was even speculated that Q* was doubtlessly harmful and performed some position in CEO Sam Altman’s hiring and firing final 12 months.
Q* was thought to mix a complicated reasoning mannequin with an AI agent able to exploring the web.
Regardless of dramatic headlines, ‘OpenAI is sitting on an apocalyptically highly effective mannequin,’ its legitimacy was very a lot contested on the time.
Extra particulars of the Q* venture emerged in Might and June this 12 months, which noticed it renamed to Undertaking Strawberry or simply Strawberry. In keeping with Reuters, Strawberry entails a specialised technique of coaching AI fashions to discover the web autonomously and conduct ‘deep analysis.’
The Q possible refers to Q-learning, a long-established reinforcement studying (RL) approach. As for the star (*), there’s extra uncertainty. Reuters mentioned it’s much like a way developed at Stanford referred to as “Self-Taught Reasoner” or “STaR.” Others say it pertains to a search algorithm named A*.
Sources talked about that OpenAI needs the mannequin to conduct analysis by autonomously looking the online, assisted by a “computer-using agent” (CUA) – which can be a key part of SearchGPT.
In keeping with these sources, OpenAI needs Strawberry to carry out “long-horizon duties” (LHT), which contain advanced planning and execution over prolonged intervals.
Stanford professor Noah Goodman, one in every of STaR’s creators, instructed Reuters in regards to the tech, “I believe that’s each thrilling and terrifying…if issues maintain getting in that route we now have some severe issues to consider as people.”
When requested about Strawberry, an OpenAI spokesperson offered a basic assertion in regards to the firm’s AI improvement targets:
“We wish our AI fashions to see and perceive the world extra like we do. Steady analysis into new AI capabilities is a standard apply within the business, with a shared perception that these techniques will enhance in reasoning over time.”
Social media stirs the pot
Not lengthy after the Reuters report, in early August, Altman posted a photograph of strawberries accompanied by the caption “i really like summer season within the backyard,” reigniting hypothesis in regards to the Strawberry venture.
i really like summer season within the backyard pic.twitter.com/Ter5Z5nFMc
— Sam Altman (@sama) August 7, 2024
Then, the person iruletheworldmo, a sort of AI-focused meme/satire account (with a profile picture of Theodore Twombly, performed by Joaquin Phoenix, from the AI-themed movie Her, which has turn out to be related to Altman), started posting strawberry-related content material, hinting at a possible ‘stage two’ breakthrough in AI.
The person posted: “welcome to stage two. how do you are feeling? did I make you are feeling?” Altman, CEO of OpenAI, responded with “superb tbh”.
This alternate set off a sequence response of strawberry-themed posts and mass hypothesis throughout X and Reddit.
welcome to stage two.
how do you are feeling?
did I make you are feeling?
— 🍓🍓🍓 (@iruletheworldmo) August 7, 2024
Strawberry takes one other flip
Only in the near past, The Data revealed that OpenAI is gearing as much as launch a model of Strawberry as a part of a chatbot and presumably combine it into ChatGPT as quickly as this fall.
OpenAI additionally allegedly demonstrated Strawberry’s capabilities to US nationwide safety officers.
Curiously, in line with The Data, OpenAI is creating two distinct variations of Strawberry:
- This smaller, simplified model is meant for integration into chat-based functions like ChatGPT. It goals to boost reasoning capabilities in situations the place customers require extra considerate, detailed solutions fairly than fast responses.
- This bigger, extra highly effective model is used to generate high-quality “artificial” coaching information for OpenAI’s subsequent flagship language mannequin, codenamed “Orion.”
Artificial information generated by Strawberry might cut back reliance on internet-scraped textual content and pictures for coaching.
That would doubtlessly result in extra correct and dependable AI fashions, addressing persistent points like AI “hallucinations” or mannequin collapse.
Surprisingly, although, these characterizations of Strawberry don’t align that nicely with the sooner descriptions of Q*.
Maybe we might speculate that Strawberry, the autonomous agent, surfs the online autonomously and makes use of its ‘deep analysis’ to finally synthesize information.
Perhaps that’s extra computationally environment friendly and helpful for mannequin coaching than merely scraping the uncooked information itself??
AI doesn’t know what number of R’s are in strawberry
Now, right here’s the place the story takes a weird and ironic twist.
Strawberry could be named after a phrase that present AI fashions, together with a few of the most superior ones, usually battle to spell accurately.
Ask an AI what number of ‘r’s are in “strawberry,” and there’s an opportunity it’ll confidently reply “two” as an alternative of the right “three.”
— Rob DenBleyker (@RobDenBleyker) August 26, 2024
Sounds ridiculous, proper? I didn’t consider it myself till I attempted it with Claude.
When this primary got here to gentle, some alleged that this was some form of ‘easter egg’ or joke inside OpenAI’s techniques.
However seeing as Claude reacts the identical as ChatGPT, then until AI corporations are colluding on area of interest strawberry jokes behind the scenes, that appears unlikely.
The reason behind that is elegant in its simplicity.
Language fashions, regardless of the title, are math-based techniques. They don’t ‘really’ perceive phrases. Textual content is translated into code, thus risking the lack of context and that means on the phrase stage.
Why strawberry reliably triggers this shortcoming is the extra mystifying query.
In any case, whether or not OpenAI selected the title “Strawberry” as a playful nod to this frequent AI stumbling block or pure coincidence stays unclear. It looks like one thing Altman may do, whether or not Strawberry is actual or not.
What’s subsequent on this weird however berry fascinating (…) strawberry story is anybody’s guess. To be sincere, I get the sense, at this stage, that not one of the speculatory ‘proof’ we now have from main information retailers is wholly consultant of what’s occurring at OpenAI.
We’ll have to attend for SearchGPT and/or GPT-5 to see simply how developed OpenAI’s merchandise turn out to be off the again of Strawberry and their different tasks.
