A startup known as Letta has simply emerged from stealth with tech that helps AI fashions keep in mind customers and conversations. Created in UC Berkeley’s famed labs startup manufacturing facility, it additionally introduced $10 million in seed cash led by Felicis’ Astasia Myers, at a $70 million post-money valuation.
Letta can be backed by a who’s who of angel buyers in AI, like Google’s Jeff Dean, Hugging Face’s Clem Delangue, Runway’s Cristóbal Valenzuela, and Anyscale’s Robert Nishihara, amongst others.
Founded by Berkeley PhD college students Sarah Wooders and Charles Packer, this can be a extremely anticipated AI startup launch. That’s as a result of it’s a baby of Berkeley’s Sky Computing Lab and is the industrial entity of the favored MemGPT open supply venture.
Berkeley’s Sky Computing Lab, led by acclaimed professor and Databricks co-founder Ion Stoica, is the descendent of RISELab and AMPLab, which spawned such firms as Anyscale, Databricks, and SiFive. Sky Lab, specifically, birthed quite a few well-liked open supply massive language mannequin (LLM) initiatives just like the Gorilla LLM, vLLM, and the LLM structured language SGLang.
“A ton of projects very quickly, within a year’s time frame, came out of the lab. Just people sitting next to us,” described Wooders. “So it was kind of an incredible time.”
MemGPT is one such venture and is such a scorching commodity that it really went viral earlier than it even launched.
“Someone scooped us,” Packer informed TechCrunch. The founders had posted a whitepaper on Thursday, October 12, 2023, and deliberate to launch a extra in-depth paper and the code to GitHub the next Monday. Some random particular person discovered the paper, posted it to Hacker News on Sunday, and it “went viral on Hacker News before we had a chance to properly release the code, release the paper, or, like, do a tweet thread or anything like that,” he mentioned.
The purpose for the thrill was that MemGPT mitigates a pernicious downside for LLMs: In their native kind, fashions like ChatGPT are stateless, which means they don’t retailer historic knowledge in long-term reminiscence. This is problematic for AI apps that rely on attending to know and be taught from a consumer over time — every little thing from buyer help bots to healthcare symptom-tracking apps. MemGPT manages knowledge and reminiscence in order that AI brokers and chatbots can keep in mind earlier customers and conversations.
The publish on the paper stayed atop Hacker News, the favored website for programmers run by Y Combinator, for 48 hours, Packer recounted. So he spent his weekend and the following few days answering questions on the positioning whereas attempting to get the code able to be launched. Once the venture was out there on GitHub, a hyperlink to it went viral on Hacker News, once more. YouTube interviews and tutorials, Medium posts, 11,000 stars and 1.2K forks on GitHub occurred rapidly.
VC Felicis’ Myers found Wooders and Packer by studying about MemGPT, too, and instantly acknowledged the tech’s industrial potentialities.
“I saw the paper when it was released,” she informed TechCrunch, and he or she promptly reached out to the founders. “We had an investment theme around AI agent infrastructure and appreciated that a really important component of that was the data and memory management to make these conversational chat bots and AI agents effective.”
The founders nonetheless just about traipsed round Sand Hill Road doing Zoom calls with VCs earlier than going with the one which liked them first.
Meanwhile, Stoica brokered introductions to Dean, Nishihara and different big-name Silicon Valley angels. “A lot of the professors at Berkeley, just as a consequence of being at Berkeley, are very well connected,” Packer recalled about how straightforward the angel investor course of was. “They have their eye on projects out of this lab that are going to be commercialized.”
Competition and the risk of OpenAI o1
While MemGPT is already out within the wild and getting used, Letta’s industrial variant, Letta Cloud, will not be but open for enterprise. As of Monday, Letta is accepting requests for beta customers. It will supply a hosted agent service that enables builders to deploy and run stateful brokers within the cloud, accessible through REST APIs, a programming interface that may preserve state. Letta Cloud will retailer the long-term knowledge obligatory to take action. Letta can even supply developer instruments for constructing AI brokers.
With MemGPT, Wooders sees a big span of makes use of. “I think the number one use case that we see is basically, highly personalized, very engaging chatbots,” she says. But there are additionally cutting-edge makes use of like “a chatbot for cancer patients” the place sufferers add their historical past after which share ongoing signs so the bot can be taught and supply steerage over time.
Worth noting that MemGPT isn’t alone in engaged on this. LangChain might be its finest recognized competitor, and it already provides industrial choices. The largest mannequin makers additionally supply AI agent-making instruments as properly, like OpenAI’s Assistants API.
And OpenAI’s new o1 mannequin might make the necessity to repair state a moot level for its customers. As it's a multistep mannequin, it essentially should preserve state to a point so as to “think” and reality verify earlier than it replies.
But Wooders, Packer, and Myers see just a few key variations to what Letta is providing versus what 800-pound market gorilla OpenAI is doing. Letta claims it would work with any AI mannequin and expects its customers to make use of many of them: OpenAI, Anthropic, Mistral, their very own homegrown fashions. OpenAI’s tech at present solely works with itself.
More importantly, Letta is utilizing open supply MemGPT and leaping firmly into the open supply aspect of the FOSS vs. black field LLM debate, saying open supply is a better option for AI utility programmers.
“We are positioning ourselves as the open alternative to OpenAI,” Packer says. “I think it’s actually very, very hard to build very good AI applications, especially when you care about, like hallucination, if you can’t see what’s going on under the hood.”
