Delve implemented security compliance for LiteLLM, an AI project hit by malware

This is one of those real Silicon Valley episodes that seems ripped from an HBO satire show. This week, another truly malicious malware was discovered in an open source project created by Y Combinator graduate LiteLLM.

LiteLLM gives developers easy access to hundreds of AI models and offers features such as currency management. It’s a hit, being uploaded 3.4 million times a day, according to Snyk, one of the many security researchers monitoring the incident. The project had 40K stars on GitHub and thousands of forks (those who used it as a base to modify it made their own).

The malware was discovered, documented, and exposed by research scientist Callum McMahon of FutureSearch, a company that provides AI agents for web research. The malware entered through a “dependency,” meaning other open source software that LiteLLM relies on. It then steals the login information of everything it touches. With those credentials, the malware gained access to open source packages and accounts to harvest more information, and more.

The malware caused McMahon’s machine to shut down after downloading LiteLLM. That incident made us investigate and find out. Ironically, a bug in the malware caused his machine to explode. Because that damn code was so sloppily designed, he (along with renowned AI researcher Andrej Karpathy) concluded that it must have been coded for vibe.

The developers of LiteLLM have been working non-stop this week to fix this situation and the good news is that it was caught relatively quickly, about a few hours.

There is another part of this saga that people in X can’t stop talking about. LiteLLM, as of March 25 when we checked, still proudly states on its website that it has achieved two major security compliance certifications, SOC2 and ISO 27001.

But it used an implementation called Delve for those certificates.

Delve is a Y-Combinator AI-powered compliance startup accused of misleading its clients about their true compliance by allegedly generating false data, and using auditors to rubber-stamp reporting. Delve denies these allegations.

There is one point of nuance here that deserves to be understood. Such certifications are intended to show that the company has strict security policies in place to limit incidents like this. Certificates do not automatically prevent a company, such as LiteLLM, from being attacked by malware. Although SOC 2 should include policies that address software dependencies, malware can still get in.

Anyway, as developer Gergely Orosz said to X when he saw people making fun of it online, “Oh damn, I thought this was a joke. …

As for LiteLLM, CEO Krrish Dholakia did not comment on the use of Delve. You are still busy cleaning up the mess from being the victim of an attack.

“Most important to us right now is the active investigation associated with Mandiant. We are committed to sharing technical lessons learned with the developer community once our investigation review is complete,” he told TechCrunch.