Technology & AI

Content moderation of Facebook’s internal structure for the AI ​​era

When Brett Levenson left Apple in 2019 to lead corporate integrity at Facebook, the social media giant was in the throes of the Cambridge Analytica fallout. At the time, he thought he could just fix Facebook’s content moderation problem with better technology.

The problem, he quickly learned, ran deeper than technology. Human reviewers are expected to memorize a 40-page policy document that was machine-translated into their language, he said. Then they have about 30 seconds per piece of flagged content to decide not only that that content violated the rules, but what to do about it: block it, block the user, limit the spread. Those quick calls were “slightly better than 50% accurate,” according to Levenson.

“It was like flipping a coin, whether human reviewers could address the policies correctly, and this was many days after the damage had already been done,” Levenson told TechCrunch.

That kind of slow, reactive approach is unsustainable in a world of adversarial and well-funded actors. The rise of AI chatbots has only compounded the problem, as failure to moderate content has resulted in a series of high-profile incidents, such as chatbots giving teenagers guidance on self-harm or AI-generated images escaping security filters.

Levenson’s frustration led to the concept of “policy as code” – a way to transform static policy documents into a workable, reviewable logic tightly coupled with enforcement. That insight led to the launch of Moonbounce, which announced it had raised $12 million in funding on Friday, TechCrunch has learned exclusively. The round was co-led by Amplifier Partners and StepStone Group.

Moonbounce works with companies to provide an extra layer of security wherever content is generated, whether by the user or AI. The company trained its large language model to look at customer policy documents, evaluate content at runtime, provide feedback in 300 milliseconds or less, and take action. Depending on the customer’s preference, that action might look like Moonbounce’s system limiting distribution while the content awaits human review later, or it might block high-risk content for now.

Today, Moonbounce operates in three main verticals: Platforms dealing with user-generated content such as dating apps; AI companies that create characters or companions; and AI image generators.

Techcrunch event

San Francisco, CA
|
October 13-15, 2026

Moonbounce supports more than 40 million daily updates and serves more than 100 million daily users on the platform, Levenson said. Clients include AI startup Channel AI, photo and video production company Civitai, and role-playing platforms Dippy AI and Moescape.

“Security can actually be a product benefit,” Levenson told TechCrunch. “It’s never been that way because it’s always an afterthought, it’s not something you can build into your product. And we’re seeing our customers find exciting and innovative ways to use our technology to make security a difference, and part of their product story.”

Tinder’s head of trust and security recently explained how the dating platform uses these types of LLM-enabled services to achieve a 10-fold improvement in findability.

“Content moderation has always been a problem plaguing large Internet platforms, but now with LLMs at the heart of every application, the challenge is even more difficult,” said Lenny Pruss, general partner at Amplifaya Partners, in a statement. “We invested in Moonbounce because we envision a world where objective, real-time guardrails become the backbone that enables all AI applications.”

AI companies are facing growing legal and reputational pressure after chatbots were accused of pushing teenagers and vulnerable users to suicide and image producers like xAI’s Grok were used to create illegal nude images. Clearly, internal security systems are failing, and it is now a question of liability. Levenson said AI companies are increasingly looking outside their walls for help to supplement their security infrastructure.

“We’re a third person sitting between the user and the chatbot, so our system isn’t as full of context as the conversation itself is,” Levenson said. “The chatbot itself has to remember, possibly, tens of thousands of tokens that have come before…We’re more concerned with enforcing the rules at runtime.”

Levenson runs the 12-person company with former Apple colleague Ash Bharwaj, who previously built the big cloud and AI infrastructure for all of the iPhone-maker’s core offerings. Their next focus is a skill called “recursive guidance,” developed in response to cases like the 2024 suicide of a 14-year-old Florida boy who became obsessed with a Character AI chatbot. Instead of automatically rejecting when dangerous topics arise, the system will intercept the conversation and redirect it, adjusting the information in real time to push the chatbot to the most supportive response.

“We hope to be able to add to our action toolkit the ability to direct the chatbot to a better place, to, in effect, take the user’s input and process it to force the chatbot to be not just a sympathetic listener, but a useful listener in those situations,” said Levenson.

When asked if his exit strategy involves being acquired by a company like Meta, which brings his work to complete content moderation, Levenson said he sees how well Moonbounce can fit into his old employer’s stack, as well as his fiduciary duties as CEO.

“My investors would kill me for saying this, but I would hate to see someone buy us and block the technology,” he said. Like, ‘Okay, this is ours now, and no one else can benefit from it.'”

Related Articles

Leave a Reply

Your email address will not be published. Required fields are marked *

Back to top button