.png)
.jpeg)
Scroll through your feed. Can you confidently tell which video is real and which is a sophisticated AI deepfake? As generative AI tools become commonplace, the line blurs, creating a potential "high garbage, low trust" internet[2]. Scams get slicker, misinformation spreads faster, and even creators worry about their original work being stolen or manipulated[2]. It's a growing headache for online platforms trying to maintain integrity.
For one startup founder, a seasoned AI researcher with deep experience in computer vision and fraud detection from a previous major tech venture, this wasn't just a headline – it was a technical challenge he'd grappled with before[1]. Teaming up with a co-founder steeped in 15 years of trust and safety policy work for giants like Facebook and numerous advisory boards, they saw a critical gap[1]. In a conversation with Misfits, they shared their journey building a solution aimed squarely at this problem[1][2].
"The speed at which problematic content and scams are being created is just so high that the future of the internet is actually a threat that we are going to high garbage load trust... [our venture] is trying to build positive AI which is actually going to create a trustworthy internet, a safe internet."[2]
Their venture focuses on detecting AI-generated or manipulated media – videos, images, and audio – not just based on text or metadata, but by analyzing the actual pixels and waveforms using advanced computer vision techniques[1].
Why Platforms Need Help
Why is simply labeling AI-generated content (AIGC) becoming so crucial? It boils down to a few key pressures. Firstly, regulations like the EU AI Act are starting to mandate labeling, forcing platforms to identify AIGC viewed within their borders[1][2]. Secondly, there's the issue of originality and copyright. If a creator's content is easily manipulated or face-swapped, who gets paid? Platforms need to distinguish original work to maintain fairness and potentially command premium ad placements[2].
Finally, it's about platform integrity. As one founder noted, platforms like Vimeo are implementing policies because they need to assure creators and viewers about content authenticity[2]. While AI can empower creators, its misuse for scams or manipulation erodes user trust, making robust detection mechanisms essential[1][2]. The problem isn't AI itself, but its potential for misuse at scale.
Detecting the "Fingerprints"
So, how does this venture tackle detection? They explained it's not about simple filters like those on Instagram, which are basic image processing. True AI manipulation or generation, especially in images and audio ("sparse data"), leaves behind subtle "fingerprints" or artifacts invisible to the human eye but detectable by specialized AI models[1][2].
"When we transform image into something like a Fourier transformation... which is a frequency domain... there is a very clear pattern that comes out... a real distribution looks like this versus the AI distribution looks like this."[2]
Their approach involves training AI models on vast amounts of real-world and synthetically generated data, using techniques like analyzing frequency domains and focusing on specific regions of interest (like faces in deepfakes) to identify these AI signatures[1][2]. This computer-vision-first method contrasts with older, text-reliant moderation techniques, allowing them to analyze the media directly[2]. They believe that while generative models advance, detection models ("discriminators") can often catch up faster and more cheaply, especially for image and audio data[1].
Go-to-Market
Building sophisticated tech is one thing; selling it is another. Their strategy involves stages. They started by providing their tool to fact-checking organizations, gaining initial validation, media visibility (as their name gets credited in reports), and generating early, albeit small, revenue[1][2].
The main game, however, is enterprise clients – content aggregators (like Vimeo, Flickr) and upcoming social media platforms (like Blue Sky, Cantina) who face these challenges acutely but find it inefficient or too specialized to build top-tier detection in-house[1][2]. The founders leverage their extensive US-based networks in the trust and safety space for founder-led, high-touch sales, recognizing these sensitive conversations often happen in person[1][2].
"It is cheaper for [a platform like] Vimeo to buy than to build constantly reiterate and focus their energies... safety and security is something which has traditionally also been outsourced... A, the behavior, B, keeping this department, they can totally build a [solution] inside [their company] but this is not the ROI they're looking for, it is cheaper to hire us..."[1]
They acknowledge that securing these larger contracts requires proof. They're currently raising capital (aiming for around $1M, with a significant portion already committed) primarily to finalize the engineering needed for scalable enterprise pilots (estimated 8-12 weeks post-funding) and execute their US go-to-market plan[1][2]. They need case studies from successful pilots with initial enterprise clients to unlock deals with larger players, creating a staged pathway to growth[1].
While acknowledging the "grey area" challenges in content policy and the constant cat-and-mouse game of detection vs. generation, the founders are betting on their deep expertise and focused approach to carve out a critical niche in maintaining digital trust[1].
Key Takeaways:
- Leverage Niche Expertise: Combining deep technical skill (AI/CV) with domain knowledge (Trust & Safety policy) creates a strong foundation for tackling complex problems[1].
- Validate Incrementally: Starting with smaller clients (fact-checkers) can provide crucial validation, visibility, and early revenue before tackling larger enterprise deals[1][2].
- Address the "Build vs. Buy": Clearly articulate why platforms benefit from buying specialized solutions (expertise, cost, focus, network effects) rather than building everything in-house[1][2].
- Founder-Led Sales are Key (Initially): For complex, high-trust B2B sales, especially entering a new market (US), direct founder involvement and existing networks are invaluable[1][2].
- Case Studies Unlock Growth: Enterprise clients often need proof of performance ("in the wild" success) before committing. Plan for pilots designed to generate these case studies[1].