Which detection techniques do platforms use to identify AI-generated child sexual abuse material in 2025?
This fact-check may be outdated. Consider refreshing it to get the most current information.
Executive summary
Platforms in 2025 use a mix of legacy and emerging techniques: hash-matching against databases for known CSAM, predictive/machine‑learning classifiers to flag novel or modified images and videos, and specialised AI tools that attempt to detect synthetic (AI‑generated) media — including classifiers trained on curated datasets like the UK CAID [1] [2] [3]. Reporting and law‑enforcement pilots show huge volumes: the National Center for Missing & Exploited Children logged 485,000 AI‑related CSAM reports in the first half of 2025 [4].
1. Hash matching: the first line of defense, but blind to novelties
Platforms still rely on perceptual and cryptographic hashing to find previously identified CSAM by matching digital “fingerprints” against curated databases such as PhotoDNA or SaferHash [3] [2]. Hashing is reliable for duplicates and known victims but cannot find first‑generation AI images or content that has been substantially modified or newly generated, limiting its effectiveness against rapidly proliferating AI CSAM [3] [5].
2. Predictive AI classifiers: finding the unknown
To address unknown content, organisations deploy machine‑learning classifiers trained to recognise underlying patterns in abusive imagery and video. Thorn and other groups describe “predictive AI” and CSAM classifiers as the only scalable way to surface novel CSAM at volume, enabling triage and referral to law enforcement [3] [6]. The Australian Institute of Criminology and academic projects are also testing deep‑learning models to differentiate synthetic from real media [7] [8].
3. AI provenance and synthetic‑media detectors: a new arms race
Responding specifically to AI‑generated CSAM, companies and investigators are building tools that attempt to detect whether content was produced by generative models. Vendors and government pilots — including the Roke Vigil AI CAID classifier used by Resolver’s Unknown CSAM Detection Service and contracts reported for Hive AI — claim the ability to classify previously unseen and AI‑generated material [1] [9] [10]. These tools attempt to identify pixel‑level artefacts or statistical signatures of synthesis, but reporting shows they are experimental and being piloted by law enforcement rather than deployed as foolproof filters [10] [11].
4. Multi‑signal systems and human review: combining tools for accuracy
Experts and NGOs stress that no single method suffices; platforms combine hashing, classifiers, synthetic‑media detectors, text analysis, and contextual signals (e.g., user reports, account behaviour), then route high‑risk hits to human moderators and investigators [3] [12]. Thorn and the Tech Coalition discuss integrated programs — Lantern and Safer Predict — that share signals across platforms and couple automated triage with human curation to limit false positives [13] [6].
5. Limitations, false positives and legal risk
Sources warn that classifiers can have higher error rates than hashing, and platforms fear the consequences of misidentifying users — a barrier to wider deployment of aggressive automated CSAM classifiers [2]. Academic and NGO research emphasises that many detection tools lack real‑world validation and that adversaries can alter or fine‑tune models to evade detection [8] [14]. Cleaning model training data is proposed as mitigation, but reports note that removing CSAM from massive datasets is difficult and not universally effective [15] [16].
6. Scale of the problem drives innovation and policy response
The spike in AI‑related reports — 485,000 to NCMEC in H1 2025 versus 67,000 for all of 2024 — has prompted new vendor services, government pilots, and legislative proposals to force or incentivise safer model training and vetting [4] [1] [16]. Industry coalitions and law enforcement training programmes are expanding to share tools and operational practices [13] [12].
7. Competing perspectives and implicit agendas
Vendors marketing new detection services (Resolver, Roke, Hive) emphasise their proprietary classifiers and access to curated datasets as competitive advantages [1] [9] [10]. NGOs and academic groups (Thorn, IWF, UNSW) push for shared, validated methods and caution that vendor claims may outpace independent validation; they also highlight the human‑harm dimension and urge policy and victim protections [3] [17] [8]. Government filings and pilot programmes signal urgency but also the experimental nature of many tools [10] [11].
Limitations: available sources do not mention fully reliable, universally accepted AI‑generation detectors that eliminate false positives; many tools are in pilots or newly marketed services with limited independent validation [10] [1] [2].