What technical signs do deepfake detection tools look for in video and audio?

1. Visual artifacts and frequency-domain irregularities

Automated video detectors hunt for pixel-level and spectral artifacts that are hard for generative models to eliminate, including inconsistent edges, odd textures, and color abnormalities visible both in raw pixels and in transformed frequency domains; frequency-domain analysis and CNNs trained on those artifacts are common approaches ^{[5] [6] [1]}.

2. Temporal coherence and spatio‑temporal texture evolution

Beyond single frames, detectors check how textures and facial features evolve over time: genuine footage shows smooth temporal texture transitions while manipulated clips often produce abrupt or unnatural changes across successive frames, so temporal-texture and spatio-temporal coherence methods flag discontinuities and gesture or expression repetition ^{[1] [7]}.

3. Physiological and behavioral biometrics (blink, pupil, pulse, micro‑expressions)

Some tools exploit human physiological signals that synthesis struggles to reproduce reliably — irregular blinking, unnatural pupil responses to light, or the subtle blood‑volume‑pulse patterns detectable in skin color fluctuations — using biological-trait models to separate real from synthetic content ^{[8] [6] [1]}.

4. Audio spectral fingerprints: Mel spectrograms, CQT and synthesis artifacts

Audio detectors transform waveforms into time–frequency representations (Mel spectrograms, STFT variants, or Constant-Q Transform) to expose synthesis artifacts: even high-quality TTS and voice conversion systems leave acoustic inconsistencies and unnatural spectral patterns that classifiers can learn to spot; CQT can give better resolution for certain nonlinear frequency features and has shown improved generalization in some studies ^[2].

5. Audio–visual synchronization and lip‑sync analysis

Multi-modal detectors test alignment between speech and visual articulatory cues — mismatched lip movement and speech timing is a strong indicator — and models that fuse audio and video streams look for desynchronization or improbable viseme–phoneme patterns that synthetic systems may produce ^{[1] [9] [6]}.

6. File and provenance signals: metadata, codecs, timestamps, device signatures and watermarks

Forensic systems examine container-level metadata, codec anomalies, timestamps and even driver/device signatures that can indicate injected synthetic streams; provenance approaches include digital watermarks embedded at creation so edits remove or alter those marks, providing another route to detect tampering ^{[3] [10] [4]}.

7. Machine learning architectures, explainability and evaluation metrics

Detection stacks commonly use CNNs, attention-based networks and hybrid classifiers that ingest frame-level features, spectrograms and landmarks; performance is measured with metrics like EER and task-specific t-DCF for audio anti-spoofing, but explainability is limited and detectors can be brittle outside curated datasets ^{[6] [11] [2] [1]}.

8. Practical limits: compression, low resolution, adversarial attacks and generalisation gaps

Real-world conditions — low resolution, heavy compression, noisy backgrounds and novel generative techniques — reduce detector accuracy, while adversarial manipulations and domain shift mean many algorithms that perform well in lab tests fail in the wild; watermarking and detection help but do not by themselves stop misuse or fully close the generalisation gap ^{[6] [4] [12]}.

9. How practitioners combine signals and why multi‑modal fusion matters

Robust systems merge visual artifacts, temporal coherence, audio spectral clues and provenance evidence into heatmaps and probability scores so investigators get forensic context rather than a simple binary flag; multi-modal fusion raises accuracy but still requires continuous updates as synthesis improves ^{[13] [3] [7]}.