Feature Fusion for
Audio Deepfake detection.

01Overview

Audio deepfakes have become nearly impossible to distinguish from real speech by ear, and they are increasingly used for identity fraud, financial scams, and political disinformation. This study builds and evaluates a detection model that improves on a prosody-only baseline by fusing two complementary types of speech analysis into a single neural network.

The central contribution is not just that fusion improved detection, but a controlled ablation study that proves the improvement came from the feature fusion itself, not merely from a larger network — evidence prior fusion studies in this space did not provide.

02The problem

• Audio deepfakes are now convincing enough to fool human listeners and enable real-world fraud.
• Existing detectors typically rely on a single feature domain, leaving blind spots.
• The acoustic-prosodic baseline detects unnatural voice rhythm, but is blind to frequency-domain synthesis artifacts modern deepfakes leave behind.
• No prior study had fused acoustic-prosodic features with CQCC spectral features in a single LSTM detector.

03The solution

A two-stream feature fusion model — the AP-CQCC LSTM — combining:

• Acoustic-prosodic features (pitch, jitter, shimmer, harmonics-to-noise ratio) — the natural variation in a human voice that AI struggles to reproduce.
• CQCC spectral features (Constant-Q Cepstral Coefficients) — the frequency fingerprint where AI voice generators leave detectable artifacts.

The two feature sets are concatenated into a single 66-dimensional input and processed by a scaled LSTM, allowing the model to catch fakes that either feature domain alone would miss.

04Methodology

Three models were compared under identical conditions to isolate exactly what drove the improvement:

The ablation is the methodological backbone: because the proposed model changed two things at once (added features and a bigger network), it separates the two effects.

• Data — ASVspoof 2021 DeepFake, an international benchmark with 600,000+ evaluation samples, including attack types unseen during training.
• Training / validation — ASVspoof 2019 Logical Access set.
• Evaluation — held-out ASVspoof 2021 DeepFake set (out-of-distribution, novel attacks).
• Primary metrics — Equal Error Rate (EER) and AUROC, threshold-independent measures chosen for the heavily imbalanced (26:1) evaluation set.

05Key findings

• Feature fusion measurably improved detection. The AP-CQCC LSTM cut Equal Error Rate from 27.37% to 24.32% (−3.05 pp) and lifted AUROC from 0.7973 to 0.8359 on the held-out set.
• The ablation proved fusion was the primary driver. Of the 3.05 pp improvement, only 1.16 pp came from the larger network — 1.89 pp (the majority) came from the feature fusion itself, confirming prosodic and spectral features occupy complementary detection spaces.
• Better at protecting genuine speakers. Bonafide recall improved 7.2 pp (56.8% → 64.1%), so far fewer real speakers are wrongly flagged as fake — a deployment-relevant gain.
• Honest documentation of trade-offs. A slight accuracy trade-off under class imbalance, training instability in the high-capacity model, and the train/eval distribution-shift gap were all documented as actionable findings.

06Contribution to the field

• First study to fuse acoustic-prosodic and CQCC spectral features in an LSTM-based deepfake detector.
• First in this space to use a controlled ablation to decompose feature contribution from architectural contribution.
• Establishes multi-domain feature fusion as a viable, evidence-backed direction for prosody-based audio deepfake detection.