CodecFake+: A Large-Scale Neural Audio Codec-Based Deepfake Speech Dataset

With the rise of neural audio codecs, codec-based speech generation (CoSG) systems can produce highly realistic speech, posing new risks of deepfakes—referred to as CodecFake. While detection is increasingly critical, existing systems are mainly designed for traditional synthesis methods and often fail against CodecFake. To address this gap, we introduce CodecFake+, the largest and most diverse dataset for codec-based deepfake detection. It includes training data re-synthesized from 31 open-source neural codecs and evaluation data collected from 17 advanced CoSG models. The codec re-synthesized data is not fake speech, but serves as a proxy for CoSG data detection. We also propose a comprehensive taxonomy that categorizes codecs based on three key components: vector quantizer, auxiliary objectives, and decoder type. This taxonomy enables multi-level analysis to uncover key factors for detection. At the codec level, we validate the effectiveness of using codec re-synthesized speech (CoRS) for large-scale training. At the taxonomy level, we find that disentanglement-based objectives and frequency-domain decoders improve detection. Finally, we show that balancing training data across decoder types further enhances performance, even surpassing models trained solely on CoSG data. Overall, CodecFake+ provides a valuable foundation for advancing both general and fine-grained detection of codec-based deepfakes.