Apollo: Band-sequence Modeling for High-Quality Music Restoration in Compressed Audio
Arxiv 2024
Kai Li
Yi Luo
Tsinghua University
Tencent AI Lab
[arXiv 📝]
[code ⚙️]
[poster 🖼️]
[checkpoint 🤗]

Apollo uses a frequency band-split module, band-sequence modeling, and frequency band reconstruction to restore the audio quality of MP3-compressed music.


Abstract

Apollo is a novel music restoration method designed to address distortions and artefacts caused by audio codecs, especially at low bitrates. Operating in the frequency domain, Apollo uses a frequency band-split module, band-sequence modeling, and frequency band reconstruction to restore the audio quality of MP3-compressed music. It divides the spectrogram into sub-bands, extracts gain-shape representations, and models both sub-band and temporal information for high-quality audio recovery. Trained with a Generative Adversarial Network (GAN), Apollo outperforms existing SR-GAN models on the MUSDB18-HQ and MoisesDB datasets, excelling in complex multi-instrument and vocal scenarios, while maintaining efficiency.

Overall pipeline of the model architecture of Apollo and its modules.

Bitrate Impact Analysis

This figure compares the performance of the Apollo model and the Stochastic-Restoration-GAN (SR-GAN) at different bitrates (ranging from 24 kHz to 128 kHz).

Music Genre Impact Analysis

This figure further illustrates the performance of both models across different music genres.

Audio Demo

MP3 Codec(24kbps)

Ground Truth Codec Wav Apollo

MP3 Codec(32kbps)

Ground Truth Codec Wav Apollo

MP3 Codec(48kbps)

Ground Truth Codec Wav Apollo

MP3 Codec(64kbps)

Ground Truth Codec Wav Apollo

MP3 Codec(96kbps)

Ground Truth Codec Wav Apollo

BibTeX:
@article{li2024spmamba,
      title={Apollo: Band-sequence Modeling for High-Quality Music Restoration in Compressed Audio},
      author={Li, Kai and Luo, Yi},
      year={2024},
      journal={xxxxx}
   }

Acknowledgements

Website template was borrowed from Colorful Image Colorization and Nerfies; the code can be found here and here. Thank you (.❛ ᴗ ❛.).
▶ cslikai.cn's clustrmaps 🌎.