Abstract
In the domain of music production and audio processing, the deployment of automatic singing pitch correction, also known as Auto-Tune, has significantly transformed the landscape of vocal performance. While auto-tuning technology has offered musicians the ability to tune their vocal pitches and achieve a desired level of precision, its use has also sparked debates regarding its impact on authenticity and artistic integrity. As a result, detecting and analyzing Auto-Tuned vocals in music recordings have become essential for music scholars, producers, and listeners. However, to the best of our knowledge, no prior effort has been made in this direction. This study introduces a data-driven approach leveraging triplet networks for the detection of Auto-Tuned songs, backed by the creation of a dataset composed of original and Auto-Tuned audio clips. The experimental results demonstrate the superiority of the proposed method in both accuracy and robustness compared to Rawnet2, an end-to-end model proposed for anti-spoofing and widely used for other audio forensic tasks.
Note: The related manuscript is currently under review. Preprint paper link.
Performance on some external tracks
Ed Sheeran - Photograph (Acoustic Version): This song doesn’t contain any type of auto-tuning and the vocals are raw and untuned.
The Detector Predictions:
| Segment Numbers | Auto-Tuned Segments (Number) | Auto-Tuned Segments (%) | Average Likelihood |
|---|---|---|---|
| 23 | 0 | 0% | 0.04% |
You & I feat. Kata Kozma: There is low amount of pitch correction in limited segments of the song.
The Detector Predictions:
| Segment Numbers | Auto-Tuned Segments (Number) | Auto-Tuned Segments (%) | Average Likelihood |
|---|---|---|---|
| 16 | 3 | 18.75% | 18.67% |
Timmy Trumpet, SwedishRedElephant, 22Bullets - The City: This song has undergone more intense auto-tuning; however it remains barely noticble for non-professional listeners.
The Detector Predictions:
| Segment Numbers | Auto-Tuned Segments (Number) | Auto-Tuned Segments (%) | Average Likelihood |
|---|---|---|---|
| 13 | 3 | 23.07% | 21.95% |
Lua freestyle2: In this track, there is a significant presence of intense auto-tuning throughout. The effect of auto-tuning is fully audible even to non-professional listeners.
The Detector Predictions:
| Segment Numbers | Auto-Tuned Segments (Number) | Auto-Tuned Segments (%) | Average Likelihood |
|---|---|---|---|
| 9 | 8 | 88.88% | 88.85% |