Manipulating vocal signal in mixed music sounds using side information based on the fundamental frequency

We propose a system that enables a listener of streaming audio to control the volume (magnitude of the signal) of independent part (specifically the vocal signal) in a mixed audio signal in real-time. In the proposed method, fundamental frequency (F0) of the vocal signal is used as side information. F0 information is estimated from the target signal before being mixed with backing track signals. After receiving the mixed music signal, vocal sound manipulation is performed using a comb filter using F0 information. In addition to the F0 information, we added side information considering the ratio between the level of the signal to be manipulated and the backing signal. As an experimental result, we obtained that the proposed method improved the quality of the manipulated signal compared with sending the information of vocal signal using the existing MP3 encoder.