Real-time pitch shifting using the phase vocoder

[cepstrum]

I have implemented a phase vocoder (as described in "Improved Phase Vocoder Time-Scale Modification of Audio", by Jean Laroche and Mark Dolson, IEEE) in MATLAB, and what I want it to do, is, in real-time, to pitch shift a signal. It is important, that each frame (max 1024 samples, 44100 khz, latency of 23 ms) can be shifted individually, since I want to make it as an AudioUnit, and therefore processing the incomming signal i real-time.

Unfortunately, when I resample each of the frames individually, there is some kind of discontinuallity between the frames resulting in a kind of clicking, even if I use 1024 samples of the next frame (and also 2048 of the previous frame), resample all 1024*4 samples, and only return the 1024 sample "middle frame". is there a way to smoothen the transition?

[neuronaut]

Hello,

thank you for your interest in our web site.

I'm afraid you can't just resample the iDFT'd transform frames on a frame by frame basis - this would speed them up and destroy phase continuity between them. There are two possible solutions that I can think of right now:

1. Resample the output signal after splicing the individual transform frames together, ie. resample the output stream

2. Change the bin frequencies in the Fourier domain. Our smbPitchShift code does this, you can find it in the "Pitch Shifting Using the Fourier Transform" article on http://www.dspdimension.com

Hope this helps!

Kind regards,
Stephan Bernsee

[cepstrum]

Thank you very much for your fast reply! The article and code is indeed useful, but I have some more questions about the method.

Actually, what I want to do, is a real-time autotune as an AudioUnit. That implies that, besides pitch shifting one frame at a time, I need to pitch shift the individual frame different. Would that be possible with your method described in "Pitch Shifting Using the Fourier Transform"?

[neuronaut]

Yes, smbPitchShift works with time-varying parameters as well. However, if you're looking for the "original" (musically monophonic) AutoTune™ effect this is done entirely in the time domain.

Changing pitch with the phase vocoder is also typically done in the time domain after iDFT'ing the signal and not on the individual transform frames.

Regards
--smb

[cepstrum]

Thank you, I will give your method a try! I am doing an assignment at the Technical University of Denmark, and therefore I would like to support your solution with some additional theory. I have been reading some articles about ABS/OLA (Speech Analysis/Synthesis and Modification Using an Analysis-by-Synthesis/Overlap-Add Sinusoidal Model, by E. Bryan George and Mark J. T. Smith), though the theory is a bit tough, so before proceeding, I would like to know if your method is actually build on this theory?

[neuronaut]

Thank you, I will give your method a try!

With "my method" you're referring to smbPitchShift()?

I have been reading some articles about ABS/OLA (Speech Analysis/Synthesis and Modification Using an Analysis-by-Synthesis/Overlap-Add Sinusoidal Model, by E. Bryan George and Mark J. T. Smith), though the theory is a bit tough, so before proceeding, I would like to know if your method is actually build on this theory?

No. Analysis by synthesis is an iterative procedure that resynthesizes the modified signal to determine the outcome of a particular parameter change. smbPitchShift() changes the partial frequencies by modifying both bin location and instantaneous frequency.

The procedure is explained here: http://www.dspdimension.com/admin/pitch ... ng-the-ft/

HTH
--smb