extracting ultra accurate phase with FFT

[Yansi]

Will not work correctly this way, as you have quantized the zero crossings to nearest sample position. Which is incorrect and will yield more phase measurement error.

The correct way is to multiply both signals by cos(wt) and sin(wt) of desired frequency of interest, integrate and then get the phase difference from the  arctan of it. (Which in fact is calculation of the single term of the DFT).

By no means I am a DSP expert, but your "+- sign algortihm" sounds as dodgy as it can get.

It works - I've done it - 30 years ago on 286 PC and today on this laptop (though I now do quadrature detection). Companies have made lots of money doing this in some lock-in amplifiers. Like I say, I've measured phase differences of doppler-shifted laser light using this very approach. I got me my PhD and has been commercialized by a number of companies. I've done a similar dodgy thing with autocorrelation (reducing data to single bit) again with great success. The zero crossing is a red herring. Choose your sampling period to be a non-integer factor of the signal period, average over enough periods and the zero crossing issue goes away.

Like I say below, tell me it can't be done and I'll do it - which I did.

That something was done or is being done does not mean  it is the best way to be done. And I do not want to be mean, but am really not sure, how integrating just the sign of the signal will yield better results than calculating the full DFT term. Especially, when the frequency of interest approaches the nyquist frequency. Taking just the sign will yield very inaccurate results in that case I guess.

I think it is not that hard to imagine what accuracy can be obtained, if you integrate just the sign of a signal, where you have like just a few samples per period.

Otherwise I really did not get how your algorithm works. Of course, cheaper RLC bridges /impedance analyzers/lockins may just only integrate the input signal with the polarity switched according to the reference. But is it the best approach? I'd say certainly not.

[coppice]

Will not work correctly this way, as you have quantized the zero crossings to nearest sample position. Which is incorrect and will yield more phase measurement error.

The correct way is to multiply both signals by cos(wt) and sin(wt) of desired frequency of interest, integrate and then get the phase difference from the  arctan of it. (Which in fact is calculation of the single term of the DFT).

By no means I am a DSP expert, but your "+- sign algortihm" sounds as dodgy as it can get.

Quadrature correlation is a faster approach, but if you can let the +- approach run for long enough it will converge on the right answer. You throw away a lot of information, and then need to recover it through the massive damping that integration provides. Its good for simplicity, and in RF applications the cycles go past so rapidly that the settling time is fine for many uses.

[JS]

Quadrature correlation is a faster approach, but if you can let the +- approach run for long enough it will converge on the right answer. You throw away a lot of information, and then need to recover it through the massive damping that integration provides. Its good for simplicity, and in RF applications the cycles go past so rapidly that the settling time is fine for many uses.

Not fast enough if your signal keeps changing, you need quadrature information to demodulate fm, lucky enough those usb synchronizes gives you rude quadrature information in base band already, if you syntonize them correctly you just need to use the phase in them. If the frequency isn't prefect a ramp in phase will appear and screw the quantization.

  Too bad I didn't have the time to play around longer with the toy as it might be a very nice and cheap HF tool. A few bux for GHz analog data to the computer. I don't do much HF but as soon as I come across needing to measure something I order one to be my first tool. Really nice spectrum analysis right out of the box with the included software, front end required to be a usable lab tool but some attenuators and adapters might be good enough to start.

JS

Enviado desde mi LG-M250 mediante Tapatalk

[JohnnyMalaria]

But is it the best approach? I'd say certainly not.

I didn't say it was. But for measuring a fixed time delay between two identical signals (which I assume is the OP's need) doesn't require DFT processing. What I described works. It doesn't mean it's better. It simply works if done right and is trivial to implement. My experience over many years is that people dismiss an idea as being impossible because either they have never tried or, if they have tried, didn't do it right.

[Kleinstein]

The simple FFT / DFT approach can become tricky if the signal frequency is not related to the sampling frequency. In the FFT the signal is not in one frequency bin. So it gets really tricky to determine a phase from the FFT. Using date with an integer number (or simple fraction) of samples per period is kind of the simple case only.

Doing a fit of a sine function to the signals might be the easier way. It is not as complicated as it might sound. Even if the frequency is only approximately known, the right functions can be close to a linear fit. It would thus only need a very few (e.g. 2) iterations for a good fit if done right (choosing the right base functions).

If computational speed is an issue, e.g. with real time data using the Goertzel algorithm (or similar) might be an option, that is probably hard to beat and may be good enough.

The frequency of the signal or ADC is another important parameter to determine on how much time can be spend on calculations.

[Yansi]

But is it the best approach? I'd say certainly not.

I didn't say it was. But for measuring a fixed time delay between two identical signals (which I assume is the OP's need) doesn't require DFT processing. What I described works. It doesn't mean it's better. It simply works if done right and is trivial to implement. My experience over many years is that people dismiss an idea as being impossible because either they have never tried or, if they have tried, didn't do it right.

Well tbh I have toyed with Goertzel and the quadrature correlation quite a bit, in an attempt to make a decent diy RLCG bridge, and later had some fun with DTMF decoding. Hence the bit of my dislike for the simple methods, as none of them seemed intereseting enough to try, as they were widely used in cheap instrumentation and I wanted to attack something better and different.  (I wasn't fully satisfied either, but due to many other things mostly).

[JohnnyMalaria]

Hence the bit of my dislike for the simple methods, as none of them seemed intereseting enough to try, as they were widely used in cheap instrumentation and I wanted to attack something better and different.  (I wasn't fully satisfied either, but due to many other things mostly).

That's strikes me as an odd reason. If a solution exists, is proven for your needs and inexpensive, why would you dismiss because it isn't interesting? If you're developing a commercial product, it seems rather luxurious and a waste of resources.


An attitude I see today is that people never look back at old solutions to the same problems. In particular, some people (I'm guessing half my age) would completely discount the option of performing some precision mathematical signal processing in the analog domain. My experience is that it can be more accurate and precise, faster (obviously), cheaper and quicker to implement. For example, I use two $30 high-precision multiplier/divider ICs to do quadrature detection plus a couple of standard op-amps to do some filtering before digitizing. I can get away with a much lower ADC sampling rate which, in turn, saves $1000s for what I would otherwise need.

[rhb]

What @JohnnyMalaria  suggested several posts back is a variation of sign bit recording.  This will actually recover full amplitude broadband  data from records which are only a single bit.  Sam Allen did this 40 years ago with Vibroseis data.  The price tag is some extra compute. But analog crosscorrelation works quite well.

If you go the FFT route, use a linear fit to the phase difference between the signals.

If you really want to be good at signal processing (or anything else), you need to know all the ways ti can and has been done.  My dissertation supervisor's main claim to fame was an analog filter using a revolving drum and adjustable playback heads to cancel water bottom reflections in seismic data in the late 50's and early 60's when seismic data was still recorded in analog format on 2" wide tape.

I filed and lapped a 0.75" x 1.0" x 3.125" block of aluminum plane and parallel to within 0.0015"  this morning.  I'm about to fix the remaining error by scraping.  All this is *very* old school.  But for what I wanted, it was quicker and easier than setting up to use my Clausing milling machine.

[Yansi]

Hence the bit of my dislike for the simple methods, as none of them seemed intereseting enough to try, as they were widely used in cheap instrumentation and I wanted to attack something better and different.  (I wasn't fully satisfied either, but due to many other things mostly).

That's strikes me as an odd reason. If a solution exists, is proven for your needs and inexpensive, why would you dismiss because it isn't interesting? If you're developing a commercial product, it seems rather luxurious and a waste of resources.


An attitude I see today is that people never look back at old solutions to the same problems. In particular, some people (I'm guessing half my age) would completely discount the option of performing some precision mathematical signal processing in the analog domain. My experience is that it can be more accurate and precise, faster (obviously), cheaper and quicker to implement. For example, I use two $30 high-precision multiplier/divider ICs to do quadrature detection plus a couple of standard op-amps to do some filtering before digitizing. I can get away with a much lower ADC sampling rate which, in turn, saves $1000s for what I would otherwise need.

I didn't say I was designing a commercial product, did I? It was a diy, mainly for the educational purpose, which it fulfilled at the satisfactory level.  Yes, there are solutions, like the synchronous detection using just polarity switching, which has obvious flaws I did not want to have in my design.

[ali_asadzadeh]

Thanks guys for your feadback

If your measurement is for a single, fixed frequency then I'd not bother with FFT, curve fitting etc. There's a much simpler way.

In the following, you will have to decide whether you can perform the measurements on all the channels at once or one at a time (+ reference channel) to allow for time delays if your ADC does not perform true simulataneous sampling. In the latter case, timing differences between channels may vary depending on how many channels you are sampling.

1. Use your first ADC channel as the reference for the rest
2. Capture the reference channel and other channel(s)
3. Convert the data to simple +1 or -1 (i.e., the sign of the data) (or, for speed, assign +1 = True, -1 = False and calculate NOT XOR)
4. Multiply a test channel's sign data by the reference sign data
5. Integrate (i.e., simple summation)

From the integral, it is trivial to get the phase.  Why?

If the reference and test channel are exactly in phase, the multiplication of the signs will always be +1. If the reference and the test channel are exactly out of phase, the multiplication of the signs will always be -1. If you sample at n samples per period, the integral will be +n if in phase and -n if 180° out of phase. There's a direct linear relationship between the integral, sampling rate and the phase. The more periods you integrate this over, the higher the precision. This allows you to use a reasonably low.

One limitation of this is that you cannot determine which quadrant your phase is in. If you are looking at small phase differences it wouldn't be a problem. Otherwise you need to do the same thing with a second reference that is 90° out of phase with the first one.

Because MATLAB is matrix-based, the above calculations are trivial and fast. I've used this approach with great success to measure phase differences of less than 0.1° for a very noisy signal. This is basically a software implementation of a phase sensitive detector.

This leads to a question: are you trying to measure phase differences or the time delay between channels at a variety of single frequrencies? If the latter, I agree with the suggestion of correlation. I'm pretty sure MATLAB can do it (cross-correlation).

I'm doing true simulataneous sampling on the signals,JohnnyMalaria would you please give me a pesudo code? I do not get your Idea fully,Also I want to get the Phase info from the signals not the time delay between channles.

The simple FFT / DFT approach can become tricky if the signal frequency is not related to the sampling frequency. In the FFT the signal is not in one frequency bin. So it gets really tricky to determine a phase from the FFT. Using date with an integer number (or simple fraction) of samples per period is kind of the simple case only.

Doing a fit of a sine function to the signals might be the easier way. It is not as complicated as it might sound. Even if the frequency is only approximately known, the right functions can be close to a linear fit. It would thus only need a very few (e.g. 2) iterations for a good fit if done right (choosing the right base functions).

If computational speed is an issue, e.g. with real time data using the Goertzel algorithm (or similar) might be an option, that is probably hard to beat and may be good enough.

The frequency of the signal or ADC is another important parameter to determine on how much time can be spend on calculations.

Kleinstein,How the sine fitting would work? I have no Idea? do you have any refrence or pesudo code?

[JS]

Sine fitting is a recursive algorithm, first you try to find the frequency, then the phase and if you are interested then you can find the amplitude. The more efficient approach h is just to find the freq using a sin and a cos and then extract the phase from there, as it would be a coef from the fft.

To find the freq you should start with a good approximation and then maximize the intercorrelation between n the signal and the sin and cos. The efficiency of the algorithm will depend on the efficiency of that search. Then with the relation between the intercorrelation of the sin and the cos you can extract the phase.

Matlab has a function to do fitting, maybe you can try to understand in the pc to later build up in the embedded.

JS

Enviado desde mi LG-M250 mediante Tapatalk

[ali_asadzadeh]

Thanks JS for the update, what's the name of the MATLAB function?

[JS]

Thanks JS for the update, what's the name of the MATLAB function?

You are welcome.
The function name is "fit".

JS

Enviado desde mi LG-M250 mediante Tapatalk

[JohnnyMalaria]

I'm doing true simulataneous sampling on the signals,JohnnyMalaria would you please give me a pesudo code? I do not get your Idea fully,Also I want to get the Phase info from the signals not the time delay between channles.

Please see that attached spreadsheet.

If your signal is steady over 100 cycles then this will work.

The calculations in the spreadsheet are very easy to do in MATLAB.

You can change the period and the phase to see how the calculation is affected. The calculated phase using the algorithm is shown. The key is to sample at a frequency that is not an exact multiple of your signal frequency. This is why the period is 99.9 in the example.

The example uses 3,000 samples. In practice, you could go much higher and increase the precision of the phase estimate.

Even with 3,000 samples, it is possible to get better than 1 degree precision for a signal sampled only 5 times per cycle

You should extend the number of rows significantly since this forum limits the size of the spreadsheet.

Note, this will only give phase results from 0 to 180 degrees (i.e., 270 = 90, 315 = 45 etc). If you need to discriminate then you have to employ the same algorithm but using a second reference signal that is 90 degrees out-of-phase with the first reference signal.

[ali_asadzadeh]

Thanks JohnnyMalaria, can the calculations be done on a single cycle of samples?

[JohnnyMalaria]

Thanks JohnnyMalaria, can the calculations be done on a single cycle of samples?

No.

[ali_asadzadeh]

Guys I have made some measurements with my System, I have a precision source injecting Voltages and currents to my ADC which is connected to  CT and PT's, the first 4 channels are CT’s and the last 4 channels are PT's. I have Injected some sin waves to the system from 40Hz to 400Hz and in the 50Hz zone I have incremented the frequency by 0.1Hz, also I have tested the system with various phase angles with the channels, the sampling frequency of my ADC is 6400 sample/sec and my ADC is a true simultaneous one! Here is the results, when I extract the phases respect to the V0 channel and in the 50Hz region, with different phases the extracted Phase is with in good ranges, but as the frequency changes the error would increase, I have only 128 samples in a full 50Hz cycle and I cannot increase my sample rate higher due to my system over head, also I should extract all the phase and frequency of the waves from this 128 samples, roughly 5ms Regarding that I have a ton of other things to do! fill free to play with these data to see if you could suggest a good enough solution to extract phase and frequency info with under 0.1 degree for phases and 0.01Hz for frequency. and all the extracted waves are ready to be injected into matlab

[JS]

But you don't have 0.01Hz data on 5ms sampling time, so you cant have that resolution from an FFT. Or you could adding enough padding or reconstruction from the signal you have but not a task for a limited overhead system.

Sorry if I missed, what resolution does your ADC have? Trying to extract some data that isn't there is possible but not useful, if your sampled data doesn't have it you can't get it.

  I might take a look at matlab but as said, I'd like to know if is theoretically possible to do so. For the 50Hz phase it probably is, depending on the bit depth of the samples.

JS

Enviado desde mi LG-M250 mediante Tapatalk

[coppice]

Guys I have made some measurements with my System, I have a precision source injecting Voltages and currents to my ADC which is connected to  CT and PT's, the first 4 channels are CT’s and the last 4 channels are PT's. I have Injected some sin waves to the system from 40Hz to 400Hz and in the 50Hz zone I have incremented the frequency by 0.1Hz, also I have tested the system with various phase angles with the channels, the sampling frequency of my ADC is 6400 sample/sec and my ADC is a true simultaneous one! Here is the results, when I extract the phases respect to the V0 channel and in the 50Hz region, with different phases the extracted Phase is with in good ranges, but as the frequency changes the error would increase, I have only 128 samples in a full 50Hz cycle and I cannot increase my sample rate higher due to my system over head, also I should extract all the phase and frequency of the waves from this 128 samples, roughly 5ms Regarding that I have a ton of other things to do! fill free to play with these data to see if you could suggest a good enough solution to extract phase and frequency info with under 0.1 degree for phases and 0.01Hz for frequency. and all the extracted waves are ready to be injected into matlab

It sounds like you are measuring 3 phase power. It would have been helpful to say this from the start. Is there a special reason why you asked about measuring from a single sample of the waveform (e.g. you are trying to do something tricky, like load disaggregation), or are longer term measurements OK? Per cycle measurements would be fine with pure sine wave signals, but you never see pure sine waves in power measurements, and that complicates things.

Most people trying to get an accurate phase measurement for mains power don't need a fast response. They synthesize a pure sine fundamental voltage waveform, and lock it to the voltage signal in both amplitude and phase. This will lock it to the fundamental component of the voltage waveform if you filter hard enough in the tracking loop. Its trivial to produce the sine wave with both in phase and quadrature components. Now you quadrature correlate the pure sine wave voltage signal with the current waveform. Only the fundamental of the current will correlate with the pure voltage signal, so you don't need a separate pure sine current waveform. From the correlation you can get both the phase angle and the fundamental component of the power (if that is of interest to you). You can't get fast updates, like per cycle results, though. The filters take time to settle.

Using the above approach its easy to get 0.01 degree accuracy in the phase measurement using the hardware and sampling rates you are using.

[jbb]

Yes, if you had said three-phase power we could have gotten here sooner. It’s the XY problem. We’ve all done it.

Over a 5ms window, the term ‘instantaneous frequency’ doesn’t mean much, because it’s not a full line cycle. Zero crossing techniques are not recommended because noise in the voltage of current waveforms will lead to the measurement going all over the place. Furthermore, the frequency of a typical AC power grid doesn’t change that fast because it’s set by large rotating generators that don’t change speed instantly.

In addition, your line currents are likely to be distorted by the presence of nonlinear loads.

So for accurate line frequency I thoroughly recommmend the use of a PLL. Because you have three phases you can use phasor extraction (Park’s Transformation) to get largely ripple-free measurements of V_D and V_Q, which allows for more aggressive tuning of the PLL loop.

If you want to measure power,  VARs etc at high rates, might I suggest looking into Akagi’s instantaneous power theory?

[coppice]

So for accurate line frequency I thoroughly recommmend the use of a PLL. Because you have three phases you can use phasor extraction (Park’s Transformation) to get largely ripple-free measurements of V_D and V_Q, which allows for more aggressive tuning of the PLL loop.

Using a PLL is good. Relying on the presence of all three phases is bad. People usually expect measurements to proceed correctly on the remaining phases if one or two phases are down.

[CopperCone]

good equipment will have a phase missing detect circuit

[ali_asadzadeh]

But you don't have 0.01Hz data on 5ms sampling time, so you cant have that resolution from an FFT. Or you could adding enough padding or reconstruction from the signal you have but not a task for a limited overhead system.

Sorry if I missed, what resolution does your ADC have? Trying to extract some data that isn't there is possible but not useful, if your sampled data doesn't have it you can't get it.

  I might take a look at matlab but as said, I'd like to know if is theoretically possible to do so. For the 50Hz phase it probably is, depending on the bit depth of the samples.

JS

My ADC resolution is 16bit

It sounds like you are measuring 3 phase power. It would have been helpful to say this from the start. Is there a special reason why you asked about measuring from a single sample of the waveform (e.g. you are trying to do something tricky, like load disaggregation), or are longer term measurements OK? Per cycle measurements would be fine with pure sine wave signals, but you never see pure sine waves in power measurements, and that complicates things.

Most people trying to get an accurate phase measurement for mains power don't need a fast response. They synthesize a pure sine fundamental voltage waveform, and lock it to the voltage signal in both amplitude and phase. This will lock it to the fundamental component of the voltage waveform if you filter hard enough in the tracking loop. Its trivial to produce the sine wave with both in phase and quadrature components. Now you quadrature correlate the pure sine wave voltage signal with the current waveform. Only the fundamental of the current will correlate with the pure voltage signal, so you don't need a separate pure sine current waveform. From the correlation you can get both the phase angle and the fundamental component of the power (if that is of interest to you). You can't get fast updates, like per cycle results, though. The filters take time to settle.

Using the above approach its easy to get 0.01 degree accuracy in the phase measurement using the hardware and sampling rates you are using.

Thanks coppice, I'm not measuring the 3 phase power, I'm doing Directional protection for a 3phase system , Since I'm doing protection, I need a cycle by cycle protection,So I need phase and frequency info with in each cycle, So based on those facts do you recommend an algorithm?

[JS]

Ok, even if you need cycle by cycle data, there is useful info in previous data to help ease things a bit. Frequency shouldn't change much, so you could have a better aproimation to start with, if you sync the sampling window with the input, phase shouldn't change much either.

So, with that in mind, seems easier to do the fit, as you know a good starting point for frequency and phase. If you know how much it could change from once cycle to the other (phase and freq) even better, as you have a limited range to work. All those things make the fitting of a curve more efficient, knowing where to look.

JS

Enviado desde mi LG-M250 mediante Tapatalk