EEVblog #683 - Rigol DS1000Z & DS2000 Oscilloscope Jitter Problems

[Howardlong]

Did anyone else notice after upgrading the DG,500uV,Power Ana and 100MHz show up as Official.  they weren't listed at before.

The 1054Z does not have the 500uV front end hardware.

I don't know about a DS1054Z, but the 500uV does sort of work in the MSO1074Z-S I have here. DC offsets are a bit wild though, and it looks like the range is achieved digitally judging from the quantisation in straight sampling. Maximum averaging appears limited to 2 for some reason.


Quantisation without averaging, source signal 1mV RMS (2.83Vpp).


With averaging = 2


Note the large DC offset (Vmid in the measurements) and trigger offset.

[MarkL]

So no reports of the the fix NOT working so far?
Is it safe to say Rigol have fixed the issue yet?

"Fixed" is such a strong word.

I still want to do a little bit of temperature testing, but it appears Rigol has buried the 5us issue deep enough that no one is going to see it for routine use.

Are you trying to NOT say it is fixed?

"Fixed", to me, says the root cause has been found and corrected.  The root cause in this case is the hardware loop filter, and it has not been corrected.

Sales or marketing can call it "fixed".  I would call it an effective workaround.

I think Rigol is lucky to have found a combination of PLL parameters to bring the instability under control.

[Bud]

A shiny oficcial car dealer sold you a 6 cylinder car with ony 4 cylinders working. You at first were happy, the car could take you from point A to point B, that was all you wanted. One day some guy passing by said - something is wrong with your car, seems it only runs on 4 cylinders. You became sort of concerned and brought the car for repair. And the dealer "repaired" it for you, it now runs on 5 cylinders. You are so much excited because now you can get from point A to point B faster than before.

Can this 6 cylinder car be called "fixed" ?

[QuadFritz]

Sportscar A: 100$
Sportscar B: 500$

Both take me from A to B.

Sports car A only running on 5 cyl... 

YEAH- for me A is a clear winner because it does the job ;-)
If I am into racing or a professional driver: different story ;-) I also like the Agilent stuff...

[Bud]

Is it safe to say Rigol have fixed the issue yet?

In my example above, the 4 cylinder issue was fixed. Overall, the car was still defective. It depends how you want to wrap up this Blog.

[fusebit]

Hi Bud,

is your Scope (with the update) out of the Rigol product spec or not?

Are the 8 cylinders stated somewhere or just the 250 km/h top speed...

[dmmartindale]

Sorry to tell you Dave, the clock is still junk. The 68.3kHz spurs just at 35dBc is a garbage PLL. 

Yes, but is it adequate for the task in a $399 scope?
The jitter is gone on first inspection, and unless other people can find issues with their scopes (there could still be hardware tolerance issues), I think anyone would have a real hard time arguing that it's a real practical problem.
Perhaps this is the best compromise Rigol could up with for the fix barring a total recall of all scopes?
I suspect Rigol might do a hardware fix on future scopes without telling anyone. Their problem at present is the scopes already out there and having a fix available for it.

Try this experiment:
- Start with an oscillator that you expect to be very clean, like an OCXO with sine wave output.
- Measure its spectrum on a spectrum analyzer you trust, using the same horizontal scale as the PLL output
- Then capture the oscillator output with the Rigol scope, and do an FFT on it
- Does the scope's FFT output show the same "shoulders" on the sides of the central spike as the PLL's spectrum?
- If so, you are seeing the unstable PLL affect the FFT output in a way that is indistinguishable from an unstable oscillator on the input

Just looking at the sequence of numbers coming out of the A/D convertor in the scope, there's no way to tell the difference between unstable frequency in the input signal and unstable frequency in the sample clock.  If the clock itself has a spectrum like you captured, then I think every FFT spectrum the scope calculates is going to be contaminated by this.  Sometimes the X scale may be set in a way that you can't see it, but I'll bet the error is visible at some scale settings.

Dave

[orin]

Try this experiment:
- Start with an oscillator that you expect to be very clean, like an OCXO with sine wave output.
- Measure its spectrum on a spectrum analyzer you trust, using the same horizontal scale as the PLL output
- Then capture the oscillator output with the Rigol scope, and do an FFT on it
- Does the scope's FFT output show the same "shoulders" on the sides of the central spike as the PLL's spectrum?
- If so, you are seeing the unstable PLL affect the FFT output in a way that is indistinguishable from an unstable oscillator on the input

The shoulders are in the order of -70 dBc on a 100 MHz signal.  The 12 Mpt FFT plot is in post #683.  You'll not see them on the FFT the scope itself does.  Certainly there was no way I could see them on the scope's FFT when I did those plots with Alessandro's program since the noise level on the scope's FFT was more like -40 dBc.

[dmmartindale]

The shoulders are in the order of -70 dBc on a 100 MHz signal.  The 12 Mpt FFT plot is in post #683.  You'll not see them on the FFT the scope itself does.  Certainly there was no way I could see them on the scope's FFT when I did those plots with Alessandro's program since the noise level on the scope's FFT was more like -40 dBc.

OK.  So it seems the PLL is stable enough that you won't see a problem when the scope is used as a standalone instrument, with either time or frequency domain display.  But the wandering sample clock will show up in some cases when you're using the scope as a digitizer and analyzing the result on an external computer.

Would I avoid buying a scope if I knew it worked well enough as a scope, but had problems as a digitizer?  I'd probably still buy it, unless there was something else with similar features and similar price that did not have the problem.  But even if I did buy it, I'd want to know about the timebase problem.

- Dave

[mtdoc]

One thing that I'm pretty sure it worked before on the DS2000 series, was reinsertion of a memory stick.

Now I can insert it once but once you remove it it doesn't recognize re-insertions.

I don't quite understand what you mean by this?  What do you mean by "doesn't recognize re-insertions"?

[Bud]

So, there ya go.  PLL is still definitely not locked, but not wildly flapping either.  I don't think we have any examples of it being completely unstable with the new firmware.

MarkL, thank you for going into all these troubles to do this tests. If you still have access, can you try one thing for me: measure the SPI bus lines with no and with a 10K resistor connected between a line and ground. I'd like to see if in idle mode the logic High comes from the processor or from pullup resistors somewhere. If from the processor, the level would not drop much. If from pullup resistors, you should see a drop proportional to the resistor divider ratio. In case the processor SPI bus is in high impedance when idling, it should make it possible to highjack the bus with a external PIC and reprogram the PLL.

[pickle9000]

So as everyone is playing "What if?", let's try this one.

To be fair and upfront I like the scope. I purchased one when the first fix failed, technically I purchased a defective scope knowing that it was defective. I do think Rigol screwed up and could have done better. I also know equipment is shipped all the time with known faults, that is a way of life. In this case they got very lucky it could have killed them. I expect they will do better in the future or get nailed to the wall for it.

A pretend interview with Rigol about the DS1000Z. This is completely fake but close to my estimate of the situation. I have "cheated" the "No comment" I basically used it where there is no real answer to the question or implied fault.   

==================================
Interviewer: Good afternoon, let's get at it.

Rigol: "Good afternoon."

Question: Was the PLL copied from another scope?

Rigol: "Yes we used an existing design as the basis for this unit"

Question: Did you know about the jitter issue prior to your first production run?

Rigol: "No Comment"

Question: Could the issue be fixed with a hardware solution?

Rigol: "It could be fixed with a hardware modification however a firmware fix was possible and therefore more desirable."

Question: Did you know about clock / jitter issues from other newly released scopes?

Rigol: "There where reports."

Question: Had you already started working on the issue?

Rigol: "Yes"

Question: Did the discovery of the issue increase the urgency for a solution?

Rigol: "It did"

Question: How did you feel when you released the beta firmware that failed?

Rigol: "We should have not released the firmware on a public forum, it was a mistake"

Question: How long did the firmware fix take?

Rigol: "Less than a week"

Question: Why so long for the release (3 weeks)?

Rigol: "Testing to help ensure a repeat of the other (beta) release would not occur."

Question: The firmware appears to be working, will the hardware be modified?

Rigol: "Hardware is always modified during a production run. I can not say if there will be a modification but there could be and that will be up to the engineering team."

Question: Where you considering a recall?

Rigol: "A great number of unit's where in the field, a recall would have been very costly. The decision was made to come up with a solution that would be acceptable. Had it been necessary it would have been a very large undertaking spanning many models."

Question: Will new units be tested for jitter?

Rigol: "These functional tests are part of the regular production run"

Question: Does that mean all units will be tested?

Rigol: "No comment"

Question: If a customer installs the upgrade will this void the warranty?

Rigol: "No, the warranty will remain in effect however we urge non-professionals to follow instructions carefully so that they do not damage the scope. If there is a problem contact Rigol so we can help."

Question: What happens if a customer finds a fault after the firmware upgrade?

Rigol: "All units are currently under warranty, Rigol will honour the warranty and repair the unit according to the warranty agreement"

Question: How are sales?

Rigol: "Sales are currently beyond our production capabilities. There is a backlog of orders. We are pleased with the popuarity of the scope"

============



 

[tautech]

So as everyone is playing "What if?", let's try this one.

That's a fair assumption of events IMO. 

This tread is only 68 pages so far and a few more for a full analysis should not hurt wouldn't it?

The same should be expected of any brand who's gear is"not quite right".

[Bud]

Hi Bud,

is your Scope (with the update) out of the Rigol product spec or not?

I do not have a 1054Z but made a mistake to buy a 2072A, which I have not opened  because I consider getting rid of it. If I decide to open it will be another can of worms and another thread.

Are the 8 cylinders stated somewhere or just the 250 km/h top speed...

Often reading the datasheet does wonders. I will save you time and have attached a side by side picture of what MarkL measured, what I measured on a PLL I built, and what the Datasheet says. The screenshots are scale leveled to the same base line of -80dBc.

You can see my PLL (middle) follows the ADI picture (right) , and Rigol (left) is an utter garbage. I used the same ADF4360 chip with a proper loop filter. The PFD frequency was same as Rigols (2.5MHz)

[fusebit]

Hi Bud,

you're right with the PLL datasheet, but what's about the Rigol spec for the scope performance?
As far as I understand and know all 10xxZ units performing according the Rigol datasheet for the scope, at least after installing the new firmware. My 1074Z never had an issue, so I'm not even sure if I want to install the new firmware.

The way how Rigol did it might be not the best and the PLL might create other issues, but right now you get what you've paid for.
And it's not proven that there are any other issues affecting the specified performance of the scope.

[JDubU]

I do not have a 1054Z but made a mistake to buy a 2072A, which I have not opened  because I consider getting rid of it. If I decide to open it will be another can of worms and another thread.
Often reading the datasheet does wonders. I will save you time and have attached a side by side picture of what MarkL measured, what I measured on a PLL I built, and what the Datasheet says. The screenshots are scale leveled to the same base line of -80dBc.
You can see my PLL (middle) follows the ADI picture (right) , and Rigol (left) is an utter garbage. I used the same ADF4360 chip with a proper loop filter. The PFD frequency was same as Rigols (2.5MHz)

Since the DS1072A samples at 2Gsps vs the 1054Z at 1Gsps, wouldn't its PLL likely be designed for a 2GHz output (assuming that it doesn't use an additional PLL to later double the frequency somewhere else)?  Since the ADF4360-7 that the 1054Z uses is only rated for a maximum of 1.8GHz I would think they would need to use a different PLL chip for the 2072A. 

The board photos that I've seen of the 2072A have the top of the PLL chip ground off so it is not clear exactly what IC they are using but, given the surrounding components, it does look to have a similar pinout to the ADF4360-7, so a likely candidate might be the ADF4360-2.  Comparing data sheets, the ADF4360-2 would necessarily use a completely different loop filter design than the ADF4360-7.  Has there been any spectral analysis done on the PLL output of the DS2000 series scopes?

[Bud]

The way how Rigol did it might be not the best and the PLL might create other issues, but right now you get what you've paid for.
And it's not proven that there are any other issues affecting the specified performance of the scope.

I do not understand how cheap price can be an excuse for not getting a part perform to its base line.

I do not care about Rigols specification because they were measured on a defective product.

And it's not proven that there are any other issues affecting the specified performance of the scope.

Because no one bothered to check. To do this one would need a reference scope that you can trust ,with similar capabilities, i.e. math, FFT, etc. and may need other gear such as spectrum analyzer, frequency counter, power splitters. Many people even do not have an extra coax cable. 

[Bud]

Since the ADF4360-7 that the 1054Z uses is only rated for a maximum of 1.8GHz I would think they would need to use a different PLL chip for the 2072A. 

The board photos that I've seen of the 2072A have the top of the PLL chip ground off so it is not clear exactly what IC they are using

Guess what, they used the same 4360-7 PLL there too, stretched beyond the manufacturers specs.
Check my reply #618 earlier in this thread

https://www.eevblog.com/forum/blog/eevblog-683-rigol-ds1000z-ds2000-oscilloscope-jitter-problems/msg563030/#msg563030

[MarkL]

So, there ya go.  PLL is still definitely not locked, but not wildly flapping either.  I don't think we have any examples of it being completely unstable with the new firmware.

MarkL, thank you for going into all these troubles to do this tests. If you still have access, can you try one thing for me: measure the SPI bus lines with no and with a 10K resistor connected between a line and ground. I'd like to see if in idle mode the logic High comes from the processor or from pullup resistors somewhere. If from the processor, the level would not drop much. If from pullup resistors, you should see a drop proportional to the resistor divider ratio. In case the processor SPI bus is in high impedance when idling, it should make it possible to highjack the bus with a external PIC and reprogram the PLL.

No problem - the DS1054Z owner had (and is having) a great time digging into all this.  He's actually done all the probing.

I looked at the possibility of hijacking the SPI bus weeks ago, but I just checked it again in case anything changed with the new release.

Even with a 1k resistor, the voltage level doesn't move much, from 3.30V to 2.93V, and visa-versa.  All 3 SPI lines going to the AD4360 are being driven hard high and low.

I also noted that forcing a reset on only the main processor did not affect the drive to these lines.  I didn't expect that since they should have all gone to hi-z temporarily.  It's possible they're being driven by the nearby FPGA acting in part as a port expander.


Also, I forgot to grab a span large enough to show the reference spurs to go with your post #1020.  See below.

EDIT: Added another screen shot with a more optimal span and RBW to show the reference spurs and some of the modulation/noise in the main carrier.

[Bud]

The SPI bus was probed.  For comparison, here are the bytes from the beta for the AD4360-7 PLL registers:

THanks again, MarkL. Here is my analysis of it.

- no change in PFD frequency of 2.5MHz from Beta

- no change in lock detect setting of Digital Lock Detect (High), and as you said, the PLL is still not locked and as the result puts out garbage

- change in Band Select bits, now properly set. See my reply #575 earlier in this thread for details

- no change in Antiback slash pulse of 6nS from Beta, change from original firmware (3nS). Useless because only has effect when PLL is in locked state

-no change in charge pump current of 0.31mA. The part's specifications are based on 2.5mA current in the Datasheet, but Rigol did not read it anyway.
 
-change in Core Power Level from 20mA in Beta to 15mA back as it was in original firmware. Same as before, outside of the part manufacturer specification and could be the culprit of all this mess beside the invalid loop filter. See my reply #582 earlier in this thread for details. ADI treats this setting critical ("...In particular check the core power, it is critical that this be set to 5 mA") but that was of no meaning to Rigol.

Providing the Core Power Level was correctly set to 5mA, simulation shows that with the above settings and existing loop filter the phase margin value is 36 degrees which is a fair cry from recommended 45 degrees an most PLL book, the others recommend 50 degrees minimum. Therefore do not expect Rigol's PLL be stable to begin with. Cranking up the Core Power Level to 15mA makes all and any thing unpredictable, which is what we all are witnessing in this case.

[Bud]

Oops, was still typing when you posted your last one.
What can I say, they even could not get the PFD spurs right, most likely because of the out of spec programming. The ADI datasheet in my triple screenshot above shows PFD at -79dBc, and on my PLL I measured -75dBc (screenshot attached). Rigol's is -53dBc. One needs to really try hard to get it like that 

EDIT: OK on no easy way to hijack the SPI bus...

[tautech]

@ Bud
Do you imagine the HW & Firmware settings Rigol has used to impact on possible ADF4360-7 failure in the future?

[EEVblog]

You can see my PLL (middle) follows the ADI picture (right) , and Rigol (left) is an utter garbage. I used the same ADF4360 chip with a proper loop filter. The PFD frequency was same as Rigols (2.5MHz)

Yeah but does it cause a problem with the scope sampling in any way?

[orin]

You can see my PLL (middle) follows the ADI picture (right) , and Rigol (left) is an utter garbage. I used the same ADF4360 chip with a proper loop filter. The PFD frequency was same as Rigols (2.5MHz)

Yeah but does it cause a problem with the scope sampling in any way?

Not that I could tell.

To abuse the car analogy some more, it's like your car has a shimmy that you can't feel at 70 mph on the freeway, but at 100 mph on the track, it becomes obvious.  In normal use, driving to work, going to the grocery store, going on vacation, you'd never know it was there.

I'll leave the math to someone else.  The only timebase spec I could find was 25ppm.  Does the jitter implied by the -35 dBc sidebands violate it?

[orin]

The SPI bus was probed.  For comparison, here are the bytes from the beta for the AD4360-7 PLL registers:

THanks again, MarkL. Here is my analysis of it.

- no change in PFD frequency of 2.5MHz from Beta

- no change in lock detect setting of Digital Lock Detect (High), and as you said, the PLL is still not locked and as the result puts out garbage

- change in Band Select bits, now properly set. See my reply #575 earlier in this thread for details

- no change in Antiback slash pulse of 6nS from Beta, change from original firmware (3nS). Useless because only has effect when PLL is in locked state

-no change in charge pump current of 0.31mA. The part's specifications are based on 2.5mA current in the Datasheet, but Rigol did not read it anyway.
 
-change in Core Power Level from 20mA in Beta to 15mA back as it was in original firmware. Same as before, outside of the part manufacturer specification and could be the culprit of all this mess beside the invalid loop filter. See my reply #582 earlier in this thread for details. ADI treats this setting critical ("...In particular check the core power, it is critical that this be set to 5 mA") but that was of no meaning to Rigol.

Providing the Core Power Level was correctly set to 5mA, simulation shows that with the above settings and existing loop filter the phase margin value is 36 degrees which is a fair cry from recommended 45 degrees an most PLL book, the others recommend 50 degrees minimum. Therefore do not expect Rigol's PLL be stable to begin with. Cranking up the Core Power Level to 15mA makes all and any thing unpredictable, which is what we all are witnessing in this case.

You seem to be quoting the publicly available data sheet and publicly accessible forums.  It is interesting to note that the preliminary information for the ADF4360 series stated that higher VCO core power resulted in lower noise.

Datasheets are marketing documents.  There is nothing stopping Rigol picking up the phone and asking an Analog applications engineer "If we did this, what would happen?" and Analog replying "Well, yes, that would work, but we aren't going to guarantee it will work in the future - it's up to you to test that incoming devices work".  Nothing unusual about that.  HP would often have an HP part number that cross-referenced to a standard part with the proviso that the part was 'selected'.  Nothing is stopping Rigol selecting -7 devices that work at 2GHz (and lasering off the part #) for the 20xx scopes and using the rest for the 10xx scopes.