EEVblog Electronics Community Forum
Electronics => Repair => Topic started by: RF_fanatic on November 17, 2016, 10:11:38 pm
-
As the topic mentions this deals with the CMU200 from Rohde & Schwarz which hangs during startup.
This is a quite often occuring bug so I will deep into some details of it and will restrict the fault to plugin level.
The CMU is packed with modules and each module also packed with plugins so how to search for a fault....
The problem manifests itself as a hanging serial number detection during BaseDiscoverOptionsBegin eventually it will continue and even will mention the correct serial number but it takes time..... a lot of time..... while this normal would be done instantly.
So what is holding it?
As I have sort of specialized myself in the CMU and want to get to know every detail of it I was able to make a CMU test bench so I can quickly identify broken of defective modules and plugins.
It was not directly my intention to sort this specific problem out but as others on this board mentioned the same problem I decided to give it some more priority and narrow the problem down to plugin level and hope others will pick up where I stopped. Eventually I will also focus on repair of it but as I have so many projects running I just have a lack of time.
So lets cut the crap and start digging :
When you had enough patience to eventually load the serial number you will often find problems like dsp.cpp error or similar red screens on the CMU.
This allready gives you information that focussing on the dsp module would be interesting. By swapping the DIG module with a know good unit it an be confirmed that the DIG module is causing the problem.
Opening the DIG module you will see multiple plugins, the number will depend on the installed options.
The unit with 1100.3220.02 is the cause of the hickup
I have tested multiple not starting DIG modules and all come down to the 1100.3220.02 unit
-
I got a similar behaviour (with the same message. Perhaps about 10 minutes of delay. I will check tonight on the exact file name. That was with SW 5.03) in the course of fixing mine, but that issue went away when I decided to make a disk image (with several reading in case of read errors, and use a different drive. It has been working for over a month now with no problems.
Could it be that the same error message screen be due to multiple causes, module hardware and/or corrupted software read from the HDD, bad data loaded to the eeproms or something else too?
Also posted here to be able to follow the thread.
Cheers and have fun! :)
-
I got a similar behaviour (with the same message. Perhaps about 10 minutes of delay. I will check tonight on the exact file name. That was with SW 5.03) in the course of fixing mine, but that issue went away when I decided to make a disk image (with several reading in case of read errors, and use a different drive. It has been working for over a month now with no problems.
Could it be that the same error message screen be due to multiple causes, module hardware and/or corrupted software read from the HDD, bad data loaded to the eeproms or something else too?
Also posted here to be able to follow the thread.
Cheers and have fun! :)
Yes, I think so. I have also a faulty CMU-200 with a similar problem. I also opened the DIGITAL board, which on my case was plenty of black dust, and carefully cleaned it along all the modules. But it still fails.
I have in the mail a new hard disk with working software for the CMU, so I will check if it is a matter of software of hardware fault. I guess the later... but I will know for sure soon.
I will let you know my findings. I guess that we can team to find a solution for this problem :)
Regards,
Jose
-
Our (RF Lab) just obtained a CMU200 from another lab. We're trying to set it up and use it , so unsure of it's status so far. Will report back any problems we see...
-
I got a similar behaviour (with the same message. Perhaps about 10 minutes of delay. I will check tonight on the exact file name. That was with SW 5.03) in the course of fixing mine, but that issue went away when I decided to make a disk image (with several reading in case of read errors, and use a different drive. It has been working for over a month now with no problems.
Could it be that the same error message screen be due to multiple causes, module hardware and/or corrupted software read from the HDD, bad data loaded to the eeproms or something else too?
Also posted here to be able to follow the thread.
Cheers and have fun! :)
In my case (serveral units) the problem was 100% produced by a dead plugin module in the DIG board. In my CMU test bench I have kept the harddrive identical during all tests. just swapped the plugin as mentioned before.
But I have seen many problems with CMU's, a failing HDD is not very common, (only seen it once), one of the things I noticed when I suspected software issues I could "repair" them by using the function "delete non volatile RAM" and using scandisk both in the version manager environment.
It would be interesting if you place your original HDD back and do these things. If it starts hanging again than it can be confirmed that a HDD can also cause this.
Back to the dead plugin unit: The components are very dense and also some ball grid arrays are applied, so I'm a little bit affraid that one of the difficult components are the cause.
-
I had a look yesterday and indeed the behavior was similar to what was described. A very long delay after the initial screen and then it would resume loading and give me the error message.
However, the file mentioned was "hw_init.cpp" at line 400. So you are right, it is a different issue. I assume this means hardware initialisation problem.
For good measure, since I now have a good disk image, I low level formatted the HDD and reinstalled the image on it. It has booted up OK several times today. Probably a bad sector on the HDD that got mapped out when I did the original check disk with the repair option. Also considering how old the HDD units are getting, having a good image backup is a really good idea I would think. They probably get less shocks than when used in a laptop but, on the other hand, my CMU has over 16500 hours logged in on that HDD. That is about 2000 8-hour work days. I don't think the HDD has a sleep function either.
:)
-
Well, some more info to the pile ;)
I have got today a hard disk with a working copy of the CMU software (it is not too recent, just V3.60). But it does the same than with the other hard disk. So, in my case, it seems there is some damage on the famous DSP board |O
I have been talking with a good friend who has a BGA rework machine (and indeed a very good German one). We are going to reball the BGA devices on the DSP board. My idea is to do it one by one, so we know for sure if any of the ICs makes a difference.
I will let you know what we find. Things are getting pretty interesting :-+
Regards,
Jose
-
The CMU200 I trying to restore now had the same problem that was fixed (started new thread about remaining troubles - https://www.eevblog.com/forum/repair/rohde-schwarz-cmu200-repair-(analyzer-levels)/msg1145649/#msg1145649 (https://www.eevblog.com/forum/repair/rohde-schwarz-cmu200-repair-(analyzer-levels)/msg1145649/#msg1145649) ).
Was able to figure out what happening after I switched to oldest version of FW that was on disk (3.06 ? ). That version not crashed during boot (not every time, though) and I was able to pass to self test. After I seen that FE EEPROM is not accessible, I looked thoroughly to cables, connecting FE and found defective IDC connector.
-
The CMU200 I trying to restore now had the same problem that was fixed (started new thread about remaining troubles - https://www.eevblog.com/forum/repair/rohde-schwarz-cmu200-repair-(analyzer-levels)/msg1145649/#msg1145649 (https://www.eevblog.com/forum/repair/rohde-schwarz-cmu200-repair-(analyzer-levels)/msg1145649/#msg1145649) ).
Was able to figure out what happening after I switched to oldest version of FW that was on disk (3.06 ? ). That version not crashed during boot (not every time, though) and I was able to pass to self test. After I seen that FE EEPROM is not accessible, I looked thoroughly to cables, connecting FE and found defective IDC connector.
Please, could you elaborate a bit more, perhaps with some picture of the offending connector location?
Thanks!
Jose
-
Please, could you elaborate a bit more, perhaps with some picture of the offending connector location?
Unable to take a shot - it is already assembled and put to a shelf over my workbench (the only place where it can be used for something useful). ;)
Concerning connector - it was a moderately wide flat cable that connected frontend to mainboard and then routed to top side of instrument. Problem connector was the middle one (2 mm 2 rows IDC) that, connects to motherboard.
Looked at connection diagram in service manual, probably cable was W215, but can't be 100% sure.
PS: Error was not in dsp.cpp though, but in stuc.cpp (cannot find line number now).
-
Hello
my cmu200 has now a similar problem. The error apears on every start up.
I removed all unnecessary options and modules without any success. Has anyone a guess what part may be defective?
-
It might seem strange but check the connector between the Frontend module and the mainboard (the grey multiwire cable)
If I remember correctly I have also seen such message when that cable was not making proper contact, but I can be wrong..
-
I checked the connectors but that didnt helped. But i updated all the software to the latest version. Now the cmu is bootin up. But the spectrum display is empty. There must be a fault in the DIG module. Either the adc or the DDC/DSP board.
-
I have a machine that throws up errors like this when i try to install B17 and B21 modules im looking for an install version of "version manager" so i can wipe my drive and reinstall the version manager and then the software.
I would also like to install an earlier software if anyone has copies of these i have all the latest 5.1 and 5.2 versions. I have no idea what versions were previous.
-
Hi,
I received two CMU200 with broken DSP board 1100.3220.02.
Just a quick question (I am in the office and can't test it now): what happens if I remove this board? Will I just lose the WDCMA functionality or will the bas spectrum/generator functions stop working, too?
Does the CRTU have this board, too? Are they exchangeable between CMU200 & CRTU?
Regards,
Vitor
-
I answer myself: without the digital board, booting is normal, but WDCMA options is missing and RF spectrum/generator will crash the machine. :(
-
1) Opened one of my CRTU and removed the two DSP modules from the DIGI board (CRTU has two)
2) Put each in my new two CMU200 (both had broken DSP module inside DIGI board)
3) Both CMU200 now start correctly, showing the serial number and spectrum works
But:
1) CRTU does not power up anymore, no matter if:
- remove DIGI board
- remove 2x broken DSP modules but insert DIGI board
- insert 2x broken DSP modules and insert DIGI board
It seems that the DIGI board is doing the power up and decides if the PC module starts or not.
2) Both CMU200 lost WDCMA capabilities. I think this is integrated in the DSP module.
So I traded one working CRTU to get two CMU200 running.
Anyone knows why the DSP modules break?
Regards,
Vitor
-
2) Both CMU200 lost WDCMA capabilities. I think this is integrated in the DSP module.
Did you run "hardware change" procedure? Maybe DSP firmware should be updated to match main SW?
-
I did.
Also installed latest FW and put in some keys...
No WDCMA options anymore.
I think that the DSP modules are all different with different options inside, though they are interchangeable.
-
I'm going to put the following remark not as a statement of fact but rather to provide additional data for possible future analyses. I may be 100% off base or this may just be a coincidence.
I recently got a CMU200 back to work (display issue). Tested it when half-assembled and it worked fine. Put it back together and it didn't anymore. It orginally threw an error "unable to load FPGA from file" but after some tinkereing it now gives me the "stuc.cpp" error.
Taking it apart again, I noticed I swapped two of the Compact Flash flat cables. May have caused the issue or not. Restoring the correct order or just taking out the CF didn't help anymore so the damage appears permanent.
-
Hi,
I don't think that swapping the CF cables would cause any harm - you would just map slot 1 to CF2 and vice-versa.
Unless, of course, you put the flat cable the wron side around, which again I don't think is an easy thing to do...
You might have:
- broken HDD
- corrupted file system
- broken module
Or, and this is a long shot: because you connected the CF slots in the wrong order, the C, D, E, ..., letters are now messed up and the BIOS/filesystem tries to load files from the wrong unit?
Good luck!
Regards,
Vitor
-
Well, figured it out in the end. There was another small flatcable not fully inserted somewhere in the chasis.Hadn't touched it but must have gotten a bit loose during assembly ;)
For my understanding; there's 3 flatcables for the CF card. How do you come to the conclusion swapping them could not cause physical harm?
-
Great news! I am glad it was a simple thing.
Please allow me to apologize - I forgot that the card-reader does feature three flat cables and they indeed all have the same connectors. Plus the length of the cables is ALMOST identical - W216 is shorter. I went to my spare parts box and looked at it.
I would say you could manage to swap W217 and W222.
Anyway, I stand by it that you probably would not break anything if you swapped the cables, as I think that there is no dangerous voltage in any of those pins and ground is the same. Some pins are not even connected, as the resistors are missing.
I suspect the third cable is not even used? The PCB is missing an IC that connects to W216.
Regards,
Vitor
-
I'd give it a try but I don't want to jinx it ;D
Anyways, using the same cables 3x on the same location is just asking for trouble IMOH.
For those stumbling upon this thread somewhere down the line: atached is a picture of the offending flatcable.
-
If I am not mistaken, that is the flatcable that connects the RF-Frontend module to the main board.
It is best accessed from the other side, when you remove the RF-Frontend module (I think).
I can imagine all sorts of errors if this cable is not propperly fitted and yes, it could get loose while messing around with RXTX-modules... (they need a lot of unscrewing of the waveguide connectors).
I wonder how long it took to develop the CMU200, considering its complexity and engineering...
Regards,
Vitor
-
I wonder how long it took to develop the CMU200, considering its complexity and engineering...
Think the better question is "how big was the team" ;-)
-
Yes, of course. I meant "how long" in "man years"...
Still, it is impressive when you are messing inside a CMU200: so many modules, each with several boards, custom IC's, FPGA, software, etc.
Then all the PCB layout, circuit design, materials (silver plated modules, etc.).
I can't begin to imagine how much work, research, development went into such a device.
No wonder they were sold new at around 100.000 Euro (?).
Regards,
Vitor
-
Pretty sure for 100k you got a base unit and not much else...
-
Yesterday one of my CRTU units booted first fine. Then I switched it off but remembered I needed to measure something else, so I turned it on again. Unfortunately the serial number recognition during BaseDiscoveryOptions took much longer than usual and the dreaded error message appeared.
Strange, though, it said "error# 1095 overlay.c system error" instead of the usual "dsp.c" error.
I restarted and then the CRTU booted OK, but on successive boots, even after one night off, it now always produces this "overlay.c" error.
I don't have the time and nerve right now to start swapping HDD and DIGITAL board, though I am pretty sure this will be DIGITAL board related.
Any idea what this could be?
It disturbs me that these fine units are starting to fail due to the DIGITAL board and the small PCB's attached inside, which are basically impossible to repair.
Regards,
Vitor
-
If you figure out which board crapped out, contact me. I can probably offer a spare.
-
Update:
Swapped DIGITAL board with a known good unit --> same error message! This is NOT related to the DIGITAL board...
Run scandisk and defrag from Service Menu --> No issues found
Swapped HDD with one of another CRTU --> device booted OK, but with fW 3.x instead of FW 5.x (which was active on that HDD -> because of compatibility with Windows 2000 partition)
Went inside Service Menu and installed "Another FW", selecting an alternative FW5.x version (less options installed) --> device boots OK on the original HDD
Conclusion: I think some file/data got corrupted and reinstalling the FW fixed it?
Strange...
Regards,
Vitor
-
Another update:
It is not the HDD or some file corruption.
Activated the original FW5.x with many options and the error comes up again!
So now I come to the conclusion that this error is related to some hardware option that got broken. If the FW does not have support for said option, the unit powers up ok.
Back to FW 5.x with just the GSM option, the device works ok.
Since I swapped the DIGITAL board, it must be some other board/module.
This is a 83 unit. The other broken unit I have is a 82 unit, so I cannot swap all boards, as they are different. I don't want mess with the other working units (CMU200 and CRTU 82).
I don't suspect the RXTX board at this moment.
However, I would be grateful for any idea regarding which board/module might be malfunctioning.
As I really only use this CRTU for spectrum and signal generator, I will leave it as is for now until new ideas come up.
Regards,
Vitor
-
I also encountered a similar problem, and the TL;DL version is: Be aware of the bad IDE ribbon cable!
While replacing the HDD with an SSD, I encountered the hang-on-boot issue on my CMU200. If the hard drive bracket was not screwed in, the boot always succeeded. But as soon as I screwed the bracket in, I had a hang-on-boot problem, sometimes with random system error messages! :wtf: :wtf: :wtf: It turned out that the SSD enclosure was little bit shorter (or longer? I'm not sure) than the equivalent HDD, so when the hard drive bracket was fully mounted, it put a lot of stress on the IDE ribbon cable and bent it to a sharper angle, causing an intermittent electrical connection, creating the random hang-on-boot issue. If anyone else has similar issues, I think it's a good idea to also check the IDE cable - one way to test it - unscrew the hard drive bracket, flip the HDD, and let the HDD sit on top of the screen with the PCB pointing up (be careful not to accidentally hit or short the HDD while it's spinning), and see if the boot succeeds. If so, the IDE cable is bad. If system failure is mysteriously fixed after removing and reinstalling it back, this can be a reason.
Another issue I encountered was a black-screen-of-death problem when entering Version Manager (even with the original disk), it really drove me crazy - I don't have any of the original installation medium! Scandisk didn't help. But my PC intuition told me it was likely a software compatibility problem, and it could be fixed by using the most ancient way to access the disk. First I entered BIOS and tried setting the HDD access mode from Auto to CHS, and set its speed to PIO 0, the slowest configuration possible. But it just led to an error on boot - CHS is too ancient, so I changed CHS to LBA. And the Version Manager came back to life. So, LBA + PIO 0 is also worth trying. :-+ The side-effect is an extremely slow disk access, but it can always be changed back after troubleshooting.
-
Hello
I now have a CMU200 with a similar problem:
The unit boots up, which it does really slow, if I want to go into "Version Manager" it just shows me a black screen until i turn it off again.
If the unit comes on eventually everything seems to work, altough the cursor sometimes lags, but when I turn on "spectrum analyzer" the CMU shows me a red warning which says:
System Error
....
Filename: tsk.cpp
Line: 2146
....
Add. Info
Str: ISR_C:HWIRQ::get_actual_stack_frame_number()
Can anyone give a hint on what this exactly means?
Help would be very much appreciated.
-
With such erratic behaviour I would firstly:
- check all ribbon cables. A lot of people have been bitten by badly inserted cables, and there are a lot of them
- the "computer board" (which is right behind the display) sometimes suffers from worn out capacitors. Check them. That would fit rather well with bad boot behaviour.
-
"sometimes suffers from worn out capacitors. Check them."
That would be appropriate indeed.
On mine the small fan was completely seized up. I had to test all the caps but only found one that was marginal. There was a tell-tale temperature sticker on one of the chip that was indicating an overtemperature to its max indication.
Also some of the ribbon connectors were at an angle. So I systematically reseated everything on the whole unit that I could get my hands on.
-
Thanks for the hints so far, and a happy new year all around!
I did so far:
Cleaned all modules and checked all connectors
Replaced BIOS battery etc.
I found that one module on the DIG board causes the CMU to hick up, as described above.
I started with removing all extra modules from the DIG board, now the CMU boots up quick as normal an even passes selftest, as long as it doesnt get to a "spectrum analyzer" function, here it will show me an error "dsp.cpp" "ddc_drv" or similar.
According to the service manual the DIG board hosts the following modules:
-ADC
-DDC
-TXDSP
-AUC (directly above TXDSP with the cooling mat on it, havent marked it in the image)
it is possible to have each module twice as an option, therefore the extra connectors.
The unit boots up normally only if the DDC module isnt fitted, otherwise it will take an eternity..
As said, without the DDC module everything seems fine until I start the spectrum analyzer or the unit itself tries to do so (selftest) which seems logical as the DDC module is there to preprocess the datastream it gets from the ADC.
From the service manual: (courtesy of Rohde & Schwarz)
"The sandwich DDC MODULE1 is directly plugged onto the DIGITAL
BOARD via three multipoint connectors and processes the digital data
stream of the receiver. In a special ASIC chip, the I/Q shaping, the
matching of the data rate and the respective filtering (bandwidth
shaping) of the digital data stream are produced. Then follows a DSP
(RX DSP) with further evaluation of the digital I/Q data for
measurement purposes. Besides, the digital I/Q data are passed on via
the MOTHERBOARD1 to the LINKHANDLER MODULE for evaluation."
I started to reverse engineer the module, and checked all the supply voltages which seem to be correct.
So I guess this isnt repairable at all, at least not without schematics and programmed spare parts.
What would the experts do?
Change the whole DIG assembly or is there a way to get hold of just the DDC board?
Regards
[attachimg=1]
-
Thanks RF_fanatic. I recently bought this tester. It looks like I have the same problem with the TXDSP module. CMU200 hung during BaseDiscoverOptionsBegin for about 5 minutes. I disassembled and cleaned the DIG board. And then by the method of elimination, I realized that the matter was in the indicated board. Without it, the device boots immediately. But now the problem is that the signal generator does not work (it did not work before).
Initially, I thought that the problem was in broken blocks on, or it was in the firmware. But it looks like a hardware problem.
I have not yet found where to buy a replacement board. And I want to try to find out what is wrong with it and maybe try to fix it. Does anyone have any ideas on this where to start or experience with a successful repair?
There are definitely no BGA chips on it. I looked at the board under a microscope - all the elements and connections are in order. At first glance, everything looks working.
-
My attempt to repair the DIG-TXDSP board
I bought a CH341a programmer and tried to read the EEPROM with a clip - it was read without problems. From which you can see the following information.
DIG-TXDSP
VER ALTERA=05.01
BD_RAM_K=0256
DS_CKMHZ=100
DS_TYPEID=56303
DS_MASKNR=0K36ADS_HASVSL
Judging by the fact that this information corresponds to the board, I concluded that it is read correctly, there are no reading errors and there is no point in changing the EEPROM. Also, the most likely reason, as it seemed to me, is the failure of the SRAM. My CMU200 worked for more than 15k hours. And it is likely that the number of read/write cycles of SRAM has exceeded a certain limit. After which the crashes begin. Having made this assumption I decided to change all the memory chips on the board. They are relatively inexpensive. But I didn’t want to guess which one had failed and how long the other chips would last. Therefore, I ordered 6 microcircuits from Ali and replaced them immediately.
I must to say that original chip was CY7C1019CV33-10VC and I bought almost the same CY7C1019CV33-12VC. The only difference is a time for rising edge. For the first one it is 10ns, for the second one it's 12ns. I didn't know would it will work or not, but I dicided to risk.
To replace it, I also purchased an Aoyue 1257 hot air nozzle to heat only the ICs contacts. Aluminum foil were also useful to protect the plastic elements from overheating. To solder new SRAM ICs to PCB I used 1.0 mm microwave solder tip.
But these actions had no effect. Any ideas?
-
Congratulation for your efforts, I am reading this with great interest, as I have several of these boards broken.
Unfortunately, I fear that the culprit is the Altera FPGA. Not sure if these are pre-programmed or if their FW is loaded during boot. This is just a guess, though.
-
I have a few questions. Does anyone know the pinout of the main connectors and two additional unpinned ones on this board? Is it possible to move the TXDSP board on the main board to the left or right? It seemed to me that this was possible. But I haven't tried it yet
How do you know if the FPGA is showing signs of life? (I've never encountered them). The complexity of all debugging manipulations lies in the fact that there are no extension cords to place the DIG board on the table...