Electronics > Repair
Tektronix MSO4054 - no boot [SOLVED] + not working front panel [SOLVED]
(1/1)
dbator:
Hi all,

in December 2022 I purchased a faulty MSO4054 which did not boot. Below is the investigation I went through so far and I hope provided information will be helpful for others trying to repair their MSO4000 series oscilloscopes.
Symptom was: after powering on, LEDs and display backlight blinked once, fans go on full speed, no splash screen, that's all.

Around February this year I motivated myself to finally start doing anything in order to repair the scope.

First what I did was measuring supply voltages and they are as follows.

Description
Value
N/A
0
N/A
0
N/A
0
N/A
0
N/A
+2.1 Volts
N/A
+3.3 Volts
N/A
0
N/A
4 Volts
N/A
-2.5 Volts
N/A
0
+17V
0
+12V
+12 Volts
+10.8
+10.76
+5V
+5 Volts
+5VA
+5 Volts
+5VARLY
+5 Volts
+3.3V
+3.3 Volts
+2.5V
+2.5 Volts
+1.8V
+1.8 Volts
+1.5V
+1.5 Volts
-5V
-5.18 Volts
-12V
-11.88 Volts

DDR termination and DDR Vref voltages are correct.

I wonder if the missing +17V is an issue or not. Maybe someone of you already knows the answer.
Second, I confirmed faulty TPS2023 at U310. Its output voltage dropped from +5 Volts below 4 Volts rapidly.
During examination of the boards and trying to confirm Tx signals are at CMOS levels I confirmed Rx and Tx routed to a ISL3221 transceiver on I/O board. After taking a look at datasheet of the transceiver I also confirmed all required signals are present to make the data pass through and you will probably not guess where the data signals are connected to. To VGA connector.
I made a VGA to DB-9 cable and tried to read debug info - without success. I probed the Tx line and confirmed there is no data traffic there (via VGA connector and also on debug connector). Probably program execution does not go that far. So I started to probe at CPU, and memory section using just a 100 MHz scope - it's better than nothing though.
I started with CPU oscillator which was working fine, but I don't understand why signal amplitude of RTC crystal drops and goes high just so while holding the probe still. It does not look for me like probe causes that. However later I will confirmed that RTC clock is working properly.

After taking a look at datasheets of CPU, DRAM and EEPROM and searching for connections between these I confirmed that EEPROM and FLASH share the same address and data buses and RAM has a separate address and bus lines, but I really am confused because all data/address and command lines of DRAMs are connected between all 4 ICs. I completely don't understand why only 27 Ohm series resistors (used against ringing) separate the address and data lines. I mean, how CPU selects different DRAMs. However, probing two separate data lines simultaneously confirmed, the read/written data is different in time line so it's a relief. Only DRAM command lines have no resistors in series so the routings are direct.
When probing at address or CS lines on two DDRs I also confirmed that these signals are the same/aligned in time period.

Now the most interesting part.
I confirmed, there is data traffic on EEPROM address, data, OE and CS lines for a short period of time (about couple of seconds) and EEPROM command lines go high and stay high after couple of seconds and most of its address and data lines go high and the rest low after that period of time.
On DRAM instead the readings were random. Sometimes there was less traffic sometimes more data traffic.
I used a thermal camera on DRAMs, because I have enough faulty old Micron DDR2 RAMs replaced in my time which showed difference on thermal image. Unfortunately either the thermal camera is of poor quality, or it is not a rule in this case of DDR.
Watching DRAMs on front of the board (opposite side to CPU), U0731 starts to warm up as first and there is a significant (visual) difference in comparison to U0742 which is less warm. Shown temperature for U0731 is 35 degrees C (95 degrees F) and for U0742 about 30 degrees C (86 degrees F).
RAM ICs on back of the board (rear of the scope, CPU side) are warming up simultaneously and have visually the same temperature, but I would say less than U0731 on front side of the board although thermal camera measures about 38 degrees C (100.4 degrees F). That's why I do not trust the exact temperature measurement. Besides, there are two things I have in mind. First is the data traffic which may be different between all 4 ICs and second causing DRAM may be warmer of cooler. Second, not necessary the warmest IC must be the faulty one, it could be coldest just because it is for example not working at all.

Anyway, second day of probing and after another couple of hours of investigation, I noticed the data traffic on DRAM bus is heavy, transfer does not stop and heatsinks of ADCs (or MUXes) got hot. So I powered down the scope quickly. Then connected the I/O board to front panel and you should see me happy, seeing Tektronix splash screen and then completely booted scope, but with an acquisition board configuration error at startup.
The only thing left (at least I thought so) was to confirm which component fails depending on temperature. I was worried if there will be something wrong with FLASH soldering because I do not have neither right equipment and nor skills at BGA reflow.
First I warmed up the CPU side with just 50 degrees Celsius (122 degrees Fahrenheit) and 10% airflow - the minimum parameters I could set. I tried DRAMs and FLASH ICs and always when any of them was warmed up the scope booted up. I repeated the process after cooling down the board using a compressed air. I did not use freeze spray because I was worried it could be too much temperature shock and also it would cost more time to bring back the board to room temperature - compressed air was a perfect solution in this case.
Opposite side of the CPU the same situation - any of the DRAMs warmed up, the scope booted. I got confused. Then I came back to the CPU side, but warmed up every of the DRAMs and FLASH ICs with hot air nozzle kept from a greater distance and for maximum 30 seconds. And voila! U0731, opposite side to CPU, the bottom with higher temperature indication on thermal imaging.
Just as about 15 years old DDR2 DRAMs from Micron which I replaced dozens in the past - they showed the difference of thermal immediately after supply voltage was on and usually started to work, when they were warmed up and stopped to work after cooling them down.

For the record.
There are two LEDs on the board. One is green and one is red. If boot sequence reaches the self test and the scope fails the self test procedure or does not detect fan tacho, red LED goes on. I other cases green LED goes on.
Not every failed self test means a relevant information is being shown on display which I will explain later.
There are many vias on PCBA near the four FLASH ICs - these I checked for continuity between EEPROM and FLASH first and then probed for data and address signals.
The DRAMs mounted on the board are DDR1, MT46V16M16P-5B from Micron Technology Ltd. The problem is, they are obsolete. Luckily I found some offers on Aliexpress so I decided to purchase 5 pcs. (for about 1 EUR a pc. plus 2.5 EUR tracked shipping) so the remaining 4 pcs, will serve as spare parts for future.
After exactly two weeks of waiting time for delivery I received them and got confused. Their date codes are 2318. Too new for an old DDR1 I would say. They look perfectly as original ICs (part number, the package, the font, and the color is almost perfect as on original). In my case it is MT46V16M16P-5B:K.
I contacted Micron and also two distributors for Europe. There was a lot of writing so I'll spare serving you all answers, but according to the information I got, that part went EOL in 2015.
Oh, there are 3 substitutes for the original with some timing differences which have to be taken into consideration before purchasing.
They are:
1) IS43R16160F-5TL (ISSI)
2) W9425G6KH-5TR (Winbond)
3) AS4C16M16D1A-5TCN or AS4C16M16D1A-TIN (Alliance Memory)
Below differences of parameters which I find are important. I put the information here since I already spend time for comparing them - maybe it will be useful for someone.

Micron
ISSI
Winbond
Alliance Memory
Parameter
MT46V16M16P:5B
IS43R16160F-5TL
W9425G6KH-5TR
AS4C16M16D1A-5TCN/AS4C16M16D1A-TIN
VDD
2.3-2.7 Volts
2.3-2.7 Volts
-
-
VDDQ
2.3-2.7 Volts
2.3-2.7 Volts
-
-
Auto precharge
A10/AP
2.3-2.7 VoltsA10/AP
A10/AP
A10/AP
CL=3
5-7.5 ns 200 MHz
5-10 ns 200 MHZ
5-12 ns
5-10 ns
CL=2.5
6-13 ns 167 MHz
6-10 ns 167 MHz
6-12 ns
6-12 ns
CL=2
7.5-13 ns 133 MHz
7.5-10 ns 133 MHz
7.5-12 ns
7.5-12 ns
Row address
8k(A0-A12)
8k(A0-A12)
A0-A12
A0-A12
Column address
512(A0-A8)
512(A0-A8)
A0-A8
A0-A8
Refresh count
8k
8k/64 ms
-
-
Burst length
2, 4, 8
2, 4, 8
2, 4, 8
2, 4, 8
tAC
-0.7-0.7 ns
-0.7-0.7 ns
-0.7-0.7 ns
-0.7-0.7 ns
tDQSCK
-0.6-0.6 ns
-0.6-0.6 ns
-0.6-0.6 ns
-0.7-0.7 ns
tCL
0.45-0.55 tCK
0.45-0.55 tCK
0.45-0.55 tCK
0.45-0.55 tCK
tCH
0.45-0.55 tCK
0.45-0.55 tCK
0.45-0.55 tCK
0.45-0.55 tCK
tDQSQ
_-0.4 ns
_-0.4 ns
_-0.4 ns
_-0.4 ns
tDIPW
1.75-_ ns
1.75-_ ns
1.75-_ ns
1.75-_ ns

On the other hand, while searching for that DRAMs on Aliexpress I found some offers with Micron logo on pictures being blurred. Maybe those sellers do not want to get sued by Micron because they are selling counterfeit parts?
Since there was not much time left for free refund, we (seller and Aliexpress customer support) agreed I will solder one of them and check if it will work (even if it will, that does not mean they will be that reliable like originals and that other 4 pcs. are also good). After replacing the DRAM oscilloscope booted.

As for the TPS2023D at U310, I replaced it, but did not want to check whether there is +5 Volts at its output or not. USB port is working though (which will be confirmed later) so I assume that problem is solved.

Now, I will tell you about place where I am currently stuck.
Just when I thought the oscilloscope is repaired, I noticed none of the front panel buttons and encoders do work. My first thought was bad connection on front panel connector, a stuck button on front panel or a configuration issue (maybe the above mentioned Acquisition board configuration error warning could have something to do with that) which could be solved by re-seating the board, checking/removing keypad membranes and updating of the firmware, respectively. None of that helped.
So I prepared an extension 20-wire cable - I soldered its wires to solder points on acquisition board (CPU side of course) and investigated what is going on there. Below are measurement results I confirmed - numbering starts from the top of the board (up side of the scope) left side pin to right side pin and then one row lower left side pin, right side pin and so on. Due lost USB cable I can't currently provide picture which could be helpful. But I hope the description is clear. I hope I did not make any mistakes when checking pinout order order. Note, voltages were not noted precisely.

Upper left
Upper right
+12VA
+12VB
GND
GND
GND
+12VC
+3.3VA
+5V
GND
GND
I2C SCL
+12VD
+12VE
I2C SDA
+3.3VB*
Reset**
+3.3VC
GND
+3.3VD***
+3.3VE****

* - That rail is not a DC signal, there is a negative peak (voltage drop) to about +1.8 Volts for about 8 us. Maybe some kind of sync pulse. Not confirmed how often it occurs.
** - It looks like reset line. It seems reset is asserted twice. First time for about 100 ms and second time (maybe during front panel programming attempt) for the second time which last then about 200 us.
*** - That rail is solid +3.3V. However, it behaves in a moment like reset. I did not measure its period. Maybe probing error.
**** - That rail is +3.3 Volts. However it is not stable (it has a bit of voltage ripple).

Now, during continued investigation which I really have enough of, I confirmed communications to front panel board, but not much. There seems to be one byte of data from front panel followed by ACK and that's all.
POST self test result is pass, but I decided to purchase a RS-232 to TTL transceiver in order to read the boot log. I probed the debug connector for Tx signal because I still could not confirm Tx output on VGA connector. Although it should be since there are Rx and Tx connections to RS-232 transceiver located on I/O board. I don't understand that.

Below is the boot log.

--- Code: ---

U-Boot 1.1.4 (Jan  8 2007 - 11:12:14) Tektronix, Inc. V1.06

CPU:   AMCC PowerPC 440EP Rev. C at 333.333 MHz (PLB=133, OPB=66, EBC=66 MHz)
       I2C boot EEPROM enabled
       Internal PCI arbiter enabled, PCI async ext clock used
       32 kB I-Cache 32 kB D-Cache
Board: Tektronix Route66 IBM 440EP Main Board
        VCO: 666 MHz
        CPU: 333 MHz
        PLB: 133 MHz
        OPB: 66 MHz
        EPB: 66 MHz
I2C:   ready
DRAM:  128 MB
FLASH: 64.5 MB
PCI:   Bus Dev VenId DevId Class Int
        00  13  10b5  9056  0680  18
        00  15  1002  4c59  0300  17
DISP:  Type 1
In:    serial
Out:   serial
Err:   serial
Enter password - autobooting in 3 seconds
## Booting image at f0000000 ...
   Image Name:   Linux-2.4.20_mvl31-440ep_eval
   Image Type:   PowerPC Linux Multi-File Image (gzip compressed)
   Data Size:    1441010 Bytes =  1.4 MB
   Load Address: 00000000
   Entry Point:  00000000
   Contents:
   Image 0:  1033953 Bytes = 1009.7 kB
   Image 1:   407042 Bytes = 397.5 kB
   Verifying Checksum ... OK
   Uncompressing Multi-File Image ... OK
cmdline is console=ttyS0,9600 quiet bigphysarea=519 panic=2 root=/dev/mtdblock7 rw mem=131072k
   Loading Ramdisk to 07f2b000, end 07f8e602 ... OK
Checking for firmware update...
No USB mass storage devices found to update from.
Linux 2.4.20_mvl31-440ep_eval V 1.15 Tektronix Route66 Tue Jun 22 15:19:50 PDT 2010
stat of /var/log/dmesg failed: No such file or directory
rm: cannot remove '/var/run/*': No such file or directory
Warning: loading NiDKEng-1.6 will taint the kernel: non-GPL license - Proprietary
  See http://www.tux.org/lkml/#export-tainted for information about tainted modules
Warning: loading NiDUsb-1.6 will taint the kernel: non-GPL license - Proprietary
  See http://www.tux.org/lkml/#export-tainted for information about tainted modules
Warning: loading tek will taint the kernel: non-GPL license - Proprietary
  See http://www.tux.org/lkml/#export-tainted for information about tainted modules
Scope application starting (normal mode)
---------------------- startScopeApp() running Init code ----------------------
versionBuildFWVersionString(): TimestampString: 25-Apr-12  11:13
                               VersionFIRMWAREVERSIONversion: v2.68
                               Major ver num: 2 Minor ver num: 68
     Initializing Mia[0]
     Initializing Tek0005[0]
     Initializing Ibm440[0]
     Initializing HFD204ADC[1]
     Initializing HFD204ADC[0]

Model id: 0x02

     Initializing Ltc1658Dac[0]

Board id: 0x01

     Initializing M859[3]
     Initializing M859[2]
     Initializing M859[1]
     Initializing M859[0]
     Initializing Ltc1660Dac[1]
     Initializing Ltc1660Dac[0]
      HFD144[0] ID_REG = 0x00001440
      HFD144[1] ID_REG = 0x00001440
      HFD144[2] ID_REG = 0x00001440
      HFD144[3] ID_REG = 0x00001440
     Initializing Hfd144[0]
     Initializing Hfd144[1]
     Initializing Hfd144[2]
     Initializing Hfd144[3]
        Open Max5362 successful.
     Initializing Max5362[0]
     Initializing Ltc1661Dac[1]
     Initializing Ltc1661Dac[0]
 Init ADT7468 and locking.
 Factory Checksum: Stored: 58538, Calculated: 58538  - OK
 Spc CheckSum: stored: 15479 calculated: 15479  - OK
Demux initialization

    Dram calibration
    Dram calibration complete

Dram Calibration results:
-------------------------
DramCal has PASSED on all Demuxs

Demux initialization complete
Starting POST diags

Finished POST diags
 1st purge loop: Remote I/O error
 1st purge loop: Remote I/O error
 1st purge loop: Remote I/O error
 1st purge loop: Remote I/O error
 1st purge loop: Remote I/O error
 1st purge loop: Remote I/O error
 1st purge loop: Remote I/O error
 1st purge loop: Remote I/O error
 1st purge loop: Remote I/O error
 1st purge loop: Remote I/O error
 1st purge loop: Remote I/O error
 1st purge loop: Remote I/O error
 1st purge loop: Remote I/O error
 1st purge loop: Remote I/O error
 1st purge loop: Remote I/O error
 1st purge loop: Remote I/O error
ERROR in fpSharedPrivate.cpp at fpReadFpHwAndSwIdAndVersion: IIC requested 3 bytes - got -1 bytes.
 firmware version read: Remote I/O error
 2nd purge loop: Remote I/O error
 2nd purge loop: Remote I/O error
 2nd purge loop: Remote I/O error
 2nd purge loop: Remote I/O error
 2nd purge loop: Remote I/O error
 2nd purge loop: Remote I/O error
 2nd purge loop: Remote I/O error
 2nd purge loop: Remote I/O error
 2nd purge loop: Remote I/O error
 2nd purge loop: Remote I/O error
 2nd purge loop: Remote I/O error
 2nd purge loop: Remote I/O error
 2nd purge loop: Remote I/O error
 2nd purge loop: Remote I/O error
 2nd purge loop: Remote I/O error
 2nd purge loop: Remote I/O error
ERROR in fpSharedPrivate.cpp at fpReadFpHwAndSwIdAndVersion: IIC requested 3 bytes - got -1 bytes.
 platform version read: Remote I/O error
 3rd purge loop: Remote I/O error
 3rd purge loop: Remote I/O error
 3rd purge loop: Remote I/O error
 3rd purge loop: Remote I/O error
 3rd purge loop: Remote I/O error
 3rd purge loop: Remote I/O error
 3rd purge loop: Remote I/O error
 3rd purge loop: Remote I/O error
 3rd purge loop: Remote I/O error
 3rd purge loop: Remote I/O error
 3rd purge loop: Remote I/O error
 3rd purge loop: Remote I/O error
 3rd purge loop: Remote I/O error
 3rd purge loop: Remote I/O error
 3rd purge loop: Remote I/O error
 3rd purge loop: Remote I/O error
ERROR in fpSharedPrivate.cpp at fpReadFpHwAndSwIdAndVersion: IIC requested 2 bytes - got -1 bytes.
 common code version read: Remote I/O error
Installing 27 in front panel.

fpReprogramFrontPanelCpuPrivate(/usr/local/perm/route66_fp.s19)
  fpSRecCheckSum returning 0
  fpSRecRaw: FP_UPDATE_CODE_START_CMD write failed - -1 bytes of 3 written
 Reprogram start: Remote I/O error
Fp software update function reports failure.
 1st purge loop: Remote I/O error
 1st purge loop: Remote I/O error
 1st purge loop: Remote I/O error
 1st purge loop: Remote I/O error
 1st purge loop: Remote I/O error
 1st purge loop: Remote I/O error
 1st purge loop: Remote I/O error
 1st purge loop: Remote I/O error
 1st purge loop: Remote I/O error
 1st purge loop: Remote I/O error
 1st purge loop: Remote I/O error
 1st purge loop: Remote I/O error
 1st purge loop: Remote I/O error
 1st purge loop: Remote I/O error
 1st purge loop: Remote I/O error
 1st purge loop: Remote I/O error
ERROR in fpSharedPrivate.cpp at fpReadFpHwAndSwIdAndVersion: IIC requested 3 bytes - got -1 bytes.
 firmware version read: Remote I/O error
 2nd purge loop: Remote I/O error
 2nd purge loop: Remote I/O error
 2nd purge loop: Remote I/O error
 2nd purge loop: Remote I/O error
 2nd purge loop: Remote I/O error
 2nd purge loop: Remote I/O error
 2nd purge loop: Remote I/O error
 2nd purge loop: Remote I/O error
 2nd purge loop: Remote I/O error
 2nd purge loop: Remote I/O error
 2nd purge loop: Remote I/O error
 2nd purge loop: Remote I/O error
 2nd purge loop: Remote I/O error
 2nd purge loop: Remote I/O error
 2nd purge loop: Remote I/O error
 2nd purge loop: Remote I/O error
ERROR in fpSharedPrivate.cpp at fpReadFpHwAndSwIdAndVersion: IIC requested 3 bytes - got -1 bytes.
 platform version read: Remote I/O error
 3rd purge loop: Remote I/O error
 3rd purge loop: Remote I/O error
 3rd purge loop: Remote I/O error
 3rd purge loop: Remote I/O error
 3rd purge loop: Remote I/O error
 3rd purge loop: Remote I/O error
 3rd purge loop: Remote I/O error
 3rd purge loop: Remote I/O error
 3rd purge loop: Remote I/O error
 3rd purge loop: Remote I/O error
 3rd purge loop: Remote I/O error
 3rd purge loop: Remote I/O error
 3rd purge loop: Remote I/O error
 3rd purge loop: Remote I/O error
 3rd purge loop: Remote I/O error
 3rd purge loop: Remote I/O error
ERROR in fpSharedPrivate.cpp at fpReadFpHwAndSwIdAndVersion: IIC requested 2 bytes - got -1 bytes.
 common code version read: Remote I/O error
After Fp software update version is -1, expected 27.
  fpSendCmd: queued I2C write error on second attemp -1
 fpSendCmd queue: Remote I/O error
  bytes 3 - 0x6 0x0 0x1
 bytes in queue 0
---------------------- startScopeApp() running Start code ---------------------
  fpSendCmd: queued I2C write error on second attemp -1
 fpSendCmd queue: Remote I/O error
  bytes 3 - 0x6 0x0 0x2
 bytes in queue 1
  fpSendCmd: queued I2C write error on second attemp -1
 fpSendCmd queue: Remote I/O error
  bytes 3 - 0x6 0x1 0x2
 bytes in queue 0
---------------------- startScopeApp() running Run code -----------------------
SIOCSIFFLAGS: Link has been severed
SIOCSIFFLAGS: Link has been severed
OK to connect by: telnet MSO4054-05CFYG 1072
  fpSendCmd: queued I2C write error on second attemp -1
 fpSendCmd queue: Remote I/O error
  bytes 2 - 0x2f 0x1
 bytes in queue 0
 SocketServerService: Socket server daemon started on port 4000.
    Protocol: Raw
-----------------------------------------------------------------
startScopeApp() complete; duration = 32.409003 seconds
=================================================================
PID to Task info written to /tmp/threads.txt
  Power Up Completed at 16:53:29
Enter 'ctrl-\' to quit scopeApp

16:53:29  fpSendCmd: queued I2C write error on second attemp -1
 fpSendCmd queue: Remote I/O error
16:53:29  bytes 2 - 0x2e 0x4b
 bytes in queue 0
16:53:30  fpSendCmd: queued I2C write error on second attemp -1
 fpSendCmd queue: Remote I/O error
16:53:30  bytes 3 - 0x6 0x0 0x2
 bytes in queue 0
16:53:31  fpSendCmd: queued I2C write error on second attemp -1
 fpSendCmd queue: Remote I/O error
16:53:31  bytes 2 - 0x2e 0x4b
 bytes in queue 3
16:53:31  fpSendCmd: queued I2C write error on second attemp -1
 fpSendCmd queue: Remote I/O error
16:53:31  bytes 2 - 0x2b 0x2
 bytes in queue 2
16:53:32  fpSendCmd: queued I2C write error on second attemp -1
 fpSendCmd queue: Remote I/O error
16:53:32  bytes 2 - 0x2a 0x2
 bytes in queue 1
16:53:32  fpSendCmd: queued I2C write error on second attemp -1
 fpSendCmd queue: Remote I/O error
16:53:32  bytes 2 - 0x2e 0x4b
 bytes in queue 0


--- End code ---

The micro controller on the front panel board at U1 is marked internally by Tekronix with firmware version called "1651-65901" programmed into it. It is an old and obsolete 8-bit HCS08 Freescale (now NXP) which P/N is MC9S08GB32A CFUE.
Acquistion board communicates with the front panel via I2C bus. It tries to read its firmware version and eventually updates it accordingly. So the front panel micro controller has to have a bootloader firmware programmed at factory stage (with I2C support) and must accept commands via the bus. The file which main CPU tries to send it a S-Record file route66_fp.s19.

First, what I discovered during the investigation of my front panel board.
AD244 at U2 transceiver serves only for control of front panel LEDs, both OE lines of the IC are short together.
There seems to be a +2.5 Volts rail at U4 - maybe some kind of voltage reference or additional HCS08 supply voltage.
U11 by U17 - DALC112 low capacitance diode array for ESD protection.
U5 - not determined.
U6 by U10, near the probe ID connectors - not determined.
Encoders work in this case not as usually (AB/BA low state order). Their connections to U1 are divided into 3 sections I believe where let's call them common pins of each section are connected to one port of U1 and the rest of the pins are connected to other ports of U1. However, not directly, but through a diode each remaining pin and after the diodes additionally through a 10 kOhm resistor. Additionally they may have 4 different state depending on which pins are short with which. Actually not important in this case.
I have no idea where the +12 Volts is routed to and to where is the LS1 filter routed/connected. Its one pin is connected to GND, but the second, I don't know. There is any voltage on the second pin.
The most significant information in my case is that crystal at Y1 (32.768 kHz) does not work at all and HCS08 provides to it no supply voltage - possible micro controller hardware failure or just internal oscillator setting active since the HCS08 has such a configuration option.
During the front panel measurements I experienced few times scope not booting and the was no short on the front panel lines I could do, so either the soldered cable turned into an antenna or oscilloscope probe caused and impedance change which somehow affected to boot up process. Or the replaced DRAM is not that perfect or one of the other DRAMs is starting to fail. Maybe after repairing of the scope I will just now replace the remaining DRAMs.

I contacted Tektronix support and asked for providing of firmware for the front panel micro controller or for front panel board programming quotation (not that I believed they will be willing do that) and they refused claiming it is not the part of the process. Of course. The only chance is purchase a front panel board, but since they do not have them much they rather would keep them instead of selling. The prices for used boards are too high for me.

Now, I wonder whether the HCS08 by any chance has not a read protection enabled.
From the firmware update package I extracted front panel firmware file and I am about to obtain PE-Micro Multilink ML-12 or similar and also can purchase 1 pc. of the HCS08 microcontroller since it is still available for a fair price. I already have a free USBDM software downloaded which should at least help me spare money on a very expensive Code Warrior software license.

But what when reading of the firmware from the micro controller via BDM will be not possible due read protection or due IC hardware failure or even if the controller is not faulty, but software is corrupt? Maybe someone of you has the right equipment to attempt readback of that firmware before I spend money for something which will be useless for me.

I know I could use the scope via USB plus TekVisa, but it will be probably not convenient (never used that software). Besides, if there are still options I would not give up if I went that far.

Note, above details I share are not complete and may be partially incorrect. Maybe there will be time and willing to confirm more information. At this stage I think it is not necessary to be worried about that anyway.
Please let me know what do you think about that.

Congratulations if you reached the end of my thread and thank you for taking your time for reading it, since it took me much time to gather all the information, aswell putting it here so it looks nice and clear and I hope it is not difficult to understand.

edit: Information about details of front panel investigation will be put here soon. Please, be patient.
dbator:
Hello,

I'm back with front panel issue investigation summary.

As you already know, in my case HCS08 MCU at U1 had no EXTAL signal (the frequency should be that of crystal at Y1: 32.768 kHz). My thinking was like, probably MCU just lost clock source configuration which could eventually change to internal clock source or eventually faulty MCU. I knew, without professional setup I would probably not gonna diagnose it. I searched the Internet for PE Micro debugger Multilink USB-ML-12E and luckily found it locally. It's much more expensive than open source USBDM debugger, but at least I'm sure it will be compatible with Freescale (now NXP) HCS08 microcontrollers. Besides I am familiar with the hardware and with CodeWarrior IDE software a bit, since I was using such a setup at my previous workplace for programming purposes. I've chosen CW IDE v6.3 as it is compatible with the HCS08 microcontrollers. Luckily I still own an old laptop with Windows XP Professional OS, so any operating system/drivers incompatibility/issues should be not occur. I did not install any of the patches available for download from NXP website, because there are many of them and I don't know in which order they should be installed. The software is working without any updates so I am not considering making any changes to my installation.
Note, if you consider going the same way, please be careful using the CW software and its components. Especially after Entering Advanced Programming Mode and in debugger mode (hiwave.exe), to not accidentally erase the FLASH module or make any changes before you are able to backup the program.

BDM connector on the front panel board has too short pins which could not fit enough deep into plug of PE Micro debugger, so I made extension wires from scratch.
I connected the debugger to the front panel board (just the board) providing +3V3 from a CR2032 battery (for safety reasons) which turned out to be enough for power consumption of U1. Then established connection by choosing [Connect (Reset)] button.
Then using PROGHCS08 (Advanced Programming Mode) called from debugger mode I was able to readback FLASH content (Upload module) and save a backup copy. PROGHCS08 saves the FLASH module content as a S-record (*.s19 file).
Note, please do not use CW IDE directly and create a project, because a demo code is generated. If you accidentally load the code to FLASH, you will loose chance to backup the original code.
Values in FPROT and NVPROT registers indicate no read protection set (desired bits were set to 1).

Further investigation confirmed incorrect clock source configuration (ICGC1, ICGC2 control registers). Changing ICGC1 and ICGC2 registers to proper values worked only until next reset and that's because these registers are unfortunately volatile registers which means POR (power-on reset) configuration is for internal reference generator (IRG) set as clock source. Then executed code re-configures mentioned registers on fly and if I understand correctly uninterrupted program execution is possible thank DCO block (Digitally-controlled Oscillator) within FLL (Frequency Locked Loop) which keeps the operating frequency in case of loss of clock which allows change from internal to external clock smoothly. Since ICG (Internal Clock Generator) provides clock (BUSCLK) to most of the HCS08 modules, incl. FLASH, RAM, I2C etc. it's obvious that incorrect BUSCLK provided to these modules will cause unexpected errors.

Here are several commands (windows) available in hiwave.exe if someone of you should decide for CW.
Among others: "Mcu registers" (not active by default, you have to choose it from the "Component" menu). It's a very useful command for reading and altering of the MCU registers.
These commands are in form of windows as you see on the attached pictures.
Remaining, useful commands are:
1) "Command" - console for controlling target (text mode). Commands (as whole CW functionality) depend on version of the CD IDE software. Useful commands: reset, go (program execution), load (program load).
2) "Memory" - showing whole memory map of the MCU, for instance FLASH (0x8000 to 0xFFFF), RAM (0x0080 to 0x087F) for MC9S08GB32A.
3) "Assembly" - showing disassembled code in assembler.
Note, apparently CW software supports open source hardware, like USBDM.

Now, debugging attempt of the program ended with "Illegal_BP" error" (illegal breakpoint) while there are no breakpoints set. I will not provide many details about possible factors causing this type errors because it's not important right now.
My attention caught many FFs disassembled as STX OpCode. But I'm not an expert (not a programmer) so I decided to not worry about that.

Next step was to install the front panel board into the scope, assembly the scope and try to debug the code in hope this time program will not break and I will see ICG registers are changing while the program is being executed. Luckily my extension wires I soldered to the BDM connector turned out to be long enough, so I could pass them through application modules slot on front panel enclosure.
Note, with this setup you have to wait until the scope boots because master CPU holds reset line of the slave CPU (front panel MCU) low for about half minute (I did not count it) from power cycle and then click [HotSync] button instead of [Connect (Reset)] button. HotSync does not asserts reset to the target (MCU), but just performs hot sync which still allows monitoring and debugging of a running program.
Result was the same: "Illegal_BP" error after start of the code execution.

I decided to manually analyze the S-record file I uploaded from the FLASH module and the S-record file (route66_fp.s19) extracted from the firmware image using 7zip (in my case the file name: firmware.img, but other MSO, MDO series have different file names) and noticed the lines in both S-record files were pointing the same addresses within FLASH module range, but the content was different. I was like, how is it possible that U1 is running bootcode and master CPU (PPC440EP) on acquisition board while communicating to the HCS08 with hypothetically properly running bootcode is able to overwrite it. Then I noticed in the backed up S-record file that every 20h of FLASH address there are few or sometimes just couple bytes of data to be loaded from a FLASH memory address looking correctly (they are not FF) and the rest and the most of the data are just FFs - it's like that part of the FLASH were blank. The non-FF values were the same in both files (backed up and route66_fp.s19). I recalled the FFs I noticed at the beginning in Memory command (window) of hiwave.exe. I thought it could mean NOP (No Operation), but HCS08 instruction set says that OpCode for NOP is 9D). It was the moment I started to suspect there are no separate codes like bootcode and application, but just one code which can be - I will call it - "hot" updated. It looks like necessary part of the code to keep the HCS08 running (I2C support etc.) executed in internal RAM of HCS08 while updating FLASH is enough and that's why the FLASH content can be replaced then. I'm not a programmer, but it just makes sense to be working that way.

Anyway, after a bit of hesitation I decided to finally erase FLASH module and load the route66_fp.s19 file (as mentioned, extracted from the downloaded firmware package) into the FLASH module via BDM (unchecking first: code run after programming, but with Verify checked). I'm telling you, I was expecting something will go wrong since I'm quite a pessimistic type. But FLASH programming and verification ended successful. I powered down the scope, disconnected the debugger and powered on the scope. After noticing "Fine" button being backlighted this time just before scope booted I was like, please, let it work. Scope booted and it was time to test the panel. And yes! Front panel is working! I really succeeded. What a huge relief :)
So the program was corrupted, but in a very specific way. I don't understand what could cause it. Normally I would say it could happen during failed/interrupted firmware update, but the scope was not booting at all due a bad DRAM. I purchased the scope already as non-functional, so it's hard to say in which circumstances the issue started to occur. And since the seller does not answer any of my messages just from the moment they dispatched the package I will probably not get my questions answered.

By the way, "hot" update of the program is a possible explanation why the front panel MCU has no read protection set - it would be more complicated to update the HCS08 via I2C if read protection would be enabled. There are such options like range protection, backdoor keys, so it's not impossible. But such a communications would be a bit harder and complex.

Next step was to read the debug console output which now does not indicate anything about read attempt of front panel program version. Here it is.

--- Code: ---U-Boot 1.1.4 (Jan  8 2007 - 11:12:14) Tektronix, Inc. V1.06

CPU:   AMCC PowerPC 440EP Rev. C at 333.333 MHz (PLB=133, OPB=66, EBC=66 MHz)
       I2C boot EEPROM enabled
       Internal PCI arbiter enabled, PCI async ext clock used
       32 kB I-Cache 32 kB D-Cache
Board: Tektronix Route66 IBM 440EP Main Board
VCO: 666 MHz
CPU: 333 MHz
PLB: 133 MHz
OPB: 66 MHz
EPB: 66 MHz
I2C:   ready
DRAM:  128 MB
FLASH: 64.5 MB
PCI:   Bus Dev VenId DevId Class Int
        00  13  10b5  9056  0680  18
        00  15  1002  4c59  0300  17
DISP:  Type 1
In:    serial
Out:   serial
Err:   serial
Enter password - autobooting in 3 seconds
## Booting image at f0000000 ...
   Image Name:   Linux-2.4.20_mvl31-440ep_eval
   Image Type:   PowerPC Linux Multi-File Image (gzip compressed)
   Data Size:    1441010 Bytes =  1.4 MB
   Load Address: 00000000
   Entry Point:  00000000
   Contents:
   Image 0:  1033953 Bytes = 1009.7 kB
   Image 1:   407042 Bytes = 397.5 kB
   Verifying Checksum ... OK
   Uncompressing Multi-File Image ... OK
cmdline is console=ttyS0,9600 quiet bigphysarea=519 panic=2 root=/dev/mtdblock7 rw mem=131072k
   Loading Ramdisk to 07f2b000, end 07f8e602 ... OK
Checking for firmware update...
No USB mass storage devices found to update from.
Linux 2.4.20_mvl31-440ep_eval V 1.15 Tektronix Route66 Tue Jun 22 15:19:50 PDT 2010
stat of /var/log/dmesg failed: No such file or directory
Warning: loading NiDKEng-1.6 will taint the kernel: non-GPL license - Proprietary
  See http://www.tux.org/lkml/#export-tainted for information about tainted modules
Warning: loading NiDUsb-1.6 will taint the kernel: non-GPL license - Proprietary
  See http://www.tux.org/lkml/#export-tainted for information about tainted modules
Warning: loading tek will taint the kernel: non-GPL license - Proprietary
  See http://www.tux.org/lkml/#export-tainted for information about tainted modules
Scope application starting (normal mode)
---------------------- startScopeApp() running Init code ----------------------
versionBuildFWVersionString(): TimestampString: 25-Apr-12  11:13   
                               VersionFIRMWAREVERSIONversion: v2.68
                               Major ver num: 2 Minor ver num: 68
     Initializing Mia[0]
     Initializing Tek0005[0]
     Initializing Ibm440[0]
     Initializing HFD204ADC[1]
     Initializing HFD204ADC[0]

Model id: 0x02

     Initializing Ltc1658Dac[0]

Board id: 0x01

     Initializing M859[3]
     Initializing M859[2]
     Initializing M859[1]
     Initializing M859[0]
     Initializing Ltc1660Dac[1]
     Initializing Ltc1660Dac[0]
      HFD144[0] ID_REG = 0x00001440
      HFD144[1] ID_REG = 0x00001440
      HFD144[2] ID_REG = 0x00001440
      HFD144[3] ID_REG = 0x00001440
     Initializing Hfd144[0]
     Initializing Hfd144[1]
     Initializing Hfd144[2]
     Initializing Hfd144[3]
Open Max5362 successful.
     Initializing Max5362[0]
     Initializing Ltc1661Dac[1]
     Initializing Ltc1661Dac[0]
 Init ADT7468 and locking.
 Factory Checksum: Stored: 58538, Calculated: 58538  - OK
 Spc CheckSum: stored: 15479 calculated: 15479  - OK
Demux initialization

    Dram calibration
    Dram calibration complete

Dram Calibration results:
-------------------------
DramCal has PASSED on all Demuxs

Demux initialization complete
Starting POST diags

Finished POST diags
---------------------- startScopeApp() running Start code ---------------------
---------------------- startScopeApp() running Run code -----------------------
SIOCSIFFLAGS: Link has been severed
SIOCSIFFLAGS: Link has been severed
OK to connect by: telnet MSO4054-05CFYG 1072
 SocketServerService: Socket server daemon started on port 4000.
    Protocol: Raw
-----------------------------------------------------------------
startScopeApp() complete; duration = 30.884302 seconds
=================================================================
PID to Task info written to /tmp/threads.txt
  Power Up Completed at 18:36:42
Enter 'ctrl-\' to quit scopeApp

--- End code ---
Of course, there are some errors, but they are probably not significant since the scope is working. Besides I saw these on logs from others.

Then I connected once again the debugger and powered on the scope. After it booted, I connected to the MCU also by choosing [HotSync] and I was able to see this time "Target Ready RUNNING" and memory and registers (for instance ports) values changing live. And the ICG registers now have correct values set of course.

My suggestions for everyone who is encountering the front panel issue is check following (depending on the symptoms and investigated scope model):
1) Make sure, no button is stucked.
a) power button in case of B-series scopes which may turn on automatically and buttons and encoders do not work,
b) any other button if the buttons and encoders just do not work).
2) Analyze debug console output.
3) Make sure front panel IDC connector and FFCs are sitting well.
4) Probe voltage rails (especially +3V3), reset, I2C (answer from front panel MCU) and other signals on front panel 20-pin IDC connector.
5) Probe crystal - stable voltage level means that crystal is probably faulty; no voltage at all means that probably MCU is faulty or its program is corrupted.

If you get to the point, you think re-programming of the front panel MCU should help, then obtain debugger (you may try the USBDM open source hardware and software if you don't want to risk spending that much money for PE Micro debugger) or use other hardware and software of you your choice if there is such you think you can trust it/it may work. Obviously try to backup the FLASH content first. Erase FLASH and load front panel S-record file extracted from the *.img file using 7zip (other programs may also be able to extract the type of files) you will find in firmware package.

Attention! If you will be connecting a debugger or other programming device, to avoid damaging the hardware, it is important to connect the USB cable to PC first, wait few seconds until it is properly detected make sure all drivers are installed properly and then power on the target. It is a general rule.

Note, you may need to convert the S-record file to a hex or bin format in some cases. There are apparently dedicated applications for such conversions. I did never found any and did not use any at all. It is possible to convert the S-record file to hex of course, but it's time consuming convert each line of the script.

If someone of you will use different setup then please share/confirm it is working. Such an information may help others.

I've attached here:
1) Corrupted (backed up) program as S-record file. I added *.txt extension to be able to upload the file here.
2) Front panel program extracted from the firmware package (route66_fp.s19) for MSO/DPO4000 (non-B series). I added *.txt extension to be able to upload the file here. You have to remove it in order to use it.
3) Pictures of my setup and scope running properly.
4) Screenshots of hiwave.exe with ICG registers shown (non-working and working front panel).
5) Debug console output of the scope after repair (above in the text).

Note, in cases of some MSO4000/MDO4000 series of scopes there is also a route66_fe.s19 file. That's why I suggest you to read debug console output first to confirm route66_fp.s19 is the file that master CPU is attempting to program into MCU on front panel board.
I wonder what actually is the route66_fe.19 for. Maybe other MSO4000/MDO4000 series of scopes have an additional MCU for additional functions on front panel, like number keypad. For instance due limited number of ports on one MCU. If anyone of you does have knowledge about the program (file) please share it.

For me the exciting, stressful and teaching experience ends. It is very satisfying to see my scope finally working. Hopefully there are no more issues because that would be too much for me.

I hope, the above portion of information will help others to repair their MSO4000 series of scopes.
If you have any questions, need more pictures or screenshots, please feel free to ask. PM me after sending a post to be sure I will read it. My setup is still not disconnected in case you have any questions and you want me to check something for you. Besides I am considering replacing remaining three DRAMs. Even if they are still working, they may fail sooner or later. On the other hand, if the counterfeit DRAMs I purchased are not that reliable I would trust them, maybe it's a good idea to replace them one by one an check everytime whether the scope is working. If one replaces them all by one and the scope would not boot, then you didn't know which one is faulty.
Sorama:
I didn’t understand half of it, but congrats anyway.
Navigation
Message Index
There was an error while thanking
Thanking...

Go to full version
Powered by SMFPacks Advanced Attachments Uploader Mod