Author Topic: Tektronix DPO4054 stucked at tektronix splash screen  (Read 16417 times)

0 Members and 1 Guest are viewing this topic.

Offline analogRF

  • Frequent Contributor
  • **
  • Posts: 955
  • Country: ca
Re: Tektronix DPO4054 stucked at tektronix splash screen
« Reply #50 on: February 14, 2021, 06:27:17 pm »
analogRF, I just found your thread regarding the dpo4104. I dont know how I didnt find it beforehand |O

I tried the firmware update....so it gets past the boot screen, but only on firmware lower than 2.68. It still crashes afterwards with not being able to handle button presses because of errors and finally kernelpanic :-DD

I'd still think there is also a problem in the CPU/computer area and not just the acquisition.
can you post the new boot log after forcing the firmware refresh
 

Offline pcwrangler

  • Regular Contributor
  • *
  • Posts: 73
  • Country: us
Re: Tektronix DPO4054 stucked at tektronix splash screen
« Reply #51 on: February 15, 2021, 12:52:24 am »
did you check all the crystal oscillators? there are several of them. at least two I remember were on the bottom side. Remember the CPU clock is 333MHz which is not directly generated by any crystal so there must be a PLL somewhere to multiply one of those crystal frequencies up to 333MHz
or maybe the CPU has that PLL implemented inside of itself.
All oscillators on both sides are present and stable. The CPU does have an integrated PLL.

Quote
I'd start by CPU architecture and pin out.
IF everything is suddenly stopped and there is no activity, either a clock has stopped or the CPU is halted or reset. So finding those pins is important
I checked every pin imaginable from the vias under the BGA. The only suspect was clkEn which enables the RAM clock. It is steady at 2.2v instead of the documented 2.5v in the CPU datasheet. However,  I believe that is still within SSTL 2 specs.  :-//

Quote
I think this CPU has no internal ROM (check?), so I guess that I2C EEPROM provides the small bit of code needed to start a boot loader but from where? or perhaps the EEPROM actually contains the bootloader itself? 1KB seems too small for that I guess but who knows.
You were right. I confirmed the CPU bootstrap is configured to gather boot configuration from the 16 bytes on the I2C EEPROM. I believe that is telling the CPU to get the bootloader from the 4MB flash chip at U822. The CE and OE pins between the CPU and U822 start enabled and then go high at the same time the boot log stops. I think bootloader instructions are interrupted because of this. I just don't know why the CPU is doing this.

Quote
your system is halted in the middle of that small code (or boorloader) for some reason. at that point nothing has been initialized yet I think
but DDR memory is probably already in play here so checking that supply termination is a good start. That is a specialized regulator IC soldered very near the CPU I dont remember the P/N but just look around the CPU and you will see it.
I confirmed the DDR termination regulator is at U650 and is behaving as specified in the datasheet. :(

Quote
maybe it's a good idea to use I2C decoding on the SDA/SCL lines of that chip and see what you get and compare with what the datasheet says about the signaling of that EEPROM.

But first and foremost is (1) all voltages including the DDR termination chip (2) all crystal oscillators (3) finding out if the CPU is halted or reset
Sadly, I have ruled out voltages, DDR termination and oscillators. I will attempt an I2C decode just to see what it looks like but I think we are beyond the I2C EEPROM step in the boot process.

Is it possible that the bootloader on the flash is damaged or missing at my halt point due to a failed FW update? I do have the means to program that chip if necessary. It feels like I'm so close!
 

Offline analogRF

  • Frequent Contributor
  • **
  • Posts: 955
  • Country: ca
Re: Tektronix DPO4054 stucked at tektronix splash screen
« Reply #52 on: February 15, 2021, 02:11:53 am »
Quote
I think this CPU has no internal ROM (check?), so I guess that I2C EEPROM provides the small bit of code needed to start a boot loader but from where? or perhaps the EEPROM actually contains the bootloader itself? 1KB seems too small for that I guess but who knows.
You were right. I confirmed the CPU bootstrap is configured to gather boot configuration from the 16 bytes on the I2C EEPROM. I believe that is telling the CPU to get the bootloader from the 4MB flash chip at U822. The CE and OE pins between the CPU and U822 start enabled and then go high at the same time the boot log stops. I think bootloader instructions are interrupted because of this. I just don't know why the CPU is doing this.


excellent piece of troubleshooting  :-+ This is the most important information that we have by now and you should focus on this flash memory U822. I also think it does contain the bootloader code. Can you remove the sticker and post the part number?

you are right, based on what you have gathered, the I2C EEPROM has done its job and I dont think it is at fault.

I'd guess the bootloader flash is corrupted. At least at this point and with the other information that we have, this seems to be the most likely cause. It happens  :palm:
That chip is easy to desolder and solder back in. If I remember correctly it is QFP package with J-leads, right? so one course of action would be to
just desolder it and read it and perhaps get a similar chip off another scope and re program this one.


one last thing to check, if you let the scope run for while, does any component get hot? I mean only the CPU, DRAM or the flash chips?
 
 

Offline pcwrangler

  • Regular Contributor
  • *
  • Posts: 73
  • Country: us
Re: Tektronix DPO4054 stucked at tektronix splash screen
« Reply #53 on: February 15, 2021, 05:04:51 am »
excellent piece of troubleshooting  :-+ This is the most important information that we have by now and you should focus on this flash memory U822. I also think it does contain the bootloader code. Can you remove the sticker and post the part number?
U822 is a Am29LV040B PLCC. Does anyone have a dump of this chip? This MSO4104(non-B) has the same version header as others have posted from different versions of the 4000. "U-Boot 1.1.4 (Jan  8 2007 - 11:12:14) Tektronix, Inc. V1.06". I'm not sure if that means it's universal and interchangeable but I also found this same version digging through the downloadable FW update from Tek. Maybe I'll desolder and compare the two.
Quote
one last thing to check, if you let the scope run for while, does any component get hot? I mean only the CPU, DRAM or the flash chips?
Unfortunately, no. This was the first thing I checked with an IR cam, hoping to find a simple short.  :--
 

Offline m k

  • Super Contributor
  • ***
  • Posts: 1924
  • Country: fi
Re: Tektronix DPO4054 stucked at tektronix splash screen
« Reply #54 on: February 15, 2021, 06:42:43 pm »
My guess is that F-line is good and if that flash has a checksum it's good also.
Based on an assumption that the programmer is not doing the text letter by letter.

There seems to be a leap from F-line to next.
If there is a watchdog, is it so accurate that it will never let through more or less than that F.

Processes seems to be asyncronous, many processing entities I guess.
How long is one letter sent, is that time too long, can it really pin point anything.

If PCI-signals are easily accessed I'd check that the bus is alive.
Advance-Aneng-Appa-AVO-Data Tech-Fluke-General Radio-H. W. Sullivan-Heathkit-HP-Kaise-Kyoritsu-Leeds & Northrup-Mastech-REO-Simpson-Sinclair-Tektronix-Triplett-YFE
(plus lesser brands from the work shop of the world)
 

Offline pcwrangler

  • Regular Contributor
  • *
  • Posts: 73
  • Country: us
Re: Tektronix DPO4054 stucked at tektronix splash screen
« Reply #55 on: February 16, 2021, 09:49:51 pm »
After desoldering and dumping the existing code on U822 I can see it contains more than just the bootloader. I compared it to the bootloader.img obtained via 'binwalk -e' of the latest firmware from the Tektronix website. Some fancy dd work on this might merge them, overwriting the bootloader portion to make things work but I would rather receive a U822 dump from someone else in the community so I know it is a functional copy. The results are below.

binwalk of U822 dump (524KB)
Quote
DECIMAL       HEXADECIMAL     DESCRIPTION
--------------------------------------------------------------------------------
295           0x127           Unix path: /vol/ibunix01/r66tools/r66tools/mvserve/filesys
1370          0x55A           U-Boot version string, "U-Boot 1.0.0 (Sep  8 2006 - 08:50:07) Tektronix, Inc. V1.02"
262144        0x40000         uImage header, header size: 64 bytes, header CRC: 0x552D426F, created: 2029-04-03 03:29:53, image size: 774975028 bytes, Data Address: 0x20284A61, Entry Point: 0x6E202038, data CRC: 0x20323030, image name: "11:12:14) Tektronix, Inc. V1.06"
262148        0x40004         U-Boot version string, "U-Boot 1.1.4 (Jan  8 2007 - 11:12:14) Tektronix, Inc. V1.06"
381312        0x5D180         CRC32 polynomial table, big endian
386984        0x5E7A8         U-Boot version string, "U-Boot 1.1.4"
414044        0x6515C         Unix path: /vol/ibunix03/r66tools/r66tools/mvserve/filesys

binwalk of bootloader.img from Tek (262KB)
Quote
DECIMAL       HEXADECIMAL     DESCRIPTION
--------------------------------------------------------------------------------
0             0x0             uImage header, header size: 64 bytes, header CRC: 0x552D426F, created: 2029-04-03 03:29:53, image size: 774975028 bytes, Data Address: 0x20284A61, Entry Point: 0x6E202038, data CRC: 0x20323030, image name: "11:12:14) Tektronix, Inc. V1.06"
4             0x4             U-Boot version string, "U-Boot 1.1.4 (Jan  8 2007 - 11:12:14) Tektronix, Inc. V1.06"
119168        0x1D180         CRC32 polynomial table, big endian
124840        0x1E7A8         U-Boot version string, "U-Boot 1.1.4"
151900        0x2515C         Unix path: /vol/ibunix03/r66tools/r66tools/mvserve/filesys
« Last Edit: February 16, 2021, 09:53:06 pm by pcwrangler »
 

Offline analogRF

  • Frequent Contributor
  • **
  • Posts: 955
  • Country: ca
Re: Tektronix DPO4054 stucked at tektronix splash screen
« Reply #56 on: February 16, 2021, 10:21:11 pm »
looks like there are two different versions of u-boot in U822? but why?

if the flash was really corrupted, I believe the chances that binwalk could extract even that much information
was slim. Still it could be corrupted in just a few bytes somewhere but that is very unlikely in my view.
The fact that binwalk can still extract this much info is 'probably' a sign that flash is not at fault
« Last Edit: February 18, 2021, 02:19:38 am by analogRF »
 

Offline analogRF

  • Frequent Contributor
  • **
  • Posts: 955
  • Country: ca
Re: Tektronix DPO4054 stucked at tektronix splash screen
« Reply #57 on: February 18, 2021, 02:17:16 am »
assuming the flash chip is ok, I think the cpu stalls when she wants to communicate with a peripheral device/chip and perhaps does not get the right response/acknowledge. At this early stage I think it is more likely to be a device on the PCI bus
Code: [Select]
PCI:   Bus Dev VenId DevId Class Int
        00  13  10b5  9056  0680  18
        00  15  1002  4c59  0300  17

the first device is the PCI Bus Master chip made by PLX (PCI9056) and the second device in the list is the ATI Mobility Radeon graphics processor
but how to troubleshoot their connection? is beyond me... :-// :scared:

 
The following users thanked this post: pcwrangler

Offline m k

  • Super Contributor
  • ***
  • Posts: 1924
  • Country: fi
Re: Tektronix DPO4054 stucked at tektronix splash screen
« Reply #58 on: February 18, 2021, 06:46:04 pm »
If there is a real PCI connector its IDSEL should blink.
If it does then at least the controller is alive.


looks like there are two different versions of u-boot in U822? but why?

My guess is that the older one is a backup bootloader.
Something like when writing a new bootloader fails backup one is still ok.

So can one think that good behaving Tek has done the same with everything?
Advance-Aneng-Appa-AVO-Data Tech-Fluke-General Radio-H. W. Sullivan-Heathkit-HP-Kaise-Kyoritsu-Leeds & Northrup-Mastech-REO-Simpson-Sinclair-Tektronix-Triplett-YFE
(plus lesser brands from the work shop of the world)
 
The following users thanked this post: pcwrangler

Offline pcwrangler

  • Regular Contributor
  • *
  • Posts: 73
  • Country: us
Re: Tektronix DPO4054 stucked at tektronix splash screen
« Reply #59 on: February 18, 2021, 10:38:22 pm »
assuming the flash chip is ok, I think the cpu stalls when she wants to communicate with a peripheral device/chip and perhaps does not get the right response/acknowledge. At this early stage I think it is more likely to be a device on the PCI bus
Code: [Select]
PCI:   Bus Dev VenId DevId Class Int
        00  13  10b5  9056  0680  18
        00  15  1002  4c59  0300  17

the first device is the PCI Bus Master chip made by PLX (PCI9056) and the second device in the list is the ATI Mobility Radeon graphics processor
but how to troubleshoot their connection? is beyond me... :-// :scared:
vcc and clock are good on the PCI9056 at U560 but the IDSEL(thanks m k) line is stuck low. The datasheet of this chip and the 440EP CPU show their IDSEL pins as input. Anyone know the source?

BTW: I programmed the flash with the downloaded and extracted bootloader.img just to see what would happen but it wouldn't even provide a boot log, completely dead. I reprogrammed it with the original code and it's back to the previous behavior. I thought I would add this here just in case someone is playing with firmware in the future. I suppose a merge of that img file with the original code might work but I believe you all are correct; if it was damaged I wouldn't get much useful info out of it.
 

Offline analogRF

  • Frequent Contributor
  • **
  • Posts: 955
  • Country: ca
Re: Tektronix DPO4054 stucked at tektronix splash screen
« Reply #60 on: February 18, 2021, 10:46:29 pm »
could it be that CPU is receiving a false interrupt from a device onboard?
 

Offline m k

  • Super Contributor
  • ***
  • Posts: 1924
  • Country: fi
Re: Tektronix DPO4054 stucked at tektronix splash screen
« Reply #61 on: February 19, 2021, 12:32:02 pm »
Good, or bad, now it's at least known that something excact is clearly something it shouldn't.
Is it a cause is another thing.

I don't know the correct PCI naming policies and the whole PCI hierarchy has always been a bit messy.
A controller can also be PCI to PCI and then another PCI controller is what you think is the first one.
Did I read it right that this 440EP can also be a PCI controller, so the 2nd in PCI line.

One construction model is CPU-PLB-bridge-PCI-controller/bridge-stuff.
There is also a method called PCI boot, where PCI memory can be accessed without configuration, it can obviously be misused for what ever purposes.

In this case it's quite possible that PCI something is predefined.
Since the system knows that some parts are there, or should be, always, it can expect them.

So if PCI 9056 is not first its missing ID request can be fine, but since that other boot log has it as first, one must follow that.
Though it doesn't exclude that there isn't something else before that but would it be completely silent in the log.

For the interrupt,
if it's PCI and not configured it should be ingnored.

One method for IDSEL is to decode one address line for each PCI connection IDSEL.
PPC440EPx-GRx manual says that device 1 is AD11.
If those addresses are used elsewhere there must be a chip with enable in between somewhere.

Advance-Aneng-Appa-AVO-Data Tech-Fluke-General Radio-H. W. Sullivan-Heathkit-HP-Kaise-Kyoritsu-Leeds & Northrup-Mastech-REO-Simpson-Sinclair-Tektronix-Triplett-YFE
(plus lesser brands from the work shop of the world)
 

Offline ApertureScience

  • Contributor
  • Posts: 23
  • Country: de
Re: Tektronix DPO4054 stucked at tektronix splash screen
« Reply #62 on: May 30, 2021, 09:48:18 am »
So i'm back on my project!
I got some compatible RAM chips and found a shop that can swap them for me.
To understand the error messages better I reversed the firmware, but nothing especially helpful came up. To tackle the 10_calDramStep29 error i wanted to narrow down the selection of the 4 ram chips associated with DEMUX 3 (aka U370).

Now the part that is pure speculation. The error message format goes:
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
What had me confused is that four RAM chips are listed with three memory adresses. I thought those were supposed to be the memory addresses currently on the DEMUX buses, even though there should be four then. But I think these are linked to the DRAM groups D and DC quoted by an error message before:
dmx 03 ERROR: 10_calDramStep29: Dram1 GrpD 29.2.4 byte 97: exp/act 0x06/0x0e (bits: 3) 
dmx 03 ERROR: 10_calDramStep29: Dram1 GrpDC 29.2.4 byte 99: exp/act 0x06/0x36 (bits: 4 5)
Hence why the second and third address are always the same. My guess would be that that group D and C are the third and fouth RAM probably, so U470 and U0462.

Does somebody have an opinion on that?
 

Offline ApertureScience

  • Contributor
  • Posts: 23
  • Country: de
Re: Tektronix DPO4054 stucked at tektronix splash screen
« Reply #63 on: May 30, 2021, 09:52:03 am »
I will also try to reball the DEMUX.
 

Offline ApertureScience

  • Contributor
  • Posts: 23
  • Country: de
Re: Tektronix DPO4054 stucked at tektronix splash screen
« Reply #64 on: May 30, 2021, 10:02:57 am »
Regarding the acqUpWriteLong failed error, in the firmware i found the function outputting it. I have no idea what it does! Raw decompiled code ahead :scared:
Code: [Select]
undefined8 acqUpWriteLong(ulong *param_1,ulong param_2,uint param_3,int param_4)

{
  undefined4 uVar1;
  uint uVar2;
  undefined4 in_LR;
 
  uVar2 = CDmxUpXfr::write(*(CDmxUpXfr **)
                            (*(int *)(*(int *)(CDemuxs::m_physicalToLogicalIndexMap + param_4 * 4) *
                                      4 + CDemuxs::m_dmxs) + 0x40),param_1,param_3,param_2);
  uVar1 = 0;
  if (uVar2 != param_3) {
    errPrintf("acqUpWriteLong failed\n");
    uVar1 = 0xffffffff;
  }
  return CONCAT44(uVar1,in_LR);
}
 

Offline ApertureScience

  • Contributor
  • Posts: 23
  • Country: de
Re: Tektronix DPO4054 stucked at tektronix splash screen
« Reply #65 on: May 30, 2021, 10:08:34 am »
Also interesting function that tells a bit about the memory layout. Not sure why it seems to show gaps though...
Code: [Select]
ushort convByteIdxToGrpIdx(uint offset)

{
  int local_20 [7];
 
  local_20[0] = 0x10003;
  local_20[1] = 2;
  local_20[2] = 0x50007;
  local_20[3] = 0x40006;
  return *(ushort *)((int)local_20 + (offset & 7) * 2);
}

This looks weird, maybe some magic on the address bus. 0x10000 is very close to 2^16, implying some 16bit wide bus.

Or.. I am totally wrong about my assumptions and everything is specified down to the SDRAM architecture to achieve the necessary acess times.
« Last Edit: May 30, 2021, 11:18:43 am by ApertureScience »
 

Offline MarkF3837

  • Newbie
  • Posts: 7
  • Country: us
Re: Tektronix DPO4054 stucked at tektronix splash screen
« Reply #66 on: January 04, 2022, 01:06:50 am »
On my DPO4104, there is an opening in the bottom of the metal shield. Inside is a 2-row, 20-pin 1.27 mm pitch connector. With the 'scope upside down & facing away from you, the second pin from the left in the top row is the TxD pin (data from 'scope). The pin to its right is ground. I haven't searched for the RxD pin.

Signal level is 3.3V. Data rate is 9600 bps.
 
The following users thanked this post: martinr33

Offline Skyhi

  • Newbie
  • Posts: 4
  • Country: ca
Re: Tektronix DPO4054 stucked at tektronix splash screen
« Reply #67 on: July 30, 2022, 03:42:06 pm »
Hi all,  just wanted to say I joined this forum because I was looking for pointers on repairing a DPO 4104.
In my case re-flowing the BGA flash IC's U840, U832, U821 and U820 solved my 'stuck at boot' problem.  I flooded them with flux and used a heater underneath and used a hand air station with a small noozle until I saw them bubble and 're-seat' themselves. 

If it happens again I plan on removing them and re-balling (with leaded solder of course).

Been working great for a while now!  :-+
Best regards,
Will
« Last Edit: July 30, 2022, 08:00:31 pm by Skyhi »
 
The following users thanked this post: analogRF, m k

Offline analogRF

  • Frequent Contributor
  • **
  • Posts: 955
  • Country: ca
Re: Tektronix DPO4054 stucked at tektronix splash screen
« Reply #68 on: August 01, 2022, 11:33:51 am »
Hi all,  just wanted to say I joined this forum because I was looking for pointers on repairing a DPO 4104.
In my case re-flowing the BGA flash IC's U840, U832, U821 and U820 solved my 'stuck at boot' problem.  I flooded them with flux and used a heater underneath and used a hand air station with a small noozle until I saw them bubble and 're-seat' themselves. 

If it happens again I plan on removing them and re-balling (with leaded solder of course).

Been working great for a while now!  :-+
Best regards,
Will

may I ask what temperature/air speed did you use? Did you also put the board on pre-heater?
This is a huge and thick board, I am very reluctant (read scared  :-//) to do reflowing
 

Offline Skyhi

  • Newbie
  • Posts: 4
  • Country: ca
Re: Tektronix DPO4054 stucked at tektronix splash screen
« Reply #69 on: August 01, 2022, 12:16:40 pm »
I did use a pre-heater underneath the board - it is a fairly small hot air setup and it had it set all the way on for heat.  As for the hand air station the nozzle I used was around 7mm in diameter - I had the heat to max and kept the air flow fairly low to keep things from blowing around.  I also removed the backup battery before I started heating things just to make it a little safer.  When I get back to work on Tuesday I will list the equipment that I used and give you a better idea of the airspeed I used.  I had initially figured out something was up in that area of the board just by using my heat gun at home just on the top of the board while trying to power up the unit and it booted!  But as soon as it cooled off again it would freeze as described.  I don't normally like putting as much heat into the parts as I did but I figured the die in the packages where insulated by the plastic and I really did need to get them hot enough for the solder underneath to liquefy and re-harden to achieve a good connection after the process.
 
The following users thanked this post: analogRF

Offline Jwalling

  • Supporter
  • ****
  • Posts: 1517
  • Country: us
  • This is work?
Re: Tektronix DPO4054 stucked at tektronix splash screen
« Reply #70 on: August 02, 2022, 09:03:43 am »
I also removed the backup battery before I started heating things just to make it a little safer.

Can you test the scope to see if it still gets an IP address when attached to a network? On a DPO3000 I worked on that had a dead battery, the MAC address was lost and couldn't be recovered.
Thanks.
Jay

System error. Strike any user to continue.
 

Offline Skyhi

  • Newbie
  • Posts: 4
  • Country: ca
Re: Tektronix DPO4054 stucked at tektronix splash screen
« Reply #71 on: August 03, 2022, 11:30:53 am »
Hmm I was seeing an error (when not connected to my network) as per this image when I booted the scope and I could not seem to set a new IP address but as soon as I plugged it into my router I could update the settings and the connection test passed. :)
 
The following users thanked this post: Jwalling

Offline Skyhi

  • Newbie
  • Posts: 4
  • Country: ca
Re: Tektronix DPO4054 stucked at tektronix splash screen
« Reply #72 on: August 03, 2022, 11:35:44 am »
This is the equipment that I used for my fix - I think I had the airspeed set to around 3.5 on the hand held hot air station.
 
The following users thanked this post: analogRF

Offline analogRF

  • Frequent Contributor
  • **
  • Posts: 955
  • Country: ca
Re: Tektronix DPO4054 stucked at tektronix splash screen
« Reply #73 on: August 03, 2022, 11:41:14 am »
I also removed the backup battery before I started heating things just to make it a little safer.

Can you test the scope to see if it still gets an IP address when attached to a network? On a DPO3000 I worked on that had a dead battery, the MAC address was lost and couldn't be recovered.
Thanks.

I think, similar to DPO4000, the MAC address is stored in a u-boot environment variable (ethaddr) and thus stored in the flash.
I dont think it has anything to do with the battery. I think only volatile settings such as IP address or user setups might be stored in the battery back up NVRAM.
 
The following users thanked this post: Jwalling


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf