Tektronix DPO4054 stucked at tektronix splash screen

• #50 Reply
  Posted by analogRF on 14 Feb, 2021 18:27
• Quote from: ApertureScience on 14 Feb, 2021 17:34

  analogRF, I just found your thread regarding the dpo4104. I dont know how I didnt find it beforehand
  
  I tried the firmware update....so it gets past the boot screen, but only on firmware lower than 2.68. It still crashes afterwards with not being able to handle button presses because of errors and finally kernelpanic
  

  
  I'd still think there is also a problem in the CPU/computer area and not just the acquisition.
  can you post the new boot log after forcing the firmware refresh
• #51 Reply
  Posted by pcwrangler on 15 Feb, 2021 00:52
• Quote from: analogRF on 12 Feb, 2021 01:41

  did you check all the crystal oscillators? there are several of them. at least two I remember were on the bottom side. Remember the CPU clock is 333MHz which is not directly generated by any crystal so there must be a PLL somewhere to multiply one of those crystal frequencies up to 333MHz
  or maybe the CPU has that PLL implemented inside of itself.
  

  All oscillators on both sides are present and stable. The CPU does have an integrated PLL.
  
  

  Quote

  I'd start by CPU architecture and pin out.
  IF everything is suddenly stopped and there is no activity, either a clock has stopped or the CPU is halted or reset. So finding those pins is important
  

  I checked every pin imaginable from the vias under the BGA. The only suspect was clkEn which enables the RAM clock. It is steady at 2.2v instead of the documented 2.5v in the CPU datasheet. However,  I believe that is still within SSTL 2 specs. 
  
  

  Quote

  I think this CPU has no internal ROM (check?), so I guess that I2C EEPROM provides the small bit of code needed to start a boot loader but from where? or perhaps the EEPROM actually contains the bootloader itself? 1KB seems too small for that I guess but who knows.
  

  You were right. I confirmed the CPU bootstrap is configured to gather boot configuration from the 16 bytes on the I2C EEPROM. I believe that is telling the CPU to get the bootloader from the 4MB flash chip at U822. The CE and OE pins between the CPU and U822 start enabled and then go high at the same time the boot log stops. I think bootloader instructions are interrupted because of this. I just don't know why the CPU is doing this.
  
  

  Quote

  your system is halted in the middle of that small code (or boorloader) for some reason. at that point nothing has been initialized yet I think
  but DDR memory is probably already in play here so checking that supply termination is a good start. That is a specialized regulator IC soldered very near the CPU I dont remember the P/N but just look around the CPU and you will see it.
  

  I confirmed the DDR termination regulator is at U650 and is behaving as specified in the datasheet.
  
  

  Quote

  maybe it's a good idea to use I2C decoding on the SDA/SCL lines of that chip and see what you get and compare with what the datasheet says about the signaling of that EEPROM.
  
  But first and foremost is (1) all voltages including the DDR termination chip (2) all crystal oscillators (3) finding out if the CPU is halted or reset
  

  Sadly, I have ruled out voltages, DDR termination and oscillators. I will attempt an I2C decode just to see what it looks like but I think we are beyond the I2C EEPROM step in the boot process.
  
  Is it possible that the bootloader on the flash is damaged or missing at my halt point due to a failed FW update? I do have the means to program that chip if necessary. It feels like I'm so close!
• #52 Reply
  Posted by analogRF on 15 Feb, 2021 02:11
• Quote from: pcwrangler on 15 Feb, 2021 00:52

  Quote

  I think this CPU has no internal ROM (check?), so I guess that I2C EEPROM provides the small bit of code needed to start a boot loader but from where? or perhaps the EEPROM actually contains the bootloader itself? 1KB seems too small for that I guess but who knows.
  

  You were right. I confirmed the CPU bootstrap is configured to gather boot configuration from the 16 bytes on the I2C EEPROM. I believe that is telling the CPU to get the bootloader from the 4MB flash chip at U822. The CE and OE pins between the CPU and U822 start enabled and then go high at the same time the boot log stops. I think bootloader instructions are interrupted because of this. I just don't know why the CPU is doing this.
  
  

  
  excellent piece of troubleshooting  This is the most important information that we have by now and you should focus on this flash memory U822. I also think it does contain the bootloader code. Can you remove the sticker and post the part number?
  
  you are right, based on what you have gathered, the I2C EEPROM has done its job and I dont think it is at fault.
  
  I'd guess the bootloader flash is corrupted. At least at this point and with the other information that we have, this seems to be the most likely cause. It happens 
  That chip is easy to desolder and solder back in. If I remember correctly it is QFP package with J-leads, right? so one course of action would be to
  just desolder it and read it and perhaps get a similar chip off another scope and re program this one.
  
  
  one last thing to check, if you let the scope run for while, does any component get hot? I mean only the CPU, DRAM or the flash chips?
   
  
• #53 Reply
  Posted by pcwrangler on 15 Feb, 2021 05:04
• Quote from: analogRF on 15 Feb, 2021 02:11

  excellent piece of troubleshooting  This is the most important information that we have by now and you should focus on this flash memory U822. I also think it does contain the bootloader code. Can you remove the sticker and post the part number?
  

  U822 is a Am29LV040B PLCC. Does anyone have a dump of this chip? This MSO4104(non-B) has the same version header as others have posted from different versions of the 4000. "U-Boot 1.1.4 (Jan  8 2007 - 11:12:14) Tektronix, Inc. V1.06". I'm not sure if that means it's universal and interchangeable but I also found this same version digging through the downloadable FW update from Tek. Maybe I'll desolder and compare the two.
  

  Quote

  one last thing to check, if you let the scope run for while, does any component get hot? I mean only the CPU, DRAM or the flash chips?
  

  Unfortunately, no. This was the first thing I checked with an IR cam, hoping to find a simple short. 
• #54 Reply
  Posted by m k on 15 Feb, 2021 18:42
• My guess is that F-line is good and if that flash has a checksum it's good also.
  Based on an assumption that the programmer is not doing the text letter by letter.
  
  There seems to be a leap from F-line to next.
  If there is a watchdog, is it so accurate that it will never let through more or less than that F.
  
  Processes seems to be asyncronous, many processing entities I guess.
  How long is one letter sent, is that time too long, can it really pin point anything.
  
  If PCI-signals are easily accessed I'd check that the bus is alive.
  
• #55 Reply
  Posted by pcwrangler on 16 Feb, 2021 21:49
• After desoldering and dumping the existing code on U822 I can see it contains more than just the bootloader. I compared it to the bootloader.img obtained via 'binwalk -e' of the latest firmware from the Tektronix website. Some fancy dd work on this might merge them, overwriting the bootloader portion to make things work but I would rather receive a U822 dump from someone else in the community so I know it is a functional copy. The results are below.
  
  binwalk of U822 dump (524KB)
  

  Quote

  DECIMAL       HEXADECIMAL     DESCRIPTION
  --------------------------------------------------------------------------------
  295           0x127           Unix path: /vol/ibunix01/r66tools/r66tools/mvserve/filesys
  1370          0x55A           U-Boot version string, "U-Boot 1.0.0 (Sep  8 2006 - 08:50:07) Tektronix, Inc. V1.02"
  262144        0x40000         uImage header, header size: 64 bytes, header CRC: 0x552D426F, created: 2029-04-03 03:29:53, image size: 774975028 bytes, Data Address: 0x20284A61, Entry Point: 0x6E202038, data CRC: 0x20323030, image name: "11:12:14) Tektronix, Inc. V1.06"
  262148        0x40004         U-Boot version string, "U-Boot 1.1.4 (Jan  8 2007 - 11:12:14) Tektronix, Inc. V1.06"
  381312        0x5D180         CRC32 polynomial table, big endian
  386984        0x5E7A8         U-Boot version string, "U-Boot 1.1.4"
  414044        0x6515C         Unix path: /vol/ibunix03/r66tools/r66tools/mvserve/filesys
  

  
  binwalk of bootloader.img from Tek (262KB)
  

  Quote

  DECIMAL       HEXADECIMAL     DESCRIPTION
  --------------------------------------------------------------------------------
  0             0x0             uImage header, header size: 64 bytes, header CRC: 0x552D426F, created: 2029-04-03 03:29:53, image size: 774975028 bytes, Data Address: 0x20284A61, Entry Point: 0x6E202038, data CRC: 0x20323030, image name: "11:12:14) Tektronix, Inc. V1.06"
  4             0x4             U-Boot version string, "U-Boot 1.1.4 (Jan  8 2007 - 11:12:14) Tektronix, Inc. V1.06"
  119168        0x1D180         CRC32 polynomial table, big endian
  124840        0x1E7A8         U-Boot version string, "U-Boot 1.1.4"
  151900        0x2515C         Unix path: /vol/ibunix03/r66tools/r66tools/mvserve/filesys
  
• #56 Reply
  Posted by analogRF on 16 Feb, 2021 22:21
• looks like there are two different versions of u-boot in U822? but why?
  
  if the flash was really corrupted, I believe the chances that binwalk could extract even that much information
  was slim. Still it could be corrupted in just a few bytes somewhere but that is very unlikely in my view.
  The fact that binwalk can still extract this much info is 'probably' a sign that flash is not at fault
• #57 Reply
  Posted by analogRF on 18 Feb, 2021 02:17
• assuming the flash chip is ok, I think the cpu stalls when she wants to communicate with a peripheral device/chip and perhaps does not get the right response/acknowledge. At this early stage I think it is more likely to be a device on the PCI bus
  

  Code: [Select]

  PCI:   Bus Dev VenId DevId Class Int
        00  13  10b5  9056  0680  18
        00  15  1002  4c59  0300  17
  the first device is the PCI Bus Master chip made by PLX (PCI9056) and the second device in the list is the ATI Mobility Radeon graphics processor
  but how to troubleshoot their connection? is beyond me...
  
  
• #58 Reply
  Posted by m k on 18 Feb, 2021 18:46
• If there is a real PCI connector its IDSEL should blink.
  If it does then at least the controller is alive.
  
  
  

  Quote from: analogRF on 16 Feb, 2021 22:21

  looks like there are two different versions of u-boot in U822? but why?
  

  
  My guess is that the older one is a backup bootloader.
  Something like when writing a new bootloader fails backup one is still ok.
  
  So can one think that good behaving Tek has done the same with everything?
  
• #59 Reply
  Posted by pcwrangler on 18 Feb, 2021 22:38
• Quote from: analogRF on 18 Feb, 2021 02:17

  assuming the flash chip is ok, I think the cpu stalls when she wants to communicate with a peripheral device/chip and perhaps does not get the right response/acknowledge. At this early stage I think it is more likely to be a device on the PCI bus
  

  Code: [Select]

  PCI:   Bus Dev VenId DevId Class Int
        00  13  10b5  9056  0680  18
        00  15  1002  4c59  0300  17
  the first device is the PCI Bus Master chip made by PLX (PCI9056) and the second device in the list is the ATI Mobility Radeon graphics processor
  but how to troubleshoot their connection? is beyond me...
  

  vcc and clock are good on the PCI9056 at U560 but the IDSEL(thanks m k) line is stuck low. The datasheet of this chip and the 440EP CPU show their IDSEL pins as input. Anyone know the source?
  
  BTW: I programmed the flash with the downloaded and extracted bootloader.img just to see what would happen but it wouldn't even provide a boot log, completely dead. I reprogrammed it with the original code and it's back to the previous behavior. I thought I would add this here just in case someone is playing with firmware in the future. I suppose a merge of that img file with the original code might work but I believe you all are correct; if it was damaged I wouldn't get much useful info out of it.
• #60 Reply
  Posted by analogRF on 18 Feb, 2021 22:46
• could it be that CPU is receiving a false interrupt from a device onboard?
• #61 Reply
  Posted by m k on 19 Feb, 2021 12:32
• Good, or bad, now it's at least known that something excact is clearly something it shouldn't.
  Is it a cause is another thing.
  
  I don't know the correct PCI naming policies and the whole PCI hierarchy has always been a bit messy.
  A controller can also be PCI to PCI and then another PCI controller is what you think is the first one.
  Did I read it right that this 440EP can also be a PCI controller, so the 2nd in PCI line.
  
  One construction model is CPU-PLB-bridge-PCI-controller/bridge-stuff.
  There is also a method called PCI boot, where PCI memory can be accessed without configuration, it can obviously be misused for what ever purposes.
  
  In this case it's quite possible that PCI something is predefined.
  Since the system knows that some parts are there, or should be, always, it can expect them.
  
  So if PCI 9056 is not first its missing ID request can be fine, but since that other boot log has it as first, one must follow that.
  Though it doesn't exclude that there isn't something else before that but would it be completely silent in the log.
  
  For the interrupt,
  if it's PCI and not configured it should be ingnored.
  
  One method for IDSEL is to decode one address line for each PCI connection IDSEL.
  PPC440EPx-GRx manual says that device 1 is AD11.
  If those addresses are used elsewhere there must be a chip with enable in between somewhere.
  
  
• #62 Reply
  Posted by ApertureScience on 30 May, 2021 09:48
• So i'm back on my project!
  I got some compatible RAM chips and found a shop that can swap them for me.
  To understand the error messages better I reversed the firmware, but nothing especially helpful came up. To tackle the 10_calDramStep29 error i wanted to narrow down the selection of the 4 ram chips associated with DEMUX 3 (aka U370).
  
  Now the part that is pure speculation. The error message format goes:
  Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
  What had me confused is that four RAM chips are listed with three memory adresses. I thought those were supposed to be the memory addresses currently on the DEMUX buses, even though there should be four then. But I think these are linked to the DRAM groups D and DC quoted by an error message before:
  dmx 03 ERROR: 10_calDramStep29: Dram1 GrpD 29.2.4 byte 97: exp/act 0x06/0x0e (bits: 3) 
  dmx 03 ERROR: 10_calDramStep29: Dram1 GrpDC 29.2.4 byte 99: exp/act 0x06/0x36 (bits: 4 5)
  Hence why the second and third address are always the same. My guess would be that that group D and C are the third and fouth RAM probably, so U470 and U0462.
  
  Does somebody have an opinion on that?
  
• #63 Reply
  Posted by ApertureScience on 30 May, 2021 09:52
• I will also try to reball the DEMUX.
• #64 Reply
  Posted by ApertureScience on 30 May, 2021 10:02
• Regarding the acqUpWriteLong failed error, in the firmware i found the function outputting it. I have no idea what it does! Raw decompiled code ahead
  

  Code: [Select]

  undefined8 acqUpWriteLong(ulong *param_1,ulong param_2,uint param_3,int param_4)

{
  undefined4 uVar1;
  uint uVar2;
  undefined4 in_LR;
 
  uVar2 = CDmxUpXfr::write(*(CDmxUpXfr **)
                            (*(int *)(*(int *)(CDemuxs::m_physicalToLogicalIndexMap + param_4 * 4) *
                                      4 + CDemuxs::m_dmxs) + 0x40),param_1,param_3,param_2);
  uVar1 = 0;
  if (uVar2 != param_3) {
    errPrintf("acqUpWriteLong failed\n");
    uVar1 = 0xffffffff;
  }
  return CONCAT44(uVar1,in_LR);
}

• #65 Reply
  Posted by ApertureScience on 30 May, 2021 10:08
• Also interesting function that tells a bit about the memory layout. Not sure why it seems to show gaps though...
  

  Code: [Select]

  ushort convByteIdxToGrpIdx(uint offset)

{
  int local_20 [7];
 
  local_20[0] = 0x10003;
  local_20[1] = 2;
  local_20[2] = 0x50007;
  local_20[3] = 0x40006;
  return *(ushort *)((int)local_20 + (offset & 7) * 2);
}
  This looks weird, maybe some magic on the address bus. 0x10000 is very close to 2^16, implying some 16bit wide bus.
  
  Or.. I am totally wrong about my assumptions and everything is specified down to the SDRAM architecture to achieve the necessary acess times.
• #66 Reply
  Posted by MarkF3837 on 04 Jan, 2022 01:06
• On my DPO4104, there is an opening in the bottom of the metal shield. Inside is a 2-row, 20-pin 1.27 mm pitch connector. With the 'scope upside down & facing away from you, the second pin from the left in the top row is the TxD pin (data from 'scope). The pin to its right is ground. I haven't searched for the RxD pin.
  
  Signal level is 3.3V. Data rate is 9600 bps.
  

  ---

  
• #67 Reply
  Posted by Skyhi on 30 Jul, 2022 15:42
• Hi all,  just wanted to say I joined this forum because I was looking for pointers on repairing a DPO 4104.
  In my case re-flowing the BGA flash IC's U840, U832, U821 and U820 solved my 'stuck at boot' problem.  I flooded them with flux and used a heater underneath and used a hand air station with a small noozle until I saw them bubble and 're-seat' themselves. 
  
  If it happens again I plan on removing them and re-balling (with leaded solder of course).
  
  Been working great for a while now! 
  Best regards,
  Will

  ---

  

  
• #68 Reply
  Posted by analogRF on 01 Aug, 2022 11:33
• Quote from: Skyhi on 30 Jul, 2022 15:42

  Hi all,  just wanted to say I joined this forum because I was looking for pointers on repairing a DPO 4104.
  In my case re-flowing the BGA flash IC's U840, U832, U821 and U820 solved my 'stuck at boot' problem.  I flooded them with flux and used a heater underneath and used a hand air station with a small noozle until I saw them bubble and 're-seat' themselves. 
  
  If it happens again I plan on removing them and re-balling (with leaded solder of course).
  
  Been working great for a while now! 
  Best regards,
  Will
  

  
  may I ask what temperature/air speed did you use? Did you also put the board on pre-heater?
  This is a huge and thick board, I am very reluctant (read scared  ) to do reflowing
• #69 Reply
  Posted by Skyhi on 01 Aug, 2022 12:16
• I did use a pre-heater underneath the board - it is a fairly small hot air setup and it had it set all the way on for heat.  As for the hand air station the nozzle I used was around 7mm in diameter - I had the heat to max and kept the air flow fairly low to keep things from blowing around.  I also removed the backup battery before I started heating things just to make it a little safer.  When I get back to work on Tuesday I will list the equipment that I used and give you a better idea of the airspeed I used.  I had initially figured out something was up in that area of the board just by using my heat gun at home just on the top of the board while trying to power up the unit and it booted!  But as soon as it cooled off again it would freeze as described.  I don't normally like putting as much heat into the parts as I did but I figured the die in the packages where insulated by the plastic and I really did need to get them hot enough for the solder underneath to liquefy and re-harden to achieve a good connection after the process.
• #70 Reply
  Posted by Jwalling on 02 Aug, 2022 09:03
• Quote from: Skyhi on 01 Aug, 2022 12:16

  I also removed the backup battery before I started heating things just to make it a little safer.
  

  
  Can you test the scope to see if it still gets an IP address when attached to a network? On a DPO3000 I worked on that had a dead battery, the MAC address was lost and couldn't be recovered.
  Thanks.
• #71 Reply
  Posted by Skyhi on 03 Aug, 2022 11:30
• Hmm I was seeing an error (when not connected to my network) as per this image when I booted the scope and I could not seem to set a new IP address but as soon as I plugged it into my router I could update the settings and the connection test passed.

  ---

  
• #72 Reply
  Posted by Skyhi on 03 Aug, 2022 11:35
• This is the equipment that I used for my fix - I think I had the airspeed set to around 3.5 on the hand held hot air station.

  ---

  

  
• #73 Reply
  Posted by analogRF on 03 Aug, 2022 11:41
• Quote from: Jwalling on 02 Aug, 2022 09:03

  Quote from: Skyhi on 01 Aug, 2022 12:16

  I also removed the backup battery before I started heating things just to make it a little safer.
  

  
  Can you test the scope to see if it still gets an IP address when attached to a network? On a DPO3000 I worked on that had a dead battery, the MAC address was lost and couldn't be recovered.
  Thanks.
  

  
  I think, similar to DPO4000, the MAC address is stored in a u-boot environment variable (ethaddr) and thus stored in the flash.
  I dont think it has anything to do with the battery. I think only volatile settings such as IP address or user setups might be stored in the battery back up NVRAM.
• #74 Reply
  Posted by ApertureScience on 26 Apr, 2024 07:37
• After a long break I reinvested some time in my broken scope and a new hot air station. I reflowed U370 and something changed.
  

  Code: [Select]

  

U-Boot 1.1.4 (Jan  8 2007 - 11:12:14) Tektronix, Inc. V1.06

CPU:   AMCC PowerPC 440EP Rev. B at 333.333 MHz (PLB=133, OPB=66, EBC=66 MHz)
       I2C boot EEPROM enabled
       Internal PCI arbiter enabled, PCI async ext clock used
       32 kB I-Cache 32 kB D-Cache
Board: Tektronix Route66 IBM 440EP Main Board
VCO: 666 MHz
CPU: 333 MHz
PLB: 133 MHz
OPB: 66 MHz
EPB: 66 MHz
I2C:   ready
DRAM:  128 MB
FLASH: 64.5 MB
PCI:   Bus Dev VenId DevId Class Int
        00  13  10b5  9056  0680  18
        00  15  1002  4c59  0300  17
DISP:  Type 1
In:    serial
Out:   serial
Err:   serial
Enter password - autobooting in 3 seconds
## Booting image at f0000000 ...
   Image Name:   Linux-2.4.20_mvl31-440ep_eval
   Image Type:   PowerPC Linux Multi-File Image (gzip compressed)
   Data Size:    1441010 Bytes =  1.4 MB
   Load Address: 00000000
   Entry Point:  00000000
   Contents:
   Image 0:  1033953 Bytes = 1009.7 kB
   Image 1:   407042 Bytes = 397.5 kB
   Verifying Checksum ... OK
   Uncompressing Multi-File Image ... OK
cmdline is console=ttyS0,9600 quiet bigphysarea=519 panic=2 root=/dev/mtdblock7 rw mem=131072k
   Loading Ramdisk to 07f2b000, end 07f8e602 ... OK
Checking for firmware update...
No USB mass storage devices found to update from.
Linux 2.4.20_mvl31-440ep_eval V 1.15 Tektronix Route66 Tue Jun 22 15:19:50 PDT 2010
stat of /var/log/dmesg failed: No such file or directory
Warning: loading NiDKEng-1.6 will taint the kernel: non-GPL license - Proprietary
  See [url]http://www.tux.org/lkml/#export-tainted[/url] for information about tainted modules
Warning: loading NiDUsb-1.6 will taint the kernel: non-GPL license - Proprietary
  See [url]http://www.tux.org/lkml/#export-tainted[/url] for information about tainted modules
Warning: loading tek will taint the kernel: non-GPL license - Proprietary
  See [url]http://www.tux.org/lkml/#export-tainted[/url] for information about tainted moScope application starting (normal mode)
---------------------- startScopeApp() running Init code ----------------------
versionBuildFWVersionString(): TimestampString: 25-May-11  15:07   
                               VersionFIRMWAREVERSIONversion: v2.58
                               Major ver num: 2 Minor ver num: 58
     Initializing Mia[0]
     Adg523 detected, patching reg offsets
     Initializing Tek0005[0]
     Initializing Ibm440[0]
     Initializing HFD204ADC[1]
     Initializing HFD204ADC[0]

Model id: 0x02

     Initializing Ltc1658Dac[0]

Board id: 0x00

     Initializing M859[3]
     Initializing M859[2]
     Initializing M859[1]
     Initializing M859[0]
      HFD144[0] ID_REG = 0x00001440
      HFD144[1] ID_REG = 0x00001440
      HFD144[2] ID_REG = 0x00001440
      HFD144[3] ID_REG = 0x00001440
     Initializing Hfd144[0]
     Initializing Hfd144[1]
     Initializing Hfd144[2]
     Initializing Hfd144[3]
Open Max5362 successful.
     Initializing Max5362[0]
     Initializing Ltc1661Dac[1]
     Initializing Ltc1661Dac[0]
 Init ADT7468 and locking.

CANNOT SET FAN PARAMETERS - CONTROLLER LOCKED

Factory Checksum: Stored: 36701, Calculated: 36701 - OK
Spc CheckSum: stored: 24980 calculated: 24980 - OK
Demux initialization

    Dram calibration

dmx 03 ERROR: 06_calDriveStrength:   didn't find good value for DramDqPullUpStrength
ERROR: DramDqPullUp can not pull up.
Bit Fields A - H: (All bytes would be 0xff for a good cal.
    0xff  0xff  0xf7  0xf7  0xff  0xff 0xff 0xff
dmx 03 ERROR: 06_calDriveStrength:   didn't find good value for DramDqPullDownStrength
ERROR: DramDqPullDown can not pull down.
Bit Fields A - H: (All bytes would be 0xff for a good cal.
    0x0  0x0  0x0  0x3  0x0  0x0 0x0 0x0
dmx 03: DramCal FAILED *****
    Dram calibration complete

Dram Calibration results:
-------------------------
dmx 02: PASSED
dmx 03: FAILED *****
 ERROR: 06_calDriveStrength:   didn't find good value for DramDqPullUpStrength

Demux initialization complete
Starting POST diags

Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000002 0x00000002
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000002 0x00000002
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000002 0x00000002
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000002 0x00000002
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000002 0x00000002
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000002 0x00000002
Failure at address 0x0
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000002 0x00000002
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000002 0x00000002
Failure at address 0x10
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000002 0x00000002
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000002 0x00000002
Failure at address 0x20
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000002 0x00000002
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000002 0x00000002
Failure at address 0x30
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000002 0x00000002
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000002 0x00000002
Failure at address 0x40
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000002 0x00000002
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000002 0x00000002
Failure at address 0x50
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000002 0x00000002
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000002 0x00000002
Failure at address 0x60
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000002 0x00000002
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000002 0x00000002
Failure at address 0x70
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000002 0x00000002
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000002 0x00000002
Failure at address 0x80
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000002 0x00000002
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000002 0x00000002
Failure at address 0x90
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000002 0x00000002
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000002 0x00000002
Failure at address 0xa0
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000002 0x00000002
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000001 0x00000001
Diag Failure: Mode 100 4, U370,U480,U0471,U470,U0462, 0x00000000 0x00000002 0x00000002
Failure at address 0xb0
Failure at address 0xc0
Failure at address 0xd0
Failure at address 0xe0
Failure at address 0xf0
dmx 03 ERROR: 06_calDriveStrength:   didn't find good value for DramDqPullUpStrength
ERROR: DramDqPullUp can not pull up.
Bit Fields A - H: (All bytes would be 0xff for a good cal.
    0xff  0xff  0xf7  0xf7  0xff  0xff 0xff 0xff
dmx 03 ERROR: 06_calDriveStrength:   didn't find good value for DramDqPullDownStrength
ERROR: DramDqPullDown can not pull down.
Bit Fields A - H: (All bytes would be 0xff for a good cal.
    0x0  0x0  0x0  0x3  0x0  0x0 0x0 0x0
dmx 03: DramCal FAILED *****

Diag Failure: Mode 100 7, U370,U270, 0x0000ffff 0xffffffff 0xffff0000
Diag Failure: Mode 100 7, U370,U270, 0x00000000 0x0000ffff 0x0000ffff
Diag Failure: Mode 100 7, U370,U270, 0x0000fefe 0xfefeffff 0xfefe0101
Diag Failure: Mode 100 7, U370,U270, 0x00000101 0x0101ffff 0x0101fefe
Diag Failure: Mode 100 7, U370,U270, 0x0000fdfd 0xfdfdffff 0xfdfd0202
Diag Failure: Mode 100 7, U370,U270, 0x00000202 0x0202ffff 0x0202fdfd
Diag Failure: Mode 100 7, U370,U270, 0x0000fbfb 0xfbfbffff 0xfbfb0404
Diag Failure: Mode 100 7, U370,U270, 0x00000404 0x0404ffff 0x0404fbfb
Diag Failure: Mode 100 7, U370,U270, 0x0000f7f7 0xf7f7ffff 0xf7f70808
Diag Failure: Mode 100 7, U370,U270, 0x00000808 0x0808ffff 0x0808f7f7
Diag Failure: Mode 100 7, U370,U270, 0x0000efef 0xefefffff 0xefef1010
Diag Failure: Mode 100 7, U370,U270, 0x00001010 0x1010ffff 0x1010efef
Diag Failure: Mode 100 7, U370,U270, 0x0000dfdf 0xdfdfffff 0xdfdf2020
Diag Failure: Mode 100 7, U370,U270, 0x00002020 0x2020ffff 0x2020dfdf
Diag Failure: Mode 100 7, U370,U270, 0x0000bfbf 0xbfbfffff 0xbfbf4040
Diag Failure: Mode 100 7, U370,U270, 0x00004040 0x4040ffff 0x4040bfbf
Diag Failure: Mode 100 7, U370,U270, 0x00007f7f 0x7f7fffff 0x7f7f8080
Diag Failure: Mode 100 7, U370,U270, 0x00008080 0x8080ffff 0x80807f7f
Diag Failure: Mode 100 7, U370,U270, 0x0000ffff 0xffffffff 0xffff0000
Diag Failure: Mode 100 7, U370,U270, 0x00000000 0x0000ffff 0x0000ffff
Diag Failure: Mode 100 7, U370,U270, 0x0000ffff 0xffffffff 0xffff0000
Diag Failure: Mode 100 7, U370,U270, 0x00000000 0x0000ffff 0x0000ffff
Diag Failure: Mode 100 7, U370,U270, 0x0000ffff 0xffffffff 0xffff0000
Diag Failure: Mode 100 7, U370,U270, 0x00000000 0x0000ffff 0x0000ffff
Diag Failure: Mode 100 7, U370,U270, 0x0000ffff 0xffffffff 0xffff0000
Diag Failure: Mode 100 7, U370,U270, 0x00000000 0x0000ffff 0x0000ffff
Diag Failure: Mode 100 7, U370,U270, 0x0000ffff 0xffffffff 0xffff0000
Diag Failure: Mode 100 7, U370,U270, 0x00000000 0x0000ffff 0x0000ffff
Diag Failure: Mode 100 7, U370,U270, 0x0000ffff 0xffffffff 0xffff0000
Diag Failure: Mode 100 7, U370,U270, 0x00000000 0x0000ffff 0x0000ffff
Diag Failure: Mode 100 7, U370,U270, 0x0000ffff 0xffffffff 0xffff0000
Diag Failure: Mode 100 7, U370,U270, 0x00000000 0x0000ffff 0x0000ffff
Diag Failure: Mode 100 8, U271, calloc failed,    0
Diag Failure: Mode 100 8, U271, calloc failed,    0
Finished POST diags
 Class 2 Level 2  acq error from sysTaskCreate in acqInit 0
 01:11:45 Fri Apr 26 2024 (thread: tUsrRoot)
 Class 2 Acquire Id 99 Instance 0 Severity 2 01:11:45 Fri Apr 26 2024 (thread: tUsrRoot) file errPublic.c line 663 (null)
  errAddErrorLogDataEntry: sysAppState != SYS_STATE_ACTIVE
 Class 2 Acquire Id 4 Instance 0 Severity 1 01:11:45 Fri Apr 26 2024 (thread: tUsrRoot) file acqPublic.c line 483 (null)
  errAddErrorLogDataEntry: sysAppState != SYS_STATE_ACTIVE
  All tasks suspended by errLogDataMsg call from tUsrRoot
 errSuspendAll called from tUsrRoot

Backtrace: (depth of 6 shown)
/usr/local/bin/scopeApp.ppcep(errSuspendAll+0x6c)[0x10c40f74]
/usr/local/bin/scopeApp.ppcep(errLogDataMsg+0x3d0)[0x10c3f8c4]
/usr/local/bin/scopeApp.ppcep(errLogMsg+0x5c)[0x10c3f944]
/usr/local/bin/scopeApp.ppcep(_Z8acqErroriiiiPci+0xbc)[0x1039c790]
/usr/local/bin/scopeApp.ppcep(_Z7acqInitv+0x3b0)[0x1039ba24]
/usr/local/bin/scopeApp.ppcep[0x10c52628]

 Class 22 System Id 0 Instance 22 Severity 2 01:11:45 Fri Apr 26 2024 (thread: tUsrRoot) file sysError.c line 191 (null)
 cmdParmGetIntMsgPoll() error processing cmd for parm 151, clientId: 0, retCode: -1
 
System status:
SheetMemSize=0, TotalNumSheets=0, bus state=-1, xop state=0
MeasAbort=0, MathAbort=0
CurrentTransactionState=-1, CurrentTransactionParm=Invalid parm id
CurrentTransactionType=-1, arg1Type=0, arg2Type=0


Sys tasks and their state:
Name TaskId Pri Pipe Msgs/Max Handler
---- ------ --- ---- -------- -------
ext 0 n/a n/a

 errSuspendAllThread(): could not suspend fpIntTask, retCode = -1
 errSuspendAllThread(): could not suspend fpUnloadCmdQueueThread, retCode = -1
 errSuspendAllThread(): could not suspend fpIrqMonitor, retCode = -1
 errSuspendAllThread(): could not suspend usbHotplug, retCode = -1
 errSuspendAllThread(): could not suspend UsbOutputMgr, retCode = -1
 errSuspendAllThread(): could not suspend piUsb, retCode = -1
 errSuspendAllThread(): could not suspend piVGpib, retCode = -1
 errSuspendAllThread(): could not suspend probesSharedUnloadCmdQueueThread, retCode = -1
 errSuspendAllThread(): could not suspend autoset, retCode = -1
 errSuspendAllThread(): could not suspend cal, retCode = -1
 errSuspendAllThread(): could not suspend diag, retCode = -1
 errSuspendAllThread(): could not suspend fp, retCode = -1
 errSuspendAllThread(): could not suspend hc, retCode = -1
 errSuspendAllThread(): could not suspend UsbSicInputMsgMgr, retCode = -1
 errSuspendAllThread(): could not suspend wfmMgrTest, retCode = -1
 errSuspendAllThread(): could not suspend search, retCode = -1
 errSuspendAllThread(): could not suspend periodicZoom, retCode = -1
 errSuspendAllThread(): could not suspend periodicClockAnimation, retCode = -1
 errSuspendAllThread(): could not suspend periodicBusyIndicAnimation, retCode = -1
 errSuspendAllThread(): could not suspend math, retCode = -1
 errSuspendAllThread(): could not suspend meas, retCode = -1
 errSuspendAllThread(): could not suspend measImmed, retCode = -1
 errSuspendAllThread(): could not suspend piCmdIntfc, retCode = -1
 errSuspendAllThread(): could not suspend ref, retCode = -1
 errSuspendAllThread(): could not suspend rtl, retCode = -1
 errSuspendAllThread(): could not suspend tHttpd, retCode = -1
 errSuspendAllThread(): could not suspend tVxi11SRQd, retCode = -1
 errSuspendAllThread(): could not suspend tVxi11Rpcd, retCode = -1
 errSuspendAllThread(): could not suspend bus, retCode = -1
 errSuspendAllThread(): could not suspend mitlRb, retCode = -1
 errSuspendAllThread(): could not suspend VgpibRead, retCode = -1
 errSuspendAllThread(): could not suspend VgpibWrite, retCode = -1
 errSuspendAllThread(): could not suspend tSocketServerService, retCode = -1
 errSuspendAllThread(): could not suspend tSocketResponseThread, retCode = -1
 errSuspendAllThread(): could not suspend UsbTmcEventDispatcher, retCode = -1
 errSuspendAllThread(): could not suspend inputOverload, retCode = -1
 errSuspendAllThread(): all threads in list suspended
 errSuspendAll(): TASK tUsrRoot SHOULD NOT BE HERE!!!