Intel Atom C2000 Failures

[ericloewe]

Hi everyone! greetings from Spain.
I have a DELL S4048-ON switch crashed after shutdown and reboot. I think I'm affected by this Clockgate from Intel, I cannot get any data via serial console and switch try to boot in a loop. I search all the web and I saw same problem on this model (2015 factory date).
I was wondering if anyone has S4048-ON's schematics so I could find LPC_CLKOUT on the mainboard. I see a connection pinout near microprocessor (pic attached), probably one of those pins is LPC_CLKOUT.

These switches have an ASPeed BMC, don't they? Look for a daughterboard with an American Megatrands [sic] MegaRAC sticker. The BMC connects via LPC (and USB and PCIe) to the host, so that would at least narrow down your search a bit.

[pat0te]

Stick a scope probe to it?

Unfortunaly, I don't have any scope.

[pat0te]

Hi everyone! greetings from Spain.
I have a DELL S4048-ON switch crashed after shutdown and reboot. I think I'm affected by this Clockgate from Intel, I cannot get any data via serial console and switch try to boot in a loop. I search all the web and I saw same problem on this model (2015 factory date).
I was wondering if anyone has S4048-ON's schematics so I could find LPC_CLKOUT on the mainboard. I see a connection pinout near microprocessor (pic attached), probably one of those pins is LPC_CLKOUT.

These switches have an ASPeed BMC, don't they? Look for a daughterboard with an American Megatrands [sic] MegaRAC sticker. The BMC connects via LPC (and USB and PCIe) to the host, so that would at least narrow down your search a bit.

I think this switch has not BMC, but I'm going to take a look. Thanks for your suggestion.

[pat0te]

Hi everyone! greetings from Spain.
I have a DELL S4048-ON switch crashed after shutdown and reboot. I think I'm affected by this Clockgate from Intel, I cannot get any data via serial console and switch try to boot in a loop. I search all the web and I saw same problem on this model (2015 factory date).
I was wondering if anyone has S4048-ON's schematics so I could find LPC_CLKOUT on the mainboard. I see a connection pinout near microprocessor (pic attached), probably one of those pins is LPC_CLKOUT.

These switches have an ASPeed BMC, don't they? Look for a daughterboard with an American Megatrands [sic] MegaRAC sticker. The BMC connects via LPC (and USB and PCIe) to the host, so that would at least narrow down your search a bit.

Zero chips or stickers found with an American Megatrends / MegaRAC ::sadface::

[ericloewe]

Ok, further digging and a series of fortuitously timed posts by a few people led to further useful info about Supermicro A1SAi/A1SRi boards, and likely other vendors' board too:

Well, the gist of it is that there are two clock lines, one feeds the TPM header and is involved in the boot ROM selection; the other runs to the BMC. This is why fixing the boot failures often still left the BMC unresponsive to the LPC bus, it just wasn't getting any clocks.

Supermicro's fix for this is a pair of 150 Ohm bodge resistors, one to pull up each of these two clock lines to 3.3 V. The fix is actually pretty neat on these boards and could pass casual inspection, depending on how well the soldering turns out.

[Sacodepatatas]

Unfortunaly, I don't have any scope.

You can buy a DIY 50MHz Frequency counter for less than 5€ from AliExpress/eBay. Or you can build a temporary one by yourself if you've got a PIC16F628A and some led displays in your spare box.

Edit: Or one of those EZ-USB FX2LP boards that can be programmed as a logic tracer and costs about the same price.

[wraper]

Unfortunaly, I don't have any scope.

You can buy a DIY 50MHz Frequency counter for less than 5€ from AliExpress/eBay. Or you can build a temporary one by yourself if you've got a PIC16F628A and some led displays in your spare box.

Edit: Or one of those EZ-USB FX2LP boards that can be programmed as a logic tracer and costs about the same price.

And they will be totally useless for the job.

[Sacodepatatas]

Why? I have used such frequency meter for monitoring the stability of 32MHz clock signals, I don't see why a 25MHz Signal can't be counted with these. Maybe related to the logic voltaje levels, but nothing that couldn't be solved quickly. The most simple manner that I think for watching if there is a clock signal at any port, without any equipment at all but a multimeter, is just a high pass filter followed by a peak detector (a schottky, a capacitor  and a high value resistor) and then measure the voltage level at the output.

[wraper]

Why? I have used such frequency meter for monitoring the stability of 32MHz clock signals, I don't see why a 25MHz Signal can't be counted with these. Maybe related to the logic voltaje levels, but nothing that couldn't be solved quickly. The most simple manner that I think for watching if there is a clock signal at any port, without any equipment at all but a multimeter, is just a high pass filter followed by a peak detector (a schottky, a capacitor  and a high value resistor) and then measure the voltage level at the output.

Because there is no signal at all or it's extremely weak once it fails.

[pat0te]

Ok, further digging and a series of fortuitously timed posts by a few people led to further useful info about Supermicro A1SAi/A1SRi boards, and likely other vendors' board too:

Well, the gist of it is that there are two clock lines, one feeds the TPM header and is involved in the boot ROM selection; the other runs to the BMC. This is why fixing the boot failures often still left the BMC unresponsive to the LPC bus, it just wasn't getting any clocks.

Supermicro's fix for this is a pair of 150 Ohm bodge resistors, one to pull up each of these two clock lines to 3.3 V. The fix is actually pretty neat on these boards and could pass casual inspection, depending on how well the soldering turns out.

I saw this repair on Internet, but Dell S4048 motherboard is very different. There is TPM header near microprocessor but only with 1 line of pins.

Video for Supermicros:

[ericloewe]

Ok, further digging and a series of fortuitously timed posts by a few people led to further useful info about Supermicro A1SAi/A1SRi boards, and likely other vendors' board too:

Well, the gist of it is that there are two clock lines, one feeds the TPM header and is involved in the boot ROM selection; the other runs to the BMC. This is why fixing the boot failures often still left the BMC unresponsive to the LPC bus, it just wasn't getting any clocks.

Supermicro's fix for this is a pair of 150 Ohm bodge resistors, one to pull up each of these two clock lines to 3.3 V. The fix is actually pretty neat on these boards and could pass casual inspection, depending on how well the soldering turns out.

I saw this repair on Internet, but Dell S4048 motherboard is very different. There is TPM header near microprocessor but only with 1 line of pins.

Are you sure it's for a TPM? Do you have any documents saying that?

If so, my suggestion is that you figure out which of the pins are not power pins. From there, try pulling up each of them to 3.3 V (one at a time!) until the thing boots. Once it boots, you'll have identified your clock pin and can do a more permanent fix.

Pictures are also welcome for the next person to come along with such a dead switch.

[pat0te]

Ok, further digging and a series of fortuitously timed posts by a few people led to further useful info about Supermicro A1SAi/A1SRi boards, and likely other vendors' board too:

Well, the gist of it is that there are two clock lines, one feeds the TPM header and is involved in the boot ROM selection; the other runs to the BMC. This is why fixing the boot failures often still left the BMC unresponsive to the LPC bus, it just wasn't getting any clocks.

Supermicro's fix for this is a pair of 150 Ohm bodge resistors, one to pull up each of these two clock lines to 3.3 V. The fix is actually pretty neat on these boards and could pass casual inspection, depending on how well the soldering turns out.

I saw this repair on Internet, but Dell S4048 motherboard is very different. There is TPM header near microprocessor but only with 1 line of pins.

Are you sure it's for a TPM? Do you have any documents saying that?

If so, my suggestion is that you figure out which of the pins are not power pins. From there, try pulling up each of them to 3.3 V (one at a time!) until the thing boots. Once it boots, you'll have identified your clock pin and can do a more permanent fix.

Pictures are also welcome for the next person to come along with such a dead switch.

Well, no, I'm not sure about TPM. I don´t have any documentation, that's the problem. I posted this pic before regarding TPM style connector:
https://www.eevblog.com/forum/index.php?action=dlattach;topic=95943.0;attach=1713377;image

[charly]

Just registered (a programmer here, my electronics skills are below zero ) to say thanks for this thread and keep it going for anybody still around (this hardware is way more than what many NAS boxes offer today).
I've had a 2550F since 2014 and it died on me like a year ago (using it with FreeNAS at home). Tried everything, it just turned some lights on the motherboard, but no video, no nothing. I even distrusted the power supply. Now this bridge that I made with audio friends help made it back to life! I didn't want to throw away this beauty even if it just works for a few extra months.
Thanks a lot!!

[Gil]

Where is the pin located on the 251+ ??  many thanks

[JakubPL]

Hey. I managed to (at least I think I did) revive a Dell S4048-ON. It took some effort and I had to sacrifice one motherboard.

But to the point:

You need to solder a wire to the R389 resistor and add a resistor (I used a 330 Ohm one) and connect it to the pins near the CPU where there's 3.3V.

So far, I've been testing it by repeatedly unplugging it and resetting it, and it's working fine.

I hope this helps someone. 

[uski]

Supposedly the Western Digital DL2100, DL4100, and DX4200 are also susceptible to these catastrophic disasters.
However Western Digital has apparently taken the attitude that they are not going to do anything other than let the chips fall where they may and people can use their warranty protection (or not).
https://translate.google.com/translate?sl=nl&tl=en&u=https%3A%2F%2Fnl.hardware.info%2Fnieuws%2F51213%2Fwestern-digital-geeft-statement-over-c2000-defecten-onderneemt-geen-actie

However, I do have a dead DL4100 and I'm game to try wiring in a 110 ohm resistor between +3.3V and LPC-CLKOUT0/1. Why not, the box is dead anyway.
However some smart wag said that he heard there is no pin connection from anywhere on the motherboard to either of the LPC-CLKOUT pins. Would us DX4200/DL2100/DL4100 owners therefore consequently all be SOL?
Rats.

Hi,

Chiming here on the DL4100. I had one with this failure, there are dozens of posts on Internet with users facing the same problem (blank screen, boot loop every few minutes).

I spent a lot of time trying to make it work. I looked on the PCB if I could find any 25MHz clock that I could pull up, painstakingly looking at every accessible via. Initially I thought I saved it, because, I identified two vias with a ~0.5V 25MHz clock. Pulling it up through a 100 ohms resistor made the signals 0-3.3v as expected. However, it made the symptoms even worse... so these were not the LPC clocks.

After a LOT of digging on the internetz I found this forum post:
https://community.wd.com/t/how-to-boot-from-a-usb-stick-dl2100/247447/21

The user says:

I looked at one of my DL4100 boards (4060-705194-000) and I can establish the offset needed, but looking at this under x-ray, this looks to be a 10 layer board and aside from the expected power and gnd planes, there are also traces passing directly under the place where you need to drill. This is for LPC_CLK_0. There is also an LPC_CLK_1. I don’t know if that one is needed to be pulled up too. I doubt it considering it wasn’t needed for the similar Synology NAS fix.

So, for this board, there isn’t any way that I can see to connect a pull-up resistor to LPC_CLK_0 short of desoldering the CPU itself and somehow connecting a rework wire to the pad on the board and then re-installing the CPU.

If that user is to be believed, and there's no reason not to - looks like the DL4100 cannot be fixed in any practical way, because the LPC_CLK0 signal doesn't go anywhere accessible externally.

Not the outcome I expected, but I hope this will save time if someone else tries to revive a DL4100 - I'd say, don´t try. To save the data, I bought a 4-bay USB non-RAID enclosure and connected it to a computer running Linux. I could then reconstruct the RAID array and mount it and copy the data elsewhere...

Not happy with WD here for not doing anything to help with the issue. I intend to avoid buying any WD product in the future.

[artyfarty]

Ok, further digging and a series of fortuitously timed posts by a few people led to further useful info about Supermicro A1SAi/A1SRi boards, and likely other vendors' board too:

Well, the gist of it is that there are two clock lines, one feeds the TPM header and is involved in the boot ROM selection; the other runs to the BMC. This is why fixing the boot failures often still left the BMC unresponsive to the LPC bus, it just wasn't getting any clocks.

Supermicro's fix for this is a pair of 150 Ohm bodge resistors, one to pull up each of these two clock lines to 3.3 V. The fix is actually pretty neat on these boards and could pass casual inspection, depending on how well the soldering turns out.

Thanks, stranger from the internet. I've just revived a board that served me for 10 years before going out by replicating this picture. Soldering 0402s was not fun though.

[dtsystems]

Well done to those who've managed to revive their dead systems - how's it lasting?
Just got another failure after I forgot about this issue for a few years.

FreePBX (Centos)

System has run since 2018.

[Wed May 20 07:11:25 2026] NMI: IOCK error (debug interrupt?) for reason 60 on CPU 0.
[Wed May 20 07:11:25 2026] CPU: 0 PID: 0 Comm: swapper/0 Kdump: loaded Tainted: G           OE  ------------   3.10.0-1127.19.1.el7.x86_64 #1
[Wed May 20 07:11:25 2026] Hardware name: MSI MS-7877/J1900I, BIOS V1.5 01/29/2015
[Wed May 20 07:11:25 2026] task: ffffffff89018480 ti: ffffffff89000000 task.ti: ffffffff89000000
[Wed May 20 07:11:25 2026] RIP: 0010:[<ffffffff88b9358c>]  [<ffffffff88b9358c>] irq_entries_start+0x4c/0x400
[Wed May 20 07:11:25 2026] RSP: 0018:ffffffff89003e38  EFLAGS: 00000002
[Wed May 20 07:11:25 2026] RAX: 005e5db77cf7b46d RBX: ffff9fb83fc23608 RCX: 0000000000000017
[Wed May 20 07:11:25 2026] RDX: 0000000225c17d03 RSI: ffffffff89003fd8 RDI: 005e5db77cf7b46d
[Wed May 20 07:11:25 2026] RBP: ffffffff89003e90 R08: 0000000000001ecd R09: ffff9fb83fc19cd8
[Wed May 20 07:11:25 2026] R10: 7fffffffffffffff R11: 7fffffffffffffff R12: 0000000000000001
[Wed May 20 07:11:25 2026] R13: 005e5db77cf78911 R14: 0000000000000001 R15: ffffffff890de6f0
[Wed May 20 07:11:25 2026] FS:  0000000000000000(0000) GS:ffff9fb83fc00000(0000) knlGS:0000000000000000
[Wed May 20 07:11:25 2026] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[Wed May 20 07:11:25 2026] CR2: 000026f14d1c2000 CR3: 000000000c410000 CR4: 00000000001007f0
[Wed May 20 07:11:25 2026] Call Trace:
[Wed May 20 07:11:25 2026]  [<ffffffff889c6227>] ? cpuidle_enter_state+0x57/0xd0
[Wed May 20 07:11:25 2026]  [<ffffffff889c621d>] ? cpuidle_enter_state+0x4d/0xd0
[Wed May 20 07:11:25 2026]  [<ffffffff889c637e>] cpuidle_idle_call+0xde/0x230
[Wed May 20 07:11:25 2026]  [<ffffffff88437c8e>] arch_cpu_idle+0xe/0xc0
[Wed May 20 07:11:25 2026]  [<ffffffff88501c7a>] cpu_startup_entry+0x14a/0x1e0
[Wed May 20 07:11:25 2026]  [<ffffffff88b6e6a7>] rest_init+0x77/0x80
[Wed May 20 07:11:25 2026]  [<ffffffff8918b1cf>] start_kernel+0x44b/0x46c
[Wed May 20 07:11:25 2026]  [<ffffffff8918ab84>] ? repair_env_string+0x5c/0x5c
[Wed May 20 07:11:25 2026]  [<ffffffff8918a120>] ? early_idt_handler_array+0x120/0x120
[Wed May 20 07:11:25 2026]  [<ffffffff8918a738>] x86_64_start_reservations+0x24/0x26
[Wed May 20 07:11:25 2026]  [<ffffffff8918a88e>] x86_64_start_kernel+0x154/0x177
[Wed May 20 07:11:25 2026]  [<ffffffff884000d5>] start_cpu+0x5/0x14
[Wed May 20 07:11:25 2026] Code: 58 eb 16 6a 57 eb 12 6a 56 eb 0e 6a 55 eb 0a 6a 54 eb 06 6a 53 eb 02 6a 52 e9 81 5c ff ff 90 6a 51 eb 16 6a 50 eb 12 6a 4f eb 0e <6a> 4e eb 0a 6a 4d eb 06 6a 4c eb 02 6a 4b e9 61 5c ff ff 90 6a

It's still running / processing calls. Will replace motherboard ASAP. AFAIK it's another Gigabyte J1900-D2H.