8bit CPU project had problems - ***SOLVED***

[PauloConstantino]

Hi there. I have built a CPU on breadboards, and I been testing the design to see how everything works. All was working fine until today I had a strange bug.

I am still using a manual clock to test each instruction. A problem happened today that if I try to execute the instructions "too quick" with the manual clock, that is, maybe 2 cycles per second, then the instructions start to go crazy. If I go slowly with the manual clock, they execute fine. I have a few bypass capacitors here and there (100nF), but not at every IC or ROMS/RAMS. What could this problem be? Could it be that I need one bypass capacitor per IC? Could this be voltage swing problems? My power supply is a good one.

Any ideas?


********Problem SOLVED********

[Zero999]

Of course you need decoupling capacitors with a low frequency clock, which will still have fast rise/fall times, thus still have high frequency components in the multi-GHz range.

[PauloConstantino]

Why does it work when I go very slowly in the manual clock then? The difference is 2 cycles per second between working and not.

[tautech]

Scope the power pin on the IC and its output too. 

[DaJMasta]

Could there be some questionable grounding that's making what logic high and low really are?  Maybe floating inputs?

Maybe there's a big cap on a data pin by mistake or an output that has too much current draw and is being pulled down to be ambiguous?

It's an odd problem, but if I'm remembering right and this is the CPU core built up on solderless breadboards.... it would be VERY easy for a loose connection or a mismatched wire somewhere in there to create this kind of problem.

[mikerj]

Why does it work when I go very slowly in the manual clock then? The difference is 2 cycles per second between working and not.

How are you generating this "manual" clock?

[PauloConstantino]

Why does it work when I go very slowly in the manual clock then? The difference is 2 cycles per second between working and not.

How are you generating this "manual" clock?

A set/reset latch, why ?

[PauloConstantino]

Could there be some questionable grounding that's making what logic high and low really are?  Maybe floating inputs?

Maybe there's a big cap on a data pin by mistake or an output that has too much current draw and is being pulled down to be ambiguous?

It's an odd problem, but if I'm remembering right and this is the CPU core built up on solderless breadboards.... it would be VERY easy for a loose connection or a mismatched wire somewhere in there to create this kind of problem.

It's odd because when I go very slowly on the clock it works, but if I accelerate a little it bugs. There's no loose wire. No cap on data pins. I am going nuts here

[MK14]

Could there be some questionable grounding that's making what logic high and low really are?  Maybe floating inputs?

Maybe there's a big cap on a data pin by mistake or an output that has too much current draw and is being pulled down to be ambiguous?

It's an odd problem, but if I'm remembering right and this is the CPU core built up on solderless breadboards.... it would be VERY easy for a loose connection or a mismatched wire somewhere in there to create this kind of problem.

It's odd because when I go very slowly on the clock it works, but if I accelerate a little it bugs. There's no loose wire. No cap on data pins. I am going nuts here

Have you tried to build something, way, way too big for breadboards (in at least some peoples opinion) ?

Have you got any unconnected inputs ?

Have you mixed logic types (e.g. LS and HC), without regard to compatibility of input threshold voltages and drive current (i.e. fan in and fan out, between the relevant TTL series) requirements ?

Have you missed out any decoupling capacitors and/or implemented them poorly ?

Are you using breadboards, despite their huge stray capacitance, potential cross-interference, unsuitability for higher frequencies and potential loose wires/connections issues ?

Are any switches (and similar) suitably debounced (where necessary, such as when going to counters) ?

Are you using a suitable quality of power supply, at the correct voltage ?

Are any of your displays or LEDs, overloading any TTL gates ?

If you have broken any of the above and/or many other decent design rules for using TTL. Then I'm not at all surprised if it is unreliable and you get odd results, at times.

[Ian.M]

This one?

I'm surprised it works at all.   Pre-made jumpers like that are almost invariably absolutely crappy (see https://en.wikipedia.org/wiki/The_Market_for_Lemons).   The length of the jumpers is adding quite a bit of inductance. Also the inductive coupling between the power jumpers and nearby signal jumpers of a similar orientation is probably enough to cause significant glitches.

[mikerj]

Why does it work when I go very slowly in the manual clock then? The difference is 2 cycles per second between working and not.

How are you generating this "manual" clock?

A set/reset latch, why ?

Because if the clock wasn't clean due to contact bounce it could cause issues, but a latch should prevent this.

You should fixed the lack of bypass caps before you go any further though.  Even if they don't turn out to be causing this issue it will cause problems at some point.

[PauloConstantino]

Why does it work when I go very slowly in the manual clock then? The difference is 2 cycles per second between working and not.

How are you generating this "manual" clock?

A set/reset latch, why ?

Because if the clock wasn't clean due to contact bounce it could cause issues, but a latch should prevent this.

You should fixed the lack of bypass caps before you go any further though.  Even if they don't turn out to be causing this issue it will cause problems at some point.

I added caps everywhere but it still bugs I will do some debugging later today.

[PauloConstantino]

This one?

I'm surprised it works at all.   Pre-made jumpers like that are almost invariably absolutely crappy (see https://en.wikipedia.org/wiki/The_Market_for_Lemons).   The length of the jumpers is adding quite a bit of inductance. Also the inductive coupling between the power jumpers and nearby signal jumpers of a similar orientation is probably enough to cause significant glitches.

Inductance has nothing to do with why it isn't working. It's something else.

[Ian.M]

You *should* be using jumpers that are cut to length from insulated tinned solid copper wire, and formed to fit neatly, following the shortest route that doesn't block access to the chips for testing.  take a look at http://hackaday.com/2017/04/10/8-bit-breadboard-computer-is-up-to-8-hours/ for inspiration.

Adding decoupling (0.1uF disc ceramic) directly across each chip's power and ground pins is likely to reduce the incidence of unexpected behaviour by keeping the transient switching currents localised, but you also need a low impedance ground connection for *EVERY* logic chip to minimise ground bounce.   For a breadboard project of this scale, to get that essential low impedance ground, I'd probably construct a grid of 2.5mm² bare copper wire, and connect each end of each ground bus strip to the ground grid using short tails of 22 AWG tinned copper wire wrapped round and soldered to the nearest ground grid then plugged into the bus strip.  Once you've got a good ground, power distribution is less critical, as long as you minimise loop area, run individual feeds to boards with higher current consumption, and use between 10uF and 100uF bulk decoupling between the power rail and ground on each board. . . .

Inductance has nothing to do with why it isn't working. It's something else.

You have no way of proving or demonstrating that inductive coupling isn't causing problems.  Of course, if there is a bad contact somewhere, capacitive coupling could also be an issue.

A glitch as short as 5ns can be expected to cause unwanted clocking of a 74HC latch or flipflop.  As there is no effective ground plane, there is no way of checking signal integrity without an expensive compact differential probe, and a high bandwidth deep memory DSO.

[PauloConstantino]

You *should* be using jumpers that are cut to length from insulated tinned solid copper wire, and formed to fit neatly, following the shortest route that doesn't block access to the chips for testing.  take a look at http://hackaday.com/2017/04/10/8-bit-breadboard-computer-is-up-to-8-hours/ for inspiration.

Adding decoupling (0.1uF disc ceramic) directly across each chip's power and ground pins is likely to reduce the incidence of unexpected behaviour by keeping the transient switching currents localised, but you also need a low impedance ground connection for *EVERY* logic chip to minimise ground bounce.   For a breadboard project of this scale, to get that essential low impedance ground, I'd probably construct a grid of 2.5mm² bare copper wire, and connect each end of each ground bus strip to the ground grid using short tails of 22 AWG tinned copper wire wrapped round and soldered to the nearest ground grid then plugged into the bus strip.  Once you've got a good ground, power distribution is less critical, as long as you minimise loop area, run individual feeds to boards with higher current consumption, and use between 10uF and 100uF bulk decoupling between the power rail and ground on each board. . . .

Inductance has nothing to do with why it isn't working. It's something else.

You have no way of proving or demonstrating that inductive coupling isn't causing problems.  Of course, if there is a bad contact somewhere, capacitive coupling could also be an issue.

A glitch as short as 5ns can be expected to cause unwanted clocking of a 74HC latch or flipflop.  As there is no effective ground plane, there is no way of checking signal integrity without an expensive compact differential probe, and a high bandwidth deep memory DSO.

Thank you. I remember once I had a problem using breadboards and jumper wires, and I posted on a forum asking for help. I got dozens of people telling me that the reason my circuit was failing was because of breadboards and jumper wires, and that it would never work. The problem had nothing to do with inductance or jumpers or breadboard capacitance etc. People are quick to preach about all these issues when the problem is actually something else. I am skeptical the design is failing for these reasons. I have only a few clock wires because my design is "robust". All my signals are level signals and they can't cause unexpected clocking... It must be something else, maybe a loose contact who knows. I appreciate the help but when people start preaching about soldering, ground planes and etc for this kinda project I am skeptical.

As for the ben eater computer, his computer is the very simple sap1. My cpu is quite different and much more complex.

[Ian.M]

The issue with 'birdsnest' wiring like that, is that it make it very very difficult to check the integrity of each signal at each chip.   

Consider that hypothetical 5ns glitch I mentioned earlier - if you treat it as a half-cycle of a sinewave, its fundamental frequency is 100MHz, and a quarterwave is about 750 mm.   A wire of 10% of that length - 75mm - is about as long as can be tolerated without matched termination.  You'll need a 300MHz or better scope and probe to see a 5ns glitch, and without a good (local) ground and an active probe, the additional loading of the probe will be likely to either suppress the glitch or introduce one that wasn't there previously.

[Siwastaja]

If you're surprised it works at all then you really know nothing so why are you giving advise? I don't welcome these kinds of rude comments.

Unfortunately, he's right, and not being rude.

The reason he's replying "without knowing anything" is simple: no one in the world has the answer to your exact question.

A circuit like this has orders of magnitudes too many unknown variables, and the operation is so chaotic that it will be seemingly random. Finding exact physical mechanisms at play, and the exact place of the failure, to answer the question "why this works/doesn't work", could take thousands of man*years, and I'm not exaggerating.

This is why we have basic engineering practices, to keep impossible variables down, to be able to focus our effort to where it comes into fruition.

Therefore, you really need to start from these basic lines:

- Eliminate solderless breadboard
- Eliminate "dupont" wire connectors

These are both just OK for prototyping small things, since it appears that both have reliability rate of about 99% per contact. But as you can see, it goes exponentially down with increasing number of contacts. Parasitic L,C circuit elements are another issue with this style, and it's not going to work well with binary signals without serious edge rate limitation.

Then, you really need to implement proper power supply bypassing. Without it, things can be very chaotic and seemingly random.

Yes, it really is very surprising it seemingly seems to work at all at some operating point.

No one can give you an answer why (on what physical principle) it fails on another operating point. There are too many places it can and will go wrong.

[PauloConstantino]

If you're surprised it works at all then you really know nothing so why are you giving advise? I don't welcome these kinds of rude comments.

Unfortunately, he's right, and not being rude.

The reason he's replying "without knowing anything" is simple: no one in the world has the answer to your exact question.

A circuit like this has orders of magnitudes too many unknown variables, and the operation is so chaotic that it will be seemingly random. Finding exact physical mechanisms at play, and the exact place of the failure, to answer the question "why this works/doesn't work", could take thousands of man*years, and I'm not exaggerating.

This is why we have basic engineering practices, to keep impossible variables down, to be able to focus our effort to where it comes into fruition.

Therefore, you really need to start from these basic lines:

- Eliminate solderless breadboard
- Eliminate "dupont" wire connectors

These are both just OK for prototyping small things, since it appears that both have reliability rate of about 99% per contact. But as you can see, it goes exponentially down with increasing number of contacts. Parasitic L,C circuit elements are another issue with this style, and it's not going to work well with binary signals without serious edge rate limitation.

Then, you really need to implement proper power supply bypassing. Without it, things can be very chaotic and seemingly random.

Yes, it really is very surprising it seemingly seems to work at all at some operating point.

No one can give you an answer why (on what physical principle) it fails on another operating point. There are too many places it can and will go wrong.

Well the only person who can really find the problem is myself. Probing the birds nest is a nightmare. Theres no prob in the logic since it works at low frequencies.

Do I need termination resistors maybe? But if so, what kind of signals require it? Edge signals or level or both ?

[DaJMasta]

It working below 2Hz does not guarantee that the logic is fine - normally even with a fair amount of stray parasitics and a fairly large layout, hundreds of kHz should be possible with modern logic families.  We're not saying that breadboards aren't suitable to this type of prototyping because we're trying to be mean or because we have a problem with them, it's because there are real signal integrity issues.

At this frequency, I'm willing to effectively ignore inductance and capacitance - a few dozen nH on every line and 15pF extra on every breadboard point aren't going to be the difference between 2Hz and 10Hz - but you can't ignore the connection issues.  Even if you build it up and it works fine, because all of the connections are loose and susceptible to oxidization and whatnot, there is a very real potential for loose connections causing intermittent issues or other problems.  As you've experienced, it's also extremely hard to troubleshoot because of how many things are in the way of you finding the spot you want to probe.  I recently brought a simple project from storage and did a little update programming and I found that the headers on the board that were joined with these individual ebay connectors were dirty enough to significantly effect ADC readings on those pins, and that just by touching the cables I could get half a volt swing intermittently or more.

These sorts of problems are common when you don't have things fully stuck down (soldered, crimped, screw terminals, etc.) and can cause the gamut of problems.  Because everything is setup in a prototyping stage, that also means that it's extremely possible for a connection to be accidentally pulled out or replaced one row over... so even if it were working fine earlier, it's not guaranteed to continue that way.


Probing power lines is a decent place to start, but instead of just saying that there isn't a problem with X, I would go back and just recheck the things suggested because even if they were fine once, they may not be now.  If it were me, I would start with making sure the ground connections are solid and low impedance, then I would check to ground any floating inputs, then i would check the power rail integrity, and then I would check for loose connections by just sort of wiggling things... but there are plenty of valid places to start and check.  It will simplify things greatly if you can move some of it onto a permanently wired board - a couple of breadboards worth of circuit moved onto a permanent board takes out dozens, if not more, points for potential intermittent and unexpected failures due to solderless breadboards and flying cables.

By the way, you initially suspected bypass caps... if they're the cause, then you should be able to see a lot of noise on the power rails at the chip - a basic measurement from a scope should give you an idea if that's a significant portion of the problem.

[jonovid]

This one?

I'm surprised it works at all.   Pre-made jumpers like that are almost invariably absolutely crappy (see https://en.wikipedia.org/wiki/The_Market_for_Lemons).   The length of the jumpers is adding quite a bit of inductance. Also the inductive coupling between the power jumpers and nearby signal jumpers of a similar orientation is probably enough to cause significant glitches.

OMG take it one step at a time  , get some of it on to a pcb.    too many jumpers

[Ian.M]

If the circuit is only TTL, I would agree there is no way that you could get the magnitude time constant needed to discriminate between a 1Hz and 2Hz clock, but the combination of the very low quiescent and input currents of SSI CMOS logic, 100nF decoupling caps and a very high resistance connection, either to an input of a gate that is missing its Vdd connection so is being powered parasitically through its internal protection diodes, or between the bus bar and the strip its Vdd pin and decoupling cap are in, could result in a large enough time constant for the observed effect.

Solderless breadboards aren't *that* evil but you do have to keep your design neat, modular and testable, which means the 'birds-nest' wiring *MUST* be minimised.  You need to be able to verify good continuity between each component leg and the contact strip its in, and through each jumper so should ideally have a spare unused hole in each strip (which may not be possible when working with 0.6" wide ICs).

At least one logic probe capable of distinguishing between high, low and hi-Z with appropriate threshold levels and capable of detecting narrow pulses is *ESSENTIAL*.  Scopes and DMMs are poor substitutes as their input impedance can pull floating CMOS inputs low enough to be seen as logic '0'.  Many affordable logic analysers cannot reliably detect hi-Z signals, though a 16 channel logic analyser + a DIP16 test clip (for ease of access to all the pins of a 14 or 16 pin logic chip) is still a very useful thing to have.

[ebclr]

If you want to increase the clock the 1st thing you need to do is to cut that spider web of jumpers you have.

Buy a standard PCB like one of those


Using wirewrap wire mount using the shortest possible connections

A copper plate ground plane will also help.

A second path is to make the same circuit on an FPGA using the same schematics, this way you get really faster clocks

[klaff]

I can say with certainty that circuit would never work for long in my house.  I would wake up to find a few jumpers laying in the floor, and the cat grinning at me.

[rstofer]

One 100 nF capacitor per chip mounted as close to the pins as possible.  In the old days, we could buy IC wire-wrap sockets with integral capacitors.

http://www.arieselec.com/products/data/12027-open-frame-capacitor-collet-socket.htm

There are others...

[rstofer]

You mentioned that the clock didn't go very many places.  Is it safe to assume that all the registers are clocked?  If you are trying to use R-S flops, you may be in for a ride.  There's a reason that FPGAs implement synchronous logic and highly discourage asynchronous logic.