I've played with the arduino and raspberry pi and I didn't find them very challenging, with these devices it largely came down to programming and that's not really what I'm interested in learning. What I want to learn are things like how digital electronics work, how a CPU works, how memory works, how to interface with memory and perform operations on the bits stored in it, how buses work and how data is transferred around the system, how you assemble bits into higher ordered data structures, how to program in machine code and assembly language, how bootloaders work, and last but not least why operating systems are designed the way they are. Basically I want to learn in great detail how you get from electrons to a functional computer.
That sounds great; I wish more people had that understanding.
You could get a good understanding of most of those by starting at the atmega328 chip level and initialising all the hardware yourself. Get an Atmel AVR Dragon, which will allow you to program and debug atmega328 devices on their own and in whatever board and system you design.
You need to pick a suitable project, one which is within your scope but will stretch your capabilities. An example of that would be that recently I wanted to make a date/time/pill reminder for my parents that would run for months on a set of small batteries. Arduinos would have used too much power, so I started with the atmega328, programmed it in the Dragon, and then designed and constructed the board containing the MCU, battery and peripherals (switches and an LCD display).
That is a suitable level for a hardware beginner - even with something that simple you will find that you will have a
lot to learn and
will make many mistakes. A 100MHz scope will be very valuable - but not
necessary;I didn't have one.
Be aware that, if you want to avoid subtle intermittent failures, you are going to have to learn about ground planes, decoupling, ground bounce, probing techniques and limitations, inductance, different types of capacitor, time<->frequency relationship, transmission lines, and many other topics. Those will be tractable for a beginner with a 100MHz scope
if you use arduino class MCUs such as the the atmega328. If you try to build a 486 class machine from scratch, you will almost certainly have significant problems.
(If you use solderless breadboards, be prepared to spend as much time debugging the breadboard as debugging your circuit. Reason: their general frailty plus electrical characteristics inherent in their design)
Edit: I'm more interested in the architecture then anything else, that's why I initially said I would like to build a 1970s era computer. Computers from this era still had discrete components. Thinking about it more, I think the first step would be to build a simplistic microcontroller from discrete components. This would help me learn things like gates, digital logic, adders, ALUs, microcode, etc. Does anyone know of a training kit that has all of this? I think a lot of this could be done in simulation or on a FPGA today, however, I want to be able to touch it. I learn best through hands on labs.
If you had been around 30-40 years ago, you would have loved bit slice processors such as the AMD2900 or Intel 3000 families.