Directions - computer design journey.

[paulca]

So far most of the design of my computer was fairly linear towards that MVP of a terminal interface that can load and execute programs.

How however, having reached that milestone I find myself is a rather large expansive space that only has dozens of more doors.

Where next?

If I go with the traditional flow, that would be the door marked "Floppy Disk Emulator" and "Go with throttle up for block storage layer"

But...  Several evenings burning calories and drinking coffee and there are many "nested" problems that need to be solved when it comes to discs, allocation and filesystems.  Going from "LOAD xxxx" to LOAD A:myfile.txt is not that big a step.  Where the trouble lies is when you want to: Delete and reclaim blocks, OR you want to open, add and save a file.

Actually the problem is not that difficult ... at first...  until you realise that any "useful" and "simple" allocation table WILL NOT FIT IN MEMORY!

So the allocation structure itself has to serialisable as a set of smaller pages which can be processed sequentially or hierarchically.

So...  I came back out of that door and thought about things a bit more.

Before Gary Kildal and Stutgart et. al.  brought us the "DOS" with a floppy and before AND disc existed we had "Tape" (and cards).  Originally punch tape/cards, then magnetic.

So these sequential access and "append or overwrite" only mechanics were already well established.  That was the jumping off point for "pivoting" the tape onto a disc and facilitating lower seek times resulting in "effective random access".

They didn't go from "serial loader" straight to DOS.  There was a lineage of problems solved first.  Then build upon, refined, extended and eventually "ported" to the new physical medium and taking advantage of the need "speed".  It is still "SEEK", "READ", "WRITE" effectively.

The trouble now is the "Tape" door lies in the "Really for mainframes" end of the room.  So that opens a WHOLE kettle of fish/can of worms.

In that model the CPU doesn't run a bootloader and establish a filesystem with an IO controller.  The filesystem is already determined and is "mounted" on the IO controller by the control program.  Then the CPU is told to "go ask IO controller 3 for data".

Where I currently am, that is certainly an option.  To pivot to mainframe architecture and treat my current Z80 board as a "CPU Node".  Feed, task and control it externally.

Or I could just implement a basic tape streaming TOS.

The over ruling premise here is that I am not following in peoples footsteps and I am not recreating what they created.  I am hacking my way through the same jungle, along the same lines and routes, but I am not following any one path directly.  I'm exploring.

Where would you go next?

[cfbsoftware]

We went through similar design decisions when we implemented HCFiler for microcontrollers. These days with inexpensive multi-gigabyte SDHC cards there is no need to reuse disk space - you just keep appending data. This effectively leads to unlimited backups.

The four HCFiler-specific library modules used in the example programs HCTest and HCReadSpeed showing the creation of files and directory listing comprise approx. 600 lines of source code and generate about 4 Kbytes of object code.

[xvr]

Before Gary Kildal and Stutgart et. al.  brought us the "DOS" with a floppy and before AND disc existed we had "Tape" (and cards).  Originally punch tape/cards, then magnetic.

You miss something in between - CP/M ( https://en.wikipedia.org/wiki/CP/M )
And this is could be the next step (particularly for Z80 based computer)

[paulca]

What I meant was,

CP/M is a DOS.  A disk operating system.

Gary Kildall the creator.

I did not mention that which history created from CP/M.

[RJSV]

   I did similar soul-searching, but not as much in the SOFTWARE realm as it was the most basic, fundamental hardware innovation.  The task set involved DATA TRANSFER and, of course the (primitive) storage.
The data item storage uses two-states physically,  but not directly BINARY math.  Doing each system WORD is wildly inefficient, but LED to benefits in the next level up, (from single bit).
   That's because, at the word organization level,  a designer could put a process in place fairly easy.  For an example of this, consider the usual microprocessor SHIFTS or bit shifts, where a math function like multiply by 4 can be performed in the BUS hardware, and often lacking any of the traditional 'logic' that other general multiply instructions have, up the wazoo.  (That's an Engineering term used to convey endless seeming hardware needed.)

   My favorite are the increment or decrement actions, where a designer could put two actions together, for a 'Read and Clear' flag (needed in some operations, like interrupts, etc.).
A designer would first design and build a driver, like for 100 positions, in a SINE Lookup Table, then assign the BUS connections for each answer.

   Compared with super-efficient binary machines, the road I took can be bulky:  Ok then, Enter the Nanotechnology world;  where 10,000 mechanical 'levers', wheels, etc. are practical.   It's all a big, Data Transfer scheme,  and you can view things at:
   US Patent Office,  Google,
   US20030172205 Methods and Components for Mechanical Computer

   That's some abandoned stuff...still good and ready to use.

- - Rick B.

[RJSV]

   You see;  I've replied here because you have expressed the same wondering, when faced with a wide-open project, having mostly only my own supervisory directions.  Many folks that I know do not like that open-ness or sky's the limit environment:  literally needing that daily supervision, and an environment discouraging experimentation.

   Also, I am very much into PACKAGING, on the Mechanical Engineering sense, so my 3-D senses and carpenter skills fit perfectly.  Here is a brief example, of packaging concerns, with a Mechanical Decimal Digit memory:
   Finished with doing a single package, a RAM (random access memory) unit for storage of one digit;  of 0 through 9,  I then considered the next higher step;  doing a design of a 10-pack of multiple words (plus any needed interface, AND...a simple specification, on how to assemble various sets of multiple memory devices.
   Well, it turned out a bit simple,  once the intimidation factor was overcome, when faced with such 'new' issues and problems.
Interfacing is one aspect, but the package had to have semi-standardized or uniform approaches.
   Simply stated,  it turned out I already had the design in a flat form, or plate-like and had the actual BUS interface as a linear array (zero through 9 ID on conduits).  So,  it was almost trivial to simply stack a set of identical plates:  or near-identical as I could get things.
   Result was a so-called 'Ten-Pod' which could store a 'whole total' of TEN numbers, each as a single digit.  That thing was a first time in this world memory component, literally !   Sized about like a small stool or chair.

   Well, that exercise started to provide a preliminary view, of what the whole (Mechanical micro-processor) package would look like, and how much floor space.  A medium sized machine would take up maybe half of a small closet, in volume.   That's very do-able, in a school classroom setting.   Even better, provided some 'DRAMA' effect, being so large, as to get student's attention.  (That's the REAL task, in teaching anything)!

   So, I thought to reply here, not because of the technology aspect, but because of the feelings of 'novelty',  newness,  that can genuinely intimidate a designer.  Unless they are inventors,  then they often seek such novelty.

[paulca]

Interesting.  I just try and think 1 step ahead.  "I might need X, so to do that Y would be handy."

"LOAD <optional address>"
Fiddle in IDE to launch the 'tape' python loader script to send the bin file.
"OK"
or
"Fail"

I just seen the next logical step to compress that into just:

"LOAD thisFileByName"

and have the "OS" figure out what that actually means, send requests to a much more generic device full of files.

-----------------------

I have begun thinking about my next project, the 68k.  Don't want to get too far ahead with the Z80 if it's simplicity isn't going to teach, but probably annoy.  None of it is portable to the 68k in any reasonable way.

Question:  Is going straight to a full wrap FSB architecture over ambitious?

By full wrap FSB I mean that all (or most) address and datalines flow through teh "Front side bus controller".  Effectively making CPU Address and Real address decoupled entirely.  The target would of course be "Virtual memory" for multi-tasking and isolation.

The strategy is to wrap the 68k in level shifters down to 3v3 and put a lattice ice FPGA as FSB controller or MMU if you will.  Real mem chips and peripherals will be managed by it.

Physically 1 Meg of ROM (probably a pair of 8bit ROMs) and 1 Meg of SRAM can occupy the first, "n" pages.  Probably 2048 pages per 1 meg block, 512 bytes, 9 bit.

Applications however can see and use the full 16Mbyte address space (and more internally to the CPU).  All addresses arriving at the FPGA will use LUTs to remap them.  The "OS" systems code is responsible for updating the LUTs on context switch.

To make this work in a "poor" way, I can just map the full ROM range into every process so it can call the "kernel routines".  However, that sounds rubbish.  So the question becomes, how do I split the ROM/boot code into "User access" and "Supervisor mode only". 

I suppose just adding LUTs and putting "private routines" out of the LUT for processes.

So while a process can call say, "malloc" to get a new page, that routine will self interrupt back to "supervisor mode" which allows it to enable the kernel LUTs, do the updating of LUTs and lists for the actual malloc, then disable the kernel LUTs, restore teh CPU state and return back.

utils_malloc - public
kernel_utils_malloc - private super visor mode only

There should, I believe, be ways to hardware signal "Super visor mode" via the CPU "Mode" pins such that the FPGA programming interface is only available in supervisor mode and will be ignored if a user land process tries to update it's own LUTs.

LUTs / TLBs.

One downside to the 68k base model is there is no "resume instruction" after a bus fault.  So paging memory out to disk or "sparse on demand" allocation won't work.  I can't fully recall the limitation, I believe you have a single oportunity to simply "retry" after that the context and PC contents are lost and even just allocating the memory and jumping back to the processes PC doesn't work as expected.

For my use as a "learn to write a multi-tasking OS white room", I don't really mind there can be no "on demand paging".  If a process access memory that doesn't exist in it's allocation LUTs for translations then it gets a SegFault and a GTFO.

There are still so many things to consider. 

IO Addressing.
IO Addressing + Virtual paging
IO Addressing and concurrent access.

If a device is mapped to a "page", so that all it's registers et. al. exist in that page, then the FPGA LUTs can identify it as such and route it to the correct peripheral device.  However, to control access to peripherals to be "one at a time", processes would have to "attach" or "assign" to a device, have that page mapped and then all other processes are blocked from mapping that device page - assuming non-concurrent access desired.  Persumably they can also "deattach" and the kernel might be as kind as to reset and clean the device for next use... or not.

IPC - again it sounds like the "path of least resistance" is to use .... memory mapped pages for IPC, in this case, 'shared memory'.  Semaphores, mutex structures et. al.  The same could be used to "buffer" data needing to be passed from "user" to "kernel".

Pipes, file descriptors....   eek.

I know it's not white room, I am far too tained by Unix and it shows.  Grey room then.

Boiling that down to find a "step one" is the challenge.  Do I start with the FPGA from board one, or do I ease into it with a "flat" memory model just to get a feel for the CPU.