altermatives to vt100? neater protocols?

[DiTBho]

so, I put my money into the purchase of several vt-terms, a VT220 and a Digital DEC VT525.

They are nice machines based on 8080 CPUs, they are nice when you just use them with Linux, and they are very useful when you program on the text-console, especially the VT525 with a faster serial.

With VT100, ANSI escape codes for cursor control and other tasks became the de facto standard for hardware video terminals and later terminal emulators, and when you look at the protocol ... well, it's rather patchy, especially if you consider all the DEC addictions.

I'd like to re-design the terminal in modern terms, supporting blocks like IBM 3.27K (but not too IBM-ish) to minimize the number of I/O interrupts required by transferring large blocks of data known as data streams with high an a 48Mbps optical fiber interface, udp/ip over Ethernet connection optionally available.

So, a bit smarter than a dumb-terminal, not so much, but, absolutely of primary importance, the protocol must be designed in a super polished and neat way.

I don't want to see complex ESC sequences with the quality of being so illogical, inconsistent, and unclear, that trying to implement it's fsm in HDL is practically more annoying than heaping and stacking stones in the mine,



Suggestion, advice, all welcome 

[brucehoult]

It seems to me that you are conflating two entirely different issues:

1) the commands recognised / sequences sent by special keys, and

2) the communications channel, whether 20 mA current loop, RS-232, SDLC, X.25 etc, synchronous or async, "line mode" or not.

Waaay back 40 years ago when I was using such things, I seem to recall that the VT100 by default interpreted both ANSI ("VT100") commands and VT52 commands in the same mode. You could switch to ANSI-only using I think ESC < and to the mode that did both using ... uhhhh ..  ESC [?2l   I don't think VT100 had a pure VT52 emulation mode, that I can recall.

When we wrote games on the PDP-11 or VAX, we used the VT52 codes to the maximum extent possible, because they are shorter and that made a huge difference at 9600 BPS. Especially the random cursor positioning using ESC Y row col (encoded in binary) was much shorter than the ANSI  ESC [23;75H or whatever.

Regarding communication channel: I remember at some point fairly early on a PDP-11/34 was re-purposed as a "LAT" to manage the async terminals and service their interrupts, and provide the VAX with nice packetised input and output. And to let you connect your terminal to any one of several VAXen too. That protocol is probably documented somewhere.

Yeah.

http://www.bitsavers.org/pdf/dec/ethernet/lat/AA-NL26A-TE_LAT_Specification_Jun89.pdf

That's dated 1989 but I was definitely accessing VAXen via this protocol by ... 1983 maybe.

[SiliconWizard]

Why would you want to torture yourself implementing that in HDL (so I suppose using an FPGA, unless you actually plan on making silicon out of that), when even a small modern MCU would allow you to implement this with ease? Just wondering. (And yes, even with a VGA output without any additional ICs if you're using something as cheap as a RP2040!)

As to the protocol, I've never looked at it closely enough to figure if it was really that bad. But sure if you don't need to be compatible with anything, you can devise a clean protocol, not based on clunky ESC sequences. And if you're targetting much faster interfaces than what was standard at the time, you can encapsulate everything in neat packets with nice headers and CRCs and whatnot.

[westfw]

x11?
At one point “X terminals” (tektronix and NCD were two vendors) were a wonderful thing.
They even supported a compressed Async protocol (XRemote) that made them useful over 9600bps dialup (and Cisco supported it in their terminal servers!)
X runs over tcp/ip, of course.


I think Stanford’s “V kernel” did similar things, but didn’t catch on anywhere.  I suspect SUN siphoned away all of the developers.

[DiTBho]

x11?

I own an original Tektronix xp-2xx-line with its netboot OS and license 

It's MIPS-R3000 based @ 50Mhz with 16Mbyte of fastpage 72pin ram (must be <=50nsec), fully net-booting, VxWorks v5-based, built-in mwm Windows manager, telnet, a built-in superfast Nescape browser and a font client.

First problem: it's pseudo color ---> forget everything GTK-based, only OpenMotif apps will work.
Second problem: only HTML-v1 (1991) and HTML-v2 (1995) are supported ---> forget 90% modern web sites

However, it also adds two serial lines in case you need vt100 an vt200 emulation.

And you don't need any "multi window" extension to operate with two sessions (you need >= vt420 for this), just open two of them, and connect each on a dedicated serial line, you have two physical serial lines, you can have up two sessions.

Do you want more? Use telenet, connect to a trusted telent-to-ssh bridge (WR703? a router?) and you will have many many more.

---

I like it, I like the "thin terminal" concept, and I recently replicated with a modern PPC GNU/Linux + uclibc + nanoX + DWM + (Invisible Island version) xterm solution ...

... just X11 and nanoX are toooooooooo big and complex to be implemented from scratch, and I won't for sure try to build my own.


The funny here is: the PPC@200Mhz with GNU/Linux and X11 is 1.2x slower than X11 on VxWorks on a MIPS-R3000@50Mhz

Kind of LOL

[DiTBho]

when even a small modern MCU would allow you to implement this with ease?

Well, Ania implemented a minimal vt100 in vHDL, and not only is it feasible but it's also cool and neat!

Once defined the new protocol, the plan is to meet Ania to implement new things in HDL.

Kind of interesting collaboration, you shouldn't underestimate this aspect.
Social Engineering is always awesome.

Plus, this project could even win an award for the next 2023 hack camping 
We are talking about a cool premium badge!!! and a case of Bel-cola!!!

- -

All the MPU-solutions I have ever seen around suck. Full of tricks and horrid things.

When people have an MPU, the design tends to be illogical, inconsistent, and unclear, especially for those who suck at software design, and, worse still, have an horrid C-style programming.


Yes, they write "sorry, I will polish and uncrustify in the next version"
Which 99.999999% times means gnegne ... forget it, it will never happen 

It would be interesting to write the whole project in myC, but it only supports MIPS4++ and for sure I won't put a MIPS R18200 into a vt-term ...

... I mean ... it would really be like sending a Grendizer robot to squash mosquitoes 

[DiTBho]

It seems to me that you are conflating two entirely different issues

there are two different needs

• One is character-oriented, good for the keyboard but you shouldn't limit the whole protocol to it otherwise you have to send every single char when you need to update the screen, and this way you have a lot of interrupts, and a very low throughput
• One is block-oriented, it offers higher throughput, and it's good when you have to send a whole sub-screen sessions, forms, or things like that

VT52, VT100, etc are char-oriented-only, and uses a char-device as communications channel, while I want to support a block-device like 48MBps fiber optic.

The new protocol needs to minimize the amount of data transmitted, and minimize the frequency of interrupts by ensuring the CPU is not interrupted at every keystroke. I won't for sure send one character at a time like in VT100, worse still with a lot of ESC sequences to address row and column, and optionally also the char attributes.

[SiliconWizard]

when even a small modern MCU would allow you to implement this with ease?

Well, Ania implemented a minimal vt100 in vHDL, and not only is it feasible but it's also cool and neat!

Well, everyone has a different concept of what is cool, but certainly it IS doable. My point was that it probably made little sense and would be harder to maintain.

All the MPU-solutions I have ever seen around suck. Full of tricks and horrid things.

Maybe, I dunno. Pretty much all commercial terminals have been implemented with MCUs/MPUs. That's certainly a workable approach.
The "let's throw a FPGA at this project and see if it sticks" sounds like a weird engineering approach, but whatever floats your boat.

Just because you have seen sucky implementations doesn't mean that you can't write a clean one (that stamtent sounded a bit like magical thinking), and certainly a VT100 terminal with say VGA output could be neatly implemented with just a RP2040 with even a lot of processing power to spare (yes you can implement a clean VGA output with the PIO). If you needed HDMI output, then sure you'd need additional parts though.

Just a thought. This is ultimately down to your preferences and that's alright. But just don't necessarily convince yourself and others that this is the only right way of doing it.

[DiTBho]

let's throw a FPGA at this project and see if it sticks

no, it's let define a finite state machine and see how we can make the protocol the neatest possible to minimize the finite state machine and get all the constrains satisfied.


Which is better than throw a MPU at this project, write some shitty C code, and see if it sticks

[brucehoult]

let's throw a FPGA at this project and see if it sticks

no, it's let define a finite state machine and see how we can make the protocol the neatest possible to minimize the finite state machine and get all the constrains satisfied.


Which is better than throw a MPU at this project, write some shitty C code, and see if it sticks

All MPU+program combinations ARE finite state machines.  And there's no reason to write shitty C code if you don't want to. Writing shitty asm code would be a lot easier than creating a huge FSM by hand. Or, you could write non-shitty C or asm code. That would be even better.

It only takes about 200 LUT4s and 200 FF on an iCE40 and one RAM block to make an RV32I soft core that would be 100 times faster than the 8080 in a real terminal. On Xilinx it's about 130 LUTs I think.

[DiTBho]

All MPU+program combinations ARE finite state machines.

A program on a MPU is NOT the minimal finite state machine possible.

[brucehoult]

All MPU+program combinations ARE finite state machines.

A program on a MPU is NOT the minimal finite state machine possible.

You're not going to get the minimal FSM no matter what you do.

Just the 24x80 screen with 8 bit chars and 4 attribute bits is 2²³⁰⁴⁰ states already. before you even start counting interpreting incoming control sequences.

[SiliconWizard]

Not sure I get DiTBho's point exactly - I think we can almost all agree that writing this in HDL will be harder than in software (so will take longer), and since software programming is very often discussed here, I think we can also all agree with the fact we can write decent and robust software. Especially for something like this which is not rocket science.

But as I said, whatever floats everyone's boat.

As I mentioned, I'd find this fun to implement on a RP2040, supporting multiple interfaces and using the PIO for a VGA (or similar) output. But if someone is more comfortable doing this in HDL, why not. I'm just pretty sure it will take more time.

Now one could argue that implementing it in HDL might have more chances of being more robust - if just because you need to be extra careful when using an HDL, and you also are more likely to test your code using test benches and simulation than you are for pure software - in that sense, HDLs may be better to force you to do things right. But that's also debatable.

[DiTBho]

Just the 24x80 screen with 8 bit chars and 4 attribute bits is 2²³⁰⁴⁰ states already. before you even start counting interpreting incoming control sequences.

Not the way you interpret it! You are not working at gate level but rather above the rtl level, you have vHDL and you can describe a finite state machines in the VDU stage that perfectly knows the concept of "row" and "column" and EA to correctly address the video ram 

In Ania's vt100-lapop you have hardware two-fish encryption directly connected to the optical fiber circuits (Manchester encoded) and to the VDU.

What arrives from the optical cable enters the protocol link engines, gets accepted or rejected (corrupted, go back n), gets payload extracted, enters the encrypt unit, gets decrypted, enters the VT100 engine (minimal set), gets interpreted, decomposed to operations for the VDU unit which address character to the video ram, and, on refresh event, you see it on the LCD.

Very clean! No software on no Softcore implemented, all modules in pipelining, 100% vHDL

Ania showed that all these units work in parallel and there are many bypassing circuits that would consume many more resources if implemented in software on general purpose CPUs, and commented that it's not because CPUs suck but rather because they are *generic* and filled with other things like pipelining, caching, etc. to go faster. Also, parallelism on a single core needs interrupts at least and context save/restore, which ... is good but not so much when compared to modules hardware pipeling.

[DiTBho]

Meanwhile, talking about "multiprocessor programming"

I just ordered this book
                    The Art of Multiprocessor Programming, Revised Reprint
                    by Maurice Herlihy, Nir Shavit

[rstofer]

The new protocol needs to minimize the amount of data transmitted, and minimize the frequency of interrupts by ensuring the CPU is not interrupted at every keystroke. I won't for sure send one character at a time like in VT100, worse still with a lot of ESC sequences to address row and column, and optionally also the char attributes.

Actual keystroke interrupts occur at something less than 10 per second.  100 ms is a very long time.  A 600 MHz Teensy 4.1 won't even notice the handler running.  ARM Cortex M7 https://www.pjrc.com/store/teensy41.html

Telnet usually operates one character at a time (Character Mode) unless Line Mode is negotiated.  Operator input obviously uses character mode since a single character response is often used (thinking of the response to 'y/N').  I don't see how you get around one character at a time when the message is only one character long.  In terms of keyboard input, you have, at the driver level, no idea how long the message will be.

There's a reason standards survive.  They are useful and very nearly optimal in their application.

[SiliconWizard]

To each their own really! That's all we can say. But all this can clearly be written cleanly in pure software on a modest MCU. Only the encryption part may require a beefier one, but even this is probably not that much of a problem due to the typical data throughput of a text-based terminal. For a graphics terminal, things would be another story.

[Jr460]

I'm really not getting what you are trying to do and why, but I'll try to play along.

In terms of less interrupts, while that is comm/serial controller's problem on the computer side, not the terminal's problem.   Someone said LAT, well that was just the transport between the CPU and and the terminal server over Ethernet.   You still came out of the terminal server at 19.2k bps or so to a terminal.   LAT is going to wait for data until it has more to fill up an ethernet packet.   I think the timer was something like 80 milliseconds, and then it had to send what not already had.   The idea was to keep the delay below the time that the human eye / brain could detect the delay.

How about this, go find an old VT1000.   That is not a typo.   I tw as monochrome X-term, so it had builtin ethernet, was based on on a 68K processor.   And if you didn't want to do full blow X, you could pop open a bunch of vt100 compatible windows that natively did LAT over the network.   Problem solved.

As far as moving less data, look at the differences between VT100, 101, 102, 200 and 220, 240, 300 code.   The later terminals allowed for 8 bit escape sequences which shorten many commands, also local line editing,, a character could be added or deleted mid line without re-painting the whole line.   Also scrolling regions on the screen a set of lines or a block.   If your program uses the right commands to the terminal, you can save a lot of output to refresh a screen.


Or go real old school.   have the terminal "screen" as part of system memoryand the video generator just cycle steals or does DMA from the screen buffer.   I had an old S100 based board that did this.   And also had a ROM on it that could be called passing it data, and it would emulate a vt100 and write to memory on the card for you.

How fast do you really need data to come to fill a screen that is 80x24.   What serial speed does it take to write the whole screen in under 80 ms?   And faster than that and you will not see the difference.   If you are blasting lines to screen just for it to scroll by and that is the delay in your program, then I'd say fix you program to only put out what need to see at the end.

[DiTBho]

Actual keystroke interrupts

yeah, not the best example, except when a keystoke causes a full page update, e.g. software vertical or horizontal scrolling.

You have them with a text-editor

let's say, worst case, you have to update the whole screen, 80x25x2 (1 char, 1 meta), one page is 4K byte of ram(1), 4kbyte/s.

• gets no-dice with a slow uart connection at 9600bps (~ 900byte/sec)
• gets better with 115kbps (~10Kbyte/sec)
• gets excellent with Ethernet 10Mbps (~800Kbyte/sec)
• gets outstanding excellent with optic link 48Mbps (400Kbyte/sec)

but here you may want larger screen, and also transport other stuff, so you want to compress and maximize the payload and get rid of all the extra things you have to transport for your terminal


The vt100-procol is ASCII based, not good for four points

• you have to parse and decode fields, e.g. row=10 is written "1" "0" rather than of 0x0a, it's OK for MPUs, but an annoying waste of resources in vHDL because you have to write a mini fsm just to manage it
• it's less efficient than a binary protocol with fixed length fields and usually takes 2.5x the length, due to its ASCII nature + ESC encoding
• it's not orthogonal, there are ESC sequences that need their own dedicated sub-fsm
• it doesn't consider when you want to update clusters of chars in a single shot

You appreciate the last point especially with IBM 3.2K terminals, which are block-oriented, and whole page updating is one shot!



(1) Ania's VDU text ram is 32Kbyte at the moment. It's a true dual port ram.

[DiTBho]

In terms of less interrupts, while that is comm/serial controller's problem on the computer side, not the terminal's problem

yes, but starting from scratch I want to solve this too since who knows? I will probably use the terminal with super fast GNU/Linux MIPS64 board @ 1.4Ghz, perhaps with a 68hc11 @ 4Mhz ...

let's make life easy on the server side too, if possible 

[DiTBho]

Someone said LAT, well that was just the transport between the CPU and and the terminal server over Ethernet.   You still came out of the terminal server at 19.2k bps or so to a terminal.   LAT is going to wait for data until it has more to fill up an ethernet packet.   I think the timer was something like 80 milliseconds, and then it had to send what not already had.   The idea was to keep the delay below the time that the human eye / brain could detect the delay.

Yup, I bought a LAT terminal, it's here in my lab. I'd like to *copy* the best form it 

[DiTBho]

As far as moving less data, look at the differences between VT100, 101, 102, 200 and 220, 240, 300 code.   The later terminals allowed for 8 bit escape sequences which shorten many commands, also local line editing,, a character could be added or deleted mid line without re-painting the whole line.   Also scrolling regions on the screen a set of lines or a block.   If your program uses the right commands to the terminal, you can save a lot of output to refresh a screen.

Yeah, and it's not supported by Ania's vt100-hdl, they are extensions, while Ania's one is a very basic vt100, like the first model appeared for the market.

Nice feature addons, I can play with them with my VT525 and see how they are pretty and nice ... just, well they made the vt-protocol more spotted than a cow, and fist thing you have to do is probing your vt-term "which version are you?" and "which feature o you support?"

flexible for different products that evolve year by year, but not good when you try to synthesize the minimal protocol for full features.

[brucehoult]

The new protocol needs to minimize the amount of data transmitted, and minimize the frequency of interrupts by ensuring the CPU is not interrupted at every keystroke. I won't for sure send one character at a time like in VT100, worse still with a lot of ESC sequences to address row and column, and optionally also the char attributes.

Actual keystroke interrupts occur at something less than 10 per second.  100 ms is a very long time.  A 600 MHz Teensy 4.1 won't even notice the handler running.  ARM Cortex M7 https://www.pjrc.com/store/teensy41.html

Well, yeah, but we used to have 50+ VT100s attached to a 1 MIPS VAX 11/780.  600 MHz Teensy is about 1000 MIPS. So the relative interrupt load on the VAX was something like a factor of 50,000 higher. That's why they started using old PDP-11s as terminal servers, to take that interrupt load off the VAX.

[brucehoult]

Nice feature addons, I can play with them with my VT525 and see how they are pretty and nice ... just, well they made the vt-protocol more spotted than a cow, and fist thing you have to do is probing your vt-term "which version are you?" and "which feature o you support?"

You haven't lived until you've programmed graphics using ReGIS / GiGi.

Code: [Select]

<ESC>P0p
S(E)(C1)
P[100,440]
V(B),[+100,+0],[+0,-10],[-100,+0],(E)
P[500,300],F(C[+100])
<ESC>\

That clears the screen then draws an outlined 100x10 rectangle, then a filled circle with radius 100.

[Jr460]

Nice feature addons, I can play with them with my VT525 and see how they are pretty and nice ... just, well they made the vt-protocol more spotted than a cow, and fist thing you have to do is probing your vt-term "which version are you?" and "which feature o you support?"

You haven't lived until you've programmed graphics using ReGIS / GiGi.

Code: [Select]

<ESC>P0p
S(E)(C1)
P[100,440]
V(B),[+100,+0],[+0,-10],[-100,+0],(E)
P[500,300],F(C[+100])
<ESC>\

That clears the screen then draws an outlined 100x10 rectangle, then a filled circle with radius 100.

Yep loved doing things with those graphics.   Wrote tuns of stuff to take files in HP-GL or Tex 41xx format and output it as Regis.