Mystery of VT100 keyboard codes

[peter-h]

I am running Teraterm (a "normal" VT100 emulator, on a COM port) to my target.

I am getting funny key codes being received.

For example, cursor up should be ESC [ A i.e. 0x1b,0x5b,0x41. But I am seeing just ESC A i.e. 0x1b,0x41

Googling around on key codes, this doesn't compute.

OTOH I recall testing this a few months ago, where I exercised all the keys to produce this table

Code: [Select]

static const char KEYS[] = {
0x0d,0x00, // ENTER
0x0a,0x00, // LF
0x08,0x00, // BS
0x1b,0x00, // ESC
0x1b,0x5b,0x31,0x7e,0x00, // HOME
0x1b,0x5b,0x34,0x7e,0x00, // END
0x7f,0x00, // DEL
0x1b,0x5b,0x35,0x7e,0x00, // PGUP
0x1b,0x5b,0x36,0x7e,0x00, // PGDN
0x1b,0x5b,0x41,0x00, // CUR_UP
0x1b,0x5b,0x43,0x00, // CUR_RT
0x1b,0x5b,0x42,0x00, // CUR_DN
0x1b,0x5b,0x44,0x00, // CUR_LF
0x09,0x00, // TAB
0x1b,0x5b,0x31,0x31,0x7e,0x00, // F1
0x1b,0x5b,0x31,0x32,0x7e,0x00, // F2
0x1b,0x5b,0x31,0x33,0x7e,0x00, // F3
0x1b,0x5b,0x31,0x34,0x7e,0x00, // F4
0x1b,0x5b,0x31,0x35,0x7e,0x00, // F5
0x1b,0x5b,0x31,0x37,0x7e,0x00, // F6
0x1b,0x5b,0x31,0x38,0x7e,0x00, // F7
0x1b,0x5b,0x31,0x39,0x7e,0x00, // F8
0x1b,0x5b,0x32,0x30,0x7e,0x00, // F9
0x1b,0x5b,0x32,0x31,0x7e,0x00, // F10
0x1b,0x5b,0x32,0x33,0x7e,0x00, // F11
0x1b,0x5b,0x32,0x34,0x7e,0x00, // F12
0x00,0x00,0x00,0x00,0x00,0x00 // prevent spurious matches with data past table end
};
and they matched the codes here
https://www2.ccs.neu.edu/research/gpc/VonaUtils/vona/terminal/vtansi.htm
although nowhere could I find the PC function keys F1-F12; I had to catch these empirically, but even those are different now e.g. F1 should be 0x1b,0x5b,0x31,0x31,0x7e but I am getting just 0x1b,0x31,0x7e.

Due to the passage of time since a VT100 was a real thing, the more obscure key codes have become fluid, while old stuff like DEL, BS, TAB, BELL, have remained.

Teraterm doesn't have a config for a different terminal.

Can anyone spot what is going on?

I am running this under win7-64, to a VCP (USB CDC function in my target).

Everything else is working fine, and if I echo the bytes back to the monitor window, cursor UP actually moves the cursor up, so Teraterm is clearly happy with what it is sending

I am getting on two machines and two targets, and one of them is the one originally used to determine the key codes.

[wek]

Maybe you should've explained your exact setup in more detail.

You're gulping every even character somewhere where you observe it (but not where you echo), probably when they arrive in one packet, as otherwise you'd notice when typing plain characters.

JW

[RoGeorge]

No need to reverse engineer the VT100 codes, they are published online in many places, random link:
http://www.braun-home.net/michael/info/misc/VT100_commands.htm
I don't think the Fn keys were a thing back then, during the time VT100 standard was made, but I'm not very sure.

Teraterm has many modes, I would check its setting to be sure Teraterm is indeed in VT100 mode.
https://ttssh2.osdn.jp/manual/4/en/about/emulations.html
https://ttssh2.osdn.jp/manual/4/en/reference/keycode.txt

[peter-h]

Interesting!

I have my 32F417 board, which among other stuff has a USB port implementing a VCP (USB CDC device profile). This is accessible to the PC like any other VCP e.g. COM3. It is also accessible to internal software as a UART port.

And this all works. I can output long chunks of text via the output to the PC (Teraterm), and I can read stuff from the PC (Teraterm keyboard). I have done a simple loopback routine in the target, which echoes keystrokes nicely.

The immediate purpose of this feature is to run a factory test screen over it. Later it will be documented for general use; it is pretty fast at > 100kbytes/sec.

Now, would you expect, if running a VT100 terminal on the PC, for the cursor keys to move the cursor on the terminal? I am not actually sure, but during a test a while ago this was working just like that. It isn't now. The terminal is doing VT100 cursor codes (it evidently supports "cursor home" etc, but the keyboard isn't generating them.

There was a problem with the VCP code back then, corrupting a few % of the data, which was never noticed on keystrokes, but when I got around to testing the VCP feature properly for not just output (which has been working for years, for debugs) but also input, I found this issue, and fixed it (various forum posts which WEK will recall).

I went looking for config options, but according to the above links, Teraterm is basically a VT100 emulator.

It does support keyboard mappings via a key mapping file, under Setup / Load key map. But among the files provided, none generate VT100 cursor codes (readily found on google e.g. ESC [ A (3 bytes) is cursor UP).

To be fair, I would not expect F keys to be specially supported, because they didn't exist on a VT100. Some DEC terminals did have F keys but mostly these sent out whole text strings. But I would expect TT to emulate "something". How do PC applications deal with F keys? Do they look for ESC sequences, or is there a specific API? If the former then I would expect to find a table somewhere, but I found nothing. It seems to be dealt with using scan codes or some such.

And only one of the .cnf files, function.cnf, which come with TT, makes the F keys do anything. Some others make all F keys generate the same code. Using the function.cnf keyboard mapping file, I get these F key codes again; F1= 1B 50, F2= 1B 51... That's OK; I can use that, but do these relate to anything whatsoever?

The cursor keys are 1B 41 to 1B 44, consistently across the .cnf files, but again I cannot see these codes relating to anything. They are not VT100 cursor codes for sure.

There is no other config I can find in TT.

Possibly I was using some keyboard mapping file which I can't find now, but I doubt it because my Teraterm installations (I use it everywhere) are straight out of the box, no config.

I cannot see how TT ever generated the VT100 cursor codes.

I uninstalled TT, deleting previous settings, and reinstalled it, and get the same thing.

I have a usable terminal now, but with keyboard F key and cursor codes which don't relate to anything I find online.

The TT "terminal ID" does not do anything in terms of changing key codes and such



The keyboard cursor codes I am seeing are e.g. here
https://docs.microsoft.com/en-us/windows/console/console-virtual-terminal-sequences
but without the '[' i.e. ESC [ A is just ESC A. But my VCP passes through the '[' fine.

I am printing the hex codes to the SWV ITM data console. I am quite confident about this part.

[westfw]

Iirc, vt100 had an escape sequence that would change the “mode” of the cursor keys…
Umm, [size=78%]https://terminalguide.namepad.de/mode/p1/[/size] ?

[Bobson]

Your VT-100 is in VT-52 mode. Need to switch to VT-100 mode. ESC <, if I remember correctly.

[peter-h]

vt100 had an escape sequence that would change the “mode” of the cursor keys…

That does not explain the cursor codes I am seeing.

Your VT-100 is in VT-52 mode. Need to switch to VT-100 mode. ESC <, if I remember correctly.

You are damn right!!!
http://toshyp.atari.org/en/VT_52_terminal.html

However that string doesn't appear to work. But now that I know where this ESC A etc comes from, I will just create a new keyboard mapping file. Actually the encoding there is damn complicated. Maybe I should ditch TT and get a proper VT100 emulator? I had a look at a load and most seem to either be crap, or need site licenses.

It seems this "keyboard is VT52 while screen is VT100" thing is not uncommon. Just tried Realterm and it does the same.

I found the ESC < here: http://www.braun-home.net/michael/info/misc/VT100_commands.htm
but it doesn't do anything on TT. The keyboard is firmly VT52.

I've concluded TT is useless. One can config a different key mapping but there is no way to save it! The teraterm.ini file has no parameter for the .cnf file. There is only a command line parameter, which is a long term nightmare for setting up a factory test procedure (the command line applies only to a specific shortcut file). Also it seems that a lot of the functionality works only on TCP/IP connections, not on serial connections.

I would be happy to buy something that actually works as-spec, but don't want to pay for each installed program, and absolutely don't want online license checking because the test laptops are not online.

[DiTBho]

Tcp/ip, try or buy mobaterm

[peter-h]

Gosh that does everything including the kitchen sink. All except serial ports

Nice tool, with RDP as well.

[bsdphk]

Check if your serial port is correctly configured.

ANSI escape codes also have a 8-bit mode:

0x1b [ A

can also be sent as

0x9b A

If your serial port strips the top bit...

[m k]

VT100 is 7-bit machine.
0x9b is Control Sequence Introduce, it came later.
VT132 is defining CSI but the actual code is still ESC [.

Can't remember how 8th bit is treated but guess is that the whole character is dropped.
Emulation can obviously do as it likes.

Correct general operation would be so that only VT52 or VT100 codes are supported, not both nor mix.

Can't remember either how VT101 was but VT102 was 8-bit machine, though can't remember what it actually meant since multinational character set came later, maybe alternative character set of some sort was available.
Operation modes were also ANSI and VT52, name VT100 was not used.

Maybe alternative keypad mode can be used for testing.
ESC = on and ESC > off.
Off is keypad numbers and on is something else, VT52 and VT100 responds differently.

Reading DEC manual pages without knowing it also can be misleading since character numbers are usually octal.

[peter-h]

My code is definitely not eating bit 7.

I think the issue is that I am testing with Tera Term and most people just use that as a dumb terminal, so they never get to care about which ESCape sequences the keyboard generates.

There is no doubt that TT can generate the VT100 cursor codes but I didn't spend enough time with it to work out how. The keyboard.cnf file has various sections in it and it isn't exactly obvious which section is in use in a given context.

The files which come with TT just do the VT52 codes.

[abyrvalg]

PuTTY sends ESC [ A/B/C/D for arrow keys correctly

[peter-h]

I could not find where Putty is configured to use a COM port. I use it only for remote server work. It's a weird program - too weird to use in production test.

I am sure somebody has done a keyboard map file for Teraterm; I just haven't found it, and have not managed to work out its file format. I can remap the function keys but not the cursor keys. One also has to do it in the keyboard.cnf file, not a different-name file.

[oPossum]

I could not find where Putty is configured to use a COM port.

Connection type: Serial

[peter-h]

Ah yes I did actually find it before.



Unfortunately it doesn't work. Just beeps once. Maybe COM9 is too high a number?

[MarkF]

Sorry, I haven't been following this topic.

However to answer your PuTTY question, I have used PuTTY to connect to a PIC32 chip with a USB_to_Serial cable without an problems.  Using Win7 and serial port 10 and the default PuTTY settings.  I also was using a 38400 baud rate.

[MarkF]

Also, the following ESC sequences worked with PuTTY:

Code: [Select]

// VT100 control macros
#define backspace                0x7f  // make sure your backspace matches this!

// PuTTY serial terminal control codes
// from https://en.wikipedia.org/wiki/ANSI_escape_code

// ANSI CSI (Control Sequence Introducer) sequences
#define clrscr()                 printf("\x1b[2J")
#define clr_right()              printf("\x1b[K")
#define home()                   printf("\x1b[H")
#define cursor_pos(line,column)  printf("\x1b[%02d;%02dH",line,column)

// ANSI SGR (Select Graphic Rendition) parameters
#define normal_text()            printf("\x1b[0m")
#define inverse_text()           printf("\x1b[7m")
#define red_text()               printf("\x1b[31m")
#define green_text()             printf("\x1b[32m")
#define yellow_text()            printf("\x1b[33m")
#define blue_text()              printf("\x1b[34m")
#define magenta_text()           printf("\x1b[35m")
#define cyan_text()              printf("\x1b[36m")



[abyrvalg]

Try \\.\COM9 instead of COM9

[T3sl4co1l]

Can't speak for Teraterm, but can also confirm PuTTY works as expected.  Has been some years now I've had a console system built around it, and works fine with PuTTY, and also HyperTerm (if you're old fashioned).  Only quirk really, I think, is force CRLF mode (my code is full of "\r" instead of "\n" or "\r\n").

Have never had to enter fully qualified / DOS path for extended COM ports in PuTTY, though have for other (less featureful?*) tools.

*Including my own; specifically, opening the device as file, requires the DOS path specifier ("\\.\") when it's a two-digit COM port.  I don't know how other programs do it (that doesn't involve OpenFile(), or not as naively).

Tim

[oPossum]

Maybe COM9 is too high a number?

No, it will work with any COM port number.

You must specify the port in the session settings.

[peter-h]

Indeed, Putty does work on serial.

However, if you disconnect the USB VCP device (unplug the USB cable) Putty renders that COM port unusable for ever. One has to reboot the PC

Teraterm doesn't do that; it merely needs the COM port reconnected in the menu.

I need to figure out how to use the TT keyboard remapping file to modify the keyboard codes. I have managed to do some keys but the file has specific named sections and you have to get this right. I am damn sure somebody has done this before...

Maybe somebody can help. I am trying to change the codes emitted by the cursor keys. This is the start of the keyboard.cnf file

Code: [Select]

; Sample of KEYBOARD.CNF for the IBM-PC/AT 101-key keyboard.
; See KEYCODE.TXT for the description of the format.
;
; This file based on IBMKEYB.CNF
; 1. send the symbol of each key top "as-is" at editor keypad.
; 2. send the symbol of each key top "as-is" at *, /, +, -, Ctrl+"+" on a Ten-Key pad.
; 3. enables using the function key from F1 to F10.
;    See [User keys] section.
;
; 10/9/2022 PH Copied from function.cnf. These are VT52 keyboard codes.
;
;
[VT editor keypad]
;Up arrow key
Up=328
;Down arrow key
Down=336
;Right arrow key
Right=333
;Left arrow key
Left=331
;Insert key
Insert=338
;Home key
Find=327
;PageUp key
Prev=329
;Delete key
Remove=339
;End key
Select=335
;PageDown key
Next=337
But nothing edited seems to make any difference. For example I am trying to edit the Down key, scan code 336. I have tried all kinds of stuff e.g.

Down=336,0,$1BABCD

to generate ESC ABCD

but it just outputs the same 1B 42 (ESC B).

I found one can remap the cursor keys to User keys e.g.

User11=336,0,$1B$1C where 336 is the scan code, and this was supposed to emit 1B 1C but it emits only 1B. If one emits 3 chars, only the 1st and 3rd come out. It is weird, and contrary to the TT's author's documentation e.g.
https://github.com/Testato/TeraTerm/blob/master/doc/en/html/reference/keycode.txt

Basically every 2nd char comes out. So a string like aabbccddee generates abcde. I was wondering if there is a "unicode" config somewhere but can't see it.

But you can tell that everybody who was involved with TT is now about 120 years old

[wek]

Basically every 2nd char comes out. So a string like aabbccddee generates abcde.

As I've said in my post above, IMO this is misinterpretation in your program, not TeraTerm's fault.

JW

[peter-h]

You were right, of course