"tftp" seems like an incomplete protocol: hacked

[Nominal Animal]

the extension I proposed does NOT break anything, so you can safely boot from a classic tftp server that does not support my extension, you simply will not have the progress bar

Unless, of course, They can even get the progress bar if they create a parallel file to bootfile named bootfile@ that contains the size of the actual boot file, in binary.

Hmm.. Would that be enough of a reason to change your protocol to pass the size in bytes as a decimal string instead?  Allowing zero or more leading ASCII whitespace (\t, \v, \f, space), one or more ASCII digits (0-9), and requiring at least one trailing whitespace including newlines (\t, \n, \v, \f, \r, space)?

That way it'd be trivial to create that file, using "stat -L -c '%s' 'bootfile' > 'bootfile@'".  (stat is part of coreutils, so it will be installed in all Linux systems.  It's a common Unix utility. The parsing overhead is neglible compared to the transfers themselves.)

[DiTBho]

the extension I proposed does NOT break anything, so you can safely boot from a classic tftp server that does not support my extension, you simply will not have the progress bar

Unless, of course, they create a parallel file to bootfile named bootfile@ that contains the size of the actual boot file, in binary.

Hmm.. Would that be enough of a reason to change your protocol to pass the size in bytes as a decimal string instead?  Allowing zero or more leading ASCII whitespace (\t, \v, \f, space), one or more ASCII digits (0-9), and requiring at least one trailing whitespace including newlines (\t, \n, \v, \f, \r, space)?

That way it'd be trivial to create that file, using "stat -L -c '%s' 'bootfile' > 'bootfile@'".  (stat is part of coreutils, so it will be installed in all Linux systems.  It's a common Unix utility. The parsing overhead is neglible compared to the transfers themselves.)

I don't think I understand the problem 

A file ending with "'@" is not valid for the filesystem.
Anyway, the trick is not to use a file that contains the filesize.

If the server supports my extension, it redirects the handler to a procedure that opens the file, gets the filesize, and sends it to the client in binary format.
If the server does NOT support the extension, it returns "file not found".

I've checked a couple of servers that do not support the extension. There are no problems.

I don't have much time right now, busy with other issues, but as soon as I get back to the tftp, I'll extend the client and server with { rfc2347, rfc2349 }

[Nominal Animal]

I don't think I understand the problem 

No problem –– I worded it wrong, now edited to fix! ––, the opposite: you can get even the progress meter back with simplest of TFTP servers.

If the size response is the same as reading a file containing the actual file size in bytes, and your own server synthesizes that file on the fly, users can use your client with any TFTP server, without losing functionality.  With other TFTP servers they only need to create the size-file by hand.  If they don't do that, the progress meter is not shown.

That is, when the TFTP client wants to load "bootfile", it first requests file "bootfileSIZE".

Your own TFTP server synthesizes its contents, the size of file "bootfile" in bytes as an ASCII decimal string followed by a newline.  If "bootfile" does not exist, this file does not exist either.  In practice, your TFTP server should always check the file existence first, and then, in the cannot-read case, check the filename suffix and if correct, only then trigger your custom handler.

Your client parses the response, and uses that length for the progress bar when loading "bootfile", the proper boot data image file.

With any other TFTP server, the user can, but is not required to, to create the "bootfileSIZE" file themselves, using e.g. the command
    stat -L -c '%s' 'bootfile' > 'bootfileSIZE'
If they do, it will let the client display the progress.  If they don't, the progress is not shown.

Above, SIZE is a valid but rare or reserved file name suffix.  (In Linux, file names are simply non-empty byte sequences not containing 0/0x00 or 47/0x2F; every other value is technically allowed, just not supported by some non-Linux/POSIX/Unix filesystems.)

I believe this yields much wider compatibility than any of the other approaches for the server side, including the later RFCs.

If you wanted security or proper data integrity checks and connection resiliency, a custom transport on top of UDP using ChaCha20-Poly1305 or AES128-GCM is not difficult to implement.  It is really the key exchange, the handshake part, that is tricky; especially if you want to avoid RSA and Diffie–Hellman (because that is what makes TLS slow on old hardware).  It is why I've looked into offloading the annoying parts to Microchip ATECC608B via I²C, specifically using a small board with the IC, a supply bypass cap, and a 4-pin JST PH2.0 connector.  To change keys, one just changes the tiny board (BOM < 1€ in singles); works even for IoT MCUs.

[DiTBho]

ah, got it, brilliant 

[westfw]

when the TFTP client wants to load "bootfile", it first requests file "bootfileSIZE"Your own TFTP server synthesizes its contents

I like it.  Since I was thinking along essentially identical lines.

You could request "bootfileMETADATA" and put whatever you want in there.

[Nominal Animal]

Exactly.  I only suggested changing the content (synthesized or not) to human-readable decimal string form, because 1) that has no endianness issues, 2) the conversion overhead is absolutely neglible compared to the actual transfers over even the fastest 10G transports even if you used BASIC on an 8-bit computer to implement the TFTP client, and 3) it is very KISS and understandable to users: a single line explanation suffices to anyone who can set up a TFTP server!

And yeah, it definitely could contain more, for example two lines of form
    size-in-bytes
    sha256sum-in-base64
which would let capable TFTP clients use the size in bytes for progress meter, but also calculate the sha256sum of the transferred firmware file for integrity verification.

If you don't want to add encryption but would like to add passphrase-based authentication, you could add third line with
    sha256hash-of-above-sha256sum-and-the-passphrase
so that the sha256 hash of the concatenated sha256sum of the data file and the passphrase forms an unreversible verification that whoever sent the data, knew the passphrase too, intended exactly that data to be received, and no man-in-the-middle fiddled with it.

(Obviously, I suggest a line-based form with ASCII contents.)

I'm not exactly sure what the "SIZE" or "METADATA" filename part should be.  DiTBho has some filename restrictions, and a single character is compact, but I'm leaning towards something like .size, because even if it contained more than just the size, it would be perfectly intuitive way to associate it with the firmware.  After all, if you saw
    mygadget-fw-20250617-v203.bin
    mygadget-fw-20250617-v203.bin.size
I'm pretty sure you would have a correct intuition about their relationship.

A TFTP server that can synthesize (without creating the metadata file) the metadata on-the-fly should probably still use the file if it exists.
I don't think I'd even add a modification timestamp or size checks (to "auto-detect" stale metadata files), because, well, KISS.

(And note that a synthesizing server can choose to synthesize the size only, the size and the checksum, or the size, checksum, and verification hash, for example based on server configuration.  In other words, the above order of these three lines isn't random, it was intentional on my part – although I didn't think of the checksum and verification hash before westfw hinted at such above.  )

[mariush]

using .size would make most sense to me, provided ONLY the file size is inside the file  ... the parser would be really simple... parse 0..9 and stop at the first character that's not 0..9

use maybe .info or some other extension if you want to have more stuff ... could be something simple as 

key [ = | : ]value [ separator ]   (repeat with next key+value pair )   where separator is limited to space, tab, ";" or "," or newline characters \r \n ... ex  .. optionally maybe accept " " or ' ' for strings to support those separator characters in strings.

"size=100, crc32=afafafaf, sha256=af2134..23"

Keeps it human readable and writable and editable and is less complicated to parse than JSON or more human readable than bencoding for example.

[Nominal Animal]

I think .info would work well, but your protocol suggestion is definitely not KISSy.  It contains unnecessary fluff.

My suggestion for the contents was basically
    LWS* size [ LWS+ checksum [ LWS+ authhash ] ] LWS+
where LWS = (HT | LF | VT | FF | CR | space).  Ordered, with size required, checksum optional, and authhash optional but only allowed if checksum is also listed.  You know, one to three values, parsing trivial, minimal space overhead.
The required trailing whitespace is there to ensure the recipient is absolutely sure the data is complete and was not truncated.

If you want a keyed format, simplest is
    ( LWS* key HWS+ value HWS* NL )+ LWS*
where HWS = (HT | space) and NL = (LF | CR).  For example,
    size size
    sha256 checksum
    auth authhash
But, then you have to handle the cases where size is not specified, or authhash is but checksum isn't, and so on.  Not KISSy.

[westfw]

I think .info would work well

Note that  ".info" is a file format already in use for providing documentation/manuals for many GNU projects.  A sort of 1989-era semi-hypertext format
https://www.gnu.org/software/emacs/manual/html_node/info/index.html

[DiTBho]

I have a simple rule: one request, one service.

[paulca]

Errors?  There are no errors.  There is no failure.  There is only reset, retry.  You are either successful or retrying.  I have had modems trying to connect for years, they don't seem to mind.  Never seem to get bored and error out.  Like who they gonna call?

[bson]

If the file is too big, the bootloader allows you to choose a different profile, which has a smaller initrd.

Both the kernel and initrd at least in the past could also be compressed; not sure if that's still the case.

tftp is still used, especially by kernel developers.  You can have simply add a make target to deposit a bootable image or images in a tftp folder somewhere, then boot it on one or multiple test systems.  When I did this at Sun (Solaris kernel) I had a sun4u on my desk, a sun4u to test on, and sun4m and sun4c systems to verify compatibility on.  We also supported old versions of the system until EOLed, so sometimes when a critical bug was fixed I'd have to backport the fix to multiple older versions, build and test those.  Sometimes the fixes ended up differing substantially.  tftp was a godsend for this.

The Sun tftp openboot loader produced the classic Sun ASCII spinner.