Man, I am really struggling with the data transfer code (for sending firmware images via the ROM bootloader). It's an easy thing in theory, but I'm having trouble balancing some conflicting goals:
The source file could be raw binary or in one of a few popular ASCII formats, like Intel Hex or Motorola S. In the raw binary case, things are easy. Figure out how much RAM is available on the target, read that much, and throw it at the serial port. Easy. To determine the segment size (the total data length), just ask the underlying file system how big the file is.
But, in the ASCII-encoded case, there are (potentially) address records that break the file into segments. The only way to know how large a data segment is going to be (which the target wants to know in advance) is to pre-scan the file and look for EOF or the next address record. Not a big deal. But, all the data is encoded in records (lines) of a certain size. There's also a line checksum, which makes it a bit of a chore to read anything less than a whole line at a time.
The problem with that is twofold: 1) In the case where this library is used on a micro to talk to another micro, it necessitates dealing with buffers between 256 and 512 bytes or so. If you have only a couple KB of RAM, that's significant, so you have to be as thrifty as possible while still supporting up to the maximum possible record length. 2) It becomes necessary to consider the RAM available on the target device as well, since all data has to be staged in RAM before being copied to flash.
Then, you also have to deal with two transfer mechanisms -- 8-bit binary, and UUencoded. In the latter case, we have another buffer to worry about (because the source data has to be encoded before being transmitted), it's again line-based, and of course there's the additional requirement of having to track and send a checksum every 20 lines. The checksum is a real bear, since the micro may tell you "send that last block again please" -- and therefore, you must retain 20 lines of data in a buffer (which puts a lower threshold on the resources required to play) OR retain enough info to seek back to the starting position in the source file. With all the various file position and memory address counters, the stack size becomes consequential, not to mention the amount of data to track and keep consistent with numerous entry and exit points in the logic.
Things get really fun when you're reading from a line-based file, and have to transfer using a line-based protocol, where the lines are not guaranteed to be the same length. Again, keeping in mind the size of target RAM available, and how that ties in with line lengths, so as not to put yourself in the position of having to break segments mid-line (thus another data point to track -- offset in the source file buffer.) UUencoded lines may be up to 45 data bytes in length, and I don't know but I suspect it's OK to send less than this on any line (that is, not just the final line), which is at least a small consolation.
Truly, the easiest way to handle all of this would be to read the entire source file, decode to binary, catalog all the addresses, then transfer the blocks from RAM. But on a device whose RAM is severely limited in comparison to potential flash image size, this is not even a remote possibility.
None of this even touches on the fact that flash has to be prepared for write and erased before it can be written to. This means having to map between addresses and sectors (not too bad), and dealing with offsets within sectors, and the rigid flash block copy sizes available. Furthermore, do you erase all affected sectors in advance (after pre-parsing the whole source file) and then write? Or erase one-by-one as necessary during the write op? Imagine what fun it would be to have two separate segments in a file that affect the same flash sector. A naive routine would erase the former data to prepare for the latter write. A smarter routine would track that, requiring an in-memory database of previously-written sectors. I don't suppose there should be retry attempts if the flash verification phase fails?
Sigh. What a PITA. There are certainly a few things the bootloader could've done to make this more tidy, although I doubt they were putting much emphasis on resource-constrained platforms as the image source. When you have 1GB of RAM at your disposal, many of these thorns cease to exist.
I asked for this, though. I could simply scrap the project and use existing tools. There are even some that claim to be portable.
Nah.