Author Topic: LPC from scratch  (Read 20327 times)

0 Members and 1 Guest are viewing this topic.

Offline Jeroen3

  • Super Contributor
  • ***
  • Posts: 3399
  • Country: nl
  • Embedded Engineer
    • jeroen3.nl
Re: LPC from scratch
« Reply #50 on: October 17, 2014, 09:19:08 pm »
By that if you mean masked ROM (which is not flash BTW) then it's unlikely. ... Do you have a credible source for your information?
Yes, rom. And I worked with one of the first lpc4350's while I was at nxp for a while. Those with the many bugs.
« Last Edit: October 17, 2014, 09:22:34 pm by Jeroen3 »
 

Offline SirNick

  • Frequent Contributor
  • **
  • Posts: 589
Re: LPC from scratch
« Reply #51 on: October 17, 2014, 09:21:59 pm »
Oh, dear me, no.  I needs muh white space.  Besides, this exercise is part wheel reinventing for the sake of a tutorial (on the protocol, on crossdev...), and part practical library so I can have MCU A flash MCU B.  I could be wrong, but I don't think there's a way to run Python code on a micro.  ;)

(I'm wrong about that, aren't I?  Thinking about it, I have no doubt someone out there has done this...)

Just found out Win32 has no getopt() equivalent.  Of course it doesn't.  Why would it?  :palm:  (getopt_long(), technically.)
 

Offline Jeroen3

  • Super Contributor
  • ***
  • Posts: 3399
  • Country: nl
  • Embedded Engineer
    • jeroen3.nl
Re: LPC from scratch
« Reply #52 on: October 17, 2014, 09:23:40 pm »
You should seriously consider opening you Microsoft vendorlock and find Qt.
 

Online nctnico

  • Super Contributor
  • ***
  • Posts: 18574
  • Country: nl
    • NCT Developments
Re: LPC from scratch
« Reply #53 on: October 17, 2014, 09:44:15 pm »
Do you happen to know if this area is actually readable during normal execution?  I would think so, since it has to be executable (and thus, readable) during boot.  But, who knows, NXP could protect it from user code.  It's apparently unavailable from the boot ROM interface.
... because there is no use to read the ROM through the regular programming interface. Years ago I found this nice web page which explained the inner workings of the LPC bootrom in great detail (the author used the route I described to get a boot rom dump and dissassemble from there) but I really can't that web page any more.
There are small lies, big lies and then there is what is on the screen of your oscilloscope.
 

Offline SirNick

  • Frequent Contributor
  • **
  • Posts: 589
Re: LPC from scratch
« Reply #54 on: October 18, 2014, 04:01:22 am »
You should seriously consider opening you Microsoft vendorlock and find Qt.

That's one option.  There are many libs out there which appear to help ease the divide ... Boost, Cygwin, etc.  However, for a small utility like this, it seems like overkill.  Most of those are C++, for one, which I don't know very well, and try to avoid unless there's a good reason.  (Afterall, you can always mix C with C++ apps, but not so easily the other way.)

Mostly, I have a hard time justifying dependencies for an app this size.  I am torn what to do about getopt_long() though...  Concede to using a library (and end up with .DLLs), include the .c and .h files necessary to build it within the executable (but then I have to comply with GPL), or re-write it myself (thus reinventing a much less interesting wheel).

Ugh.  How is this kind of thing not part of the standard C library on Windows?
 

Offline SirNick

  • Frequent Contributor
  • **
  • Posts: 589
Re: LPC from scratch
« Reply #55 on: October 18, 2014, 04:39:21 am »
Hey, while I'm thinking of it...  I haven't looked too closely at Intel Hex files before, so while writing a parser for them, I found out there is addressing info encoded in there too.  Go figure.

AFAIK, raw binary images don't indicate where the data should be located.  I.e., to upload a binary image to flash, you would have to specify where you want the image to be written.  Syntax to write 32KB from the image to flash might look something like this:

Code: [Select]
--write 0x00000000,32768,bin="image.bin"or
Code: [Select]
--write image.bin --address 0x00000000 --size 32768
But then we have Hex images, which DO have this information encoded into them.  In terms of flexibility to do "weird and wonderful things", what would your expectation be, given the examples above, but with a Hex file as the source?  Alternatively, which would be most useful?

A) Write up to 32KB of data, if the file has content that is supposed to be placed at 0x0000 0000.  I.e., obey address and offset from the file, and allow the user to subset this with the CLI options.  If the address isn't found in the source file, it's a no-op.

B) Do not support address and size fields on the command line for Hex files.

C) Use the address given on the command line as a base address and the per-line offsets are relative to that.  BUT-- In this case, what should be done about address fields in the file?  (That is, the record types that set a 16-bit segment address or 32-bit linear address.)  Ignore them?  Keep them relative to the base address?  Set the base address when encountering them, possibly ignoring the CLI?  Stop processing?

D) Ignore all address / offset info and treat the file as raw data.  (This seems dangerous, since there's no guarantee the data is meant to be contiguous.)
 

Offline zapta

  • Super Contributor
  • ***
  • Posts: 6004
  • Country: us
Re: LPC from scratch
« Reply #56 on: October 18, 2014, 05:08:55 am »
A quick update, I uploaded gerber files (2 layers, using Elecrow cam file) and ordered 10 1mm boards from elecrow. Will see how it goes.

https://github.com/zapta/arm/tree/master/mini-pro/eagle
Drain the swamp.
 

Offline gmb42

  • Regular Contributor
  • *
  • Posts: 184
  • Country: gb
Re: LPC from scratch
« Reply #57 on: October 18, 2014, 11:44:19 am »
Also:  Windows programming sucks.  FormatMessage() is like strerror() and sprintf() bundled into one, with a little iconv() thrown in for good measure.  But the real surprise was finding that the C99 integer types (intXX_t) don't exist in VS2010.  2010!!  ><  And, while VS2013 finally got around to adding them, at least according to a note on one of the betas, sprintf() works now, but sNprintf() didn't make it because they "ran out of time" implementing it.  So, when it comes down to adding a byte counter in your format loop, or having user code suffer buffer overflows, I guess the choice is obvious.  Oy vey......

For the int_XX types try including stdint.h, the header where the C99 standard says they should be defined.  It exists in my copy of VS2010, note that just having this header file doesn't make VS2010 C99 compliant.

For snprintf, use _snprintf, see here for more info.

For getoptlong, first note that it's part of the POSIX standard, and usually on Linux is provided by the GNU C library, e.g. from the GNUC C manual (my added emphasis):

Quote
The getopt and getopt_long functions automate some of the chore involved in parsing typical unix command line options.

Having said that there are getopt libraries for use with Visual Studio, e.g. here and probably many others.


 

Online nctnico

  • Super Contributor
  • ***
  • Posts: 18574
  • Country: nl
    • NCT Developments
Re: LPC from scratch
« Reply #58 on: October 18, 2014, 01:03:40 pm »
Using GCC (mingw32) + Eclipse instead of Visual studio gives you a programming environment which is much more like Linux. For using a serial port in a cross-platform way I suggest to look for a component called wxCTB: https://iftools.com/opensource/ctb.en.php
« Last Edit: October 18, 2014, 01:07:46 pm by nctnico »
There are small lies, big lies and then there is what is on the screen of your oscilloscope.
 

Offline paulie

  • Frequent Contributor
  • **
  • !
  • Posts: 849
  • Country: us
Re: LPC from scratch
« Reply #59 on: October 19, 2014, 02:21:00 am »
A quick update, I uploaded gerber files (2 layers, using Elecrow cam file) and ordered 10 1mm boards from elecrow. Will see how it goes. https://github.com/zapta/arm/tree/master/mini-pro/eagle

Are those for your "arm-mini-pro"? Why here?

If so I am quite interested because this looks like a perfect board for that chip. A couple friends might also like to build one and Gerbers would be helpful. Unfortunately I can't get into that page. Would it be possible to zip them up here or better still in the other thread?
 

Offline SirNick

  • Frequent Contributor
  • **
  • Posts: 589
Re: LPC from scratch
« Reply #60 on: October 19, 2014, 02:31:40 am »
For the int_XX types try including stdint.h, the header where the C99 standard says they should be defined.  It exists in my copy of VS2010, note that just having this header file doesn't make VS2010 C99 compliant.
Ah crap.  I'm speaking from the wrong end again.  You're right, stdint.h IS included.  It's just inttypes.h that doesn't exist.  So, intXX_t are available, but PRIu32, etc, are not.  I had imported my POSIX files directly, saw a bunch of squiggly lines (the IDE's way of saying "can't find this header"), and was surprised to see inttypes.h as one of them, so I started researching and found out it doesn't exist due to incomplete C99 support.  I found the MSIntType project on Google Code that has both stdint.h and inttypes.h, so I've been using those instead.  Totally missed the fact that stdint.h did actually resolve to a valid file.  Sorry about that.

Oh, and yes, I do get that C99 is more than just integer types.  Fun fact I learned recently:  MOST compilers are not fully compliant.  (Just closer than VS.)

For snprintf, use _snprintf, see here for more info.
Hey.. great!  Thanks!

For getoptlong, first note that it's part of the POSIX standard, and usually on Linux is provided by the GNU C library
Yep, understood.  I made it a point to look up native getopt-like CLI processing functions on Win32, and from what I could find via StackOverflow and MSDN, there really is no equivalent.  It seems like a fairly obvious omission.  Even GUI apps often have command-line options, so I figured there would be a built-in API to help with the typical chores associated with parsing them.

Having said that there are getopt libraries for use with Visual Studio, e.g. here and probably many others.
I found a few.  Some are direct ports of the GNU LibC versions, which come with the responsibility to uphold GPL terms, while others are written from scratch (and may or may not have licensing terms.)  Fewer ports of getopt_long(), although I did find at least one -- but again, a port of GNU code.  Hence the thinking-out-loud about whether I want to bother with DLLs, concede to GPL compliance, or go an entirely different direction.

Using GCC (mingw32) + Eclipse instead of Visual studio gives you a programming environment which is much more like Linux.
I looked into MinGW, which I might end up using eventually, but probably not on this.  I don't like Eclipse at all though.  I'm not a huge fan of VS either, although I do want to maintain VS project files.  CMake supposedly helps with that.  Yet another tool to learn... sigh...  ;)

For using a serial port in a cross-platform way I suggest to look for a component called wxCTB: https://iftools.com/opensource/ctb.en.php

That part's already done.  Windows doesn't use POSIX I/O calls, but the native API is close enough that a few light wrappers will do the job.  I have to abstract them anyway, since I want the libraries to work on various microcontrollers as well.
« Last Edit: October 19, 2014, 02:35:21 am by SirNick »
 

Offline gmb42

  • Regular Contributor
  • *
  • Posts: 184
  • Country: gb
Re: LPC from scratch
« Reply #61 on: October 19, 2014, 08:50:12 am »
For getoptlong, first note that it's part of the POSIX standard, and usually on Linux is provided by the GNU C library
Yep, understood.  I made it a point to look up native getopt-like CLI processing functions on Win32, and from what I could find via StackOverflow and MSDN, there really is no equivalent.  It seems like a fairly obvious omission.  Even GUI apps often have command-line options, so I figured there would be a built-in API to help with the typical chores associated with parsing them.

Unfortunately it has never figured in the Win32 API, so grabbing an external piece of code, or writing your own has always been the way.

I'm not a huge fan of VS either, although I do want to maintain VS project files.  CMake supposedly helps with that.  Yet another tool to learn... sigh...  ;)

CMake does produce good VS solution files (and Makefiles for other platforms) but the learning curve for anything other than a toy project is immense.  Makes ARM toolchain setup on Windows look like a walk in the park.  Their docs are very similar to MSDN, they describe all the functions in detail, but not how to bring it together in a coherent manner.
 

Offline SirNick

  • Frequent Contributor
  • **
  • Posts: 589
Re: LPC from scratch
« Reply #62 on: October 21, 2014, 12:27:11 am »
Well, new day new problem.  I traded out the LPC4078 dev board for a minimal LPC812 on breadboard setup.  My test app did not synchronize, so I ran through the process in Minicom to see what was happening.

I sent a '?', and got "Synchronized" back, so I sent "Synchronized" in return.  But, instead of getting "OK" from the micro, the cursor just moves back to the beginning of the line (ala carriage-return).  Hm.  I tried sending the clock speed ("12000") anyway.  I get as far as "12" and then each 0 just echoes back garbage characters.  So the session goes like this:

Code: [Select]
>> ?
<< Synchronized
>> Synchronized
>> 12000
<< (garbage after the '2')

I tried swapping between a Prolific USB serial to TTL cable and an FTDI equivalent.  No change.  Pulled out another breakout PCB and a new 812 -- same behaviour.  Just in case it had something to do with line endings, I emerged lpc21isp and it had the same problem my script has (Synchronizing.... timeout).

Any ideas?
 

Offline paulie

  • Frequent Contributor
  • **
  • !
  • Posts: 849
  • Country: us
Re: LPC from scratch
« Reply #63 on: October 21, 2014, 01:05:46 am »
Following your description I was able to get all the way to display version and command list. Initially under MSDOS using a turbo C 2.0 program then Python under XP. Both about 100 times easier than my experiences with dot net and C#. It looks like you are doing everything right so IDK. You're shorting the right ISP pin for your version? I know that is different for older chips.
 

Online nctnico

  • Super Contributor
  • ***
  • Posts: 18574
  • Country: nl
    • NCT Developments
Re: LPC from scratch
« Reply #64 on: October 21, 2014, 11:24:53 am »
just moves back to the beginning of the line (ala carriage-return).  Hm.  I tried sending the clock speed ("12000") anyway.  I get as far as "12" and then each 0 just echoes back garbage characters.  So the session goes like this:

Code: [Select]
>> ?
<< Synchronized
>> Synchronized
>> 12000
<< (garbage after the '2')

I tried swapping between a Prolific USB serial to TTL cable and an FTDI equivalent.  No change.  Pulled out another breakout PCB and a new 812 -- same behaviour.  Just in case it had something to do with line endings, I emerged lpc21isp and it had the same problem my script has (Synchronizing.... timeout).
I have had similar problems with lpc21isp and it seems to be an order-of-commands or timing issue. Flashmagic gets it right so I suggest to make a trace of what Flashmagic sends and mimic that.
There are small lies, big lies and then there is what is on the screen of your oscilloscope.
 

Offline SirNick

  • Frequent Contributor
  • **
  • Posts: 589
Re: LPC from scratch
« Reply #65 on: October 21, 2014, 05:58:42 pm »
You're shorting the right ISP pin for your version? I know that is different for older chips.

On these breakout PCBs, I have Gnd, Vcc (w/ 1uF), and a 10-47k pull-up on reset.  ISP_En (the pin that has changed since the early silicon) is left floating, so it will (and does seem to) enter ISP when there's no user code in flash, but I can pull it down manually if need be.  There's no external Xtal, it's using the internal RC.

I have had similar problems with lpc21isp and it seems to be an order-of-commands or timing issue. Flashmagic gets it right so I suggest to make a trace of what Flashmagic sends and mimic that.

You're right.  That worked.  It also worked fine when I used Flash Magic's terminal to go through the init process by hand. So there must be some difference with how the port is configured (??), combined with the code differences in the 800-series bootloader.  :-//  It's a start I guess.  Thanks for the suggestion.

On that note, here are some other differences I've found, for the interested:

The 800-series transfers code as binary streams, while the 4078 is UUencoded and uses Xon/Xoff flow control.  The 800 seems to (although the docs are sparse) transfer the entire image wholesale, while the 4078 wants 20 lines at a time, then a block CRC.  The 800 compensates by offering a separate CRC check command.  Finally, the 800 ignores the clock speed sent during init, using the internal RC for the entire session.  The docs say it's still required for "compatibility", presumably with the other versions of the bootloader, although the transfer mechanism has changed so the util needs to know the new protocol regardless.  I assume they intend for the util to work this out after getting the part ID and ROM version, although a different sync string could have easily identified the discrepancy, and intentionally broken unaware utilities from trying to communicate with unsupported ICs.  *shrug*
 

Offline SirNick

  • Frequent Contributor
  • **
  • Posts: 589
Re: LPC from scratch
« Reply #66 on: October 21, 2014, 06:29:37 pm »
And now it's working from my Linux laptop (where it wasn't before.) :wtf:  I hate intermittent problems.  Sometimes.
 

Offline cv007

  • Frequent Contributor
  • **
  • Posts: 487
Re: LPC from scratch
« Reply #67 on: October 22, 2014, 01:12:51 am »
 the host should send the same string "Synchronized<CR><LF>"

maybe its possible you are/were sending only CR or LF instead of CR+LF
can be easy to get wrong (every terminal app can have its own setting, etc)
don't know if the bootloader is fussy about it, but if its looking for both, only sending one of them could cause a problem
just a guess


 
 

Offline SirNick

  • Frequent Contributor
  • **
  • Posts: 589
Re: LPC from scratch
« Reply #68 on: October 22, 2014, 11:19:55 pm »
I thought that might be the case too, but now I'm pretty sure it wasn't:  The same code worked later; the same code worked on a different series part; I do in fact use /r/n to end lines (I double-checked); and the docs say the ROM should be resilient to CR, LF, or extraneous CR/LFs.

I dunno what the deal was.  I used two different cables, and two different breakout boards, and got (AFAICT) the exact same gibberish back every time.  Awful consistent to be a fluke.  I thought maybe Flash Magic did something magical to the board to make it behave, but I only used it on one of the two PCBs I have, and when I went back to Linux, both worked.

So ... full moon? :-//  I'm at a loss.  I'm hoping I find a bad jumper wire or something so I can get closure.
 

Offline SirNick

  • Frequent Contributor
  • **
  • Posts: 589
Re: LPC from scratch
« Reply #69 on: November 04, 2014, 10:41:15 pm »
Man, I am really struggling with the data transfer code (for sending firmware images via the ROM bootloader).  It's an easy thing in theory, but I'm having trouble balancing some conflicting goals:

The source file could be raw binary or in one of a few popular ASCII formats, like Intel Hex or Motorola S.  In the raw binary case, things are easy.  Figure out how much RAM is available on the target, read that much, and throw it at the serial port.  Easy.  To determine the segment size (the total data length), just ask the underlying file system how big the file is.

But, in the ASCII-encoded case, there are (potentially) address records that break the file into segments.  The only way to know how large a data segment is going to be (which the target wants to know in advance) is to pre-scan the file and look for EOF or the next address record.  Not a big deal.  But, all the data is encoded in records (lines) of a certain size.  There's also a line checksum, which makes it a bit of a chore to read anything less than a whole line at a time.

The problem with that is twofold:  1) In the case where this library is used on a micro to talk to another micro, it necessitates dealing with buffers between 256 and 512 bytes or so.  If you have only a couple KB of RAM, that's significant, so you have to be as thrifty as possible while still supporting up to the maximum possible record length.  2) It becomes necessary to consider the RAM available on the target device as well, since all data has to be staged in RAM before being copied to flash.

Then, you also have to deal with two transfer mechanisms -- 8-bit binary, and UUencoded.  In the latter case, we have another buffer to worry about (because the source data has to be encoded before being transmitted), it's again line-based, and of course there's the additional requirement of having to track and send a checksum every 20 lines.  The checksum is a real bear, since the micro may tell you "send that last block again please" -- and therefore, you must retain 20 lines of data in a buffer (which puts a lower threshold on the resources required to play) OR retain enough info to seek back to the starting position in the source file.  With all the various file position and memory address counters, the stack size becomes consequential, not to mention the amount of data to track and keep consistent with numerous entry and exit points in the logic.

Things get really fun when you're reading from a line-based file, and have to transfer using a line-based protocol, where the lines are not guaranteed to be the same length.  Again, keeping in mind the size of target RAM available, and how that ties in with line lengths, so as not to put yourself in the position of having to break segments mid-line (thus another data point to track -- offset in the source file buffer.)  UUencoded lines may be up to 45 data bytes in length, and I don't know but I suspect it's OK to send less than this on any line (that is, not just the final line), which is at least a small consolation.

Truly, the easiest way to handle all of this would be to read the entire source file, decode to binary, catalog all the addresses, then transfer the blocks from RAM.  But on a device whose RAM is severely limited in comparison to potential flash image size, this is not even a remote possibility.

None of this even touches on the fact that flash has to be prepared for write and erased before it can be written to.  This means having to map between addresses and sectors (not too bad), and dealing with offsets within sectors, and the rigid flash block copy sizes available.  Furthermore, do you erase all affected sectors in advance (after pre-parsing the whole source file) and then write?  Or erase one-by-one as necessary during the write op?  Imagine what fun it would be to have two separate segments in a file that affect the same flash sector.  A naive routine would erase the former data to prepare for the latter write.  A smarter routine would track that, requiring an in-memory database of previously-written sectors.  I don't suppose there should be retry attempts if the flash verification phase fails?

Sigh.  What a PITA.  There are certainly a few things the bootloader could've done to make this more tidy, although I doubt they were putting much emphasis on resource-constrained platforms as the image source.  When you have 1GB of RAM at your disposal, many of these thorns cease to exist.

I asked for this, though.  I could simply scrap the project and use existing tools.  There are even some that claim to be portable.

Nah.  :-/O
 

Offline westfw

  • Super Contributor
  • ***
  • Posts: 3124
  • Country: us
Re: LPC from scratch
« Reply #70 on: November 04, 2014, 10:58:19 pm »
Quote
the easiest way to handle all of this would be to read the entire source file, decode to binary, catalog all the addresses, then transfer the blocks from RAM.  But on a device whose RAM is severely limited in comparison to potential flash image size, this is not even a remote possibility.
Multiple passes?  I'm not sure why there is a device with limited RAM in the path; if you want to have some intermediate systems ("pocket programmer"), can't you massage the data BEFORE putting it on that intermediate system?

 

Offline SirNick

  • Frequent Contributor
  • **
  • Posts: 589
Re: LPC from scratch
« Reply #71 on: November 05, 2014, 12:21:00 am »
That may end up being the direction I take.  I would like it to be full-featured enough to read images from external storage (e.g., SD) but it's starting to look like more trouble than it might be worth.

E.g., to optimize delivery over serial would mean bulk transfers as large as possible.  Resource constraints beg for processing data in the smallest increments possible.

The code is more flexible and the API is cleaner if it can allocate memory freely.  Embedded systems appreciate pre-allocated buffers and aggressive re-use of existing memory.

The utility is more useful if it supports large numbers of devices, file formats, and transfer protocols.  The cost being abstraction and a bigger API -- i.e., larger memory footprint and code size.

While I would like for this to run on an ATtiny (arbitrary example), I'm starting to think it may not be realistic.  The smallest host device would have to be capable of holding the image contents in memory, or having enough RAM to support all the overhead of at least being able to locate, seek in, and read from a file on some other medium.  That rules out a whole class of lower-end devices already.


In other news, my 812 has gone unresponsive again.  It seems to be resetting after 15 characters are transmitted to it.  Sending '?' will get the "Synchronized" prompt, but no matter what I enter at this point, after sending 15 characters, it resets.  Sending another '?' or two will prompt another "Synchronized" from the chip.  Sending anything else will throw off the auto-baud routine and return garbage until power-cycled.

Here's an actual transcript, completely repeatable:

Code: [Select]
???Synchronized
ABCDEFGHIJKLMNO??Synchronized

No idea why this is happening.  It's not my day I guess.  ::)
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf