Help Me with My PIC Code Assembly, I am falling apart! Which Byte is Which?

[SuzyC]

Here's the problem, I don't know what I am doing.

I want to program a PIC 8-bit MCU using another MCU. The target chip in this case is the PIC18F2539..

(1) I wrote a program in C
(2) The program is compiled and the output result is a ASCII Intel .Hex file ready to feed to the chip.
(3) But I am a little confused, which byte  is which??

Ok, so here's the first few lines of the .Hex file:

:040000002BEF02F0F0
:0400080047EF05F0C9
:10001800D8CF41F0E8CF42F0E0CF43F08B88F2A48C
:1000280001D001D05ED0F90ED76E9F0ED66EF29435
:100038006A50400AD8A401D001D003D02B2A2C2A18

Now I know the format of the .hex file

Take the last line above, for instance, the payload bytes(grouped in sets of four consecutive) are:

6A 50 40 0A     D8 A4 01 D0     01 D0 03 D0       2B 2A 2C 2A   

And I guess the starting address for these bytes to be wrote to is 0x38

Fine enough, but the devil is always in the details!


The Programming Datasheet for this chip I got from Microchip shows exactly how to program this chip, all I gotta do is figure out what it is trying to tell me.

I understand how to do the programming to accomplish the instructions below, except I am a bit confused on how to get from the
ASCII .hex representation to the format described below in the 'Sequence" below:

Basically my question boils down to this: Which byte is which?

To program the chip, 8-words are loaded into a buffer, then the buffer is written to the chip..no problem!

Here's the steps in question:


Again:

Load Address Pointer Upper, High and Low


4-bit Command          Payload            Core Instruction
1100                         <LSB><MSB>    Write two bytes and post increment  by 2
1100                         <LSB><MSB>    Write two bytes and post increment  by 2
1100                         <LSB><MSB>    Write two bytes and post increment  by 2
1111                         <LSB><MSB>    Write two bytes and start programming

//the code above is sending a serial-bit stream to the MCU to be programmed in a LSbit to MSbit order.

Low=Low+8;    //increment Low address pointer by 8 and loop back to Again until all bytes from .Hex are programmed into the chip.
if(Low<LastAddress2B_Programmed) goto Again;

So here's the question, as you see the first two bytes from the example line of code above are 6A 50

Now to present these bytes to the instruction shown, which byte is <LSB> and which byte is <MSB>

I get it that I must store them in an Uint16 word, lets call the variable name DW

DW is the 16bit Uint16 fed to the Load2Bytes and Increment by two Table Write  command.


Looks easy, but I am not sure how to proceed.

I will program my work in C.

If I parse and convert the ASCII hex literals into sets of two, convert the values to a 16-bit integer and
and store them in a Uint16 type array called PgmCode[(size=num of prog words ], I first must know which byte is which.

But storing the bytes in an integer would have a <Hibyte><LoByte> order of bits.  and reversing the bit order would mean a lotta code for each word write!

Exactly the opposite of what is needed.

[Buriedcode]

Ok, your first post is a little off-putting - alot to read.  So I'll dumb it down a touch just to clarify:

You wish to program a PIC, specifically a PIC18F2539, using another microcontroller, be it PIC, AVR etc.. I have done this on several occassions, but it was years ago, on much older hardware (atmega88A programming various other PICs from flash)

It might be easier if you converted the HEX to a BIN so you have the raw bytes, and I'm pretty sure the PIC is 'little-endian'

Looking at my code for the above project, after a bulk erase (programming address counter sets to 0) I read two bytes from the flash, then combined them to a 16-bit word, little endian. So read byte1, then byte2, then word = ((byte2 << | byte1). 

You could store the bytes as .. well, uint8, and increment the pointer twice in a loop, or just do a byte swap when you read it as a uint16 before programming it in.

Perhaps I'm not seeing the problem.. theres many many ways to do it, just depends on where and how you're storing the firmware.  In my case I had plenty of flash to play with so I just wrote up to the highest address needed, and stored in the first bank of flash that was the index.

[IanB]

I think you are overthinking this.

A .hex file is an ASCII file, and the data records in the file are lists of bytes.

So you have a list of bytes: 6A 50 40 0A D8 A4 01 D0 01 D0 03 D0 2B 2A 2C 2A

These bytes can be grouped into words: 6A 50 | 40 0A | D8 A4 | 01 D0 | 01 D0 | 03 D0 | 2B 2A | 2C 2A

In each word the first byte is the MSB and the second byte is the LSB. So for instance, in 6A 50, the MSB is 6A and the LSB is 50.

(Actually, maybe not. See below. I think the byte data is meant to be the order of the raw bytes in the memory of the target device.)

[IanB]

OK, maybe I am a little too sure about this.

It does seem like the data bytes in the .hex file are meant to be in consecutive byte addresses in memory.

So probably you should use that to determine whether to swap the bytes or not.

Couldn't you try it both ways and find out which one works?

Looking at documentation for whatever created the .hex file might help.

[cv007]

Intel hex data is in byte format only, from lower address to higher address (keep in mind the 'Intel' in Intel Hex format). So the byte order is the same coming in as going out (lower byte first, then upper byte).

'6A' -> 0x6A -> low byte (lower address)
'50' -> 0x50 -> high byte (higher address)

uint8_t tmp[2];
tmp[0] = 0x6A; low byte
tmp[1] = 0x50; high byte

*(uint16_t*)tmp = 0x506A;
byte order still the same-
we just see it 'reversed' so it looks normal to us
the compiler knows 0x6A is the lower address byte

If you really want to bend the mind, decode the first hex line-
:040000002BEF02F0F0

2BEF -> 0xEF2B -> 0111 1111 0010 1011 (Msb->Lsb 16bit program word)
                  0111 1111 kkkk kkkk -> GOTO
02F0 -> 0xF002 -> 1111 0000 0000 0010
                  1111 kkkk kkkk kkkk (GOTO part2)

2BEF02F0 -> GOTO 0x0022B  (that's my guess)

(I found the instruction set encoding in the datasheet- and if I am correct, my original reply is valid)

[SuzyC]

Thanks very much to everyone, especially double thanks to IanB for twice using some precious time to help me!
And thanks again to cv007 who has shown me the right place to look (first line of .hex) for the answer and to understand the the answer I am seeking is in the first line of the .Hex.

IanB:  Here's the facts:

The 18Fxx39 specsheet says that the LSByte of every program instruction word must be stored in Flash at an even address.
Further it says that only three instructions are 4-bytes and are special for use with tables, so I know I need just work with two bytes per instruction.

The first .hex line starts at an even address, address=0, so the LSB of an instruction could be stored there. Every MCU needs rreset  boot code at a fixed reset starting address whose function is just to make a jump to the main() code for initialization.
The first two bytes do look like a Goto, see attached.

The Intel .Hex spec says the payload bytes are consecutive in program memory,  but words stored always in Little Indian order.

The programming code "Sequence" above wants a reversed-order integer as a payload, but doesn't say how it will store the instruction payload in memory!

What threw me for a loop is the  <LSB><MSB> in the programming sequence. It says the 16-bit bit integer payload has to be created in reverse order.

I often find myself overthinking some problem, and I again thank IanB who kindly reminded me of this.

I quickly realized that there are really only two possibilities..but I wanted to think this out, rather than rely on an empirical  approach to solving this enigma.

Your replies helped me determine that I needed to create a reversed byte ordered 16=bit integer array of code to be programmed.

Test Program:

TRISB = 0;            //Makes only all of PORTB digital I/O instead of default 3-state.
PORTB = 0x80;     //'If  the integer <LSB><MSB> order works, only bit7 of PORTB should be high.

Compiled this C-code. It created a tiny 20-byte .Hex file.

It worked!

[dannyf]

(3) But I am a little confused, which byte  is which??

i'm not sure why you care so much about it, as long as the program works correctly.

the best way to do it is to look at the disassembly or the lst file.

[amyk]

Figure 2-6 in the PIC18FXX39 programming specification gives an example, and I agree that it's confusing because the "data payload" in the example is written as "3C 40" when what it really means is "3C40", i.e. one 16-bit word, which is actually transmitted, in consistent little-endian order, as the two bytes 40 3C, or the 16 bits 0 0 0 0 0 0 1 0 0 0 1 1 1 1 0 0.

Note that everything is consistently little-endian, even the 4-bit commands; the "1100" table write is transmitted as 0 0 1 1.

If I parse and convert the ASCII hex literals into sets of two, convert the values to a 16-bit integer and and store them in a Uint16 type array called PgmCode[(size=num of prog words ], I first must know which byte is which.

You don't need to store it in anything if you're just sending the data to the PIC. Read two bytes from the hex file and interpret them as two hex digits forming a single byte. Send the byte starting with the lowest bit. Repeat for each byte.

[Bruce Abbott]

Note that everything is consistently little-endian,

Yes. All data is transferred one bit at a time, starting with the least significant bit. This is clearly shown in figure 2-6.

Not confusing at all.

[SuzyC]

Bruce Abbott,

Maybe to you, smartypants!

[SuzyC]

(3) But I am a little confused, which byte  is which??

i'm not sure why you care so much about it, as long as the program works correctly.

the best way to do it is to look at the disassembly or the lst file.

'Cause I'm trying to understand how to understand how things work, smartypants.

[westfw]

I'm pretty sure the PIC is 'little-endian'

A couple of people have mentioned "little-endian" without really clarifying what that means...
If you are going store a quantity that is larger than 8bits in memory that is addressed in 8bit chunks, you have two choices.

• You can store the least significant byte of the value in the least significant (lowest) address.  This has a certain elegance to it.  However, if you write the bytes in natural english order (left to right for low to high), the bytes will be backwards from their numeric representation.  0x1234 will read out as 0x34 0x12.
• You can store the most significant the byte in the lowest address.  Now numbers read naturally, and this has certain elegances to it as well.

The two schemes are called "little-endian" and "big-endian", respectively, based largely on a classic paper: "On Holy Wars and a Plea for Peace": https://www.ietf.org/rfc/ien/ien137.txt  (This in turn is based on a tale from Swift's "Gulliver's Travels" between groups fighting over which end of a soft-boiled egg should be opened.)  Intel Processors are little-endian, while the motorola 68k processors were (and are) big-endian.  ARM, MIPS, and PPC can go either way, and sometimes actually switch on-the-fly (though I don't think I've seen an ARM Cortex that doesn't decide one way or the other in silicon.)   Network data traffic tends to be big-endian (10.0.0.51) appears on the wire 10 0 0 51, but most network hardware ships the bits little-endian.  So network SW engineer like to have big-endian processors.   (Not that it matters too much when most of the big chips have an instruction that will swap the bytes appropriately far faster than the fetch from memory happened.)
Wikipedia also has a good article: https://en.wikipedia.org/wiki/Endianness
bi-endian compilers have come to exist as well.  You can tag your data with the endianess, and the compiler will insert those speedy swap instructions after the fetch of anything that doesn't match the native endianness.  Very hand for networking code.  (and it sure made quicker work of thousands-of-man-hours of big-endian networking software being ported to an x86 than previous manual efforts.)

[tszaboo]

You should read this:
http://ww1.microchip.com/downloads/en/DeviceDoc/31006a.pdf
Also, PIC18 program memory width is 14 bit as far as I remember, sooooooo... I guess that create funky issues.

[eugenenine]

I think going straight from C to machine may make the learning curve a little steep.  I'd start with learning to program in assembly if you want to see how the code translates straight to bytes, then when you understand that you can try to work to a higher level language.

High level language doesn't translate to bytes as straightforward as Assembly, its sort of like trying to translate from English to Chinese then to bytes where the words don't translate one to one and the sentence structure is different.  Compiler optimizations may take the same C instruction and compile it into different code depending on the code around it so the actual bytes on the chip could be different each instance within the same program.

[JoeyG]

Just load / import the hex file in to MPLAB 8.9xxx or  MPLAB X  and read the assembler output,

MPLAB X    is....   -->new project--  microchip embedded  -  prebuilt hex file

MPLAB 8.9xxx  is    - open  blank MPLAB ,   configure  -- select device     then file  import  hex file

[Ian.M]

Essential reading if you are messing around with Hex files and MCU programming:
Intel Hexadecimal Object File Format Specification
http://microsym.com/editor/assets/intelhex.pdf

Also see section 1.7.5 Hex File Formats of Microchip's
MPASM Assembler, MPLINK Object Linker & MPLIB Object Librarian User’s Guide (DS33014L).

You should also read about the HEXMATE utility in the XC8 User Guide - its  command line 'Swiss Army Knife' for PIC IntelHEX files, and wll vastly ease the task of generating test files for your programmer code.

MPLAB 8 is much better at off-the-cuff HEX file wrangling than MPLAB X is, so if you dont have it, and have a Windows PC grab a copy.  The workflow is much easier than MPLAB X as you simply set the device, then can import, edit memory and export directly, and don't have to mess around with loadables to import HEX files then building a project including the loadables to patch and re-export it.

There are various FOSS C libraries for parsing IntelHEX.  You can write one yourself but its wasted effort if you can find a lightweight enough FOSS one with a licence that you can tolerate.

If you have to store it in an embedded host, you do *NOT* want to use any flavour of IntelHex, though you may need to support PCupload/download in IntelHEX.   Instead, I suggest taking each line as you receive it and storing it in binary form, appending it  to the current block if it is contiguous or starting a new block if not.  Each block should be prefixed by a 32 bit start address and block length.  The end of the data structure can be indicated by a block of length 0, though you may wish to make the whole structure a doubly linked list.

You then have the problem of erase and write block sizes.  The FLASH memory can only be written in blocks of 8 bytes and erased in blocks of 64 bytes (the block size varies between device sub-families, but the write block is never larger than the erase block.  You have two choices for handling unused regions not specified by the HEX file - you can fill with 0xFF, the unprogrammed state, or you can read the chip for the current contents then replace those parts each erase block that you have data for with the new contents.   You will need the latter if you want to be able to do partial firmware updates, unless you take great care to make sure memory regions in your firmware are erase block aligned. 

The simplest/easiest/quickest method is to simply force all stored data block to be N*64 bytes long and bad with 0xFF.  This means you cant simply patch a const in the firmware unless that const was block aligned and the rest of the block reserved (or used for other updatable consts, to be updated at the same time) when the firmware was compiled.  However it makes the programmer much much easier to write as you can simply erase a block then go back and write and verify it directly from your block buffer.

[SuzyC]

NandBlog

You should read this:
http://ww1.microchip.com/downloads/en/DeviceDoc/31006a.pdf
Also, PIC18 program memory width is 14 bit as far as I remember, sooooooo... I guess that create funky issues.

I know now what I am doing, but I thought I should comment on your reply..

Of course, thanks for your advice, but the document you mention seems only to deal with internal MCU register handling, so this doc unfortunately doesn't directly address the byte order problem that is the focus of my post.

In short, reading this would have just added a wild goose chase to my rabbit hunt.

[SuzyC]

I think going straight from C to machine may make the learning curve a little steep.  I'd start with learning to program in assembly if you want to see how the code translates straight to bytes, then when you understand that you can try to work to a higher level language.

High level language doesn't translate to bytes as straightforward as Assembly, its sort of like trying to translate from English to Chinese then to bytes where the words don't translate one to one and the sentence structure is different.  Compiler optimizations may take the same C instruction and compile it into different code depending on the code around it so the actual bytes on the chip could be different each instance within the same program.

Thanks for you advice, but your advice invites me to completely abandon my C-language skills and spend a few hundred hours learning how to work with assembly..only so then I would have the knowledge needed to answer my simple posting.

Hmmm, I dislike assembly language programming intensely because it forces someone to learn so much about a non-portable skill.
Assembly language programming is a step towards learning everything about something that allows someone to know everything about nothing in general.

 C-language is universally transportable to other MCU's. It adds a very necessary layer of abstraction that avoids the problem of spending tons of hours of  valuable time learning some specialized assembly language skills, then only later do you realize you have become a one-trick pony in programming.

Thanks, but your reply is just a dangerous misdirection in solving my simple question.

[SuzyC]

Ian.m ...you should read what IanB has to post!

You say, he FLASH memory can only be written in blocks of 8 bytes and erased in blocks of 64 bytes (the block size varies between device sub-families, but the write block is never larger than the erase block.  You have two choices for handling unused regions not specified by the HEX file - you can fill with 0xFF, the unprogrammed state, or you can read the chip for the current contents then replace those parts each erase block that you have data for with the new contents.   You will need the latter if you want to be able to do partial firmware updates, unless you take great care to make sure memory regions in your firmware are erase block aligned. 

OMG, you are overthinking the problem..at least with programming MCU's.

(1) Forget about partially patching code.. Just erase the whole damn cabbage and program the whole device at once. After all, programming the entire device takes in the worst case(with a slow programming device) less than a minute.
(2)Then you can forget about 8-word alignment, 64-byte limitations, unprogrammed spaces, complex partial erasures, etc. and trying to squeeze code into small places.

[SuzyC]

westfw,.thanks for your lengthy reply!

I know I have my trouble with indians, as any horse-soldier programmer has, Native code has it's problems, but even after reading carefully the expansive explanation of .Hex files you've presented, it doesn't give me the answer I sought.

The fact is that my question was essentially simple and my confusion was created by the programming instruction "Sequence" I first posted that fails to give just a small example that would disambiguate the <LSB><MSB> statement in relation to just a PIC18 single instruction or else to relate to a single line of code presented in a standard Intel .Hex file.

Why is it in technical writing that it has become style to minimize explanation and abandon example?

Speaking of omissions, It is notable to mention that the Microchip Programming specsheet first states it is possible to completely erase the chip, then fails to mention the byte code necessary to do exactly this(although it does list the byte required for the five other individual area erasures)..I had to hack the Chip Bulk Erase sequence to finally discover the magic byte was 0x80 to do a complete chip erase!

[Ian.M]

Depending on the chip and your application circuit, you may not have enough Vdd supply voltage to use chip erase. e.g. many older chips couldn't be bulk erased reliably below 4.5V Vdd.  In that case you have no choice but to wrestle with individual 64 byte page erases.  However its conceptually simpler to loop through erasing them in sequence before writing any data rather than just in time erasing on-the-fly.

OMG, you are overthinking the problem..at least with programming MCU's.

(1) Forget about partially patching code.. Just erase the whole damn cabbage and program the whole device at once. After all, programming the entire device takes in the worst case(with a slow programming device) less than a minute.
(2)Then you can forget about 8-word alignment, 64-byte limitations, unprogrammed spaces, complex partial erasures, etc. and trying to squeeze code into small places.

Its a common requirement to preserve unit specific tables in program memory - e.g MAC addresses, Serial no.s or factory calibration data.  One can put it in EEPROM for PICs that have any, but the EEPROM is often needed for user data so being able to partially reprogram the chip is a frequent requirement.  However if you don't have this requirement, I agree its nothing but hassle and is best avoided.

[eugenenine]

I think going straight from C to machine may make the learning curve a little steep.  I'd start with learning to program in assembly if you want to see how the code translates straight to bytes, then when you understand that you can try to work to a higher level language.

High level language doesn't translate to bytes as straightforward as Assembly, its sort of like trying to translate from English to Chinese then to bytes where the words don't translate one to one and the sentence structure is different.  Compiler optimizations may take the same C instruction and compile it into different code depending on the code around it so the actual bytes on the chip could be different each instance within the same program.

Thanks for you advice, but your advice invites me to completely abandon my C-language skills and spend a few hundred hours learning how to work with assembly..only so then I would have the knowledge needed to answer my simple posting.

Hmmm, I dislike assembly language programming intensely because it forces someone to learn so much about a non-portable skill.
Assembly language programming is a step towards learning everything about something that allows someone to know everything about nothing in general.

 C-language is universally transportable to other MCU's. It adds a very necessary layer of abstraction that avoids the problem of spending tons of hours of  valuable time learning some specialized assembly language skills, then only later do you realize you have become a one-trick pony in programming.

Thanks, but your reply is just a dangerous misdirection in solving my simple question.

I did not say abandon C, I said learn some assembly to help understand it.  Sort of like learning addition before multiplication.  The concepts of assembly are the same no matter what the processor architecture.

[amyk]

Why is it in technical writing that it has become style to minimize explanation and abandon example?

Because those for whom those documents are written are not beginners expecting to be spoon-fed, and the extra material would not be helpful for them. For example, you are expected to know what a bit, byte, or endianess means, and be somewhat familiar with the architecture of the processor. Microchip's documentation is not the greatest, but if every document started from nothing they would all be 1000+ pages which are useless to almost everyone who reads them. To use an analogy, a calculus textbook is not going to teach you arithmetic. Perhaps you should think about the prerequisites before you start complaining that you can't understand.

"Why is it in asking questions that it has become style to minimize learning and abandon thinking?"

[cv007]

learn some assembly to help understand it

Just looking at the C compiler output (like lst file) is a good way to to get a handle on the instruction set and how it is used. Even though I would not want to write assembler (anymore), I would say being able to read assembler output from the compiler does come in handy and eliminates the 'black box' effect of just staying on the pretty side of the C compiler.

Programming flash- you cannot flip a bit from a 0 to a 1 without an erase. Which means writing a 1 will never change a bit. That info can be helpful to keep in mind. For example, if a partial 'page' (whatever that may be) needs programming, just set the (page) bytes you don't want to change to 0xFF (I'm assuming a full erase before programming). You could do a chip erase (whatever method), then read a hex record, program it, read the next record, program it, etc. - it does not matter if the hex record is aligned to the page size, or even if the hex records are in order (excluding non-data records). I'm not saying that's how it should be done, though.

In fact, with that knowledge, there is nothing preventing anyone from programming the whole pic one bit at a time (in any order). No, no one would want to, but its possible.