Avrdude output deciphering help needed

[chipwitch]

Attached is the verbose output from my attempt to program an Atmega328p-up MCU.  I would prefer doing it with the Arduino IDE, but the problem manifests itself there as well.

Equipment:
  Tried on two computers; a win7-64 and a winxp sp3 (32-bit)
  Tried with two different ISP programmers on both computers.  A usbtiny and an avrisp mkii (hacked with Dave's usb jumper mod).
  Chip platform is a no-frills perf board I assembled.  It consists of a 28-pin Ziff socket, a 6-pin dip header.  No external clock or power.
  Both grounds are grounded.  The Vcc and the other Vin pins are both fed.  I did not connect the Aref.
  Two weeks ago, I successfully uploaded a sketch to a an Atmega 328p-up on this programmer board using the mk ii and one of the two computers. 
  I have checked continuity on the board from header to chip and the ribbon cable from jack to jack.
  I also checked for shorts on every pin in the ziff socket against every wire.  All good.
  I've tried two atmega328p-up's

I continue to get the same output for every combination.

Software:
  I've downloaded and installed libusb and generated drivers for both ISPs.  Only installing one at a time.
  Downloaded and installed winavr.
  Using the latest version of Arduino IDE
  I have attempted to download and use the AVR Studio, but I don't know my way around.  I did try to communicate with the chip.  Seems like the software sees it.
 
Clues:
  All lights on both ISPs are green.
  No comms show connected on the tool menu in Arduino. 
  Device manager correctly identifies which programmer is plugged in under the "libusb-win32 devices" folder.  Unplugging the programmer results in the folder disappearing as expected. 
  No "comms" pop up or disappear when you plug or unplug a programmer. 

I only have one actual Arduino board.  It's currently in use managing an incubator so I can't use it to test for another week.

It just seems like the ports aren't connecting properly and I don't know why.  I posted this on the Arduino forum 8 hours ago.  No replies.  I'm sure I've left something out.  I've been searching all day for a solution.  TONS of people have a problem connecting.  There are myriad reasons they can't connect.  I've read quite a few and walked through more tutorials than I care to think about.  I'm really hoping someone can tell from the avrdude output what's going on and can point me in the right direction.  This just isn't making any sense.  One thing I did on both computers that may have caused the problem is I installed AVR Studio.  The drivers that are installed, "Jungo," have been reported to have problems with avrdude.  I did that AFTER the one successful upload I did two weeks ago.  But, I was able to uninstall it without issue.  Unless there's some residual somewhere, that shouldn't be a problem.
 

 

[ve7xen]

This is avrdude's generic and useless way of telling you it can't communicate with the chip. This can be a problem with your ISP or drivers, with your hookup, or with your target. If you haven't successfully used this programmer before, and since you are using Windows, my first guess would be that your libusb installation isn't set up properly. Not having used Windows in a long time, I know little other than that this is a huge pain in the ass.

[GiskardReventlov]

Your avrdude is from 2010, can you try the latest?  The whole setup sounds a bit complex with all those different softwares. I'd revert and start simple with just avrdude.

[chipwitch]

This is avrdude's generic and useless way of telling you it can't communicate with the chip. This can be a problem with your ISP or drivers, with your hookup, or with your target. If you haven't successfully used this programmer before, and since you are using Windows, my first guess would be that your libusb installation isn't set up properly. Not having used Windows in a long time, I know little other than that this is a huge pain in the ass.

I have used the programmer (the mk II).  Since both are giving me the same error, it's unlikely they're the problem.  I am new to MCU's and getting all the drivers right is like voodoo... it's bad enough with plug and pray.  Anyway, I think you're right about the drivers (libusb).  I just don't know what's missing.

Your avrdude is from 2010, can you try the latest?  The whole setup sounds a bit complex with all those different softwares. I'd revert and start simple with just avrdude.

It sounds more convoluted than it is.  I didn't start out with all of them.  For example, the driver generator is libusb.  It's kind of a generic program that detects your programmer and then compiles the driver.  Once compiled, it installs it.  First time with one programmer didn't work.  So, I generated another for the other programmer.  Then, I uninstalled the first one so the second one could install.  Not as bad as it sounds.  2010?  Yeah, I see that now.  Thanks.   I just downloaded it from sourceforge today!  It's the latest version (WinAvr.)  Maybe avrdude hasn't been updated in a while.  It isn't like it's feature rich.  I'm guessing WinAVR is just a shell or sibling program.  Just found a couple gui versions.  They might have some tools for troubleshooting.  I'm going to give them a try.

[GiskardReventlov]

The avrdude I have is 6.0.1, so you're a rev back.
From the avrdude-6.0.1/README

"The latest version of AVRDUDE is always available here:

  http://savannah.nongnu.org/projects/avrdude"

Avrdude works for me.  That's a good feature.


It's been said a picture == 1k words, maybe show your rig?

[ve7xen]

If you have a scope, disconnect the ISP connector and confirm that the RESET and SCK lines toggle when you try to program the chip. If you get that far, chances are something is wrong with your target board or ISP connection. If you don't see any activity, then it's a driver problem. LibUSB on Windows is a farking mess, thanks to Windows' inflexible driver architecture. If you replaced the mkII libusb driver (called something else, I think?, but it's just a commercial compile of the open source driver) with the one for USBtiny, one or the other is probably broken.

If you have a mkII, I would go through Windows' "purge all the driver crap from this" process for the drivers, then reinstall AVR Studio and the USB drivers from there, and use AVR Studio to write your image. If that works, try avrdude.

[GiskardReventlov]

ve7xen has reminded me that I had not set a jumper to the RESET when I first tried to use avrdude. When I connected that I was able to program my atmega32.

[zapta]

I am using this script to program blank 328p's with avrdude

https://github.com/zapta/linbus/blob/master/analyzer/firmware/icsp_program_all.sh

Note the -B flag, the 328p comes stock with 1Mhz clock and the max ICSP clock speed you can use is 1/4 of the clock speed (per Atmel documentation). The -B reduces the programming speed. Learnt it the hard way. Maybe related to your problem.

[chipwitch]

Giskard, thanks for the info and the link, but I have a hard enough time keeping up with all the idiosyncrasies of Windows without adding Linux to the mix.  I've tried.  Besides, even according to the avrdude docs, they even recommend using WinAVR's version of avrdude for windows installations.  It worked once... it has to work again.  This has to be a problem with the driver.  I downloaded avrdudess, a gui for avrdude.  Looks like avrdude can't see the port.  It won't accept "usb" for the -P flag either.  Blank doesn't work.

Even though I still think it's a driver issue, attached are the photos of my rig.

Sorry about the glare

[chipwitch]

Ve7xen, I'll try it.  What am I looking for?  Reset will be 5v, then go low?  Same with Sck?  I'm don't know my way around the scope very well yet, but I can read a dc voltage.

[chipwitch]

zapta, tried your settings... same response.  Thanks.  It was on my list of things to do (baudrate in particular).  But, as it DID work once without changing the speed... 

[leppie]

Maybe you need an external clock?  If these have been pre-programmed with an Arduino bootloader that will most likely be the case.

[miguelvp]

Scratch that, found this link and you don't need that pin (16):
http://allaboutee.com/2011/05/11/how-to-program-an-avr-microcontroller/

some more info that might help
https://www.sparkfun.com/tutorials/93

[chipwitch]

I went back to the avrisp mkii.  Plus, I typed in four "-v" (verbose).  Someone who knows what this means might be able to confirm if there is some data being sent and received.  Looks like something is going back and forth?

[chipwitch]

Can anyone answer this question:  With the rig I'm using, (USB powered chip, usblib driver, avrisp), in the Arduino IDE, shouldn't I see a serial port just like when programming a full-fledged arduino board using the ftdi driver?

[ve7xen]

Ve7xen, I'll try it.  What am I looking for?  Reset will be 5v, then go low?  Same with Sck?  I'm don't know my way around the scope very well yet, but I can read a dc voltage.

Yeah, the ISP interface is active while ~RESET is low, so you'd see it at Vcc then drop before programming. SCK is the serial clock, so you should see it toggling at some 100s of KHz. It might only burst for some ms before giving up though, so set your trigger up at say 1V and keep your eye on the TRIG indicator when you try to program it, you might not catch it on the screen. Or if you have a digital scope, set it up to do a one-shot when RESET goes low and you should see SCK toggling shortly after.

As leppie mentions, did you (or anyone) ever program the fuse bytes? If this was once an Arduino ATmega it will require an external clock.

Can anyone answer this question:  With the rig I'm using, (USB powered chip, usblib driver, avrisp), in the Arduino IDE, shouldn't I see a serial port just like when programming a full-fledged arduino board using the ftdi driver?

No. The Arduino by default has a serial bootloader that allows loading code via the serial port (or some variants use DFU over USB instead, but both have custom code loaded on the AVR). A blank one from the factory won't have that serial bootloader, and your rig doesn't provide a serial port anyway. You're using the ISP programming interface that is built into the chip itself, which is based on SPI not UART. You can load such a bootloader if you want, but then you will need a serial converter as well.

[dannyf]

This has to be a problem with the driver.

The most important thing in debugging is to have an open mind. Rarely, the bugs are where you think they are.

I have used avrdude lots of times under windows and they are generic enough to give me any problems. So yours should work. I would check wiring, pin outs, connector orientation, etc. before doing something crazy.

[chipwitch]

Danny, I've already double checked the things you listed.  I am open to it being something else, but as the clues pile up, it looks more and more like driver.

[chipwitch]

Ve7xen, I'll try it.  What am I looking for?  Reset will be 5v, then go low?  Same with Sck?  I'm don't know my way around the scope very well yet, but I can read a dc voltage.

Yeah, the ISP interface is active while ~RESET is low, so you'd see it at Vcc then drop before programming. SCK is the serial clock, so you should see it toggling at some 100s of KHz. It might only burst for some ms before giving up though, so set your trigger up at say 1V and keep your eye on the TRIG indicator when you try to program it, you might not catch it on the screen. Or if you have a digital scope, set it up to do a one-shot when RESET goes low and you should see SCK toggling shortly after.

Just waking up here.  Please, bear with me.  I have an older Tektronix digital scope.  I'll see if it can do a one-shot.  Without a function generator or AC PS, I'll have to set up something to practice catching a burst on.  It does record... I think.  I'll try that after I've got some coffee lapping through my veins.

As leppie mentions, did you (or anyone) ever program the fuse bytes? If this was once an Arduino ATmega it will require an external clock.

Right!  I forgot to respond to leppie about that!  I meant to when I first read it. 

First off, wouldn't the verbose output from avrdude show whether or not it has a bootloader installed, somehow?  Maybe it is showing in the output.  It looks like it's stopping communication, to me.  I don't know.  If it is stopping, then it has to be communicating at least partially and maybe there is a bootloader on it.

I bought all my Arduino stuff maybe two years ago (that goes for all my electronics stuff).  Life got in the way before I actually got around to doing anything with any of it. Yes, basically, I've only been doing Arduino and electronics for the last month.  So, I'm not an air-head, though I am old and getting more forgetful.  Two years is a long time to remember, but I think I bought a few 328p's from Mouser.  They shouldn't have bootloaders on the them.  Some, have a sticker on the tops, labeling the pins.  They're likely from Sparkfun or Adafruit.  The 328p I successfully uploaded to a month ago using the rig in the photo, was without the label... so no bootloader on those.  However, I have clicked on "Upload bootloader" on the IDE menu, in frustration trying to get them to communicate.  It's conceivable that the chip I've been using does now have a bootloader.  I'm guessing you can tell by looking at the fuse settings?  Aren't they indicated in the avrdude outputs I've posted?  I don't know what that would look like.  If the fuse settings are how you erase a bootloader (I believe I read that somewhere), then I'm pretty sure I tried the right settings.  In Zapta's post, he showed the fuse settings he uses for blank 328ps.  I tried them and got similar if not the same output.  Now, if his fuse settings pre-suppose the bootloader is absent and doesn't actually erase the bootloader, I wouldn't know.  What all this means is I do have chips I'm 99% sure have no bootloader.  The two chips I've been using throughout this entire thread are NOT among them.  I'll try one of the mouser chips before proceeding.  Avrdude is like a long chain with all but the two end links screened by something impenetrable.  If any one of the many links isn't welded closed, you can pull on one end link all you want with no effect on the other... Now, find the link!  Frustrating.

Can anyone answer this question:  With the rig I'm using, (USB powered chip, usblib driver, avrisp), in the Arduino IDE, shouldn't I see a serial port just like when programming a full-fledged arduino board using the ftdi driver?

No. The Arduino by default has a serial bootloader that allows loading code via the serial port (or some variants use DFU over USB instead, but both have custom code loaded on the AVR). A blank one from the factory won't have that serial bootloader, and your rig doesn't provide a serial port anyway. You're using the ISP programming interface that is built into the chip itself, which is based on SPI not UART. You can load such a bootloader if you want, but then you will need a serial converter as well.
[/quote]

So, is there ANYTHING in the Arduino IDE that would indicate when you're connected via SPI?  When connected UART, the indication is indirect... the port will appear and disappear as you plug it in.  There's nothing like that happening with SPI?

I have a Sparkfun FTDI breakout board.  I was going to hook it up to my rig on the breadboard to try that, but noticed it has a mini A connector.  I thought I had a cable, but they're all micro and standard A's.  Maybe I'll try replacing the mini port with a micro.  I think I have one or two I've salvaged from old boards.

[chipwitch]

Ve7xen... you da man!  Got it.  I think there were several things going on.  Driver conflicts etc.  I tried a never before programmed 328p, without the label, with the MK II controller and it worked!!  I retried the usbtiny, with the exact same everything (except changing the controller on the command line, obviously) and it seems to be faulty.  This wasn't helping.  Anyway... it's now working.  Thank you so much.  16 hours this took...   

[chipwitch]

I also managed to upload a sketch with Arduino... on my 64 bit machine.  So, looks like it's all good

[ve7xen]

First off, wouldn't the verbose output from avrdude show whether or not it has a bootloader installed, somehow?  Maybe it is showing in the output.  It looks like it's stopping communication, to me.  I don't know.  If it is stopping, then it has to be communicating at least partially and maybe there is a bootloader on it.

The (potential) issue is that the chip itself has no clock signal. AVRs have 3 fuse bytes that you set to determine what their clock source is. By default, they use an internal RC oscillator, but many applications will use an external crystal or clock source instead. Once you've programmed those fuse bits to use an external clock, the chip won't do anything until you provide one, including responding to ISP requests. So yeah, if you've set the fuses to require an external clock, you need to provide one to get any response at all from ISP, and the ISP will think no chip is connected. I believe (though don't know for sure) that the Arduino has a clock crystal onboard, so the ATmegas that come with them would have the fuses programmed for an external clock. There is also a fuse that allows using the ~RESET pin as a GPIO, which basically makes the ISP interface useless. ISP itself can be disabled too. It's easy to 'brick' a chip this way, requiring you to add an external clock or in the case of disabling RESET or SPI, you need to do high-voltage programming (which neither the mkII or USBtiny can do) to change it back.

Unlike many more modern microcontrollers, the AVR can't switch clock sources from software while running and only uses the semi-permanent fuses for this. ARMs generally can for example detect whether the external clock is running (or even if it stops during operation) and switch to the internal clock as required. In fact they don't provide any way to control the boot-time clock source, you must do it from code once the chip has started up.

I'm guessing you can tell by looking at the fuse settings?  Aren't they indicated in the avrdude outputs I've posted?

You can, but so far you haven't posted any avrdude outputs with it actually communicating with the chip. I believe if you just run avrdude without any write command it will output the current fuse bytes to screen. Check the datasheet (p 295) for their meaning, remember that a '1' means UNSET and '0' means SET for the fuse bits.

If the fuse settings are how you erase a bootloader (I believe I read that somewhere), then I'm pretty sure I tried the right settings.

The bootloader is just stored in flash, you can overwrite it as you'd overwrite any other code. You'll need to set the fuse bytes correctly for your application (clock source etc.) regardless.

Avrdude is like a long chain with all but the two end links screened by something impenetrable.  If any one of the many links isn't welded closed, you can pull on one end link all you want with no effect on the other... Now, find the link!  Frustrating.

Unfortunately for troubleshooting, embedded processors are pretty primitive devices, and the interface is by design simple and low pin count, so it can't really tell if you're doing something wrong. ISP tends to work, or not. There isn't much room for diagnostics.

So, is there ANYTHING in the Arduino IDE that would indicate when you're connected via SPI?  When connected UART, the indication is indirect... the port will appear and disappear as you plug it in.  There's nothing like that happening with SPI?

I assume it shows up in Device Manager somewhere? Whether the target is connected and properly working you need to query using avrdude or AVR Studio and see if you can read e.g. the chip ID. This is pretty much the same as how serial ports work, there's no indication if something is actually connected to the port or not, it just happens with Arduino that the 'port' (the USB->serial chip) and 'device' (AVR) are on the same board and permanently connected.

I don't know if the Arduino IDE supports ISP programming other than to load the bootloader, and once that's done you'd probably need the external crystal discussed above, as well as a serial converter (like your FTDI breakout) properly connected to communicate with the bootloader.

[chipwitch]

First off, wouldn't the verbose output from avrdude show whether or not it has a bootloader installed, somehow?  Maybe it is showing in the output.  It looks like it's stopping communication, to me.  I don't know.  If it is stopping, then it has to be communicating at least partially and maybe there is a bootloader on it.

The (potential) issue is that the chip itself has no clock signal. AVRs have 3 fuse bytes that you set to determine what their clock source is. By default, they use an internal RC oscillator, but many applications will use an external crystal or clock source instead. Once you've programmed those fuse bits to use an external clock, the chip won't do anything until you provide one, including responding to ISP requests. So yeah, if you've set the fuses to require an external clock, you need to provide one to get any response at all from ISP, and the ISP will think no chip is connected. I believe (though don't know for sure) that the Arduino has a clock crystal onboard, so the ATmegas that come with them would have the fuses programmed for an external clock. There is also a fuse that allows using the ~RESET pin as a GPIO, which basically makes the ISP interface useless. ISP itself can be disabled too. It's easy to 'brick' a chip this way, requiring you to add an external clock or in the case of disabling RESET or SPI, you need to do high-voltage programming (which neither the mkII or USBtiny can do) to change it back.

Most if not all Arduino boards have a 16Mhz crystal onboard.  Obviously, the MCU's are also preprogrammed with a bootloader.  I've read about the reset/GPIO sometimes presenting a problem, but I think I read that it's only on the smaller chips, like the ATTinys. Don't quote me on that.  High voltage programming?  One sec.... going to go make some popcorn.... okay.  Go on!  You don't say?  I have not read that anywhere!  Sounds like you're not really an Arduinite.  May I ask how you know so much about them?

Unlike many more modern microcontrollers, the AVR can't switch clock sources from software while running and only uses the semi-permanent fuses for this. ARMs generally can for example detect whether the external clock is running (or even if it stops during operation) and switch to the internal clock as required. In fact they don't provide any way to control the boot-time clock source, you must do it from code once the chip has started up.

Interesting.  I realize the Atmegas are toys compared to other MCUs out there.  In fact, shortly after discovering Arduino, I bought some TI Stellaris boards and chips because of their superiority.  I figured I'd be better off learning the Arduino first... lots more support and a heavily saturated user base.

I'm guessing you can tell by looking at the fuse settings?  Aren't they indicated in the avrdude outputs I've posted?

You can, but so far you haven't posted any avrdude outputs with it actually communicating with the chip. I believe if you just run avrdude without any write command it will output the current fuse bytes to screen. Check the datasheet (p 295) for their meaning, remember that a '1' means UNSET and '0' means SET for the fuse bits.

I'll do that.

If the fuse settings are how you erase a bootloader (I believe I read that somewhere), then I'm pretty sure I tried the right settings.

The bootloader is just stored in flash, you can overwrite it as you'd overwrite any other code. You'll need to set the fuse bytes correctly for your application (clock source etc.) regardless.

Am I to understand any speed external signal will work?  I read elsewhere that (or did you write this?) that it isn't enough to just connect a crystal to the crystal pins (with caps), but that you actually need a TTL signal?  At Vcc voltage I presume?

If you erase the flash and eeprom, does that likewise erase the signature?  If not, what determines the signature?  Can it be manipulated?

Avrdude is like a long chain with all but the two end links screened by something impenetrable.  If any one of the many links isn't welded closed, you can pull on one end link all you want with no effect on the other... Now, find the link!  Frustrating.

Unfortunately for troubleshooting, embedded processors are pretty primitive devices, and the interface is by design simple and low pin count, so it can't really tell if you're doing something wrong. ISP tends to work, or not. There isn't much room for diagnostics.

So, is there ANYTHING in the Arduino IDE that would indicate when you're connected via SPI?  When connected UART, the indication is indirect... the port will appear and disappear as you plug it in.  There's nothing like that happening with SPI?

I assume it shows up in Device Manager somewhere? Whether the target is connected and properly working you need to query using avrdude or AVR Studio and see if you can read e.g. the chip ID. This is pretty much the same as how serial ports work, there's no indication if something is actually connected to the port or not, it just happens with Arduino that the 'port' (the USB->serial chip) and 'device' (AVR) are on the same board and permanently connected.

I see.  And yes, I can see the usb connection being made and broken as expected in the device manager.

I don't know if the Arduino IDE supports ISP programming other than to load the bootloader, and once that's done you'd probably need the external crystal discussed above, as well as a serial converter (like your FTDI breakout) properly connected to communicate with the bootloader.

I can answer that.  I think loading sketches via ISP from the IDE was a problem for older versions, but the latest version can upload sketches as well as the bootloader.  No bootloader is required.  Nor an external clock in that instance.

Thank you so much.  This has been most educational!

[chipwitch]

Of course, this all begs the question, what's the easiest way to implement a clock with my rig so I can use it to program both bootloader and non-bootloader chips with the same board.  If the SCK sends a clock signal, why can't that be used?  And what does that signal have to connect to?

[ve7xen]

Most if not all Arduino boards have a 16Mhz crystal onboard.  Obviously, the MCU's are also preprogrammed with a bootloader.  I've read about the reset/GPIO sometimes presenting a problem, but I think I read that it's only on the smaller chips, like the ATTinys. Don't quote me on that.  High voltage programming?  One sec.... going to go make some popcorn.... okay.  Go on!  You don't say?  I have not read that anywhere!  Sounds like you're not really an Arduinite.  May I ask how you know so much about them?

We aren't talking about "Arduino" here, we're talking about AVR, since you're starting from bare chips and trying to use ISP programming (while Arduino typically uses the bootloader for loading code). I've worked with AVR for years, before Arduino existed. The tools you're having trouble with are tools for AVR development, not tools for Arduino development. If you want to do Arduino, buy one of the many Arduino boards that comes preloaded with a bootloader and all ancillary components already installed .

I would venture that many if not most uses of AVR do not include an external crystal, and certainly if you're trying to program it on a breadboard or homemade board that might not be an obvious requirement.

The RSTDISBL fuse exists on every AVR I've worked with. If you set it, you will be unable to use ISP to program the chip. The only way to fix this is to use the AVR's high voltage programming mode to reset the fuse. See http://mightyohm.com/blog/products/hv-rescue-shield-2-x/ for an Arduin-ified version of this.

Interesting.  I realize the Atmegas are toys compared to other MCUs out there.  In fact, shortly after discovering Arduino, I bought some TI Stellaris boards and chips because of their superiority.  I figured I'd be better off learning the Arduino first... lots more support and a heavily saturated user base.

Yeah, I find the AVRs a lot easier to work with. There is a lot of "boilerplate" setup code required for ARM.

Am I to understand any speed external signal will work?  I read elsewhere that (or did you write this?) that it isn't enough to just connect a crystal to the crystal pins (with caps), but that you actually need a TTL signal?  At Vcc voltage I presume?

It depends on what the fuses are set to. I believe setting them to external clock mode disables the crystal oscillator, so in that case you would need a full TTL clock (though a simple 7404 inverter or even a transistor can be used with the crystal to make a simple oscillator). Unless you've accidentally done a stupid though, it will probably be set up for a crystal, I don't think the external clock option would be that common. Any frequency within the AVR's operating envelope (or crystal oscillator envelope for the current setting; see the datasheet on clock selection) should work. Just keep in mind that the maximum ISP speed is IIRC 1/4 of the clock rate, so if it's running really slow (especially if /8 CKDIV is enabled) you may need to slow down your programmer.

If you erase the flash and eeprom, does that likewise erase the signature?  If not, what determines the signature?  Can it be manipulated?

The signature and some calibration values are stored in a special page of flash that isn't accessible via standard ISP. In double checking some things to respond to you here I found someone reporting on an undocumented ISP command that can enable you to write this data, and that it's possible for it to get corrupted if the right bit gets flipped during programming. Other than that though it's basically read-only, as designed.

Thank you so much.  This has been most educational!

No problem!