Has anyone tried the XC8 PRO compiler with AVRs?

[rstofer]

Now I'm getting confused!  If I stay away from Atmel Studio and MPLAB and use arv-libc and the current WinAVR or the Linux equivalent toolchain, can I compile unlimited code or not?  I don't need nor want the vendor tools when there are 'free' versions in the wild.

Other than Arduino, I haven't used an Atmel chip in at least 10 years but I had always though highly of the ATmega128 and I have written a bunch of code for it.  I might like to do so again some day and I would have a tendency to want to use the Linux tools along with avrdude.

Are there chips on the market for which the latest gcc is incapable of generating code?

[technix]

Are there chips on the market for which the latest gcc is incapable of generating code?

I have samples of ATmega4809 and ATtiny1617, some new 8-bit AVR, to test this with. AFAIK the entire SMART ARM line and most chip series released before Microchip acquisition has full GCC support.

[hans]

Microchip are hardly the only ones to sell license-protected builds of GCC (eg. Montavista, IIRC Mentor Graphics/CodeSourcery too). The GCC developers seem fine with this, as long as the source code is freely available.

Will someone set up a build server that automatically builds Microchip's GCC forks with license checks patched out? That should be fully legal (as indicated by GPL) and should result in drop-in replacements for Microchip's distribution.

Also this does mean it is full legal to apply binary patches to the Microchip GCC forks to remove license checks.

I wouldn't go as far as saying "full". These terms may apply to the GCC binary from XC compiler, but what about MPLAB X? I haven't checked, but it may have a constraint saying you can't use 3rd party or modified compilers with their software. After all, MPLAB integrates the compiler package, debugger and device support dialogs all together to make a fully functional toolchain.

Nonetheless, I'm still amazed why they want to hard-split their own customer base like that. Maybe (also guessing/speculating here) Microchip is having lots of relatively small customers in the thousand quantity region. The existance of MicrochipDirect and low-quantity production programming support would suggest that.
There could be too little margin in todays chips, that they want to make money twice of their customers by also selling them a "premium" compiler. If you have spend 1k$ on a compiler and only ever sell a few thousand 1$ parts, then that's probably more earned by the compiler than the actual silicon.

[JPortici]

don't forget that they keep increasing the price of older chips in order to
- move people to newer and better chips
- squeeze more money from people who can't or don't want to use pin-to-pin compatible. (if you want you can still get the ancient chips in mask rom or OTP memory, new stock)

In the end the code generated by XC8 is FINE for a free compiler. From time to time i run comparisons between free and pro code and the difference in code size is minimal, like speeding up the startup routine or using more the indirect registers. you just have to know how to write good code for your platform, the architecture is what it is so you can't optimize it that much.
the GCC license check can be bypassed easily as we saw in the past threads, if you want to compile GCC the source code is there in the download archives as it has always been (altough the AVR part of XC8 is not there yet..)
but -O1 is usually good enough (we ran some tests on generated code here in this forum some months ago, little difference for higher optimizations, the listing file was more and more confusing to read as the optimizer put pieces of code together and rearranged instructions in order to gain some more speed or space)

[westfw]

If I stay away from Atmel Studio and MPLAB and use arv-libc and the current WinAVR

There is no "current" WinAVR.  The last AVR build was in 2010, and has avr-gcc v4.3.3.The replacement since then has been primarily the "toolchain" CLI tools downloadable from ... Atmel.(Currently gcc 5.4.0.  The most recent avr-gcc "from source" is v8.2, I think)

The XC8-AVR contains a copy of AVR-GCC with Microchip XC8 license check

Yep.   Really a depressing step...

WWHackintosh<9983> /Applications/microchip/xc8/v2.00/avr/bin/avr-gcc --version
avr-gcc (AVR_8_bit_GNU_Toolchain_3.6.2_42) 5.4.0
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

WWHackintosh<9984> /Applications/microchip/xc8/v2.00/avr/bin/avr-gcc -Os /tmp/str.c
/tmp/str.c:1:0: warning: Compiler option (Optimize for size) ignored because the free XC8 compiler does not support this feature.

[cv007]

Really a depressing step...

I guess they figure if they can get a grand from the xc16/32 'pro' users, why not the 'pro' avr (and sam) users also.

Although they don't have an easy 'backdoor' option like xc16/32 (that I know of- no source released), the binaries can be changed easily enough even without the source as they use the same method as xc16/32 to check the license. Change a -1 to a 2 in the binaries .data section (3 binaries) and you are good to go.

The arm compiler they have in xc32 also restricts optimizations, so they are going all in with this idea.

The only positive is they currently provide the source code to xc16/32 from their web site. I can imagine the next step will be to require anyone wanting the source to send in their $100 to request a copy.

I guess we all can get into the compiler business.  Get the source code, change one line in the source code (for those that will request the source code), modify a few bytes in 3 binaries (no need to compile), sell the 3 binaries for $x, offer to provide the source for a $100 fee. Repeat for every version that comes out.

Anyone want to buy some binaries from me?

[technix]

Really a depressing step...

I guess they figure if they can get a grand from the xc16/32 'pro' users, why not the 'pro' avr (and sam) users also.

Although they don't have an easy 'backdoor' option like xc16/32 (that I know of- no source released), the binaries can be changed easily enough even without the source as they use the same method as xc16/32 to check the license. Change a -1 to a 2 in the binaries .data section (3 binaries) and you are good to go.

The arm compiler they have in xc32 also restricts optimizations, so they are going all in with this idea.

The only positive is they currently provide the source code to xc16/32 from their web site. I can imagine the next step will be to require anyone wanting the source to send in their $100 to request a copy.

I guess we all can get into the compiler business.  Get the source code, change one line in the source code (for those that will request the source code), modify a few bytes in 3 binaries (no need to compile), sell the 3 binaries for $x, offer to provide the source for a $100 fee. Repeat for every version that comes out.

Anyone want to buy some binaries from me?

Maybe we really should start a build server and releasing our own version of Microchip GCC’s with license checks patched out. We need to have a build server that can produce up-to-date binaries in time for every upstream release.

* XC8-AVR: just plain AVR-GCC from upstream.
* XC16: It is GCC after all, with a custom backend. We can take MCHP’s source code, strip out the license check, and rebuild as-is. When rebuilding we can even turn on C++ support.
* XC32: The XC32-MIPS part is based on GCC and AFAIK it generates worse code than stock MIPS32 GCC, and the ARM part is stock GCC (just like XC8-AVR) so the build server would build straight upstream MIPS32-GCC as XC32-MIPS replacement and make use of the ARM’s official release for XC32-ARM, and build stock upstream GCC-AVR32

[andersm]

Maybe we really should start a build server and releasing our own version of Microchip GCC’s with license checks patched out.

The sources build with the license check disabled by default.

The XC32-MIPS part is based on GCC and AFAIK it generates worse code than stock MIPS32 GCC, and the ARM part is stock GCC (just like XC8-AVR) so the build server would build straight upstream MIPS32-GCC as XC32-MIPS replacement and make use of the ARM’s official release for XC32-ARM, and build stock upstream GCC-AVR32

Microchip have added a lot of functionality to their GCC and binutils forks that are not upstreamed. A vanilla mipsel-gcc will not work as a drop-in XC32 replacement. Their GCC compilers also used to be based on CodeSourcery's/Mentor's codebase, but I don't know if this is still the case (upstream MIPS support was severely behind CS' releases for a very long time). It remains to be seen whether Microchip will add their modifications to their AVR and ARM compilers as well.

Note that eg. Ubuntu's package repositories have an avr-gcc that's built straight from Atmel's source releases, and MacPorts have a very up to date vanilla avr-gcc 8.2.

[technix]

Maybe we really should start a build server and releasing our own version of Microchip GCC’s with license checks patched out.

The sources build with the license check disabled by default.

This might actually be a GPL violation if code change is required to bring back that license check - the distributed source code does not match the binaries if the don't behave the same.

[andersm]

This might actually be a GPL violation if code change is required to bring back that license check - the distributed source code does not match the binaries if the don't behave the same.

Don't be daft. By that logic, any program that used #ifdefs would violate the GPL.

[Eturtle]

tested on arch-linux. enjoy O3,Os,lto with unmodified xclm

Code: [Select]

cd /opt/microchip/xc8/v2.05/avr/libexec/gcc/avr/5.4.0/

printf '\x90\x90\xc7\x05\x30\xed\xf8\x08\x02' | dd of=cc1 bs=1 seek=8175160 count=9 conv=notrunc
printf '\x90\x90\xc7\x05\xd0\xad\x0c\x09\x02' | dd of=cc1plus bs=1 seek=9241176 count=9 conv=notrunc
printf '\x90\x90\xc7\x05\x90\xcb\xec\x08\x02' | dd of=lto1 bs=1 seek=7578772 count=9 conv=notrunc


[cv007]

100 ways to mod, here is a single bit mod (if I could write to one single bit in a file, anyway).
83 f1 01   xor    $0x1,%ecx -> $0x1 -> $0x0 -> free license=0, 0 xor 0 = 0, if 0 then clear nullify optimization vars previously set
I guess changing a je/jne (74<->75) also qualifies as a single bit mod, but is not as much fun and I can't use /dev/zero in that case.

cd /opt/microchip/xc8/v2.05/avr/libexec/gcc/avr/5.4.0/

mod(){ dd if=/dev/zero of=$1 bs=1 count=1 seek=$2 conv=notrunc; }
mod cc1 $((0x7cbe74)); mod cc1plus $((0x8d0294)); mod lto1 $((0x73a4d0))

[blacksheeplogic]

WWHackintosh<9984> /Applications/microchip/xc8/v2.00/avr/bin/avr-gcc -Os /tmp/str.c
/tmp/str.c:1:0: warning: Compiler option (Optimize for size) ignored because the free XC8 compiler does not support this feature.
[/tt]

I worked in a team optimizing both the C & Fortran compilers for a very large company. We gave our optimizations to GCC only after out compiler had been out in the field for a considerable period of time (it varied).

It's expensive development and we invested millions into the compilers and processors. It has to be paid somehow and competitive advantage was one of the ways we justified this development.

[westfw]

BTW, the updated version of XC8 apparently supports -Os and -O2 even in the "free" version.I'm not sure about -flto...

We gave our optimizations to GCC only after out compiler had been out in the field

In this case, Microchip was/is turning OFF the availability of common optimization options implemented by the community and/or other vendors (perhaps even you) that have been available for years (are still available, if you download the "avr-gcc" binaries, or compile from source.)  It's just ... sleazy.

[technix]

Microchip are hardly the only ones to sell license-protected builds of GCC (eg. Montavista, IIRC Mentor Graphics/CodeSourcery too). The GCC developers seem fine with this, as long as the source code is freely available.

Will someone set up a build server that automatically builds Microchip's GCC forks with license checks patched out? That should be fully legal (as indicated by GPL) and should result in drop-in replacements for Microchip's distribution.

Also this does mean it is full legal to apply binary patches to the Microchip GCC forks to remove license checks.

I wouldn't go as far as saying "full". These terms may apply to the GCC binary from XC compiler, but what about MPLAB X? I haven't checked, but it may have a constraint saying you can't use 3rd party or modified compilers with their software. After all, MPLAB integrates the compiler package, debugger and device support dialogs all together to make a fully functional toolchain.

Nonetheless, I'm still amazed why they want to hard-split their own customer base like that. Maybe (also guessing/speculating here) Microchip is having lots of relatively small customers in the thousand quantity region. The existance of MicrochipDirect and low-quantity production programming support would suggest that.
There could be too little margin in todays chips, that they want to make money twice of their customers by also selling them a "premium" compiler. If you have spend 1k$ on a compiler and only ever sell a few thousand 1$ parts, then that's probably more earned by the compiler than the actual silicon.

MPLAB X is also GPL software, being based on NetBeans 8 or lower.

Microchip is slapping their license check code into GPL code left and right here, and disabling features based on that. MPLAB X is based on NetBeans, XC8-AVR, XC16 and XC32 are all based on GCC. The only part they have license check and being illegal to patch is XC8-PIC which is based on either proprietary code or LLVM/clang.

I think I am going to try set up and release manual builds of those tools with license checks patched out. My avr-gcc is not going to be whatever fork they are using, instead it is based on FSF mainline releases. XC16 and XC32 are patched Microchip code.

[cv007]

TW, the updated version of XC8 apparently supports -Os and -O2 even in the "free" version

Up to -O2, does not include -Os (both xc8-pic, xc8-avr).

I think I am going to try set up and release manual builds of those tools with license checks patched out

In the case of xc16/xc32, its not necessary as you need a 2 line text file to get all optimizations. For xc8-avr, I just showed 2 lines of script to take care of it. Why do you want to go through all that trouble to compile these into something that will be different than what everyone else is using? I'm not saying you can't do it, but am asking why would you want to?

Microchip is slapping their license check code into GPL

The actual license check code is a separate file. The calling of that external program is done from the gpl code, and the gpl code source is available to download. I didn't just magically figure out that changing the literal of an xor instruction results in no optimization nullification, I had help in the form of source code (still had to hunt where that source code ended up in the binary).

MPLAB X is also GPL software

http://wiki.netbeans.org/FaqCommercialApp

[JPortici]

It's just ... sleazy.

according to a guy formerly in the XC8 team, it took a long time for them to convince management to give away -O2 but they finally succeded. I have a feeling that the atmel deal has some influence in the decision. I hope this gets on to XC16 and XC32.. and in the end free full optimizations for all.

[technix]

I think I am going to try set up and release manual builds of those tools with license checks patched out

In the case of xc16/xc32, its not necessary as you need a 2 line text file to get all optimizations. For xc8-avr, I just showed 2 lines of script to take care of it. Why do you want to go through all that trouble to compile these into something that will be different than what everyone else is using? I'm not saying you can't do it, but am asking why would you want to?

For the peace of heart, that my compiler carrying the GCC name is vanilla GPL code, not a package infested with proprietary code.

Microchip is slapping their license check code into GPL

The actual license check code is a separate file. The calling of that external program is done from the gpl code, and the gpl code source is available to download. I didn't just magically figure out that changing the literal of an xor instruction results in no optimization nullification, I had help in the form of source code (still had to hunt where that source code ended up in the binary).

It appeared to me that recent versions of GCC has an internal integrity check that makes it nag when binary edited it. Patching out that code results in a binary that does not nag.

[cv007]

It appeared to me that recent versions of GCC has an internal integrity check that makes it nag when binary edited it. Patching out that code results in a binary that does not nag

Yes, they check if their xclm program is the 'real' one, but who cares. No one is touching that piece of software and it is free to do what it wants, and it doesn't matter. It will just tell the compiler executable (which calls it) that we have a free license. Not important. In the case of xc8-avr, I just changed a bit in the gpl binaries so optimization nullification is 'undone' as if we have something greater than a free license.

Unless you have a lot of free time, you are not going to roll your own xc16/32 version of gcc easily unless you stick to the sources they have provided. At that point... there is no point as you are then not gaining anything. In the case of avr/arm, there probably is no point either as you can simply tell mplabx to use another compiler- point it to the latest version of gcc-avr or gcc-arm in some folder on your pc, it will use it.

The source code is available from MCHP, and you are free to look at it, study it, and understand what its doing. You are also free to compile it if you wish.

[janoc]

WWHackintosh<9984> /Applications/microchip/xc8/v2.00/avr/bin/avr-gcc -Os /tmp/str.c
/tmp/str.c:1:0: warning: Compiler option (Optimize for size) ignored because the free XC8 compiler does not support this feature.
[/tt]

I worked in a team optimizing both the C & Fortran compilers for a very large company. We gave our optimizations to GCC only after out compiler had been out in the field for a considerable period of time (it varied).

It's expensive development and we invested millions into the compilers and processors. It has to be paid somehow and competitive advantage was one of the ways we justified this development.

Nobody disputes that it is expensive. But that is not an excuse to violate/ignore the license of the software that one is (or rather Microchip)  distributing and which saved even more R&D money (writing a semi-decent compiler for any language/architecture is expensive). I am sure Microchip's lawyers would not be happy neither if someone took their paid software and started distributing it or decompiling it or doing whatever else that their license forbids.

If a company wants to keep stuff closed and locked down then they shouldn't use GPLed software as a base of the derivative work, it is that simple. Either use something else that is less restrictively licensed (e.g. LLVM) or write your own code.

In Microchip's case this has been going on for at least 10 years:
https://www.embeddedrelated.com/showthread/comp.arch.embedded/91196-1.php

I suspect that they provide the uncrippled (including optimization, etc.) source code for download, just most people couldn't be bothered to build it. In that case they are likely to escape any wrath of the FSF lawyers because they have fulfilled their obligations as both the source is available and there is no extra restriction attached (ok, you have to recompile the program but that's hardly an obstacle. Certainly not any kind of DRM). OTOH, if they ship only a source to a crippled version without the optimization and other "secret sauce", then they would be in violation of the license.

[Teemo]

Regarding the XC8 ver 2.05
I just installed and tried it out. Not with AVR but with PIC.
Seems Microchip is sinking with that too.
I had a conversation about the so called "bloat" in the free versions some time ago. Accidentally posted a question and suddenly it became clear that I unknowingly stumbled up on to the bloat issue. Link here:https://www.microchip.com/forums/m1067094-p2.aspx

So now in the new version I discover that the mentioned bloat actually still there, but they have edited the dissassembly listing generator so that it do not show the bloated parts correctly.  Bloat is still there but the addresses where it is are in some other place in the file! Coincidental bug?
Disassembly listing jumps right from the address 001E to 0022. And guess what is missing form between? (it is still in some other place in the file)

So  they are sinking very well. Not only with making XC8 for AVR but with making XC8 for PIC as well. Better stay clear.

[cv007]

If one is scared of spiders, every little thing that moves looks like a spider.

the so called "bloat" in the free versions

There is no bloat issue. The compiler starts out producing code, then refines it to remove things not needed, or to speed up or reduces size or whatever else it wants to do to optimize. The free version just does not do all the steps. They just added the -O2 option to the free version, so they are giving away more 'code refining' steps. For the average user, the lack of -O3 or -Os in xc8-pic is nothing to get excited about.

The listings you show only proves that they cannot produce any disassembly listing in a continuous order. They are not trying to hide anything, or intentionally confuse you. You can debug via the simulator (or real), debugging->disassembly (not disassembly listing file), and probably get a look at the actual instructions of interest in order (for whatever function you are in).

There are no spiders here.

[Teemo]

I was looking through Microchip's website and I realized that XC8 PRO ( the not-free and expensive version of Microchip's 8 bit compiler) is a thing for AVR with the new AVR support in MPLAB X. I find it hilarious because no one is going to buy XC8 pro for something that already has pretty good compilers. However, it got me thinking. How does the XC8 compiler compare in terms of optimizations? Does microchip plan on dropping support for the free AVR compilers with Atmel Studio and not including them in MPLAB X in hopes that people are forced to buy their stupid expensive compilers if they want better optimizations? When I first heard that AVR support was coming to MPLAB X was great since it meant a proper IDE for AVR in Linux but now it's a little terrifying.

/Begin rant

Let's face it, Microchip should release their XC8 compilers and stick to the hardware business. I'm sure they have made enough money back from their purchase of the HI-TECH C compilers already. I also don't think that "go mess with assembly if you want optimizations" is a good answer either since you would expect a hardware manufacturer to make their hardware as accessable to everyone as possible. Companies where a few thousand dollars for a compiler is a formality are exempt from this rant, although they kind of perpetuate the problem by forking over the cash.

Don't get me wrong, I've only dealt with PICs so I am by no means an AVR fan boy. It's just kind of frustrating to see this and to think that Microchip might possibly be trying to force AVR into this, too.

/End rant

Sorry for any errors, typed this out with my phone. IX

Ok lets face the current situation: Microchip bought out their former best friend and co-operator HiTech. And by doing this entered into the competition with all the other independent compiler manufacturers for the PIC-s.
So now they lost a friend via the buying. (Seems surrently trying to leach back off the buying price, from the compiler business section.)
And also lost other cooperators, the independent compiler developers, by entering into competition with them. 
Who would want to compete with the chip manufacturer itself with developing the compiler and IDE, in the situation when there is always the threat that chip manufacturer can realease its compilers for free at the any moment suitable?

Also the huge deal of buying the Atmel. Probably still digesting the big cost of that. And the same thing: entering to the competition with the compiler development, thus trying to kill off independent compilers.

So no wonder the situation is terrifying. But lets look at the bright side -- for a moment it looked like the Microchip and Atmel would get too much monopoly in the MCU market certain sectors, now it is been going more balanced.

[technix]

I wonder if this is doable:

Code: [Select]

char padding[64] = {};
int main(void) { return 6; }
Then use a mining rig to brute force a hash collision in padding array? The whole cryptocurrency proof of work is based on hash collision anyway.

The binary can be constructed in a way that the padding bytes coextends with one hash block and whatever comes after the block is less than one hash block long - better if the padding block us the last block. This will allow the brute force effort make use of partial resumption of hashes and reduce the block count to less than three, allowing some speed up of the collision process.

[Jan Audio]

Their processors are crap anyway, so no loss.

The other companys has no DIP packages, amateur that i am.