Is Microchip violating the GPL?

[Karel]

I tried to download the sourcecode of the latest (1.42) PIC32 C compiler from the Microchip website.

I couldn't find it at their page: http://www.microchip.com/mplab/compilers
tab: download archives
section: source archives
column: MPLAB XC32

Instead, I found it at their page: http://www.microchip.com/mplab/mplab-x-ide
tab: download archives
section: source archives
column: MPLAB XC32

So, I downloaded the sourcecode and had a look at the file xc32-tools-xc32-XC32-release-1_42-src/gcc/gcc/config/mchp-cci/mchp_sha.h
To my surprise, I found a wrong sha256 hash at line 30. This hash doesn't match the hash used in the binary
/opt/microchip/xc32/v1.42/bin/bin/gcc/pic32mx/4.8.3/cc1
To be sure, I did a grep -I -R <sha256hash> * to see if the hash used in the cc1 binary is present in the sourcecode.
It isn't.

So, apparently, the sourcecode provided by Microchip is not the sourcecode used to create the binary.
This would mean that Microchip violates the GPL.

My question is, am I overlooking something?

[Marco]

AFAICS only if they deny you a request for the actual source code.

More background here.

The bigger GPL violation in my opinion is that the IDE and the compiler are clearly a combined work, assuming they distribute them together.

[Karel]

AFAICS only if they deny you a request for the actual source code.

Then the question arises, what's the purpose of the link to the sourcecode with the wrong hash?
It's labeled as beeing the sourcecode voor version1.42.

[dannyf]

This would mean that Microchip violates the GPL.

they are definitely wrong.

mortgage your house, your retirement and your kids college funds and sue those bastards.

we are right behind you,

[Ian.M]

The bigger GPL violation in my opinion is that the IDE and the compiler are clearly a combined work, assuming they distribute them together.

They don't.   MPLAB X is a totally separate download from the XC compiler suites.  There may have been some legacy violations with the MPLAB 8 suite and C30/C32. 

[jesuscf]

So, I downloaded the sourcecode and had a look at the file xc32-tools-xc32-XC32-release-1_42-src/gcc/gcc/config/mchp-cci/mchp_sha.h
To my surprise, I found a wrong sha256 hash at line 30. This hash doesn't match the hash used in the binary
/opt/microchip/xc32/v1.42/bin/bin/gcc/pic32mx/4.8.3/cc1
.
.
.
My question is, am I overlooking something?

The correct sha256 is probably computed and included at build time.   That is the way I would do it.  The only way to know is by looking at the building script...

[amyk]

So, apparently, the sourcecode provided by Microchip is not the sourcecode used to create the binary.

No, it could be a different compiler they used to compile the compiler, or different compilation settings, and various other environment configurations.

Anyway, unless you're some pedantic lawyer, it doesn't really matter. Even the GPL doesn't mean they have to essentially give away their whole build environment.

[Karel]

So, apparently, the sourcecode provided by Microchip is not the sourcecode used to create the binary.

No, it could be a different compiler they used to compile the compiler, or different compilation settings, and various other environment configurations.

Anyway, unless you're some pedantic lawyer, it doesn't really matter. Even the GPL doesn't mean they have to essentially give away their whole build environment.

I wasn't talking about a build environment. I was talking about the GCC compiler which is GPL licensed.
More specific, a binary with the name cc1.
The GPL license says that if you distribute a modified version, you have to distribute (or make available) the exact sourcecode of that
modified version. There are no excuses.

If they compute and build the sha256 hash at buildtime, as jesuscf suggestst, it's still not an excuse to not provide
the sourcecode of the cc1 binary after building that binary.

[helius]

I wasn't talking about a build environment. I was talking about the GCC compiler which is GPL licensed.
More specific, a binary with the name cc1.
The GPL license says that if you distribute a modified version, you have to distribute (or make available) the exact sourcecode of that
modified version. There are no excuses.

I hope you are aware that modifying source code is not the only way that changes to a binary may be made.
https://www.ece.cmu.edu/~ganger/712.fall02/papers/p761-thompson.pdf

This issue has never been addressed by the FSF: the GPL only requires that changes to the source be shared, but those are not the whole extent of what determines binary output. The compiler, other tools, environment variables, makefiles, and libraries all affect the output in a complex way. And most organizations do not have a reproducible build process in place.

[voltsandjolts]

If Microchip have included a build date/time string in the binary (perhaps not a good thing to do), re-building the binary would never give the same hash.
Have you tried doing a make clean and rebuild to see if your hash is consistent?

[dannyf]

it's still not an excuse to not provide

if you are so convinced of your righteousness, why don't you put your money where your mouth is and sue Microchip? Then you may find out where you may not have done your due diligence.

[andersm]

If you want a legal opinion, ask a lawyer. If you believe a violation has occurred, contact Microchip or the Free Software Foundation.

[amyk]

If Microchip have included a build date/time string in the binary (perhaps not a good thing to do), re-building the binary would never give the same hash.

Actually that is the default behaviour of a lot of compilers, they will embed the version of the compiler used to compile into the binary too. It's useful for determining when and where it was built, for example.

[Karel]

If Microchip have included a build date/time string in the binary (perhaps not a good thing to do), re-building the binary would never give the same hash.
Have you tried doing a make clean and rebuild to see if your hash is consistent?

I wasn't talking about the hash of the binary. I was talking about a hash in the binary.

It works as follows. When the compiler is called, it calls another (closed source) program that checks your license.
This program returns a value to the compiler so that it knows if it can use the higher optimization settings yes or no.
In the past, people have replaced this program with another simple program that always return the value that allows the
compiler to use all optimization settings.
Now, the compiler performs a sha256 sum of the program that checks for the license. If it detects that the hash of that program
is different, it refuse to compile.
The hash value is hardcoded in a headerfile and because of that, clearly visible when you open the compiler executable with
a hexeditor.

Have a look here: https://www.eevblog.com/forum/microcontrollers/pic32-evolution/msg1007099/#msg1007099

and here: http://www.jubatian.com/articles/turning-on-optimizations-in-microchips-xc32/

[helius]

Surely a complete fool's errand? Anyone who compiles from source can simply comment out the code that checks if it is equal 

But if Microchip had any brains (fat chance that), they would realize that they could have just made their proprietary program communicate over RPC. Then when they wanted to add some code to the GPL'd base (gcc), they would simply put it in the new program instead, and leave a rpc-call stub in gcc. No GPL violation, but the feature won't work unless you buy it from Microchip.

[Marco]

RPC is likely to open them up to the claim that the calling program is a derivative work.

[Karel]

Surely a complete fool's errand? Anyone who compiles from source can simply comment out the code that checks if it is equal 

But if Microchip had any brains (fat chance that), they would realize that they could have just made their proprietary program communicate over RPC. Then when they wanted to add some code to the GPL'd base (gcc), they would simply put it in the new program instead, and leave a rpc-call stub in gcc. No GPL violation, but the feature won't work unless you buy it from Microchip.

Despite being it theoretically possible, many modifications Microchip made are very much interwoven with the original code.
Because of that it's not practical.

[AlfBaz]

Quite some versions of C32 ago, I got my hands on on the source package for C32. I didn't know about hash values but used CC1's source in conjunction with it's disassembly to hex-hack CC1's exe by placing jumps and nop's around the offending code. Took about an hour to do.

From memory i changed about three bytes in the exe and got a fully optimizing C32. At the time I remember thinking "Who the hell releases source to the code that protects their IP?"

[Karel]

At the time I remember thinking "Who the hell releases source to the code that protects their IP?"

The people who want to use GPL licensed code?

[hans]

Quite some versions of C32 ago, I got my hands on on the source package for C32. I didn't know about hash values but used CC1's source in conjunction with it's disassembly to hex-hack CC1's exe by placing jumps and nop's around the offending code. Took about an hour to do.

From memory i changed about three bytes in the exe and got a fully optimizing C32. At the time I remember thinking "Who the hell releases source to the code that protects their IP?"

The hash counter measure was only introduced at a later date.

Before you could just replace xlcm with a return statement and it would activate appropriately.
Conveniently, it could also make compiling much faster because it doesn't have to check at the server what the actual status is. But I believe they fixed this bug.

They added the hashing later, but being an open source product that has little effect to people that want to figure this stuff out.
I believe you can also just hex edit the target file and invert the nullify option if expressions. Indeed, with 1 hour spare time and a cup of coffee you could have this sorted.
Of course you could also recompile from source, but doing that on a windows box it's small little hell IMO.

[jesuscf]

Allow me to show results from the *gcc -v of several gcc compilers I have installed in my computer:

Code: [Select]

C:\>gcc --v
Using built-in specs.
Target: x86_64-w64-mingw32
Configured with: ../gcc44-svn/configure --target=x86_64-w64-mingw32 --host=x86_64-w64-mingw32 --disable-multilib --disable-nls --disable-win32-registr
y --prefix=/mingw64 --with-gmp=/mingw64 --with-mpfr=/mingw64 --enable-languages=c,c++
Thread model: win32
gcc version 4.4.3 (GCC)


Code: [Select]

C:\>avr-gcc -v
Using built-in specs.
Reading specs from c:/programs/avr8-gnu-toolchain/bin/../lib/gcc/avr/4.9.2/device-specs/specs-avr2
COLLECT_GCC=avr-gcc
COLLECT_LTO_WRAPPER=c:/programs/avr8-gnu-toolchain/bin/../libexec/gcc/avr/4.9.2/lto-wrapper.exe
Target: avr
Configured with: /home/toolsbuild/workspace/avr8-gnu-toolchain/src/gcc/configure LDFLAGS=-L/home/toolsbuild/workspace/avr8-gnu-toolchain/avr8-gnu-tool
chain-win32_x86-hostlibs/lib CPPFLAGS= --target=avr --host=i686-w64-mingw32 --build=x86_64-pc-linux-gnu --prefix=/home/toolsbuild/workspace/avr8-gnu-t
oolchain/avr8-gnu-toolchain-win32_x86 --libdir=/home/toolsbuild/workspace/avr8-gnu-toolchain/avr8-gnu-toolchain-win32_x86/lib --enable-languages=c,c++
 --with-dwarf2 --enable-doc --disable-shared --disable-libada --disable-libssp --disable-nls --with-avrlibc=yes --with-mpfr=/home/toolsbuild/workspace
/avr8-gnu-toolchain/avr8-gnu-toolchain-win32_x86-hostlibs --with-gmp=/home/toolsbuild/workspace/avr8-gnu-toolchain/avr8-gnu-toolchain-win32_x86-hostli
bs --with-mpc=/home/toolsbuild/workspace/avr8-gnu-toolchain/avr8-gnu-toolchain-win32_x86-hostlibs --enable-win32-registry=avrtoolchain --with-pkgversi
on=AVR_8_bit_GNU_Toolchain_3.5.3_1700 --with-bugurl=http://www.atmel.com
Thread model: single
gcc version 4.9.2 (AVR_8_bit_GNU_Toolchain_3.5.3_1700)


Code: [Select]

C:\>arm-none-eabi-gcc -v
Using built-in specs.
COLLECT_GCC=arm-none-eabi-gcc
COLLECT_LTO_WRAPPER=c:/programs/gnu\ tools\ arm\ embedded/5.4\ 2016q2/bin/../lib/gcc/arm-none-eabi/5.4.1/lto-wrapper.exe
Target: arm-none-eabi
Configured with: /home/build/work/GCC-5-0-build/src/gcc/configure --build=i686-linux-gnu --host=i686-w64-mingw32 --target=arm-none-eabi --prefix=/home
/build/work/GCC-5-0-build/install-mingw --libexecdir=/home/build/work/GCC-5-0-build/install-mingw/lib --infodir=/home/build/work/GCC-5-0-build/install
-mingw/share/doc/gcc-arm-none-eabi/info --mandir=/home/build/work/GCC-5-0-build/install-mingw/share/doc/gcc-arm-none-eabi/man --htmldir=/home/build/wo
rk/GCC-5-0-build/install-mingw/share/doc/gcc-arm-none-eabi/html --pdfdir=/home/build/work/GCC-5-0-build/install-mingw/share/doc/gcc-arm-none-eabi/pdf
--enable-languages=c,c++ --disable-decimal-float --disable-libffi --disable-libgomp --disable-libmudflap --disable-libquadmath --disable-libssp --disa
ble-libstdcxx-pch --disable-nls --disable-shared --disable-threads --disable-tls --with-gnu-as --with-gnu-ld --with-headers=yes --with-newlib --with-p
ython-dir=share/gcc-arm-none-eabi --with-sysroot=/home/build/work/GCC-5-0-build/install-mingw/arm-none-eabi --with-libiconv-prefix=/home/build/work/GC
C-5-0-build/build-mingw/host-libs/usr --with-gmp=/home/build/work/GCC-5-0-build/build-mingw/host-libs/usr --with-mpfr=/home/build/work/GCC-5-0-build/b
uild-mingw/host-libs/usr --with-mpc=/home/build/work/GCC-5-0-build/build-mingw/host-libs/usr --with-isl=/home/build/work/GCC-5-0-build/build-mingw/hos
t-libs/usr --with-cloog=/home/build/work/GCC-5-0-build/build-mingw/host-libs/usr --with-libelf=/home/build/work/GCC-5-0-build/build-mingw/host-libs/us
r --with-host-libstdcxx='-static-libgcc -Wl,-Bstatic,-lstdc++,-Bdynamic -lm' --with-pkgversion='GNU Tools for ARM Embedded Processors' --with-multilib
-list=armv6-m,armv7-m,armv7e-m,armv7-r,armv8-m.base,armv8-m.main
Thread model: single
gcc version 5.4.1 20160609 (release) [ARM/embedded-5-branch revision 237715] (GNU Tools for ARM Embedded Processors)


Code: [Select]

C:\>xc32-gcc -v
Microchip Language Tool Shell Version 1.42 (Build date: Jun  1 2016).
Copyright (c) 2012 Microchip Technology Inc. All rights reserved
*** Executing: "C:\Programs\xc32\v1.42\bin\bin/xc32-gcc.exe" "-v"

Using built-in specs.
COLLECT_GCC=C:\Programs\xc32\v1.42\bin\bin/xc32-gcc.exe
COLLECT_LTO_WRAPPER=c:/programs/xc32/v1.42/bin/bin/gcc/pic32mx/4.8.3/lto-wrapper.exe
Target: pic32mx
Configured with:
Thread model: single
gcc version 4.8.3 MPLAB XC32 Compiler v1.42 (Microchip Technology)

Did you notice something missing from the output of the XC-32 compiler that is present in the other ones?  Maybe I am wrong, but it looks to me that Microchip is making an extra effort to prevent people from building XC-32 from sources.  But wait, there is more.  Look what happens when you do the same with XC-32 v1.40:

Code: [Select]

C:\Programs\xc32\v1.40\bin>xc32-gcc -v
Microchip Language Tool Shell Version v1.40 (Build date: Jun  3 2015).
Copyright (c) 2012 Microchip Technology Inc. All rights reserved
*** Executing: "C:\Programs\xc32\v1.40\bin\bin/xc32-gcc.exe" "-v"

Using built-in specs.
COLLECT_GCC=C:\Programs\xc32\v1.40\bin\bin/xc32-gcc.exe
COLLECT_LTO_WRAPPER=c:/programs/xc32/v1.40/bin/bin/gcc/pic32mx/4.8.3/lto-wrapper.exe
Target: pic32mx
Configured with: ../../src48x/gcc/configure --build=i386-apple-darwin10 --host=i586-mingw32 --prefix=/Volumes/home/c11067/xc32/build/xc32-XC32_v1_40_
ranch/win32-image --includedir=/Volumes/home/c11067/xc32/build/xc32-XC32_v1_40_branch/win32-image/pic32mx/include --bindir=/Volumes/home/c11067/xc32/
uild/xc32-XC32_v1_40_branch/win32-image/bin/bin --libexecdir=/Volumes/home/c11067/xc32/build/xc32-XC32_v1_40_branch/win32-image/bin/bin --disable-sha
ed --with-libelf=/Volumes/home/c11067/xc32/build/xc32-XC32_v1_40_branch/win32-build/mingw32-libs --with-gmp=/Volumes/home/c11067/xc32/build/xc32-XC32
v1_40_branch/win32-build/mingw32-libs --without-isl --without-cloog --target=pic32mx --program-prefix=xc32- --with-bugurl=http://www.microchip.com/su
port --with-dwarf2 --with-newlib --with-gnu-as --with-gnu-ld --enable-cxx-flags=-mno-smart-io --enable-lto --enable-fixed-point --enable-gofast --ena
le-languages=c,c++ --enable-sgxx-sde-multilibs --enable-sjlj-exceptions --enable-obsolete --disable-hosted-libstdcxx --disable-libstdcxx-pch --disabl
-libssp --disable-libmudflap --disable-libffi --disable-libfortran --disable-bootstrap --disable-shared --disable-__cxa_atexit --disable-nls --disabl
-libgomp --disable-threads --disable-sim --disable-decimal-float --disable-libquadmath --without-headers 'XGCC_FLAGS_FOR_TARGET=-frtti -fexceptions -
no-enforce-eh-specs -mno-smart-io' --disable-checking --enable-poison-system-directories
Thread model: single
gcc version 4.8.3 MPLAB XC32 Compiler v1.40 (Microchip Technology)

Did they suddenly realized that displaying the 'Configured with' was helping those who tried to build XC-32 from sources and then they removed it?

[Eturtle]

Hello. I'm new to this forum and my English isn't the best, so be patient.

Microchip took the free gcc as base for XC16 and XC32. Everyone can download the modified version even without registration. This satifies the GPL, IMO. But the building of XC?? is very tricky, otherwise they couldnt sell it. To check for a valid license some parts of the modified gcc (cc1, cc1plus and lto1) call the external program "xclm". Some figured out it was easy to cheat gcc by replacing xclm with a simple "return 2". MCHP counters back by checking the sha256 of xclm. Cause xclm isnt GPL it's totally no matter how the checksum is.

Seems XC was build canadian cross with a macintosh. Tried myself and got really trouble on an actual 64bit linux. But one day i got some success with my old 32bit desktop from 2009. Build XC16 v1.25 for linux which dont do the xclm-check. You can download it from https://www.mikrocontroller.net/topic/400756, copy the binaries and enjoy O2, O3 and Os for your PIC24. Unfortunately i did not get any feedback if its working.

[dannyf]

This satifies the GPL, IMO. But the building of XC?? is very tricky

That's the basic flaw in GPL: it assumes the only thing that matters is the source code.

Couldn't have been further from the truth.

[Karel]

Everyone can download the modified version even without registration. This satifies the GPL, IMO.

And this is exactly the point why I started this thread. They did not. (at least that's what I think)
The hardcoded hash in the header file is not the same as the hash inside the executable.
Correct me if I'm wrong but to me this means that the sourcecode they provide is not the same
as used to produce the executable.

[grumpydoc]

This satifies the GPL, IMO. But the building of XC?? is very tricky

That's the basic flaw in GPL: it assumes the only thing that matters is the source code.

Couldn't have been further from the truth.

No it doesn't - the GPL V2.0 contains the following definition

"The source code for a work means the preferred form of the work for making modifications to it. For an executable work, complete source code means all the source code for all modules it contains, plus any associated interface definition files, plus the scripts used to control compilation and installation of the executable."

V3 contains a similar definition.

Stallman was well aware that you need more than just the "C" files.