After many cups of coffee, and playing around i've managed to get access to the bootloader with the ST Flash Loader Demonstrator.
First of all, the memory is protected...

and by trying to unlock, it was auto-erased... damn!
In case someone else will try it:
USART1 isn't easy availlable, so you have to solder some small wires to the STM. The RX/TX from J1 are wired to USART2 and useless in our case.
First, we need to set the BOOT0 pin to high (3.3V) to enable the bootloader. BOOT0 is at pin 44.
USART1-TX is on pin 30 and USART-RX is on pin 31, which also goes to C1 and the encoder.
You have to remove C1, cause C1 acts like a filter. I've used the pad from C1 to attach the RX-wire.
We need the RESET-pin, which is availlable at pin 7 and R1.
GND and 3.3V are availlabe from JK1.
Finally, the baudrate is 115200,8N1 with even parity.
By using the Flash Loader Demonstrator, you shoud disable ECHO and set the timeout to 5.
And here are some pics showing my attempts to get access to the bootloader.