Well, I got it mostly to work now.
It took some cutting and component-acrobatics, but I finally managed to get a decent waveform at the output that the counter can use. I also discovered that the biggest problem wasn't the board itself, but the Si5351 Adafruit board I used as a signal generator. Using it at 120MHz caused the output to contain considerable jitter which made the output of the prescaler-board look a lot worse than it actually was.
Self-Oscillation and a lot of harmonics is riding on top of the output waveform, but at as there is a circuit on the board that shuts off the output if there is no input-signal, this is no problem as long as the input-signal is handled correctly.
Which brings me to the biggest problem I'm having at the moment: It does not handle the signal correctly! The second Prescaler performs the correct /4 division, but the first one doesn't.
I already changed the chip because I suspected a faulty one from the 5 piece lot I bought on ebay, but the problem remains: The first prescaler only does division by 2. Both ICs have basically the same connection with both SW-Pins grounded.
Btw. the datasheet specifies the SW-Pin to be open for a low-condition, but if I leave the pin unconnected the voltage at the pin rises to about 3,5V and it still keeps dividing just by 2.
The picture with the counter displaying 100MHz was taken with the math function active which added a multiplier of 0.5 to the counter-result.
When searching for other prescaler-chips I also found some interesting replacements (pin-compatible).
A suitable set of chips would be the MC12026 (both control Pins set to high) together with the MC12083 (/2 divider with standby). The MC12026 could also be used standalone with some board-hacking, but that would defeat the standby-control as this chip has no standby-input. This chip is compatible with the boards I have here now.
About building the board: I've specified BAT54JFilm and BAP50-03 diodes in several places, but with a cut off leg regular BAT54C or schottky diodes in a similar package will also fit.
I also used 0402 components in several places so components ranging from 0805 down to 0402-package fit on the footprints.
With board revision D some cutting of traces and mounting resistors and capacitors on top of each other is necessary.
I don't know yet if I'm going to order another batch, as this board seems to work except for the problem with the first prescaler mentioned earlier (anyone have an idea about that?).
EDIT: Was a bit late and forgot to add the picture of the counter with the prescaler in it
