Yup, you're double-screwed...
1. See the traces coming off the speaker connector? Two ferrite beads, then a teensy black square? Looks to be a 9 ball CSP. Class D amplifier, unfiltered. So, there's full amplitude switching noise on the output, which is balanced (bridged, as they say in amplifier lingo).
2. Amplitude is whatever's needed to drive that speaker. Which... 5W, 4 ohms, so it only needs about 7V supply. So expect signal levels on the order of 5V, not ~1V that an external speaker wants!
Solution? Well, it's possible, but it's not pretty...
a. Keep a load resistor between the terminals, of maybe 4-10 ohms. Just because. The amp might have poor linearity into light loads, or an open-circuit protect or something, I don't know.
b. Attach a filter. If the switching frequency is quite high, then an RC filter might be good enough. Otherwise, you'll need an LC filter, and a matched impedance load (usually a series R to dampen the LC, since your attached amplifier probably won't have a useful input impedance). Typical values might be 1k\$\Omega\$ + 8.2nF, or (120\$\Omega\$ + 1mH) + 68nF. Ground is evidently white, and red and black are signal. Doesn't matter which one. Leave the other alone (aside from the load resistor if necessary).
c. Add an attenuator. For the RC filter, you can simply put a voltage divider after it. For the LC filter, use resistors somewhat larger than the series resistor (i.e., for 1/10th voltage, use maybe 820 ohms in series, 120 ohms parallel to ground).
d. You only get mono, unless there's another one on the other side. In which case, repeat the process.
How's that HDMI splitter looking now?...
Tim