The equivalent circuit looks like this:
The filter choke is ultimately working into the 50 ohm source impedance, but that's not where the interesting stuff happens. The interesting part is the impedance it's working against: a capacitively coupled noise source, and capacitance across the isolation barrier, with very little loss otherwise.
R1 and R3 represent the LISN, which in turn is used to represent the AC line during testing. (In general, a real AC line will not be a pair of resistors, but a complex, lossy and random transmission line. But that doesn't really help, so we blindly go on assuming it's okay to use resistors. Applying some RF theorems, it's not actually as bad an assumption as it sounds.)
C3 represents the winding-to-winding and isolation capacitance in a typical SMPS, and V1 represents the switching voltage which appears on those windings. The voltage seen by the isolation barrier is roughly the average of the voltage across the winding. So for a flyback supply that's driving 400Vpp across the primary (and has a low voltage secondary), you might expect V1 is 200Vpp.
C1, C2, C4, C5 and L1 comprise the familiar EMI filter. C5-C7 may be used in various permutations, depending. (C4 may or may not be used. Despite how it looks, it actually has little impact on what L1 is doing.)
In any case, the equivalent sum of C3 and C5-C7, is the capacitance which resonates with L1. You can simply draw the equivalent circuit: GND--25 ohms (they act in parallel)--L1--Ceq--GND.
Ceq is C5 + ((C7 + C3) || C6). That is, C3 adds to C7, so we have 2.3nF, in series with 2.2nF is 1.12nF. And this adds with 2.2nF, giving 3.32nF total equivalent.
Now that we know the capacitance and inductance, we know the impedance and frequency!
If that impedance is very different from the mains resistance, then we will have a problem: the filtering will either be very sloppy (Z < R), or very peaky (Z > R).
So we have, Zo = sqrt(5mH / 3.32nF) = 1227 ohms. Which is much larger than 25 ohms, so there will be a substantial peak at some frequency, where V1 is able to transmit energy to the line resistors (or vice versa). If everything is lossless, the Q factor is 49, so we can expect the peak to be quite strong indeed!
And, the frequency of that peak will be very close to F = 1 / (2*pi*sqrt(L*C)) ~= 39kHz. The simulation gives 39.21kHz:
Now, -40dB doesn't sound very thrilling, but the fact is, you're starting with hundreds of volts. And the regulations say it has to be millivolts or less. So this is only getting you down to single volts, and you have a whopping 60dB yet to go!
Since the Q factor is so high, we can add some resistance to that coil, worsening the attenuation at very high frequencies, but dropping the peak substantially. This looks like so:
with 5kohms strapped across one of the windings (doesn't much matter which one). Now the peak is -59dB, an improvement of 19dB.
I don't have a realistic CMC model handy for a part of this size, but the effective parallel resistance can be found in the datasheet: on a graph of Zcm vs. F, it is the peak value of the impedance. At low frequencies, Z is rising (inductive characteristic), and at high frequencies, Z is falling (capacitive characteristic). Where those ranges meet, L and C cancel, and Z is real (resistive), and equal to Rpar. Most times, it will be much too high (e.g., >10k for a part of this size), so you might consider damping the capacitors instead (e.g., replace C5 with 1nF, and put (2.2nF + 1kohm) in parallel with it).
Lastly, an SMPS really isn't perfect white noise. If you can place the transmission peak at a frequency below the switching frequency (or subharmonics, just in case), you don't need to worry about it, because there's so little power being transmitted at that frequency. That leaves plenty of time for the lowpass filtering to kick in, before the fundamental, which might be at 100kHz or so, and harmonics beyond there.
Tim