Sorry about barging in on a polite debate, but felt I had to raise my voice on this subject.
Simon,
Judging from the datasheet, then your idea is in trouble. The detector seems optimized to work in the range from 1 to 100 mm, with a difference in signal strength over this range of about three orders of magnitude. Refer to Figure 3 on page 3 of the datasheet.
Your problem is almost certainly that the detector sees the transmitter diode reflected in the *outside* air/glass surface. Each pane of glass obviously has two air/glass interfaces, and both will give reflections. Even using a transparent potting compound wouldn't do you any good, as there is no way to eliminate the outside air/glass interface. This problem is likely to get worse, if you are using thick, armored glass, as this would increase the distance between the chip and the reflecting surface.
From figure 3 it seems obvious that even a weak reflection from the glass surface a few mm away will completely overwhelm any signal from the 50-100 mm range.
Try holding the detector IC about 20mm from the glass surface, and see if it detects its presence. IE. do a detector test as shown in the test circuit, also on page 3.
You have two options, both of which has already been mentioned and ruled out(?):
1) Set transmitter and receiver at an angle to the glass surface. For a transparent solid there exist an optimal angle, the Brewster angle, where reflections in the surfaces are minimized or eliminated. This angle depends on the difference in dielectric constant between the involved substances. For air/glass this angle is about 42 degree, yet reflections are reduced in a broad range of angles about this value.
2) Separate the transmitter and receiver, so the RX cannot see the internal glass reflections from the TX.
Sorry about being the bringer of bad news...