As for measuring: For lower frequencies, less than say a couple of GHz, the easiest way is just to place a big rectangle on the PCB (stripline or microstrip) and accurately measure its (DC) capacitance to ground, from this you can derive the dielectric constant. With a microscope, measure the actual trace widths/layer heights realized. Then put both of them in a simulator. (In your design, let some internal traces run accross the edge of the board, to get a cross section for free. Or in case of a production board, run them very close to the edge/corner, so you can use a file to get to the trace)
The mentioned method is also typically used for higher frequencies, albeit that the dielectric constant is determined with ring resonators.
I do not know what your aim is for knowing the impedance:
For digital circuits: if your board has layer heights of say 0.2 mm or less and lines that are of substantial length compared to the rise time, ask the manufacturer for controlled impedance lines. This is because etching tolerance and pressing will be major contributors to the impedance (apart from the FR4 itself), If using thicker layers, do not bother.
Microwave circuitry. Do not let the manufacturer control the impedance, unless you are only using 50 ohm lines. If any fancy stuff is happening, do net let them touch it. Just ask the manufacturer for the specific board stackup, including prepregs etc. Also ask for etching tolerances. When knowing the FR4-manufacturer, the dielectric constants will have a tighter tolerance than just stating FR-4. Use these values to determine the production tolerances.
When using multilayer boards, skipping ground planes for the RF part is the easiest way to keep the effect of etching tolerance under control.