You will have to stay on them to keep using your spec'd parts. Don't be surprised to start seeing used/clone parts or the like quietly subbed. Just politely call them out, ask they never do it again or you'll go elsewhere.
To prevent this you can say "Any supplier" for non-critical parts. Seems they are doing this for you - just check the prices and make sure you're at least saving money by using these parts over name brand..
This is my experience too. But whether you spec Farnell/RS etc parts or not, and tell them specifically to use those exact parts, I will guarantee you that they won't. There are fundamental culture differences.
As an example, on one project I was using a 1mm PCB with appropriate SMA end launches, I'd specified the exact RS part number. The assembler later complained to me that the connectors didn't fit and they'd had to fudge it. Well, that's because they didn't use the part I'd specified.
Second example. They bought cheap "alternative" 1.2V regulators from somewhere, don't know where, despite my specifically giving them the Mouser part number. They had a failure rate of about 50% and the failure mode was to dump the 5V input voltage right onto the 1.2V core of the most expensive chip on the board.
Another rather less technical issue with dealing with China is that everything shuts down for two weeks over the Chinese New Year, so plan accordingly.
Personally speaking, I will not manufacture in China unless I have someone I know and trust working for me locally there looking after my interests, it's just too risky otherwise, but that is a big expense, and might very well not be worth it, the cost of your agent might well be more than having your boards made on home turf. They will use every way they can to shave a fraction of a penny off. The culture is to give you the nice warm feeling, affirming what you've agreed, and then go and do something different. Getting used to this is hard for a Westerner. You have been warned!