I found a way that partially works. Pour the ground polygon first, then autoroute the other stuff. Then regenerate the polygon. For the most part it sees the ground points are all connected to the polygon so it leaves them alone, but in a number of places it still puts tracks from one ground pad to another despite these pads beeing connected to the polygon.