I always find it interesting to hear people talk about ICs as these magical blocks that are handed down from heavens. Someone also needs to design these ICs and things are somewhat different there. And it is not like there are just a few 'elite' designers working on these things - these are massive industries.
While they still use 'standard' topologies - to some extent a least - In my experience IC designers are much more familiar with the very low-level stuff, especially when talking about analog. Because of the limitations faced in that sector I think they go through a good bit more change in terms of technology and circuit design. Granted, I work in a millimeter-wave research group, and as a result the industrial partners I talk to are not exactly the ones that can make the same chip for 20 years at 5 cents a piece, but still.
Two things are different in IC design compared to circuit design using existing ICs as building blocks: 1) In these analog ICs you are working with those fundamental building blocks. You are not even designing with fixed transistors - you care actually deciding the dimensions (and perhaps type) of this transistor. As a result you need better understanding of how all these specifications relate. 2) You need to really understand what impacts performance because you need to fully simulate and verify the design.
With time you learn to really understand how to come up with new topologies and solutions. This is not something you can just learn in a university course, and takes years to master. I haven't really mastered it yet, though I have come up with some circuits of my own. As I work in a sector where we try to do what nobody has done before, we can't use existing building blocks because they just can't do it. That doesn't mean that our circuits are fundamentally different (though some are), but can mean we might spend more time designing parts that most designers take for granted (like studying the way to connect to the drain and gate of a transistor for 3 months because at 200 GHz, every little bit of resistance is a problem).
So how do we design a circuit? We start out with the concept - what do we want/need?
From there we do what everyone here has already said: Look at what building blocks we can use. Think about what specifications matter where: in the LNA we will care much more about linearity and noise than gain. In the PA noise is not a big issue because we are dealing with high signal power. In the mixer we might want to get high linearity and low LO leakage, etc.
We often even simulate using idealized models to get an idea of what specs we need for different parts. We can see what noise is acceptable in different sections, what linearity we need for the mixer, etc... and how it all influences each other.
From there on we look at how we can design the blocks. We try to meet the specs for every block, but if this is not possible, we need to look at why we can't (over time designers will also get a bit of 'instinct' about what specs are realistic. We can start from an understanding of different topologies, we know what topology will be good at what. EG, for a variable gain amplifier I can look at making an actual variable gain amplifier. However, this might be harder to verify performance-wise, and could be more challenging to control. I can use a fixed gain amplifier with variable attenutators in between, which could be easier but requires more power. Perhaps multiple amplifiers in parallel, and switch on the amount we need to give us the right gain? But this will take a lot of area because we need to duplicate this amplifier again and again.