So they are all just electrolytic caps then?
Those in your picture, yes, they are.
Why the different color? Different manufacturer?
Whether or not they are from a different manufacturer, I can't tell from the photo. Given that the style of the negative electrode marking looks pretty much the same for all of them (apart from the colour), my guess would be that they're from the same manufacturer. You should be able to compare them, or post a photo taken from the other side?
If they are from the same manufacturer, the different colours may just mean that they're different types. There's more to the specs of a cap than just the capacity and voltage rating. There's also mounting style, size, ESR, lifespan, max. temperature etc. to be considered. As an example, just look at
this overview of all the different series in the aluminium capacitors range of just Nippon Chemicon, one of the very reputable A-brands.
The gold ones have a top that is not flat and looks like pressure may have pushed on it - do you think they are bad? This is from an old netgear router...
A bulged top is not a good sign. Even worse is when the crosses in the top (manufactured weak spots to prevent explosion) burst and start spewing electrolyte. Electrolyte may also be leaking out at the base (look for signs of fluid on the board). As an example I added an attachment of the cap inside a CFL that had failed, it had started buzzing and the light output started to drop sharply. The bulged cap was the cause.
If you intend to keep running the router, replace them with a suitable item. The type should be printed on it, often just a couple of letters and/or numbers; in my example it would be a Rubycon of the CFX-series, a quick google finds the
datasheet.
If I were to replace this cap, but wasn't able to source the same Rubycon, I'd find another cap with similar or better ratings for e.g. ripple current and dissipation factor, because in this particular application, those will be important parameters.
If you can't find a datasheet, my personal rule of thumb is that gold printing usually indicates Low-ESR types (especially with Chinese/Taiwanese caps). You will want to replace them with Low-ESR caps as general purpose caps might have a bit too high an ESR for that circuit and they will heat up more and wear out faster.
When dealing with general purpose caps, then replacing them with whatever you have lying around of similar capacity and the same or higher voltage rating will normally do just fine.