Generics should drive the process (not the VHDL process) of determining what gets included and excluded.
You might be limiting your scope/applications of generics thinking of them this way. They are a fancy constant, that can be set at compile time to differentiate the entity from other instantiations (be that within the same project, or between projects, or at different times, etc, etc).
Oh, believe me, I love to use generics for many things. I use them for setting part revisions and beta build constants (as read back over some communications link). I use them to set clock division ratios for, say, an SPI peripheral. If it's a constant that I might want to override, I make it a generic!
What you described (propagating a constant to prune conditionals/states) works ok, by wrapping it in if's at that level makes it clearer to the tools. Messy code but gets the job done.
An alternative would be to make the tools realise some of the literals of the enum aren't possible on the case expression (state machines are pruned/optimised like this). A function in the package declaring the enum that takes all the possible inputs and translates it to the set implemented in that variant?
I'm not sure what that function is supposed to do?
You need a function/assignment/process that the tools recognise will only ever have a subset of the "states" of the enum coming out of them. Think of it as the next step after constant propagation, set reduction/pruning. I've not tested this for the general sense, but the major tools are certainly doing this for state machines (particularly when spread across multiple processes). They know which states are unused in a case select and optimise them out automatically.
Ohhh, wait, I think I am getting the idea!
Let's begin with my command parser description. I create an enumeration that defines all possible command tokens for all design variants. A signal of that type is the case expression. (Just like a state machine.) Each entry in the enumeration is a selector in the case statement. Each selector drives some signals. This case statement therefore covers all possible design variants.
Even if a particular variant will never get some of the defined commands (this is guaranteed because I control both sides of the command-sending pipe), the selectors for the commands will still exist. Now if there is nothing connected to those signals (this is in the variant-specific code), then we might assume that the tools will recognize this and work backwards and ultimately optimize away the case selector.
But the problem is that in the system, every command sent to the FPGA gets a response back. Unimplemented and undefined commands get a response that says, "not supported!" (Again, because I control both ends of the communications, undefined commands are never sent ... but we should be prudent and define what happens when such an event occurs.)
It is because of the requirement that a response is returned that I have to work out what happens if the unimplemented for this variant command is mistakenly sent.
So then what is needed is a "prescreener" function: the input is the command token. The function has a variant-specific mechanism (lookup table?) that replaces unsupported commands with a default "not implemented" command. If the command is supported, the output is the input command. This lookup is all a ROM made at compile time. The output of this function is used as the case expression. (I think VHDL-2008 even allows for the function call to be part of the case expression; if not, whatever, I'll make an intermediate signal.)
Aside: I think I can use a package generic to choose which variant's lookup table is used in that function. The function will be in the same package.
Will it result in the unpossible command selectors being optimized away? There's only one way to find out: code it and synthesize it, which will be project for tomorrow.