Yes, start with core type, roughly.
Typically 20 < μ_r < 60 gives best compromise between core size and copper losses, with values outside of this range being useful in less common conditions (typically, lower mu at higher frequency; or higher mu where number of turns or wire length is weighted more heavily than energy density, HV flyback transformers being a typical example).
Typical powdered iron materials are #18 and #52 (Micrometals mix numbers), MPP, or branded formulations like Hi-Flux, Sendust/Kool-Mu, etc.
Typical ferrites are gapped shapes. Ferrite toroids are either hi-μ or hi-freq materials, and you can't easily gap a toroid, so if we're talking just toroids, yeah.
Core size: we can use an energy argument here to find minimum. We may however still need a larger core, or lower ripple fraction or lower frequency in the design (higher inductance, slower Fpwm, slower control loop) to deal with losses, in which case that needs to be fed back into the design, and the inductor design/selection process starts over.
Energy storage is given by the Maxwell stress: σ = B^2 / (2 μ) This has units of pressure, which is also units of energy density. Set B = Bsat, then divide your energy storage requirement E = 0.5 L I^2, by this, to get core volume v_e. Select parts with equal or greater volume.
Note this is inversely proportional to μ, so a lower μ core stores more energy -- but it also requires more turns for the same inductance, which increases copper losses. The preferred range of μ is driven by typical frequencies and the resistivity of copper, of all things. (After all, if we had casual room-temperature superconductors, we wouldn't need cores at all, just pile on thousands of turns of hair-fine wire and not care.

)
Core loss: determine Bpk for your application (i.e., the ripple fraction). Find the material's core loss curves, and match up Bpk and F. The power loss density will typically be 100-1000 mW/cm^3 (or kW/m^3 or however they label it). Higher losses are acceptable in smaller cores (say 500 for T50 size cores?), because the surface area/volume ratio is more favorable, and lower for larger (say 100-200 for T300 size?). You can calculate or find the core surface area, and use a typical convection rate, to solve for temperature rise, and in turn, loss density.
(Again, as a density, multiply by v_e to get watts.)
Finally, check that your wire fits the core. Typically, Bsat will require 200-2000 At, and with passive convection, over 500-1000 At is where things start getting impractical (in terms of winding area and temp rise).
If any of these conditions is not met, try the next larger size core, etc.
Should be able to find candidates in a handful of iterations, and this method can also be applied to a table of parts to automatically select or rank them. There are unfortunately too many variables to write out a closed-form solution -- although this is a bit easier when manufacturers provide core loss formulae.
Going back to ferrite: choose μ the same way, then calculate the air gap based on μ_avg or μ_eff (which, I don't remember the formula offhand but it's easily found; or use the figure from the datasheet, if pre-gapped cores are used).
Tim