ya, the price goes up for less quantity, but at some point, you'll cap out. They can only make them so cheap, vs. cost of material.
Injection molding is so quick, but milling aluminum takes operator and machine time. Plus, cleaning of the aluminum after it comes out of the machine (there's gunk on it from the cooling process as the machine cuts it.
Of course, with injection molding, there may be cleaning of edges.
Again, you just have to figure out how many you need. Would it be more economical to order what you need instead of spending the money for what the mold costs? Or is it cheaper to make the molds and have the parts injected molded?