Major aggro I find with all the Dremel-type units is the collet chucks just letting go in the middle of a job. A unit with a proper keyed chuck would save a lot of swearing.
I have the same issue with my particular Dremels because of the vibration and/or runout, but to be fair, I can't adjust the speed on my mains Dremel; it's 35K, all the time. (And my Li ion Dremel has crazy runout). The main bits I use in my Dremels are 99% cutoff wheels, 1% nondirection carbide bur, non directional cutters, felt polishing wheels, or stone grinding bits. Stuff that doesn't "dig" itself out of the tool. My Dremels are pretty much useless at freehanding with endmills. It's too herky jerky, even if the bit would stay in.
With Proxxon, I use carbide endmils, freehand, all the time. It's probably the most common bit I use in my Proxxons. And despite Proxxon having excellent 3 slot steel collets, I use the adjustable chuck adaptor for everything. Even the Proxxon in my router table wears this adjustable chuck, despite I hardly ever change that bit from 3mm endmill. It makes no difference in performance that I can tell (although it probably isn't great for the life of the tool, considering the extra length and resultant side load). Smoothness and balance fundamentally changes what you can do with a rotary tool. (I'm not suggesting that Dremel doesn't make tools that aren't smooth enough to do this stuff; just that my 2 Dremel do not fall into this category).
Re Dremel/Proxon both are good but if you are using either with side loads like a Mill then expect issues with keeping straight. The Bits and Connecting Bars will flex for a start before you even get to the tool and it's bearings.
Dunno about Dremel, but I mill plastics, sheet metal, wood, and FR-4 with my Proxxon router table. No problem at all. The main limitation is the torque, not the herkyjerk, broken bit, or magic smoke.
Dremels are for freehand work ... You wouldn't use a drill press as a milling machine, and you shouldn't try to use a Dremel as a drill press. They are very different tools for very different purposes.
Exactly. A high speed rotary tool is more like a mill motor than a drill. Milling/routing goes much better when the mill motor is solidly mounted and the material is fed in a controlled manner. Hence, accessories like router tables and drill presses can be useful (if they are well designed and manufactured). There is rare occasion one should actually choose a rotary tool to properly drill things (with a regular fluted drill bit), but drilling is not the only way to make (or to enlarge/shape/finish) "holes," including "holes" that are not round. And since this is EE forum, some of us actually have this rare application where you might want to drill at high RPM. Carbide drill bits will punch super tiny holes in PCBs very rapidly and cleanly, provided the RPM is high(er than a proper drill press). Suggestion to "buy a REAL drill press" is missing your own point. The "real" upgrade to a rotary tool is a die grinder or router or mill motor. And with proper accessories, you can do more. With just a mount, a flat table, and a fence you can mill straight slots, do radial symmetry, edge jointing/plane-ing, follow cuts parallel to an outer reference edge (that could be made with a saw or belt sander), all of this is in addition to more control and cutting speed in free-hand shaping (with the additional benefit of having a fixed perpendicular bit angle, if not adjustable). When you figure out how to take advantage of all this stuff, it means you can make more complex things. And most of this stuff can aid in shaping/cutting/milling thin, flat things like enclosure front/back/top/sides, which is presumably what the OP is about.
In short, yes. 15-35k RPM is, indeed, too fast to drill almost everything. Aside from plastic, this is a disaster for wood or metal or just about anything. But it is in the right ballpark to mill or grind almost anything. But if the tool vibrates and chatters out of control, you will never know. Esp considering the small radius of bits you will use in such a tool, a rotary tool might be too SLOW for a lot of milling operations. I would give the OP benefit of doubt and suppose he is interested in milling openings in an enclosure. If he only wanted to drill standard size round holes, you don't need a press of any kind to do that in 1/8" plastic and sheet metal. A hand drill is fine.
Fun fact: you can turn the basic Proxxon drill press into a spring loaded arbor press by moving the spring from the top to the bottom. (You would get it if you look at it). This can be useful for milling, if you don't have a router table. Lift, insert the stock, then let it drop into the hole/slot you are wanting to mill. I occasionally use one of mine* to do stuff like this when I want to do it at my main workbench, with the comfy chair and good lighting. This spring loaded arbor configuration is also useful if you ever have to flash a bunch of PCB via ICSP. But if you want to mill anything thick with such a weak tool, you will want a very quick and easy way to adjust the height of the bit. Gotta take shallow cuts. This is where Matthias's router table design and a quickly adjustable fence are key.
*I actually have found enough uses to own two of these Proxxon drill presses. And yeah, you know I have a real drill press, too.