To answer the original question as directly as possible, "Can I just run audio straight into my scope without a probe?" - the answer is yes, but lots of things are simpler and better if you just use the probe.
Signal integrity (you don't care): When dealing with high frequency systems, impedance matching is important. The scope has either a 1 MOhm input impedance or a 50 Ohm (cheaper scopes aren't selectable, and are just 1Mohm). Typical coax systems are 50, 75, or 100 ohms. When you mix impedances, the junctions can cause things like reflections and standing waves, such that what you measure in one spot (the scope itself) isn't the same as what is elsewhere in the system on the same electrical net. For example, you are measuring a 50-ohm source through 50-ohm coax into a 50-ohm load. Let's say you put a BNC tee in the middle somewhere and just run another bit of coax into your scope. If it's terminated properly, you won't see any reflections or standing waves or anything like that: what you measure at the scope is really what's happening on the line. If you have it in 1Mohm impedance mode, this isn't true: what the scope measures and displays may well be different from some arbitrary point elsewhere on that transmission line, like at the source or load (what the scope sees isn't what the load sees, perhaps). A properly engineered probe actually has losses in it to prevent this from being a problem when you're measuring real systems. Basically, what it does is present a 1Mohm or 10Mohm impedance AT THE EXACT POINT of the probe tip, so if you're measuring a transmission line you won't inadvertently add a big bit of line that can mess up the measurement by adding reflections. But at audio frequencies this isn't really a big concern.
Safety: The probe is most useful in 10x mode and people usually leave it there. In 10x, it has an impedance of 10Mohm, which is to say it's basically 10-times safer from a voltage standpoint. With a 10x probe you could expect to measure 1000v without anything blowing up, whereas if you run that into the scope it will CERTAINLY damage it. Not that doing so is SAFE by any means. The important part here is that if you don't know exactly what you're measuring, but expect it should be safe for a scope, start the measurement with a 10x probe. Only switch to 1x when you verify that it's safe and desirable (by the way, you'll reduce your measurement bandwidth by 10x in most cases, though for audio no big deal). Someone above mentioned "it's not like you're measuring 100v peak to peak," but the thing with audio is you actually might be. If you're measuring the output side of an amplifier, the voltages can actually be kind of high. Take my car subwoofer, 1200w at 4 ohms. Power=V^2/R, and if you solve for V, it's 70 in this example. That's RMS power, so RMS voltage. 70V rms on a pure sine wave is actually 100V.
The other point about safety, as someone mentioned, is that scope ground is earth ground. I measured a system once that had a negative rail which was supposed to be floating. In fact, it was held at 12V above earth by an internal short, and this was a supply capable of sourcing 100A. When I hooked up the ground lead to my scope, it literally caught fire (it turns out, scope chassis can handle a lot of abuse). The proper way to make sure this isn't a problem is to take a DC measurement of the 'negative' to earth with a multimeter. But if you're being cautious, you can make a poor man's differential probe by using channel A for positive, channel B for negative, short the grounds together, and use math to calculate B-A.