The factory default settings of the fuses on a m328 are to run using the internal 8MHz clock, additionally divided by 8, to yield a final CPU frequency of 1MHz. (This has the advantage of always "working" across the entire voltage range of the chip.) Normally you connect a crystal and set the fuses to (1) use the crystal, and (2) not divide by 8.
The "calibration" is to fine-tune the internal clock, so that it would run 8.0 MHz rather than 7.3MHz or 8.1MHz (that's supposed to be the full range of possible calibration.) Improper calibration shouldn't cause the chip to run noticeably slow.