Even a regular DC motor is going to require power to hold position. If you want to be able to completely disconnect the power, and you do, you need some way of holding position. A worm gear drive for rotation and perhaps a lead screw drive for tilt might be a good way to go. Something where the mechanics of the system hold position with or without the motor.
As you are using LDRs to track the Sun, you don't really need a stepper motor. However, the stepper has the advantage of moving a predictable amount for every step but, if you plan to depower the motor, you need to remember the state of the outputs so that the motor will cog back to the same relationship (assuming the rotor hasn't drifted too far). But this will also be a problem with a DC motor used as a servo as well.
It's interesting to use LDRs to try to locate the sun. That won't work quite as well on cloudy days but it may still be ok.
Personally, I would use one of the celestial navigation formulas that apply to my Latitude and predict where the Sun is going to be, clouds or not. Telescope mounts do this all the time.
Reed's Almanac used to have a section on celestial navigation. One of the neat features was a 5th degree polynomial used to predict the location of the Sun for a given Latitude and Longitude. The equation was derived annually by the Royal Observatory in London. I used to have it programmed into my HP 48GX calculator - back when I was involved with that kind of thing. Alas, I haven't seen it in a long while.
There are suitably accurate equations that don't need to be calculated on an annual basis. It's not like you're trying to make landfall.
I think you can get quite a bit of cogging out of a stepper motor if you short the windings. Some H-Bridges may include that capability. But you would still need to power the driver, just not the motor.
References to Sun tracking are included here:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3355400/