Sorry for pulling up an old thread but this sounds exactly like the issue I am having. I first noticed it on my laptop using the trackpad, wasn't a huge deal as I don't usually use the laptop for more than viewing designs.
I recently bought a wireless mouse as I was doing some minor changes and the trackpad wasn't up to task, this however has the same issue.
I've done a quick screen capture showing the effect when not moving the mouse and simply scrolling up and down on the wheel:
I am a bit lost as to how this is occurring when horizontal scroll is disabled and nothing other than Ctrl and the wheel are in use.
Notice it's only panning the center of the view when zooming out past a certain level.
This is occurring because the window is fitting to its maximal extents. Altium has this bizarre concept of space, where you cannot view beyond the edges of that space (I think it's 100 inches square or something like that). And for whatever reason, they start the board in the very corner, so you can't pan over the corner, especially when zoomed out.
The very first thing I do on a new board design is, move the board outline up and right by 10 or 20 inches, and set the "origin" accordingly, so this only occurs at uselessly wide zoom levels.
Occasionally, you will move objects into this "null space" as well, whether by accident or by unfortunate library imports/changes/updates or whatever. The "Select - Outside Rectangle" command does not enumerate objects out there. The only way I know of, is to locate the objects manually: either by sorting on position coordinates on the PCB List Panel (sort by any of X1, Y1, X2, Y2; whichever are the most negative), or on the PCB Filter Panel and entering a similar query (e.g., "(X1 < -1000) AND (Y1 < -1000)"). Then click in the PCB window, M, S (Move Selection), and reposition back on the screen. Or Align to something, or etc.
Tim