For the source code downloaded sometime around the end of March 2018
The relevant file is kicad-source-mirror/common/view/wx_view_controls.cpp
The pan function is called bool WX_VIEW_CONTROLS::handleAutoPanning( const wxMouseEvent& aEvent )
Somewhere around line 563 is where the wx_timer is started that handles the auto pan speed.
What is really needed is for this to be set from an external config file but this lack of configurability is something that seems like it will never be fixed in kicad it has been like this for many years.