Honestly, I'd start your son off on the "free" route, at least until he decides he wants to get more serious with it and perhaps start saving up for his own rig.
For software, have a look at
OpenShot Video Editor, it's completely free and open source. Although it's not anywhere near as polished as the big players (Adobe Premiere, Adobe After Effects, AVID Media Composer etc...) it still has a familiar interface and quite a lot of features for a free product. Later on if he wants something more powerful, you could consider getting a subscription to Adobe Creative Cloud.
As for hardware, you don't really need anything special. Whilst GPU rendering is very much a thing, CPU rendering is still quite reasonable, particularly if you're talking resolutions of 720P and below. The quality of the output won't be any different, it will just take longer to render on a slower machine and previews will probably need to be reduced in quality.
A reasonably good Intel i5 or i7 will do the trick (a Xeon would be even better). RAM is important. If your son is mostly dealing with HD content, 8 GB would be the absolute minimum I would recommend. 16 GB is good, 32 GB is even better.
Ultimately it comes down to how much you're willing to spend. We can throw specifications at you all day long, but if it's beyond your budget, it won't really help you.