That was probably just a regular pixel editing program. Gimp, Krita, or any of the commercial variants. In such programs you can import separate images in layers, and for each layer you can set opacity (transparencey), scaling, and you can limit any operation to a subset of the layers (such as drawing lines, erasers, color correction, etc). If you post a screenshot, or a link of what the GUI looks like, the program is probably easy to identify.
On a side note. in KiCad you can use
PCB Editor / Place / Add Reference Image to add such images directly into the PCB editor, and you can scale them and set opacity. This way you can place footprints and copper tracks directly over the image graphics.
It's very easy to find plenty of examples with a simple search:
https://www.youtube.com/results?search_query=pcb+reverse+engineeringFor this kind of work "high quality" pictures are important. but "high quality" here is in different areas then for general photography. Colors are not high on the list, but shadows can be annoying. Barrel distortion is one of the first things to attempt to avoid. Taking the picture from a long physical distance, and then zooming in on the PCB helps to reduce distortion. Resolution also does not need to be very high. As long as text on IC's is readable, and the thinnest tracks are 5 or more pixels wide, the resolution is high enough.
Having experience with such graphical programs is a real benefit. Rotation and barrel distortion can be corrected quite easily. It is also possible to first load images in KiCad, and when you see halfway though the process that a part of the picture can not be lined up with the PCB, you can do things like:
- Export a picture of the PCB from KiCad.
- Load it in your favorite graphic program on a separate layer.
- Use the graphic program to locally "stretch" the picture to make it fit the PCB better.
- Use these newly generated pictures and put these in KiCad's PCB editor.
But if the distortions are small enough, you can of course also just shift the picture a bit.
For completeness: There is specialized software for this, but you'd have to ask to get a price quote. There are also companies who do this commercially. You can send them a multi layer PCB, and they send you back high resolution scans or even Gerber files of each layer. The PCB is usually destroyed in the process, as each layer is removed after scanning to expose the layer beneath it.