It used to be a general purpose interface (as the name suggests) that was used for test equipment, but also some computer peripherals and plotters. These days it's almost exclusively used for interfacing with legacy test equipment. Test equipment from the eighties will usually only have a GPIB interface. Test equipment from the nineties might also feature RS-232. USB and LAN became common on test equipment somewhere in the past ten years.
For example, the HP 3478A and 3457A DMMs that Dave has showed in his videos feature a GPIB interface. This allows you to record measurements to a computer (could have been useful for the Gaussian resistance measurement experiment) or control the instrument from a computer. You need a GPIB interface (eg. a USB-to-GPIB interface) and software that can talk to the instrument. LabView is the 800 lbs gorilla here.
New instruments might have a GPIB interface for compatibility with legacy systems, but in general it's quite rare on new equipment these days. GPIB was designed for automated test equipment systems, and has some special features for ATE like SRQ interrupts (instruments signal the controller when they have data) and parallel triggers.