No one has come right out and said it yet, but you should probably start with existing components (eval boards, etc) before diving into the full design. That way you can explore the limitations of a given approach before spending a lot of time on it.
I agree with Kalvin that a good place to start is with a 24 bit, 4 channel audio codec (or just an ADC, because it sounds like you won't need any outputs). Most of these use an I2S interface to communicate with the host, with an additional I2C bus for configuration (I2C is not generally fast enough for audio rate data).
I designed a stereo audio board that's compatible with a variety of hosts (
https://hackaday.io/project/5912-teensy-super-audio-board), but it only has 2 inputs. This project extended it using a multi-channel codec (2 stereo input channels, 4 stereo outputs) (
https://hackaday.io/project/9634-linux-based-low-latency-multichannel-audio-system), but it's only compatible with BeagleBone Black.
If you are restricted to using a USB uplink to a PC host, you can explore using a small micro to communicate with the audio codec/adc, and then send it upstream to the PC using a USB audio link. I know that there was some work to get something like that working on the Teensy (
pjrc.com). The Teensy even has working quad input I2S using Paul's audio boards (
https://www.sparkfun.com/news/2055). Doing quad input I2S at 192kHz is probably outside of the capabilities of a Teensy 3.x, however.
If you don't want to go the microprocessor, or RPi route, there are ICs available that act as a bridge between an I2S interface and a USB audio interface.