Okay, I figured this out and it was kind of stupid. I checked on my Lenovo running Debian 10 and the sound was set to "HDMI/DisplayPort Built-in Audio" and I noticed on the HP it was set to "Analog Stereo Out".
Hiding out to the right in the Pulseaudio window, past the border where it wasn't visible, was one more tab for "Configuration". There was a small arrow thingy that I just hadn't noticed, but geez. Seriously the window isn't that big, why not make it so all 5 tabs are in view by default?
Anyway, on that tab was a listing for many output devices (most of which were unavailable), setting it to "Digital Stereo (HDMI) Out" fixed the problem. Not sure why "Analog Stereo Out" wasn't giving stereo out, or rather both channels to the left side, but I'm not gonna worry about that now.