You may want to consider completely redoing the input stage with something different.
Since you have access to the two differential inputs without the risk of knocking it off its your oyster so to speak. There is a
lot of opportunity to learn about RF visually with that setup.
Its really almost an ideal learning setup.
The input on the RTLSDR at pins 4 and 5 is around 3000 ohm differential input.
Its suceptible to ESD, so if you build your own whatever, keep that risk in mind.
Try to filter out everything above 24 MHz (since your dongle by itself can receive above 24 Mhz on its own.)
I would concentrate on the lower part of that range since we're near the minimum of the sunspot cycle. An LC network can match the antenna impedance to the input impedance.
Especially do whatever it takes to electrically decouple your receiving setup somewhat from the noise thats often found riding on power lines.
In a pinch, even just a bunch of wire (i.e. an air core inductor) will work.
But its best if you have an appropriate magnetic material you can use for a core.
Try to keep the length of wire between your unun and your ground connection as short as possible.
Another thing that may potentially help a lot is to decouple the coax from your receiver by wrapping it as many times as you can around a ferrite rod or toroid or even just make a tightly would coil using the coax.. creating an RF choke. before it gets to the receiver.
I have done this with RG-174 and Ive been able to get a lot of turns in there because its so thin. (The loss in RG-174 is still pretty modest for HF, so this helps a lot.)
If its convenient, try to put your ground rod somewhere where the soil stays wet year round. (like under a downspout)
An interesting antenna thats a fairly good choice for people without the room to erect a really long wire is the so called "mini-whip" e-field probe antenna.
Its an amplified antenna and it usually gets its power via its feedline.
Its feedline is used in part of the antenna but it needs to have a big RF choke to prevent the nise indoors from making its way out to it.
The active unit is tiny, its basically a capacitive probe that sticks up into the air as high as possible. The antenna itself has a very high impedance preamp.
Its performance depends on it being put in a quiet location.. (outside of a house, and as far away from e-stuff as possible.)
Then ground the coaxial cable outside before it enters the house, then put a common mode choke on the coax before it gets to the receiver.
https://www.google.com/search?hl=en&as_q=pa0rdt+mini+whip