Having given some fairly stream of consciousness background on my scope down-select I will now go into the final two scopes: The MSOS104A and the RTO2014. I will try to be terse.
Signal Integrity
As analog signal performance was core to this exercise I really focused on this when I evaluated the scopes.
Keysight has a very fancy ASIC at the core of their S-Series, that while older is still a very excellent performer. The S-Series comes fully calibrated up to the full 8-GHz (though you only can do this on 2 channels). It runs at 20 GSPS on 2 channels and 10 GSPS if you are using 4 channels. The RTO just runs at 10 GSPS for any number of channels. It is calibrated to the bandwidth you order though it is still just a software limit I believe.
I measured things like the SFDR with the S-Series and the RTO at low signal levels. I focused on repetitive signals that could be averaged as well to find the best DANL etc. The S-Series basically did better in these tests. The RTO showed spurs above the noise floor in many of these tests while the S-Series did not. Note these were not at high levels like the animation Keysight showed. The difference in the triggering systems was very noticeable. Trigger jitter while undesirable would have effectively spread the power of any component out. I did not really get to look too deeply at this in relation to phase noise measurements.
As shown in the Signal Path video, the S-Series is good at demodulating tasks. These are optional extras and not something I would be doing but a lot of care looks like it went into the S-Series's ASICs and analog design. This focus I felt was born out in the SFDR performance.
The RTO of course is well designed also but I did not feel the signal chain was as good as the S-Series. Of course it is an 8-bit converter which can process a lot more waveforms/s. They are different.
R&S make lot of their HD option which increases the ENOB of the system via digital filtering. I would call this emblematic of their approach to design. While I felt the heart of the Keysight was a very excellent set of analog and mixed signal ASICs, I felt the heart of the R&S was software/digital. In some ways this is more like LeCroy.
The HD option increases resolution for signals under 1 GHz, by applying a digital low pass filter after sampling before feeding the samples to the rest of the chain (see: High-Resolution Measurements with R&S Oscilloscopes; which compares it to a simple moving average filter).
As I am focused on < 1 GHz here this is a big deal. With HD mode on while SFDR and other tests were slightly worse than the S-Series it wasn't very dramatic. Everything on both scopes was of course inline with their datasheets.
If you had to go faster than 1 GHz and needed the best signal integrity, I would say: stop looking; go buy an S-Series. The new 5-Series doesn't have the bandwidth or sample rate. The LeCroys don't in their regular HDO series (though plenty of super fast fancy LeCroys out there that I have used in the past).
For a time I was very focused on the fact that the S-Series had better signal integrity by my testing at least.
Looking at the core specs, the upgrade path is unmatched for bandwidth at this price and the time base was excellent. I feel many people don't think about time bases enough. An error in "x" goes into "y" though the differential at the point. A good time base is super important.
User Interface
The Keysight user interface is fine. You can find most things in the menus without too much trouble and once you know where it is it is not hard to use. It has dedicated knobs for each of the 4 main channels and the digital signal bundle is on the front (the R&S MSO option is on the back if you get it).
I lot of people want dedicated knobs for each channel, but I think that might be a bad use of space if there are more important functions. I found the lack of other knobs and buttons compared to the front panel of the RTO made the S-Series harder to use.
The software for the Keysight is very mature but with that maturity comes a lot baggage. In general things are configured though dialog boxes. While you can use the touch screen to use these dialogs, they are text driven and on the smaller side. I think the UI works for the most part better with a mouse. I would say this probably is because of the legacy of the software and where it came from with touch being only added recently. Pinch and cursors work well though.
R&S have less time in the scope market (though I don't know how much Hameg is in the RTO line). The software they have created while not beautiful (looks like the 5-Series will be easier on the eyes), is clearly designed with using the touch screen most places. I appreciated the combination of most features into visual depictions of signal chains and acquisition settings together.
Fragmentation of averaging, filtering, etc is annoying. LeCroy has long put this into one big channel menu which is good. R&S is almost as good at putting it all together, perhaps better if you like graphical representation of the signal chain. While we all know what the options are for, no one gives all text slides for a presentation, more people are visual. Keysight and others should take note.
As I noted in the previous post I am colorblind (red-green), this affects perhaps 2-8% of the user market for Oscilloscopes.
R&S uses shared knobs for channels. While I doubt they had the colorblind in mind when they made the design choice, this means they backlit everything with RGB leds to match the selected channel color. This means for perhaps the first time ever, all of the colors (not just the ones I can change on the screen via menus which often is a feature on analysis scopes) actually match and I can tell them apart (the backlight matches screen color). Those with full color vision take color coding as a standard way to make things easy. Often the over reliance on it makes my life miserable. Here at least I have the ability to make it work well for me: a first. Others please do this if you cannot make things colorblind safe (which you could if you tried at all).
This is not a deciding feature, because I always have worked around this by moving the vertical offset a little to figure out which pattern is which, but it still matters.
Overall the UI for the RTO is better than the S-Series. The visual driven system with fewer text menus and drop boxes is just nicer to use.
There are some oddities in the RTO UI like channel math functions that use only one channel showing two channel inputs. Some colors in dialogs did not update with the channel color. This later issue is just a bug which I am sure they will eventually fix (I don't remember where it was).
Boot time
The RTO booted in 70-80 seconds
The S-Series booted in 220-240 seconds
The RTO and S-Series reached windows at about the same time, but the software for the Keysight is just bigger I guess. It loaded things like a Matlab kernel. I'm not sure if that occurs if you do not pay for the Matlab option or not but for these fully loaded demos this was what I saw. Boot times aren't ultra critical but that sort of time difference is obvious and probably could get annoying with time. Instrument boot time to ready varies a lot depending on what calibrations it must do etc. Worth noting though for a workhorse.
Noise
Neither scope was loud in testing under the loads I could give them, but the fans on the RTO were louder than the S-Series on my units. My old LeCroy LT344 was a lot louder than either and I did at times find it tiresome to listen to. I doubt most would be bothered by either the RTO or S-Series noise levels.
Math
As I mentioned above the S-Series has a Matlab option. That's cool but it costs extra. Normal channel math like +-*/, etc are there. Not as fancy as a LeCroy. The Matlab option looks from the datasheet nicely integrated into the traces though.
I personally started as an Engineer with Matlab, but I have mostly stopped using it in favor of Python. Python works for more things than Matlab and unless I needed a very special toolbox I would not choose to use it. While I can still write code in Matlab I try not too these days. Julia looks interesting and is like Matlab style wise maybe its fast execution would be helpful for scope like applications. Anyway... Matlab for advanced math and pay more for the S-Series.
The RTO's math can do more functions built-in and do compound expressions without chaining channels. This is good. Though I would take my data off the scope and use Python to look at it in detail for full analysis, for quick functions in real time the better math of the RTO is very valuable. This feature looks like it is built-in for the base model. It looks like the 5-Series might also have this though try as hard as I could I could not find any screen shot of it...
It is a little unclear how deep the external hooks are for the RTO. I found a Python tool, and saw the .NET/C# API along with the normal VISA like stuff. I'd love to see a direct realtime API for custom decoding and math in Python or Julia or something open and free. It might crimp the decode software market though...
Probes
Keysight has more probes than R&S but it looks like R&S has been expanding. They have the most common probes I might want, so its a draw really. Costs are almost the same.
Documentation
R&S has a proper manual for the RTO which is a few thousand pages in length. Keysight has the help file you would see in software. Well written documentation is undervalued and is frankly rare. I like single monolithic documents rather than fragmented documentation.
Option, options, options
To make the RTO2014 closer to the MSOS104A requires options. I would say the biggest one is the ovened reference (RTO-B4). I would not bother to get a scope costing this much without an excellent time base. With the time base option they are similarly good. You also need the HD mode (RTO-K17) and the Mixed signal module (RTO-B1).
Final bit
If you are going to stay below 1 GHz with these options the RTO has some advantages: faster waveform update rates, faster boot times, friendlier UI.
Above 1 GHz if signal integrity is your prime motive I would probably just select the S-Series.
For me although I was very focused on signal integrity, I felt like the easy built in math (without writing an m file), and UI would make my daily tasks faster with the RTO. Signal integrity was still good, and for the most difficult things I will still have to use RF techniques regardless.
I got some other options on my RTO quote that matched the current offers from Keysight. Some of these options on the R&S list for nearly 2x what they do on the S-Series. So think through your future need carefully! I'm not sure you have as much leverage for getting an option at a discount later.
Just ask sales folks for what you need at the price you need it, they will probably help you if they can. The Keysight and R&S sales staff were both helpful.