See user guide I use here:
https://www.batronix.com/files/Siglent/Spectrum-Analyzer/SVA1000X/SVA1000X_QuickGuide_QG0701X_E02A.pdf Look at 5.2.1.5. Filter Setup section. It mentions Nyquist and Sqrt Nyquist filters (which is what I used) but in UI they show as Raised Cosine (RC) and Root Raised Cosine (RRC), respectively. There is a small table showing what filter to select depending on the TX filter used. Looks like since your TX uses Gauss, on the Siglent SA you should select "None" for the measure filter and "Gauss" for the reference filter. In my case my TX used RRC, so on the Siglent SA I selected "RRC" for the measure filter, and I was supposed to select "RC" for the reference filter but I had better results selecting "RRC". My understanding is that RRC * RRC = RC so if both the TX and RX use RRC the total system filter is in fact RC. This matches the info in the Keysight note.
For the alpha/beta factor I experimented with 100% synthetic QAM signals form QAM4 to QAM256, generating the baseband IQ signal, upsampling, upconverting to a carrier frequency, adding artificial SNR, dithering to minimize quantization bias, then quantizing to a WAV file in 8, 16, 24, 32, float, and doubles, at various sample rates, without transmission, then reading the WAV file and decoding the message very successfully. My code chain added guards of various lengths on either side (to allow decoding of last few symbols due to edge effects), and a preamble: guard + preamble + message + guard. Whatever alpha factor I chose in the TX had to be matched in the RX. Not only that but the RRC introduces GD that had to be trimmed exactly by both TX and RX (where the GD = (len(RRC) - 1) // 2) where the length is in samples (not in symbols like Siglent), otherwise the sample selected to read the symbols would not align perfectly and all decoded symbols would be wrong. Also found that for odd SPS the offset for the optimum sample to read symbols at was 0, but for even SPS the optimum offset was 2. At least in 100% synthetic signals. My preamble was a random but known sequence so the RX could de-scale the signal being quantized to a WAV file (otherwise the RX constellation would have a scale and offset problem).
The alpha factor, even with 100% synthetic signals, makes a huge difference, going from 100% decode on QAM-256, to not decode on QAM-4 if I changed the alpha factor (while keeping it the same on both synthetic TX and RX). I think this is because I fine tuned all the other values (guard length, preamble length, SPS, and number of positive lobes NPL for my RRC filter) for that one given alpha, and different alphas cause different ISI and edge effects. In the case of my RRC the length of my RRC was 2 * rrc_npl + 1 (where rrc_npl is the RRC number of positive lobes NPL, for which I settled on 5, other values drastically affected everything). Conversely, the same NPL and ALPHA worked well for all baud rates, SPS, and WAV quantization. Surprisingly quantizing WAV to 16 or even 8 bits had little effect (I can 100% decode QAM-256).
So I don't have the answer and need to experiment more but the filter parameters are critical and obviously if the RX does not know exact values would have to fiddle with them until RX is optimum, but start by selecting the correct measure and reference filter type, then only ALPHA and LENGTH of the filter are left to fiddle with.